dashscope-confidential-maas

README

Confidential Model as a Service on Alibaba Cloud Model Studio

中文版：README_zh.md

Project Overview

CMaaS (Confidential Model as a Service) is an attested inference solution built on Trusted Execution Environment (TEE, e.g., Intel TDX) technology. It addresses data privacy challenges in AI inference through TEE isolation, end-to-end encryption, and remote attestation.

The system builds trust through three key dimensions: Secure Channel for end-to-end encrypted communication, Attested Node for verifiable execution environment, and Verified Code for auditable software provenance. Together, these dimensions provide comprehensive security guarantees:

• End-to-End Encryption An application-layer secure channel is established on top of HTTPS between the client and the confidential inference node. Each TEE node generates a random root private key for session key negotiation, with an ephemeral key derived for each session. All keys are held in memory on their respective side: client memory or the node's TEE-protected memory. Neither the SDK nor the inference service persists them to disk.

• Security Hardening & Integrity Measurement Leveraging a dual-layered defense of chip-level hardware isolation and software hardening, the system performs full-stack integrity measurement of the boot chain and runtime environment. At the block storage level, the root partition's integrity is protected by a Merkle Tree, while the data partition is encrypted with one-time keys to ensure confidentiality.

• Software & Hardware Transparency Clients can directly verify the authenticity of the hardware-generated remote attestation report with the TEE vendor. Furthermore, they can inspect a public transparency log to confirm the integrity of the server-side software artifacts.

For the complete system security architecture design, see the architecture document.

How to Use

Overview

The CMaaS Client SDK abstracts the complexity of various security hardening into simple, easy-to-use APIs, enabling developers to integrate confidential inference capabilities into their applications with minimal code changes.

• SDK Library (sdk/): Core client implementation with attestation, verification and encryption
• CLI Tool (cmd/cmaas-client/): Command-line tool demonstrating SDK usage
• OpenAI-Compatible Proxy (cmd/openai-proxy/): HTTP server that exposes /v1/chat/completions and /v1/responses backed by the SDK, so existing OpenAI clients can access the confidential inference service without code changes
• Anthropic-Compatible Proxy (cmd/anthropic-proxy/): HTTP server that exposes /v1/messages for the Anthropic Messages API, so Anthropic SDKs and Claude Code can target the confidential inference service by only changing base_url
• Audit Tool (cmd/cmaas-audit/): Offline verification of local transparency reports

The SDK manages the following areas:

1. Transport Management: Manages same-node routing, concurrent requests, and automatic session recovery over the Model Studio HTTP/SSE path
2. TEE Attestation: Verifies TEE Quote signatures and EventLog integrity, with public key caching to reduce overhead in load-balanced deployments
3. Transparency Log Verification: Verifies release manifests against transparency logs service, ensuring the inference node runs a known, audited software version
4. Encrypted Requests: Implements RA-based key negotiation and end-to-end encryption for all inference data

The design ensures that plaintext prompts and responses never leave the client's local environment or the TEE's isolated memory, providing end-to-end confidentiality throughout the inference lifecycle.

Quick Start (CLI)

The CMaaS CLI serves as a reference implementation and testing utility for confidential inference.

Build and Run

# From the repository root
make client

export DASHSCOPE_API_KEY=your_api_key
bin/cmaas-client \
  --endpoint https://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api/v1/services \
  --model    <code> \
  --prompt   "What is confidential computing?" \
  --stream

CLI Options

Flag	Required	Default	Description
--endpoint	Yes	(none)	Model Studio gateway endpoint
--model	Yes	(none)	Model name for gateway routing
--prompt	Yes	(none)	Inference prompt to send
--stream	No	false	Enable streaming mode
--transparency-report-dir	No	OS-specific	Directory for local transparency reports
--no-prompt-in-transparency-report	No	false	Exclude prompts from local transparency reports

Note: The DASHSCOPE_API_KEY environment variable must be set for authentication.

Quick Start (SDK)

The SDK provides Client and SessionManager APIs, both with attestation, request encryption, and response decryption.

The ChatCompletionRequest sampling parameters (e.g. Temperature, TopP, MaxTokens) follow the naming conventions of the OpenAI Chat Completion API for familiarity. The SDK also supports function/tool calling (Tools, ToolChoice, ParallelToolCalls), reasoning models (ReasoningContent/Reasoning fields and ThinkingTokenBudget), and text, reasoning, logprobs, and tool-call fragments through the unified Chunks streaming channel.

Client

Client selects a model at initialization and sends inference requests through Chat.

client, err := sdk.NewClient(&sdk.Options{
    Endpoint: "https://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api/v1/services", // Model Studio gateway endpoint
    APIKey:   os.Getenv("DASHSCOPE_API_KEY"),                                  // Model Studio API key
    Model:    "<code>",                                                        // Model code on Model Studio
})
if err != nil {
    return err
}
defer client.Close()

completion, err := client.Chat(context.Background(), &sdk.ChatCompletionRequest{
    Messages: []sdk.Message{
        {Role: "user", Content: "What is confidential computing?"},
    },
    MaxTokens:   sdk.Int(2048),   // maximum response tokens (adjust as-needed)
    Temperature: sdk.Float(0.7),  // temperature (adjust as needed)
})
if err != nil {
    return err
}

// Decrypted inference result
fmt.Println("Response:", completion.Content)

// Inspect attestation result
fmt.Printf("Verified At: %v\n", completion.Attestation.VerifiedAt)
fmt.Printf("TEE Type: %s\n", completion.Attestation.Bundle.TEEType)

SessionManager

SessionManager accepts a model on each Chat call and manages encrypted sessions by model and request affinity. Requests with the same model and affinity reuse a session whenever possible, reducing repeated key negotiation and providing affinity routing.

manager, err := sdk.NewSessionManager(&sdk.SessionManagerOptions{
    Options: sdk.Options{
        Endpoint: "https://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api/v1/services",
        APIKey:   os.Getenv("DASHSCOPE_API_KEY"),
    },
})
if err != nil {
    return err
}
defer manager.Close()

completion, err := manager.Chat(context.Background(), "<code>", nil, &sdk.ChatCompletionRequest{
    Messages: []sdk.Message{{Role: "user", Content: "What is confidential computing?"}},
    MaxTokens: sdk.Int(2048),
})
if err != nil {
    return err
}
fmt.Println("Response:", completion.Content)

Streaming Inference

Both APIs enable streaming with Stream: true and return incremental results through Chunks. The example below uses Client; for SessionManager, replace the call with manager.Chat(ctx, model, nil, req). Result handling is the same.

completion, err := client.Chat(ctx, &sdk.ChatCompletionRequest{
    Messages: []sdk.Message{{Role: "user", Content: "Write a story"}},
    Stream:   true,
    MaxTokens:   sdk.Int(2048),
    Temperature: sdk.Float(0.7),
})
if err != nil {
    return err
}

// Read and print text from the unified streaming channel
for chunk := range completion.Chunks {
    fmt.Print(chunk.Content)
}
fmt.Println() // Newline after streaming completes
if completion.Error != nil {
    return completion.Error
}

// Inspect attestation result
fmt.Printf("Verified At: %v\n", completion.Attestation.VerifiedAt)
fmt.Printf("TEE Type: %s\n", completion.Attestation.Bundle.TEEType)

Consume Chunks until the channel closes, then check completion.Error. Cancel the request context if you stop reading early. Both APIs cancel unfinished streams on Close() and wait for their related transparency audit reports to finish writing before returning.

See the standalone examples for reasoning output and tool calls.

Image Input

Both APIs support text and image messages through Message.ContentParts when using a model with image input support. Images are passed as Base64 Data URLs; public URLs are not currently supported as direct inputs. The following example uses a JPEG image:

imageBytes, err := os.ReadFile("image.jpg")
if err != nil {
    return err
}
dataURL := "data:image/jpeg;base64," + base64.StdEncoding.EncodeToString(imageBytes)

completion, err := client.Chat(context.Background(), &sdk.ChatCompletionRequest{
    Messages: []sdk.Message{{
        Role: "user",
        ContentParts: []sdk.ContentPart{
            {Type: "text", Text: "Describe this image."},
            {Type: "image_url", ImageURL: &sdk.ImageURL{URL: dataURL}},
        },
    }},
})
if err != nil {
    return err
}
fmt.Println(completion.Content)

SDK Options

Common Options fields are listed below. Client uses Options directly; SessionManager uses SessionManagerOptions.Options for the same gateway, security verification, and audit settings. Model is set at initialization only for Client; SessionManager takes the model in each Chat call.

type Options struct {
    Endpoint string     // Model Studio gateway endpoint (required)
    APIKey   string     // Model Studio API key (required)
    Model    string     // Model code on Model Studio (required for Client)
    ProxyURL string     // Optional HTTP(S) forward proxy for gateway and PCCS

    PubKeyCacheTTL           time.Duration // Public key cache TTL (default: 12h, -1 to disable)
    GatewayExtraHeaders      http.Header   // Additional Model Studio gateway headers

    TransparencyLogs          []sdk.TransparencyLogConfig // Trusted transparency log instance configurations
    TransparencyReportDir     string                      // Audit report directory (default: OS-specific)
    NoPromptInTransparencyReport bool                     // Exclude prompts from local transparency reports
    DisableTransparencyReport bool                        // Disable local audit report persistence

    AllowAttestationFallback bool // Allow automatic fallback on attestation failure (default: false)
}

Options Details:

• Endpoint: Model Studio gateway endpoint.

• ProxyURL: Optional HTTP(S) forward proxy for gateway and PCCS requests. Overrides environment proxy settings; leave empty to preserve default behavior.

• PubKeyCacheTTL: Remote attestation verification requires fetching collateral from the TEE vendor and querying transparency log inclusion proofs, which may introduce significant latency. To reduce this overhead, the SDK caches verified server public keys. In load-balanced deployments, the client may connect to multiple backend nodes, and the cache significantly accelerates subsequent connections to previously verified nodes. Set this value to -1 to disable the cache and force full verification every time.

• GatewayExtraHeaders: Headers added to Model Studio gateway requests.

• TransparencyLogs: Configures trusted transparency log instances. When empty, the SDK uses its built-in Rekor configuration.

• TransparencyReportDir: Specifies the directory where transparency reports which are auditable are stored. When empty, OS-specific defaults are used:

  • macOS: ~/Library/Application Support/dashscope-confidential-maas/transparency-reports/
  • Linux: $XDG_DATA_HOME/dashscope-confidential-maas/transparency-reports/
  • Windows: %APPDATA%\dashscope-confidential-maas\transparency-reports\

• NoPromptInTransparencyReport: Set to true to exclude prompts from local transparency reports while preserving all other report fields and attestation materials.

• DisableTransparencyReport: Set to true to disable local persistence of transparency reports. (Not recommended.)

• AllowAttestationFallback: Disabled by default. When enabled, failures in remote attestation reports, transparency logs, or measurement verification may fall back instead of aborting. Ephemeral-key signature verification and key derivation remain mandatory, but the returned result has a zero VerifiedAt.

SessionManager session settings:

Option	Default	Description
SessionsPerModel	8192	Maximum cached sessions per model, with LRU eviction
SessionTTL	480s	Idle session TTL

Environment variables:

The following environment variables apply to applications built on the SDK.

• CMAAS_GATEWAY_EXTRA_HEADERS: Sets headers added to Model Studio gateway requests as comma-separated name=value pairs. Its priority is lower than GatewayExtraHeaders.
• CMAAS_DEBUG: Set to true to enable SDK diagnostic logs. The two proxies also log complete plaintext HTTP request and response bodies.

Session Affinity

SessionManager automatically derives session affinity identifiers from commonly used agents' session identifiers and request characteristics such as content prefixes, helping maintain affinity between related requests where possible.

To specify session affinity explicitly, set X-DashScope-CMaaS-Proxy-Affinity: pass it through the headers argument to Chat when using the SDK directly, or as an HTTP request header when using a proxy.

Compatible HTTP Proxies

For integrations that already speak OpenAI or Anthropic protocols, the repository ships two local HTTP proxies built on the SDK. Starting with v0.3.2, both proxies are built on the SessionManager API in place of the previous Client API.

Note: Both proxies handle plaintext API requests and must run within the user's trusted environment.

Both proxies listen on 127.0.0.1:8080 by default. To run both proxies on the same host, set a different --listen port for one of them. To accept connections from other hosts, explicitly set --listen to a non-loopback address (for example, 0.0.0.0:8080) and enforce appropriate network access controls; do not expose an unprotected proxy directly to an untrusted network.

OpenAI-Compatible Proxy (cmd/openai-proxy/)

Implements the relevant subset of the OpenAI HTTP API, with support for streaming, function/tool calling, and reasoning model fields (reasoning_content, thinking_budget, enable_thinking).

# From the repository root
make openai-proxy

export DASHSCOPE_API_KEY=your_api_key
bin/openai-proxy \
  --endpoint https://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api/v1/services \
  --listen   127.0.0.1:8080

Endpoints:

• POST /v1/chat/completions — OpenAI Chat Completions (streaming and non-streaming)
• POST /v1/responses — OpenAI Responses (streaming and non-streaming)

Flags:

Flag	Required	Default	Description
--endpoint	Yes	(none)	Model Studio gateway endpoint
--model	No	(none)	Comma-separated model list
--served-model-name	No	(none)	Model alias; requires a single-model configuration
--listen	No	127.0.0.1:8080	HTTP listen address; explicitly use a non-loopback address for remote access
--sessions-per-model	No	8192	Maximum cached sessions per model
--session-ttl	No	480s	Idle session TTL
--workers	No	0	No longer has any effect
--max-concurrency	No	0	Maximum concurrent requests (0 disables the limit)
--cache-ttl	No	12h	Public key cache TTL (0 disables the cache)
--transparency-report-dir	No	OS-specific	Directory for local transparency reports
--no-prompt-in-transparency-report	No	false	Exclude prompts from local transparency reports

Parameter and endpoint removal notice: Starting with v0.3.2, the --model, --served-model-name, and --workers flags are deprecated. v0.4.0 will remove these three flags and the /v1/models and /models endpoints.

Starting with v0.3.2, each request selects its inference model through model, independently of the --model list.

DASHSCOPE_API_KEY must be set in the environment.

OpenAI-Compatible Chat

Example using curl for chat completions:

curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "<code>",
    "messages": [
      {"role": "system", "content": "You are a helpful assistant."},
      {"role": "user",   "content": "What is confidential computing?"}
    ],
    "temperature": 0.7,
    "max_tokens": 128,
    "stream": false
  }'

Example with the OpenAI Python SDK (chat completions):

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8080/v1", api_key="ignored")
resp = client.chat.completions.create(
    model="<code>",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user",   "content": "What is confidential computing?"},
    ],
    temperature=0.7,
    max_tokens=128,
)
print(resp.choices[0].message.content)

Note:

• For image input in Chat Completions, use image_url content parts. The image_url.url field currently accepts only Base64 Data URLs, not public URLs.

OpenAI-Compatible Responses

Example using curl against the responses endpoint:

curl http://localhost:8080/v1/responses \
  -H "Content-Type: application/json" \
  -d '{
    "model": "<code>",
    "instructions": "You are a helpful assistant.",
    "input": "What is confidential computing?",
    "temperature": 0.7,
    "max_output_tokens": 128,
    "store": false,
    "stream": false
  }'

Example with the OpenAI Python SDK (responses):

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8080/v1", api_key="ignored")
resp = client.responses.create(
    model="<code>",
    instructions="You are a helpful assistant.",
    input="What is confidential computing?",
    temperature=0.7,
    max_output_tokens=128,
    store=False,
)
print(resp.output_text)

Note:

• For image input in Responses, use input_image content parts. The image_url field currently accepts only Base64 Data URLs, not public URLs.
• Responses supports stateless requests only. Parameters related to stored conversations and background execution (previous_response_id, conversation, background) are unsupported. Built-in tools such as web search and code execution are unsupported.

Anthropic-Compatible Proxy (cmd/anthropic-proxy/)

Exposes the Anthropic Messages API so the official Anthropic SDKs and the Claude Code CLI can be pointed at a CMaaS backend. The proxy translates the Anthropic protocol into the SDK's OpenAI-shaped request, and translates the response back into Anthropic-style streaming events (message_start / content_block_* / message_delta / message_stop).

For text and image messages, pass images as image content blocks. The source field currently accepts only Base64 image data (type="base64", media_type, and data), not public URLs.

# From the repository root
make anthropic-proxy

export DASHSCOPE_API_KEY=your_api_key
bin/anthropic-proxy \
  --endpoint https://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api/v1/services \
  --listen   127.0.0.1:8080

Endpoints:

• POST /v1/messages — Anthropic Messages API (streaming and non-streaming)

Flags are generally the same as the OpenAI proxy. DASHSCOPE_API_KEY must also be set in the environment.

Example using curl against the messages endpoint:

curl http://localhost:8080/v1/messages \
  -H "Content-Type: application/json" \
  -d '{
    "model": "<code>",
    "max_tokens": 128,
    "messages": [
      {"role": "user", "content": "What is confidential computing?"}
    ],
    "stream": false
  }'

Example with the Anthropic Python SDK:

import anthropic

client = anthropic.Anthropic(base_url="http://localhost:8080", api_key="ignored")
resp = client.messages.create(
    model="<code>",
    max_tokens=128,
    messages=[{"role": "user", "content": "What is confidential computing?"}],
)
print(resp.content[0].text)

Both proxies log the locally generated local_request_id and, when available, the gateway-returned request_id to support troubleshooting.

Environment variables:

• CMAAS_ALLOW_ATTESTATION_FALLBACK: Set to true to enable attestation fallback. It is disabled by default, and either proxy emits a security warning when it is enabled.

Attestation and Auditing

Attestation Result

Each inference will bind an AttestationResult that provides transparency into the TEE server's identity and integrity:

• VerifiedAt: Timestamp when the attestation was performed
• Bundle: Contains the full attestation bundle including TEE type, server public key, TEE evidence, and transparency data

Call result.TEEDetailsJSON() to extract TEE-specific key fields as JSON. For Intel TDX environments (currently supported), this includes:

• MRTD: Initial TD measurement (VM configuration and initial code)
• RTMR[0-3]: Runtime measurement registers for firmware, kernel, and application layers
• TD Attributes: Trust domain configuration attributes
• Debug Enabled: Whether the TEE is running in debug mode

TDX Collateral Service and Cache

TDX Quote verification requires collateral from Intel PCS or a PCCS caching service. The SDK uses Alibaba Cloud PCCS by default and also supports specifying another service through an environment variable or the QCNL configuration file.

Related settings

• PCCS_URL: Specifies the PCS/PCCS service URL. It takes precedence over PCCS_URL in the QCNL configuration file /etc/sgx_default_qcnl.conf and the default Alibaba Cloud PCCS. For example, in a cn-beijing VPC environment, set PCCS_URL=https://sgx-dcap-server-vpc.cn-beijing.aliyuncs.com, or set PCCS_URL=https://sgx-dcap-server-vpc.cn-beijing.aliyuncs.com/sgx/certification/v4/ in the QCNL configuration file.
• PCCS_EXTRA_HEADERS: Sets headers added to PCS/PCCS requests as comma-separated name=value pairs.

Transparency Log Verification

Transparency log verification strengthens the trust chain by ensuring the inference node runs a known, audited software version—not just any code that happens to produce valid TEE measurements.

During key negotiation, the SDK automatically performs the following transparency verification steps:

1. Verify Log Entry Integrity: Validates each proof entry's DSSE envelope signature, RFC 6962 Merkle inclusion proof, and signed checkpoint against the transparency log's public keys
2. Verify Payload Hash: Confirms that the release manifest's SHA-256 hash (JCS-canonicalized) matches the payload hash recorded in the log entry
3. Validate TEE Measurements: Validates the TEE measurements from the TEE evidence (for TDX: measurements from Quote, UKI hash and container digests from EventLog) against the expected values in the verified release manifest

If any step fails, key negotiation is aborted; only explicitly enabling AllowAttestationFallback permits fallback and continuation.

Transparency Reports

After each completed inference call, the SDK appends a record to a daily, process-specific reports-YYYY-MM-DD-PID.jsonl file. Identical attestation materials are stored separately under attestation-materials/ and referenced by content hash. Records include:

• The request prompt
• Request status and error information
• Whether attestation fallback was used and the corresponding verification failure reason
• Session binding fields and verification timestamps
• A reference to the TEE evidence and transparency data when negotiation completed

These reports provide a locally persisted and verifiable trail for offline auditing and security review. For offline verification, see the CMaaS Audit Tool.