# Confidential Model as a Service on Alibaba Cloud Model Studio

> 中文版：[README_zh.md](README_zh.md)

## Project Overview

CMaaS (Confidential Model as a Service) is an attested inference solution built on Trusted Execution Environment (TEE, e.g., Intel TDX) technology. It addresses data privacy challenges in AI inference through TEE isolation, end-to-end encryption, and remote attestation.

The system builds trust through three key dimensions: **Secure Channel** for end-to-end encrypted communication, **Attested Node** for verifiable execution environment, and **Verified Code** for auditable software provenance. Together, these dimensions provide comprehensive security guarantees:

- **End-to-End Encryption** An application-layer secure channel is established on top of HTTPS, encrypting the entire communication path from the client to the confidential inference node. Each node generates a random root private key for session key negotiation, with a unique ephemeral key derived for every session. All keys reside exclusively in encrypted volatile memory, are never persisted to disk, and are destroyed upon session termination.

- **Security Hardening & Integrity Measurement** Leveraging a dual-layered defense of chip-level hardware isolation and software hardening, the system performs full-stack integrity measurement of the boot chain and runtime environment. At the block storage level, the root partition's integrity is protected by a Merkle Tree, while the data partition is encrypted with one-time keys to ensure confidentiality.

- **Software & Hardware Transparency** Clients can directly verify the authenticity of the hardware-generated remote attestation report with the TEE vendor. Furthermore, they can inspect a public transparency log to confirm the integrity of the server-side software artifacts.

## Architecture

For detailed architecture documentation, see [Architecture Documentation](docs/architecture.md).

## How to Use

### Overview

The CMaaS Client SDK abstracts the complexity of various security hardening into simple, easy-to-use APIs, enabling developers to integrate confidential inference capabilities into their applications with minimal code changes.

- **SDK Library** (`sdk/`): Core client implementation with attestation, verification and encryption
- **CLI Tool** (`cmd/cmaas-client/`): Command-line tool demonstrating SDK usage
- **OpenAI-Compatible Proxy** (`cmd/openai-proxy/`): HTTP server that exposes `/v1/chat/completions` backed by the SDK, so existing OpenAI clients (Python/JS SDKs, vLLM benchmark tools, etc.) can access the confidential inference service without code changes
- **Anthropic-Compatible Proxy** (`cmd/anthropic-proxy/`): HTTP server that exposes `/v1/messages` for the Anthropic Messages API, so Anthropic SDKs and Claude Code can target the confidential inference service by only changing `base_url`

The SDK provides a high-level `Client` API that internally manages four core aspects:

1. **Connection Management**: Handles WebSocket connections with session reuse and automatic reconnection capabilities
2. **TEE Attestation**: Verifies TEE Quote signatures and EventLog integrity, with public key caching to reduce overhead in load-balanced deployments
3. **Transparency Log Verification**: Verifies release manifests against transparency logs service, ensuring the inference node runs a known, audited software version
4. **Encrypted Requests**: Implements RA-based key negotiation and end-to-end encryption for all inference data

The design ensures that plaintext prompts and responses never leave the client's memory or the TEE's isolated memory, providing end-to-end confidentiality throughout the inference lifecycle.

### Quick Start (CLI)

The CMaaS CLI serves as a reference implementation and testing utility for confidential inference.

#### Build and Run

```bash
cd cmd/cmaas-client
go build

# Basic usage
export DASHSCOPE_API_KEY=your_api_key
./cmaas-client \
  --endpoint wss://dashscope.aliyuncs.com/api-ws/v1/inference \
  --model    <your-confidential-model-name> \
  --prompt   "What is confidential computing?" \
  --stream
```

#### CLI Options

| Flag | Required | Default | Description |
|------|----------|---------|-------------|
| `--endpoint` | Yes | *(none)* | ModelStudio WebSocket gateway endpoint (e.g. `wss://dashscope.aliyuncs.com/api-ws/v1/inference`) |
| `--model`    | Yes | *(none)* | Model name for gateway routing |
| `--prompt`   | Yes | *(none)* | Inference prompt to send |
| `--stream`   | No  | `false`  | Enable streaming mode |

**Note**: The `DASHSCOPE_API_KEY` environment variable must be set for authentication.

### Quick Start (SDK)

The `ChatCompletionRequest` sampling parameters (e.g. `Temperature`, `TopP`, `MaxTokens`) follow the naming conventions of the OpenAI Chat Completion API for familiarity. The SDK also supports function/tool calling (`Tools`, `ToolChoice`, `ParallelToolCalls`), reasoning models (`ReasoningContent`/`Reasoning` fields and `ThinkingTokenBudget`), and structured streaming via `Deltas` / `ToolCallChunks` channels.

```go
client, _ := sdk.NewClient(&sdk.Options{
    Endpoint: "wss://dashscope.aliyuncs.com/api-ws/v1/inference", // ModelStudio WebSocket gateway
    APIKey:   os.Getenv("DASHSCOPE_API_KEY"),                     // ModelStudio API key
    Model:    "your-model-name",                                  // model name on ModelStudio
})
defer client.Close()

completion, _ := client.Chat(context.Background(), &sdk.ChatCompletionRequest{
    Messages: []sdk.Message{
        {Role: "user", Content: "What is confidential computing?"},
    },
    MaxTokens:   sdk.Int(2048),   // maximum response tokens (adjust as-needed)
    Temperature: sdk.Float(0.7),  // temperature (adjust as needed)
})

// Decrypted inference result
fmt.Println("Response:", completion.Content)

// Inspect attestation result
fmt.Printf("Verified At: %v\n", completion.Attestation.VerifiedAt)
fmt.Printf("TEE Type: %s\n", completion.Attestation.Bundle.TEEType)
```

#### Streaming Inference

```go
completion, _ := client.Chat(ctx, &sdk.ChatCompletionRequest{
    Messages: []sdk.Message{{Role: "user", Content: "Write a story"}},
    Stream:   true,
    MaxTokens:   sdk.Int(2048),
    Temperature: sdk.Float(0.7),
})

// Read and print streaming chunks as they arrive
for chunk := range completion.Chunks {
    fmt.Print(chunk)
}
fmt.Println() // Newline after streaming completes

// Inspect attestation result
fmt.Printf("Verified At: %v\n", completion.Attestation.VerifiedAt)
fmt.Printf("TEE Type: %s\n", completion.Attestation.Bundle.TEEType)
```

#### SDK Options

Currently the SDK exposes a simple `Options` structure for configuration:

```go
sdk.NewClient(&sdk.Options{
    Endpoint string     // ModelStudio WebSocket gateway endpoint (required)
    APIKey   string     // ModelStudio API key (required)
    Model    string     // Model name on ModelStudio (required)

    ReuseSession   bool             // Enable connection reuse (default: false)
    PubKeyCacheTTL time.Duration    // Public key cache TTL (default: 12h, -1 to disable)

    TransparencyLogType           transparency.LogType  // Transparency Log service type (default: "rekor-v1")
    TransparencyConfig            any                   // Transparency Log Service specific config (built-in defaults)
    TransparencyReportDir         string                // Transparency Report Directory (default: depends on your operating system)
    DisableTransparencyReport     bool                  // Disable local transparency report persistence
})
```

**Options Details**:

- **`ReuseSession`**: When enabled, the WebSocket connection is kept alive across multiple `Chat()` calls, reducing the overhead of repeated handshakes. Useful for high-throughput scenarios.

- **`PubKeyCacheTTL`**: Remote attestation verification requires fetching collateral data from TEE vendors and querying transparency log proofs, which can introduce latency. To reduce this overhead, the SDK supports caching verified server public keys. In load-balanced deployments where clients connect to multiple backend servers, this caching mechanism significantly accelerates subsequent connections to previously verified servers. The cache can be disabled by setting this value to `-1` to enforce full attestation verification on every connection.

- **`TransparencyLogType`**: Specifies the transparency log service to use for verifying release manifests. Defaults to `"rekor-v1"` ([Sigstore Rekor](https://docs.sigstore.dev/logging/overview/)). The SDK uses built-in public keys for Rekor verification by default.

- **`TransparencyConfig`**: Provides transparency log service-specific configuration. For `rekor-v1`, pass a `*transparency.RekorV1ServiceConfig` to override the default Rekor server URL or verification keys, or leave as `nil` to use built-in defaults.

- **`TransparencyReportDir`**: Specifies the directory where transparency reports which are auditable are stored. When empty, OS-specific defaults are used:
  - macOS: `~/Library/Application Support/dashscope-confidential-maas/transparency-reports/`
  - Linux: `$XDG_DATA_HOME/dashscope-confidential-maas/transparency-reports/`
  - Windows: `%APPDATA%\dashscope-confidential-maas\transparency-reports\`

- **`DisableTransparencyReport`**: Set to `true` to disable local persistence of transparency reports.

### Compatible HTTP Proxies

For integrations that already speak OpenAI or Anthropic protocols, the repository ships two HTTP local proxies built on top of the SDK. Each proxy maintains a pool of `--workers` CMaaS client instances; incoming HTTP requests are multiplexed onto that pool, which caps the number of simultaneous confidential sessions and queues any excess. Session reuse is disabled in both proxies, so every request runs a full TEE attestation / key negotiation / transparency verification on its own ephemeral session before plaintext returns to the caller.

> **Note**: Both proxies should be run within the user's trusted environment.

#### OpenAI-Compatible Proxy (`cmd/openai-proxy/`)

Implements the relevant subset of the OpenAI HTTP API, with support for streaming, function/tool calling, and reasoning model fields (`reasoning_content`, `thinking_budget`, `enable_thinking`).

```bash
cd cmd/openai-proxy
go build

export DASHSCOPE_API_KEY=your_api_key
./openai-proxy \
  --endpoint wss://dashscope.aliyuncs.com/api-ws/v1/inference \
  --model    <your-model-name> \
  --listen   :8080 \
  --workers  8
```

Endpoints:
- `POST /v1/chat/completions` — OpenAI Chat Completions (streaming and non-streaming)
- `GET  /v1/models`            — minimal model list (returns the `--model` flag value)

Flags:

| Flag | Required | Default | Description |
|------|----------|---------|-------------|
| `--endpoint`  | Yes | *(none)* | CMaaS WebSocket endpoint |
| `--model`     | Yes | *(none)* | Model name passed to the gateway for routing |
| `--listen`    | No  | `:8080`  | HTTP listen address |
| `--workers`   | No  | `8`      | Size of the CMaaS client pool (= max in-flight requests) |
| `--cache-ttl` | No  | `12h`    | Public key cache TTL (`0` disables the cache) |

`DASHSCOPE_API_KEY` must be set in the environment.

Example using `curl` against the chat endpoint:

```bash
curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "<your-model-name>",
    "messages": [
      {"role": "system", "content": "You are a helpful assistant."},
      {"role": "user",   "content": "What is confidential computing?"}
    ],
    "temperature": 0.7,
    "max_tokens": 128,
    "stream": false
  }'
```

Example with the OpenAI Python SDK (chat completions):

```python
from openai import OpenAI

client = OpenAI(base_url="http://localhost:8080/v1", api_key="ignored")
resp = client.chat.completions.create(
    model="<your-model-name>",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user",   "content": "What is confidential computing?"},
    ],
    temperature=0.7,
    max_tokens=128,
)
print(resp.choices[0].message.content)
```

#### Anthropic-Compatible Proxy (`cmd/anthropic-proxy/`)

Exposes the Anthropic Messages API so the official Anthropic SDKs and the Claude Code CLI can be pointed at a CMaaS backend. The proxy translates the Anthropic protocol into the SDK's OpenAI-shaped request, and translates the response back into Anthropic-style streaming events (`message_start` / `content_block_*` / `message_delta` / `message_stop`).

```bash
cd cmd/anthropic-proxy
go build

export DASHSCOPE_API_KEY=your_api_key
./anthropic-proxy \
  --endpoint wss://dashscope.aliyuncs.com/api-ws/v1/inference \
  --model    <your-model-name> \
  --listen   :8081 \
  --workers  8
```

Endpoints:
- `POST /v1/messages` — Anthropic Messages API (streaming and non-streaming)
- `GET  /v1/models`   — single-model list reflecting the `--model` flag

Flags mirror the OpenAI proxy (`--endpoint` and `--model` are required; `--listen` defaults to `:8081`; `--workers` to `8`; `--cache-ttl` to `12h`). `DASHSCOPE_API_KEY` must be set in the environment.

Example with the Anthropic Python SDK:

```python
import anthropic

client = anthropic.Anthropic(base_url="http://localhost:8081", api_key="ignored")
resp = client.messages.create(
    model="<your-model-name>",
    max_tokens=128,
    messages=[{"role": "user", "content": "What is confidential computing?"}],
)
print(resp.content[0].text)
```

Both proxies log a per-request correlation id (`chatcmpl-…` / `msg_…`) alongside the CMaaS `task_id`, so client-side errors can be traced back through the proxy to the inference node.

#### Attestation Result

Each inference will bind an `AttestationResult` that provides transparency into the TEE server's identity and integrity:

- **`VerifiedAt`**: Timestamp when the attestation was performed
- **`Bundle`**: Contains the full attestation bundle including TEE type, server public key, TEE evidence, and transparency data

Call `result.TEEDetailsJSON()` to extract TEE-specific key fields as JSON. For Intel TDX environments (currently supported), this includes:
- **MRTD**: Initial TD measurement (VM configuration and initial code)
- **RTMR[0-3]**: Runtime measurement registers for firmware, kernel, and application layers
- **TD Attributes**: Trust domain configuration attributes
- **Debug Enabled**: Whether the TEE is running in debug mode

#### Transparency Log Verification

> Transparency log verification strengthens the trust chain by ensuring the inference node runs a known, audited software version—not just any code that happens to produce valid TEE measurements.

During key negotiation, the SDK automatically performs the following transparency verification steps:

1. **Verify Log Entry Integrity**: Validates each proof entry's DSSE envelope signature, RFC 6962 Merkle inclusion proof, and signed checkpoint against the transparency log's public keys
2. **Verify Payload Hash**: Confirms that the release manifest's SHA-256 hash (JCS-canonicalized) matches the payload hash recorded in the log entry
3. **Validate TEE Measurements**: Validates the TEE measurements from the TEE evidence (for TDX: measurements from Quote, UKI hash and container digests from EventLog) against the expected values in the verified release manifest

If any step fails, the key negotiation is aborted and the inference request will not proceed.

**Transparency Reports**

After each successful inference call, the SDK automatically persists a transparency audit record to local storage. Each report (saved as `report_YYYYMMDD_HHMMSS.000.json`) contains:
- The request prompt
- TEE attestation evidence with extracted details (MRTD, RTMR, TD attributes)
- Transparency log data (release manifest and proof entries)
- Verification timestamps

These reports provide a locally persisted, verifiable trail for future offline auditing and security reviews.
