# Architecture

## System Overview

<p align="center">
  <img src="../assets/architecture-overview.png" alt="CMaaS Architecture" width="800"/>
</p>

The system implements attested inference through a client-server architecture where the CMaaS Client communicates directly with Confidential Inference Nodes running in TEE environments. The Model Studio Gateway authenticates the client and routes requests to the appropriate nodes, while the TEE-Vendor Backed Attestation Service provides attestation verification support. Additionally, software supply chain verification mechanisms enable customers to verify the provenance of the code running in the inference nodes. This is achieved through transparency log services such as [Rekor](https://docs.sigstore.dev/logging/overview/) and open-source code.

The system builds trust through three complementary security mechanisms:

## A. Secure Channel

CMaaS establishes secure communication between the Client and Confidential Inference Node through remote attestation and end-to-end encryption. The CMaaS Client communicates with the CMaaS Proxy running inside the Confidential Inference Node, which attests the node to the client and forwards encrypted requests to the LLM Backend within the same TEE node.

<p align="center">
  <img src="../assets/e2e-inference-flow.png" alt="E2E Inference Flow" width="700"/>
</p>

**Remote Attestation based Key Negotiation**

The CMaaS Client initiates a key negotiation session with the CMaaS Proxy. The CMaaS Proxy responds with an Attestation Bundle containing the TDX Quote, EventLog, and transparency log data (release manifest and proof entries) as proof of its TEE identity and software provenance. The client performs remote attestation by verifying the Quote's signature (with support from the TEE-Vendor Backed Attestation Service), validating TEE measurements (MRTD, RTMR values, and TD Features), replaying the EventLog to ensure boot chain integrity, and verifying the transparency log entries to confirm the node runs a known, audited software version. Upon successful verification, both parties establish a shared ephemeral Session Key that exists only in volatile memory and is never persisted to disk.

**Key-aware Request Routing**

The session key exists only between the client and the specific inference node it is communicating with; it is not shared with any other inference nodes. Therefore, the Model Studio Gateway must be aware of the key negotiation process and ensure that both the key negotiation request and the subsequent inference requests are routed to the same inference node. To achieve this, we employ WebSocket Secure (WSS) protocol: all requests within the same session are consistently directed to a single node and encrypted using the same session key. This approach not only enhances security but also delivers lower latency and improved KVCache hit rates.

**End-to-End Encrypted Inference**

The CMaaS Client encrypts inference requests using the Session Key and sends them through the Model Studio Gateway to the confidential inference node. Within the TEE, the CMaaS Proxy first decrypts the request, forwards it to the LLM Backend for processing, encrypts the inference result, and returns it to the client. All plaintext data (user prompts, responses) exists only within secure boundaries—the client's memory and the TEE's isolated memory. No plaintext ever leaves these boundaries or appears in transit, on host systems, or in cloud infrastructure outside the TEE. When the connection terminates, the Session Key is immediately destroyed from memory, ensuring forward secrecy.

## B. Attested Node

CMaaS ensures the integrity and isolation of the execution environment through hardware-based TEE technology and VM image hardening.

### Hardware Security Mechanisms

The Confidential Inference Node runs as a confidential virtual machine within Intel TDX Trust Domain with NVIDIA Confidential Computing support:
- **TDX Isolation**: Provides hardware-level memory encryption, CPU state isolation, and DMA protection to ensure the TD is completely isolated from the host OS/VMM
- **CPU-GPU Secure Communication**: PCIe traffic is protected through SPDM-based key negotiation and encryption, preventing interception or tampering by the host
- **GPU Isolation**: The GPU's Compute Protected Region (CPR) isolates GPU memory from the host, with direct ingress and egress blocked by the PCIe firewall. Sensitive data and model weights remain protected within the GPU's secure memory boundaries

<p align="center">
  <img src="../assets/tdx-nvcc-hardening.png" alt="TDX & NVCC Hardening" width="750"/>
</p>

### VM Image Hardening and Measurement

The confidential VM image is built with security hardening to minimize attack surface and ensure immutability. All security-critical components are cryptographically measured during boot and recorded in measurement registers:

<p align="center">
  <img src="../assets/attestation-eventlog.png" alt="Attestation Report & EventLog" width="650"/>
</p>

- **MRTD** (Measurement Register of TD): Initial TD measurement, contains VM configuration and initial code
- **RTMR[0]**: Firmware measurement (UEFI, bootloader)
- **RTMR[1]**: Kernel measurement (Linux kernel, initrd and filesystem)
- **RTMR[2]**: Application measurement (cmaas-proxy, LLM backend)

These measurements are included in the TDX Quote, which is cryptographically signed by Intel's CPU and verified by the client during attestation. The EventLog provides a complete record of all measurement events, allowing clients to replay and verify the integrity of the boot chain.

<p align="center">
  <img src="../assets/measurements.png" alt="Measurements Coverage" width="800"/>
</p>

In addition, the VM image hardening includes:
- **RootFS Integrity**: dm-verity ensures read-only root filesystem integrity verification, preventing runtime tampering
- **RootFS Encryption**: dm-crypt provides encryption for the root filesystem and data volume, the encryption key is an One-Time Key
- **Minimal Attack Surface**: VM image is stripped down to essential components only, with cloud-init, SSH, getty, aliyun-assistant and other operation channels removed. We also disabled all user logins and removed intermediate cache data from the image build process. No dynamic container image changes after deployment

## C. Verified Code

CMaaS enables customers to verify that inference nodes run known, audited code versions through software supply chain verification mechanisms.

<p align="center">
  <img src="../assets/verifiable-provenance.png" alt="Verifiable Provenance" width="700"/>
</p>

### Baseline Release
To enable efficient verification of RTMR and MRTD values, we publish trusted baseline values to transparency log service, ensuring these values are publicly accessible, auditable, and traceable.

### Artifact Verifiability and Auditing
We believe that a trusted environment baseline, while foundational, is insufficient on its own to fully realize our security goals. Therefore, we mandate rigorous code audits for all components with access to plaintext prompts and responses.

The core objective of these audits is to ensure the complete absence of any unintended operational channels or backdoors, and to eliminate all potential paths that could lead to the leakage of plaintext data. This deep, code-level verification is a cornerstone of our commitment to security. How we navigate the balance between protecting user privacy and securing our technological assets remains a core challenge we are continuously working to address.

### Verifiable Build
Trusted baseline values and audited source code still need to be linked to the actual running binary. This is achieved through Reproducible Builds, where identical source code and build environments produce identical binaries. Reproducible builds have been widely adopted in security-focused Linux distributions, and we plan to integrate them in our next phase.
