What we hold.
What we don't. What we audit.
Card PANs are not stored. Crypto private keys never leave Coinbase CDP. AP2 keys are AES-256-GCM at rest. Every credit, every signature, every approval is a row. Here's the full inventory — including the gaps we're tracking.
Three classes of secrets. Three storage models.
Every secret in the system falls into one of three boxes — and each box has a different storage strategy. We never mix them.
① Card PAN / CVV — never stored
The full Primary Account Number and CVV are never persisted in our database. When a Principal needs to view a card, the request is signed with a short-lived nonce; the server fetches the PAN live from Stripe Issuing or Airwallex via VGS-proxied SDK call; the response is held client-side in a sandboxed iframe for 30 seconds, then cleared.
The application server only ever holds the VGS alias (e.g. tok_card_1aB2cD3eF4gH5iJ6kL7mN8oP), the last-4, and the masked PAN. The full PAN exists only:
- Inside Stripe / Airwallex (the issuer)
- In the VGS vault
- Briefly in the user's browser when explicitly revealed
CREATE TABLE cards ( id uuid PRIMARY KEY, agent_identity_id uuid REFERENCES agent_identities, provider card_provider, -- stripe | airwallexprovider_card_id text, -- ic_… or aw_…vgs_alias text, -- tok_card_…last_four varchar(4), brand text, -- visa | mcbalance_cents int, status card_status, created_at timestamptz DEFAULT now()-- NO pan, NO cvv, NO exp month/year);
② Crypto private keys — Coinbase CDP HSM
Every agent's USDC wallet is a Coinbase v2 Server Wallet. The private key is generated and held inside Coinbase's HSM. We never see, transmit, or store the raw private key — not even encrypted.
To sign a transaction, the application calls the CDP API with a CDP API key. The HSM signs and returns the signed transaction. The agent's wallet address (0xabc…) and the CDP wallet ID are the only crypto-side identifiers we persist.
x402 mechanics. When the agent pays an x402 endpoint, we construct an EIP-3009 transferWithAuthorization typed-data digest, send it to CDP for signing, and emit the signed authorization. The HSM produces the v/r/s; we never reconstruct or hold the private key.
③ AP2 mandate keys — AES-256-GCM at rest
AP2 mandates are signed with an ES256 (P-256) keypair per agent. The public JWK is stored as jsonb; the private JWK is encrypted with AES-256-GCM using a master key from environment (AP2_KEY_ENCRYPTION_KEY) and stored as a binary blob.
import { createCipheriv, randomBytes } from 'crypto';function encryptAp2PrivateKey(privJwk: JWK) {const iv = randomBytes(12);const cipher = createCipheriv('aes-256-gcm', Buffer.from(process.env.AP2_KEY_ENCRYPTION_KEY, 'hex'), iv );const enc = Buffer.concat([ cipher.update(JSON.stringify(privJwk), 'utf8'), cipher.final() ]);return { iv, authTag: cipher.getAuthTag(), ciphertext: enc };}
API keys — fingerprint, not retention
Per-agent pak_… keys are returned once at creation or rotation. We persist:
- SHA-256 fingerprint — the lookup token. Constant-time compared on every request.
- First 12 chars — UI hint (
pak_live_a1f9c2…). - Last 4 chars — UI hint.
- Last rotated timestamp — for compliance.
If a Principal loses the raw key, there is no recovery: rotate it. The new key is shown once and never logged.
Inventory
| Secret | Where it lives | How we touch it |
|---|---|---|
| Card PAN / CVV | Stripe / Airwallex / VGS | never |
| Crypto private key | Coinbase CDP HSM | never |
| AP2 private key | Postgres · AES-256-GCM | Decrypt at sign time, in-memory only |
| Agent API key | Returned once · SHA-256 stored | Constant-time fingerprint check |
| Principal session | HttpOnly · Secure cookie | 30d sliding · rotated each request |
| CDP API key | Environment variable | Read-once at boot |
| Stripe / Airwallex SK | Environment variable | Outbound through VGS proxy |
Principals authorize. Agents act. Both signatures travel.
An AP2 IntentMandate is signed by the Principal, not the agent. Subsequent CartMandate and PaymentMandate are signed by the agent under that intent. Verifiers downstream see the chain — they can prove a Principal authorized the spending envelope.
Why two-key signing matters
If an agent's API key is leaked, an attacker can attempt cart/payment mandates — but they can't forge an IntentMandate without the Principal's key. The Principal's signing key is held in their browser via WebCrypto (non-extractable), backed by a hardware authenticator if the device supports it.
Current state. Browser-side Principal signing exists for IntentMandate. The fallback path — server-held Principal keys for Principals who haven't onboarded WebAuthn — is in production for compatibility. Phasing this out, with a hard cutover to client-side signing only, is tracked in Roadmap → Q3.
Mandate verification — strict alg
Every JWS verification call locks alg: 'ES256'. The verifier rejects any incoming JWS with a different algorithm — closing the classic JWT alg: 'none' and HS-confusion attack surface.
await jwtVerify(token, jwk, {algorithms: ['ES256'], // ← only ES256, never accept othersissuer: expectedDid, audience: myEndpoint, clockTolerance: '30s'});
Replay protection
- jti uniqueness. Every mandate carries a UUIDv7 jti.
ap2_mandate_audithas a unique index on(jti, action='execute')— replay at the executor is impossible. - Short expirations. IntentMandate: 24h. CartMandate: 1h. PaymentMandate: 1h. clockTolerance capped at 30s.
- Single-use SPTs. Stripe Shared Payment Tokens are scoped to one merchant + one amount + one window.
- EIP-3009 nonce. Each authorization uses a fresh nonce; the contract enforces single-use.
Append-only. Every system.
Money moves leave permanent rows. Mandates leave permanent rows. Approvals leave permanent rows. Tool calls leave permanent rows. The platform never deletes audit records — period.
Audit tables
| Table | What's logged | Indexed by |
|---|---|---|
| agent_wallet_transactions | Every cent that crosses a wallet | agent_identity_id, created_at |
| agent_identity_activity_log | MCP tool calls, approvals, key rotations | agent_identity_id, action, created_at |
| ap2_mandate_audit | Every sign / verify / execute | jti, action, agent_identity_id |
| acp_spt_issuances | Stripe SPTs minted by an agent | agent_identity_id, stripe_token_id |
| acp_card_token_issuances | VGS card tokens issued for outbound | agent_identity_id, vgs_alias |
| admin_audit_log | Force-approves, manual top-ups, freezes | admin_id, action, created_at |
| agent_portal_session_log | Principal logins · IP · user-agent | principal_id, created_at |
What "append-only" means here
None of the audit tables have UPDATE or DELETE triggers exposed in application code. Drizzle queries are explicit insert() only on these tables. Postgres REVOKE UPDATE, DELETE at the role level for the application user is in the operations runbook for production.
Honest disclosure. The DB-level REVOKE is documented but not all environments have it applied. The application-side guarantee is strong; the DB-level guarantee is only as strong as the role grants. SOC 2 attestation cycle covers this.
Mandate audit — what each row contains
{"id": "01J9aud…","jti": "01J9mnd…","action": "verify", // sign | verify | execute | error"agent_identity_id": "ag_01J9…","mandate_type": "payment","counterparty_did": "did:agentwallet:ag_remote…","result": "ok", // ok | error"error_code": null,"claims_snapshot": { /* full payload */ },"created_at": "2026-05-04T18:42:01Z"}Runbooks. Every common failure.
Operations is one of the chapters that survived the deepest revisions. These are the runbooks that wake on-call.
Master CDP balance is low
Symptom. getMasterUsdcBalance() < 10 USDC alarm fires. Agent USDC sends start failing with INSUFFICIENT_MASTER_FUNDS.
- Top up the master CDP wallet via Bridge.xyz USDC on/off-ramp.
- Verify with
cdp.wallet(MASTER_ID).getBalance('USDC'). - Existing pending tool calls retry automatically; no replay needed.
Widget approval queue depth
Symptom. listAllApprovals().filter(p => p.status === 'pending').length > 50.
- Check Principal SMS/email delivery (AgentSim / AgentMail dashboards).
- If a single Principal owns the bulk: reach out, escalate, or use force-approve with admin audit.
- If global: a notification provider may be down. Failover to the secondary channel.
AP2 verification spike
Symptom. ap2_mandate_audit rows with result='error' spike.
- Group by
error_codeand counterparty DID. Most common:SIG_INVALID,EXPIRED,NO_INTENT. - For
SIG_INVALIDfrom a single counterparty: their key may have rotated. Re-resolve/.well-known/agent.json. - For
EXPIREDspike: clock drift on the executor. NTP check.
Process restart — what happens to in-flight
- Postgres-backed: ledger entries, mandates, payouts, invoices — all durable. No loss.
- In-memory Map: pending widget approvals are lost. The agent retries; a new approval is created. No double-spend (idempotency keys), but the Principal sees a duplicate notification.
- MCP sessions: reconnect. Tools/list re-served on reconnect.
Required environment variables
| DATABASE_URL | Neon connection string |
| CDP_API_KEY_NAME | Coinbase CDP key name |
| CDP_PRIVATE_KEY | Coinbase CDP private key (PEM) |
| X402_WALLET_ADDRESS | Master wallet 0x… on Base |
| AP2_KEY_ENCRYPTION_KEY | 32-byte hex · master AES key |
| STRIPE_SECRET_KEY | Routed via VGS outbound proxy |
| AIRWALLEX_API_KEY | Card issuing + payouts |
| AGENTSIM_API_KEY | Phone / SMS provisioning |
| AGENTMAIL_API_KEY | Email provisioning |
| VGS_VAULT_ID · VGS_USERNAME · VGS_PASSWORD | Outbound proxy auth |
| ANTHROPIC_API_KEY | Claude Sonnet for DE |
| SESSION_SECRET | Express session signing |
Testing. What's there. What isn't.
We don't claim parity that doesn't exist. Here's the honest test inventory.
| Surface | Coverage | Status |
|---|---|---|
| AP2 mandate sign + verify | Smoke test · happy path | shipped |
| Wallet allocateFunds advisory lock | Concurrency test (2 writers) | shipped |
| RBAC verifyAgentAccess | Unit · 6 role × 12 endpoint matrix | shipped |
| x402 EIP-3009 sign roundtrip | Manual + Base Sepolia testnet | partial |
| Full DE 60s-tick replay | LangSmith trace-replay | partial |
| End-to-end browser (Playwright) | Defined · not yet shipped | planned |
| Chaos: Postgres restart mid-tx | Defined · not yet shipped | planned |
| Load test: 1k concurrent MCP sessions | Defined · not yet shipped | planned |
Manual fixtures. The repo ships a scripts/ap2-smoke.ts and a scripts/x402-paywall-mock.ts for local end-to-end validation. Run them before any release that touches the mandate path.
Open risks. Tracked, named, not hidden.
The whole point of these docs is honesty. Here's the live list.
R-01 Wallet credit bypass.
The Principal-facing POST /agents/:id/wallet/credit calls addFunds directly — no advisory lock, no master-headroom check. The admin path enforces both.
Fix: route the portal endpoint through
allocateWalletFunds. Tracked.R-02 Widget approvals in memory.
Pending approvals live in a single in-memory Map. Process restart loses pending approvals. Single-instance only.
Fix: migrate to Redis or Postgres-backed work queue. Tracked.
R-03 DB-level append-only not universal.
Application code does not UPDATE or DELETE audit tables. Postgres role-level REVOKE is documented but not applied in every environment.
R-04 Server-held Principal AP2 keys (legacy).
A subset of Principals onboarded before WebAuthn signing have keys held server-side, encrypted with AES-256-GCM.
R-05 No formal pen-test on AP2.
AP2 is built on standard JWS primitives with strict alg pinning, but a third-party pen-test specifically targeting the mandate chain has not been completed.