Crux Daemon · 8. Errors

Every failure path returns an RFC 9457 problem document with Content-Type: application/problem+json, and the useful part is the flattened code extension member. This chapter lists the roughly 40 codes the daemon actually emits.

This chapter reflects the code, not docs/error-catalogue.md. That file documents a completely different taxonomy, the structured-logging and CLI error codes, presented as an HTTP API contract. There are 13 discrepancies, enumerated in §8.7. A client matching on the codes in that file will never match. Treat it as superseded.

This chapter is reference.

8.1 The canonical error body

ProblemDetails (corecrux-types/src/lib.rs:783):

JSON field	Type	Required	Notes
type	string (URI)	yes	serde-renamed from problem_type
title	string	yes	Short human summary
status	number (u16)	yes	Mirrors the HTTP status
detail	string	no	Omitted when absent
instance	string (URI)	no	Omitted when absent
extensions	any object	no	#[serde(flatten)]: members are hoisted to the top level, not nested (lib.rs:798)

The de-facto convention is a code member, a SCREAMING_SNAKE_CASE string, plus problem-specific fields.

{
  "type": "https://errors.cuecrux.com/forbidden",
  "title": "Forbidden",
  "status": 403,
  "detail": "insufficient scopes",
  "code": "MISSING_SCOPE",
  "missingScopes": ["admin:write"]
}

That example is the 403 from require_http_scopes (auth.rs:1331).

Extension key styles are not consistent. missingScopes, missingAnyScope, shardId and ownerGpuId are camelCase; tenant_id, reason_class, reason_code, operation_id and required_credits are snake_case. A client must handle both.

Read detail. The daemon puts genuinely actionable text there, including which environment variable to flip and which CLI command to run.

8.2 Content type and RFC

The content type is application/problem+json, set unconditionally (problem.rs:26). No charset parameter.

The type source calls the format RFC 9457 (corecrux-types/src/lib.rs:775); problem.rs:6 says RFC 7807. Both name the same wire format, 9457 obsoletes 7807. Prefer RFC 9457.

Type URIs are rooted at https://errors.cuecrux.com (corecrux-types/src/lib.rs:30).

Two serialisation edge cases you can hit (problem.rs:19-32):

• An out-of-range status degrades to 500 on the wire while the body keeps the original number. StatusCode::from_u16(9999) fails, and the response status becomes 500 while "status": 9999 remains in the JSON.
• A serialisation failure emits a literal fallback: {"type":"https://errors.cuecrux.com/internal","title":"Serialization Error","status":500}.

8.3 The status-to-problem mapping

Factory methods (corecrux-types/src/lib.rs:831):

Factory	Status	type suffix	title
bad_request	400	/bad-request	Bad Request
unauthorized	401	/unauthorized	Unauthorized
forbidden	403	/forbidden	Forbidden
not_found	404	/not-found	Not Found
precondition_failed	412	/precondition-failed	Precondition Failed
rate_limited	429	/rate-limited	Too Many Requests
internal	500	/internal	Internal Server Error
not_implemented	501	/not-implemented	Not Implemented
service_unavailable	503	/service-unavailable	Service Unavailable

For ad-hoc responses, problem_for_status (http/mod.rs:1721) adds 413 /payload-too-large, 409 /conflict, 422 /unprocessable-entity, 502 /bad-gateway, 402 /payment-required and 204 /no-content.

Anything unmapped becomes a 500 (http/mod.rs:1783). If you add a handler that returns an exotic status through this helper, you will get a 500 body instead.

8.4 Every code the daemon emits

Auth and tenant

Code	HTTP	Meaning	Location
UNAUTHENTICATED	401	No credential, or the credential failed verification. Covers missing dev scopes, and a missing or invalid bearer under either JWT mode. The gRPC equivalents are Status::unauthenticated with a JSON string payload	auth.rs:861, auth.rs:885
MISSING_SCOPE	403	Insufficient scopes. Carries missingScopes for an all-of check or missingAnyScope for an any-of check	auth.rs:1333
AUTH_MISCONFIGURED	500	The mode is a JWT mode but the config object is absent	auth.rs:879
TENANT_FORBIDDEN	403	The token's tenant set does not contain the requested tenant. Carries a tenantId extension	auth.rs:832
TENANT_CLAIM_MISSING	403	The token carries no tenant claim on a tenant-scoped route	auth.rs:840
TENANT_SELECTOR_REQUIRED	403	A multi-tenant token wrote without x-corecrux-tenant-id	auth.rs:1031
PASSPORT_HEADER_UNBOUND	403	X-Corecrux-Passport-Id present but the token carries no passport identity	auth.rs:1202
PASSPORT_HEADER_MISMATCH	403	The header differs from the token passport and the caller has no override scope	auth.rs:1209

401 versus 403 is the most client-relevant distinction here. UNAUTHENTICATED means "present a credential"; MISSING_SCOPE means "your credential is valid but insufficient", and the missingScopes array tells you exactly what to ask for.

Sync peer plane

Code	HTTP	Meaning	Location
SYNC_PEER_AUTH_FAILED	401	Handshake rejected. reason_class is one of missing_peer_handshake, malformed_peer_handshake, peer_trust_unavailable, peer_delegation_disabled, peer_nonce_rejected and others	sync.rs:82
SYNC_PEER_TENANT_MISMATCH	403	The authenticated peer is not authorised for the requested tenant	sync.rs:92
SYNC_PEER_AUTH_UNAVAILABLE	503	The nonce cache lock is poisoned, or verification is unavailable	sync.rs:102

Storage, sharding, replication

Code	HTTP	Meaning	Location
SHARD_UNAVAILABLE	503	Extensions shardId, ownerGpuId, currentShardMapVersion	http/mod.rs:1689
WRONG_SHARD	412	Extensions leaderGrpcAddr, currentShardMapVersion	http/mod.rs:1698
SHARDMAP_VERSION_MISMATCH	412	Client shard-map version differs from current. Extensions clientShardMapVersion, currentShardMapVersion	http/mod.rs:1706
REPLICATION_SEGMENT_HASH_MISMATCH	admin surface	A pushed segment's hash does not match	admin.rs:2090
REPLICATION_EPOCH_MISMATCH	admin surface	Replication epoch mismatch	admin.rs:2119

Throttling (gRPC)

Code	gRPC status	Location
TENANT_THROTTLE_INFLIGHT	RESOURCE_EXHAUSTED	grpc.rs:359
TENANT_THROTTLE_RATE	RESOURCE_EXHAUSTED	grpc.rs:392

These are live code but unreachable from the stubbed RPCs, see chapter 1 §1.4.

Embedding and semantic profile

Code	HTTP	Location
EMBEDDING_DELEGATION_DEGRADED	503	http/mod.rs:1802
EMBEDDING_SEMANTIC_PROFILE_MISMATCH	409 or 4xx	http/mod.rs:1815
SEMANTIC_PROFILE_MISMATCH	409	corecrux-memory/src/embeddings.rs:1458
DELEGATED_CLIENT_VECTORS_UNSUPPORTED	4xx	local_ingest.rs:258
INVALID_SEMANTIC_PROFILE	4xx	local_ingest.rs:283
EMBEDDING_DELEGATION_REQUEST_TOO_LARGE	413	local_ingest.rs:328

Compute plane

Built by compute_problem (compute.rs:338), where type is derived as https://errors.cuecrux.com/{code lowercased}. Titles: 400 gives Invalid Compute Request, 409 gives Semantic Profile Conflict, 503 gives Compute Capability Unavailable, anything else gives Compute Provider Error.

Codes: COMPUTE_CALLER_PASSPORT_REQUIRED, COMPUTE_PROVIDER_DISABLED, COMPUTE_EMBED_INVALID_REQUEST, COMPUTE_EMBED_FAILED, COMPUTE_EMBED_HASH_FAILED, COMPUTE_EMBEDDER_UNAVAILABLE, COMPUTE_INVALID_EMBEDDING_SHAPE, COMPUTE_RECEIPT_FAILED, COMPUTE_SEMANTIC_PROFILE_UNAVAILABLE, SEMANTIC_PROFILE_MISMATCH.

Cloud-witness plane: the lower_snake_case outlier

Built by cloud_witness_problem (observations.rs:1409). These are the only lower_snake_case codes in the daemon.

Code	HTTP	Location
witness_envelope_invalid	400	observations.rs:1433
witness_signature_invalid	400	observations.rs:1438
witness_stale	400	observations.rs:1473
witness_replay_rejected	400	observations.rs:1498
witness_verification_unavailable	503	observations.rs:1447

Other

Code	HTTP	Location
RESERVED_ENTITY_PREFIX	400	facts.rs:221

8.5 Errors identified only by type

These carry a distinctive type URI but no code extension. A client must branch on type.

type	HTTP	Notes	Location
…/payment-required/insufficient-credits	402		gpu1.rs:640
…/rcx-lane-denied	403	Extensions capability, reason_code, mode, token_id, token_hash	gpu1.rs:660
…/conflict/credit-operation-payload-mismatch	409		gpu1.rs:688
…/conflict/credit-operation-already-spent	409		gpu1.rs:710
…/rate-limited	429	Sets a numeric Retry-After	ingress.rs:186
…/invalid-passport-header	400	The passport header failed shape validation at ingress	ingress.rs:207
…/overloaded	503	Sets Retry-After: 1. The load-shed response	ingress.rs:504
…/payload-too-large	413	Body limit exceeded	ingress.rs:545
…/session	varies	Session-plane failures	session.rs:46

The /v1/gpu1/* rows exist only in a build with --features hosted-surfaces.

8.6 Three code namespaces, and only one is an API contract

#	Namespace	Where	Is it in an HTTP response?
1	CORE_ERROR_* constants: 11 codes, exposed as CORE_ERROR_CODES	corecrux-types/src/lib.rs:37	No
2	structured_log::ErrorCode: the same 11 codes, serialised into the error_code field of an operations log line	structured_log.rs:18	No
3	HTTP problem code extensions, the roughly 40 codes in §8.4, defined ad hoc at their throw sites	throughout crates/corecruxd/src/http/	Yes

Namespaces 1 and 2 are disjoint from namespace 3. No CORE_ERROR_* code is ever emitted as an HTTP problem code member anywhere in the daemon.

The 11 log and CLI codes, for completeness, are IO_READ_FAILED, IO_WRITE_FAILED, IO_FSYNC_FAILED, SEGMENT_CORRUPT, INVALID_FRAME, INVALID_TOC, SHARD_NOT_OWNER, EPOCH_MISMATCH, BACKPRESSURE, TIMEOUT, INTERNAL. They appear in the structured operations log and in corecruxctl verify-store output. They are not API errors.

8.7 Why docs/error-catalogue.md cannot be used

The file in the repository is 48 lines and predates the HTTP surface. Thirteen discrepancies, each verified:

#	Discrepancy
D1	It documents the wrong namespace. All 11 rows are the structured-logging and CLI taxonomy, presented in an "Error Codes" table with HTTP and gRPC status columns, implying an API contract that does not exist. A client matching code == "IO_READ_FAILED" will never match
D2	Roughly 40 real HTTP problem codes are entirely absent. Not one of UNAUTHENTICATED, MISSING_SCOPE, AUTH_MISCONFIGURED, the four TENANT_* codes, the two PASSPORT_HEADER_* codes, the SYNC_PEER_* family, SHARD_UNAVAILABLE, WRONG_SHARD, SHARDMAP_VERSION_MISMATCH, the REPLICATION_* pair, the TENANT_THROTTLE_* pair, the EMBEDDING_* family, SEMANTIC_PROFILE_MISMATCH, the COMPUTE_* family, the witness_* family or RESERVED_ENTITY_PREFIX appears
D3	SHARD_NOT_OWNER (412) is never emitted. The live code for that condition is WRONG_SHARD
D4	EPOCH_MISMATCH (412) is never emitted over HTTP. The live code is REPLICATION_EPOCH_MISMATCH
D5	BACKPRESSURE (429) is never emitted as an HTTP code. Backpressure surfaces as a rate_limited problem with no code member (http/mod.rs:1681), and as storage-layer codes BACKPRESSURE_MAX_EVENTS and BACKPRESSURE_MAX_BATCH_BYTES (append.rs:619), neither of which the catalogue names
D6	The Retry-After claim is only conditionally true. The catalogue tells clients to respect Retry-After on a BACKPRESSURE 429. The 429 that actually sets it is the ingress rate limiter (ingress.rs:196), not the backpressure path. The 503 …/overloaded also sets Retry-After: 1 and is undocumented
D7	TIMEOUT (504) never appears on the HTTP surface, only as a log error_code. No handler returns 504
D8	No mention of application/problem+json, the RFC 9457 body shape, the type/title/status/detail/instance fields, or the flattened-extensions convention. The file describes a code list, not an error shape
D9	Type URIs are undocumented, including the sub-path forms in §8.5 and the code-derived forms in the compute and witness planes
D10	The casing inconsistency is undocumented. The witness plane uses lower_snake_case; everything else uses SCREAMING_SNAKE_CASE. Extension member keys mix camelCase and snake_case
D11	Auth error semantics are undocumented. 401 versus 403, and the missingScopes and missingAnyScope arrays, are the single most client-relevant error contract and are absent
D12	The unmapped-status fallback is undocumented. problem_for_status turns any unrecognised status into a 500 body, and an out-of-range status field degrades to 500 on the wire
D13	The MCP Tool Errors table lists plausible JSON-RPC codes (-32602, -32601, -32603) that were not verified against crates/crux-mcp/src/dispatch.rs in the audit that produced this chapter. Treat those three as unverified rather than as documented contract

This is defect B7 in chapter 16.

8.8 Handling errors in a client

Six practical rules, in order.

1. Branch on code first, then on type, then on status. Roughly 40 responses carry code; nine carry only a distinctive type; everything else is generic.
2. Read detail. It is written for a human operator and frequently names the exact environment variable or CLI command that fixes the problem.
3. On 403 MISSING_SCOPE, read missingScopes or missingAnyScope. They tell you precisely what to request.
4. Distinguish 401 from 403. 401 means present a credential; 403 means the credential is valid and insufficient.
5. Honour Retry-After on 429 and on 503 …/overloaded. Those two set it; the backpressure path does not.
6. Do not assume the extension keys are camelCase. Handle both conventions.

Sources

• crates/corecrux-types/src/lib.rs:783, ProblemDetails
• crates/corecrux-types/src/lib.rs:831, the factory methods
• crates/corecruxd/src/problem.rs:19, into_response and the fallbacks
• crates/corecruxd/src/http/mod.rs:1721, problem_for_status
• crates/corecruxd/src/http/mod.rs:1788, problem_response
• crates/corecruxd/src/auth.rs:1331, MISSING_SCOPE
• crates/corecruxd/src/structured_log.rs:18, the log-only ErrorCode enum