Crux Daemon · 6. The data directory
Everything the daemon knows lives under one directory. Back it up and you have backed up the daemon. This chapter is the map: what is in there, which code writes each file, what breaks if it goes missing, and which files grow forever.
This chapter is reference.
6.0 In plain English
The daemon keeps all of its state in one directory tree. Facts, shard segments, signing keys, node identity, routing state and locks all live under it. That is the whole storage model, and it has a useful practical consequence: this directory is the backup unit. Copy it and you have copied the daemon; restore it onto another machine and the daemon comes up believing exactly what it believed before.
The reason to have a chapter about a directory is that the tree is not self-describing. A file named MANIFEST or passport.key does not tell you what depends on it, whether it is regenerated automatically, or what happens if it is missing when the process next starts. This chapter answers those three questions for every file: what writes it, what breaks without it, and whether the damage is recoverable.
You will need it when you set up backups and want to know whether it is safe to exclude something; when a disk fills and you need to know which files grow without bound (§6.7); when you are moving a daemon between hosts; and in the worst case, after something has been deleted and you need to know what you have actually lost. §6.5 is ordered by blast radius for exactly that moment.
Two things surprise people. The first is that deleting facts.jsonl is a total, silent memory loss: the daemon starts clean, reports itself healthy, and will not tell you anything is missing. Nothing in the startup path treats an absent journal as an error, because an absent journal is also what a brand-new install looks like. The second is that several subsystems you would expect to find as files are not files at all. The coordination plane, extension grants, session bindings and legal holds are all stored as facts under reserved entity prefixes, so searching the tree for a coord/ directory finds nothing and tells you nothing. §6.4 lists them, and the practical upshot is that GET /v1/facts inspects far more of the daemon's state than its name suggests.
6.1 Two roots, usually the same directory
| Setting | Env var | Config-file key | Fallback chain | Default |
|---|---|---|---|---|
data_dir | CORECRUXD_DATA_DIR | daemon.data_dir | env, then daemon.data_dir, then daemon.state_dir, then the literal | ../CoreCruxData/v1 |
state_dir | CORECRUXD_STATE_DIR | daemon.state_dir | env, then daemon.state_dir, then data_dir | equals data_dir |
Source: config.rs:834-842.
// crates/corecruxd/src/config.rs:834
let data_dir = env_string("CORECRUXD_DATA_DIR")
.map(|value| expand_path(&value))
.or(file_data_dir)
.or_else(|| file_state_dir.clone())
.unwrap_or_else(|| PathBuf::from("../CoreCruxData/v1"));The default is relative to the daemon's working directory (config.rs:838). Two daemons started from two different directories get two different data dirs and two different LOCK files, so the single-instance guard cannot help you. Set CORECRUXD_DATA_DIR to an absolute path.
Path values go through expand_path, which substitutes $XDG_STATE_HOME, $XDG_CONFIG_HOME, $HOME and a leading ~/ (config.rs:772-791). There is no implicit XDG default, an $XDG_STATE_HOME/crux layout only happens if you configure it.
Both roots are created at startup, in this order, before anything else touches disk: create_dir_all(state_dir) (main.rs:475), create_dir_all(data_dir) (main.rs:476), then acquire_lock(data_dir) (main.rs:477).
Two subsystems read CORECRUXD_DATA_DIR directly from the process environment rather than from Config, so they silently no-op if the variable is unset even when the YAML file sets a data dir: the activity journal (activity.rs:598) and the MCP sync tool (tools/sync.rs:50).
6.2 The tree
<data_dir>/ # CORECRUXD_DATA_DIR, default ../CoreCruxData/v1
├── LOCK # daemon single-instance lock (fs2 exclusive)
├── CONTROL.json # operator valve state
├── .install-uuid # session-plane install identity
├── passport.key # daemon Ed25519 passport seed (in state_dir)
├── passport.claimed # anonymous-claim marker (in state_dir)
├── audit-export-signing.key # persistent audit-export Ed25519 key
│
├── meta/
│ ├── node.json # node identity (node_id, addrs, build)
│ └── routing/
│ ├── LOCK # shard-map publish lock
│ ├── current # ASCII u64 + newline: active shardmap version
│ ├── shardmap.v<00000001>.json # immutable versioned shard maps
│ └── tmp/ # staging for atomic publish
│
├── shards/
│ └── shard-<NNNN>/ # zero-padded 4-digit shard id
│ ├── LOCK # per-shard exclusive open lock
│ ├── MANIFEST # CCMF append-only catalogue
│ ├── segments/
│ │ ├── seg-<seq:020>-<id:32hex>.ccxseg # sealed segment (CCS3/CCF3)
│ │ ├── seg-<seq:020>-<id:32hex>.ccxhead # open head segment (unsealed)
│ │ ├── seg-<seq:020>-<id:32hex>.ccxi # BM25 companion index
│ │ └── seg-<seq:020>-<id:32hex>.ccxv # dense-vector companion (not produced here)
│ ├── directory/
│ │ └── dirrun-l<level>-r<run:020>.ccxdir # LSM directory run (CCDR)
│ ├── projections/
│ │ ├── projections.meta.json
│ │ ├── artifact_living_state.snapshot.ccxs
│ │ ├── artifact_relations.snapshot.ccxs
│ │ ├── artifact_dependents.snapshot.ccxs
│ │ ├── pressure_events.snapshot.ccxs
│ │ └── cold/
│ │ ├── relations/{<2hex>/<64hex>.ccxblk, segments/}
│ │ └── dependents/{<2hex>/<64hex>.ccxblk, segments/}
│ ├── receipts/verification/<tenant>/<receipt_id>.json
│ ├── tmp/ # in-flight segment writes
│ └── quarantine/ # swept orphans; never auto-deleted
│
├── facts.jsonl # fact store journal (primary memory)
├── entities.jsonl # substrate entity journal
├── substrate-edges.jsonl # substrate edge journal
├── sessions.jsonl # memory session journal
├── cases.jsonl # case store journal
├── relations.jsonl # relations projection journal
├── witness_proofs.jsonl # pending + witnessed seal-chain heads
├── credit-meter.jsonl # credit meter journal (flag-gated)
├── session-events.jsonl # session-plane sealed event log
├── sync-outbox.jsonl # outbound sync queue
├── sync-cursor.json # sync replication cursor
│
├── observations/
│ ├── <sanitised-session-id>.jsonl # signed observation/receipt records
│ ├── __governance__::gc.jsonl # GC receipts
│ └── __agent_session__<scope>__ledger__<passport>.jsonl
│
├── sessions/
│ ├── <session_id:32hex>.json # session registry entries
│ └── <session_id:32hex>.json.tmp # transient
│
├── passports/
│ └── <passport_id>.key # per-passport Ed25519 seed
│
├── console/
│ ├── settings.json # onboarding/console state
│ ├── chunks-index.json # console content chunk index
│ └── chunks-index.lock # index mutation lock
│
├── cost/reports.jsonl # cost lens (flag-gated)
├── activity/journal.jsonl # activity log (flag-gated)
├── provenance/verification-records.jsonl # provenance verification records
│
├── integrations/
│ ├── index.json # installed pack index
│ ├── audit.jsonl # pack + extension audit log
│ ├── packs/<pack_id>/<version>/manifest.json
│ ├── grants/<passport_fpr>/<pack_id>.json
│ ├── github/{credentials.json, selected_repos.json}
│ └── openai/credentials.json
│
├── extensions/
│ ├── trusted-keys.json
│ ├── registry/index.json
│ └── <extension_id>/extension.wasm
│
└── studio/library/index.json # signed template library indexOne feature-conditional addition: with --features dense-embed-model and CORECRUXD_DENSE_MODEL=fastembed, the ONNX model download lands directly under the data-dir root (main.rs:952).
6.3 File by file
Lifecycle reads: boot means created or opened during startup; write means created lazily on first write; GC means there is an automatic reclamation path.
Root-level control and identity
| Path | What it is | Format | Written by | Lifecycle |
|---|---|---|---|---|
LOCK | Single-instance guard for the whole data dir. Held for the process lifetime | Empty file; the semantics are in the advisory flock | main.rs:2156 | boot; never removed |
CONTROL.json | Operator valve state: pause_ingest, pause_compaction, throttle, read_only, emergency_brake | JSON (ControlV1) | main.rs:479; writer control.rs:472 | boot, then rewritten on any valve change |
meta/node.json | Node identity: node_id, HTTP and gRPC advertise addresses, build info | JSON (NodeMetaV1) | main.rs:500 | boot |
.install-uuid | 32-char hex install identity for the session plane. BLAKE3-hashed before it leaves the host | Plain text, one line | crux-session/src/passport.rs:32 | first session-plane use |
passport.key (in state_dir) | The daemon's Ed25519 passport seed, the identity that signs receipts | Text-encoded 32-byte seed | path config.rs:889; writer passport.rs:232 | boot |
passport.claimed (in state_dir) | Marker that the anonymous passport claim already succeeded, so it never retries | Plain text | main.rs:151, main.rs:2622 | on first successful claim |
passports/<id>.key | Per-passport Ed25519 seed, e.g. personal-default.key, work-default.key | Text-encoded 32-byte seed | passports.rs:688; seeded at main.rs:1019 | boot, when seeding is on |
audit-export-signing.key | Persistent Ed25519 key for signing audit-export bundles. Created with owner-only permissions | Bytes, mode 0600 | audit_signing_key.rs:119 | first audit export, unless the env key is set |
Routing and the shard map
| Path | What it is | Format | Written by | Lifecycle |
|---|---|---|---|---|
meta/routing/LOCK | Serialises shard-map publishes. Blocking lock_exclusive | Empty file + flock | shard_map.rs:108 | boot; held only during a publish |
meta/routing/current | The active shard-map version number | ASCII u64 plus a newline | shard_map.rs:107 | boot, rewritten per publish |
meta/routing/shardmap.v<NNNNNNNN>.json | Immutable versioned shard map, 8-digit zero-padded | Pretty JSON (ShardMapV1) | shard_map.rs:144 | on publish; never deleted |
meta/routing/tmp/ | Staging for the write-then-rename publish protocol | - | shard_map.rs:106 | boot; emptied by rename |
Publish protocol (shard_map.rs:182-196): write tmp/shardmap.v….json.tmp, fsync, rename into routing/, fsync the directory, write tmp/current.tmp, rename over current, fsync the directory.
The shard store
Path layout is defined once in ShardPaths::for_root (corecrux-storage/src/lib.rs:180). Shard directory names are shard-{id:04}.
Path under shards/shard-NNNN/ | What it is | On-disk format | Written by | Lifecycle |
|---|---|---|---|---|
LOCK | Exclusive open lock for the shard. Retries try_lock_exclusive ten times at 5 ms to absorb the deferred-fput flock release window | Empty file + flock | lib.rs:1374 | held while the shard is open |
MANIFEST | Append-only catalogue: AddSegment, AddDirRun, RemoveDirRun, StreamMetaUpdate. The authority for which segments are live | 256-byte header, magic CCMF, version 1, then CRC32C-framed records | header manifest.rs:68; created lib.rs:1398 | boot; appended on every seal |
segments/….ccxseg | Sealed immutable segment | Header magic CCS3 with a 4096-byte header; footer magic CCF3, 256 bytes; TOC magic TOC1; frames magic CRX1; per-block 256-byte bloom filter | naming append.rs:355; constants corecrux-segment/src/lib.rs:44 | on seal; immutable afterwards |
segments/….ccxhead | The currently-appending head segment. Not tracked in MANIFEST | The same frame stream, unsealed; CCMT 64-byte commit markers delimit crash-safe boundaries | naming append.rs:518 | on first append; renamed to .ccxseg at seal |
segments/….ccxi | BM25 inverted-index companion built at seal time | .ccxi binary, BLAKE3-hashed | companions.rs:90 | on seal, only when CORECRUXD_BUILD_CCXI is on (default off) |
segments/….ccxv | Dense-vector companion. Recognised by the orphan sweeper but not produced by this build | binary | referenced lib.rs:1468 | - |
directory/dirrun-l<level>-r<run>.ccxdir | LSM directory run | Magic CCDR, 4096-byte header, 256 partitions, 12-byte entries, 32-byte extents | naming lib.rs:565 | on directory compaction, default off |
tmp/ | Staging for segment and companion writes before the atomic rename | - | lib.rs:189 | created on open; swept into quarantine/ on every open |
quarantine/ | Where crash debris goes. Three prefixes: tmp-<ns>-<name>, orphan-<ns>-<name>, dirrun-orphan-<ns>-<name> | files moved verbatim | lib.rs:1439 | created on open; never emptied automatically |
receipts/verification/<tenant>/<receipt_id>.json | Per-receipt verification report | Pretty JSON | store_v1.rs:24 | on verification |
Durability discipline. Sealed segments and directory runs are written into tmp/, fsynced, renamed into place, then the containing directory is fsynced, and only then is the MANIFEST record appended (lib.rs:1505). A crash between the rename and the MANIFEST append leaves an orphan, which the next open quarantines, the mechanism that stops segment sequence numbers being reused.
Three-place wiring. Companion files are deliberately exempted from the orphan sweep while their .ccxseg is still MANIFEST-referenced (lib.rs:1461). Without that exemption every restart would quarantine the live retrieval indexes. The matching load-at-startup half is at main.rs:774, gated on config.build_ccxi || config.local_ingest_enabled. If you introduce a new on-disk artefact type, this is the pair of places that must both know about it.
Projections
Paths defined in ProjectionFiles (runner.rs:55).
Path under shards/shard-NNNN/projections/ | What it is | Written by | Lifecycle |
|---|---|---|---|
projections.meta.json | Projection cursors, schema versions, module ref list, commit id. The recovery anchor for every projection. Written via a temp file and rename | meta.rs:316 | boot; rewritten per commit |
artifact_living_state.snapshot.ccxs | Living-state projection snapshot | runner.rs:63 | on commit |
artifact_relations.snapshot.ccxs | Relations projection snapshot | runner.rs:64 | on commit |
pressure_events.snapshot.ccxs | Pressure-events projection snapshot | runner.rs:65 | on commit |
artifact_dependents.snapshot.ccxs | Dependents projection snapshot | runner.rs:66 | on commit |
cold/{relations,dependents}/<2hex>/<64hex>.ccxblk | Content-addressed cold blocks; BLAKE3 hex names, sharded by first byte | runner.rs:902 | on spill |
cold/{relations,dependents}/segments/ | Content-addressed cold segments, 64 MiB cap | runner.rs:919 | on spill; GC'd by gc_cold_segments_dir_v1 (runner.rs:846) |
The memory-plane journals
All are append-only JSON lines, replayed at boot, rebuilt entirely in memory. There is no index file.
| Path | What it is | Written by |
|---|---|---|
facts.jsonl | The fact store. Store, Delete and Supersede events. The single most important user-data file, coordination announces, punchcards, extension grants, session bindings and every store_fact call live here as facts, not as separate files | fact_store.rs:730 open and replay; fact_store.rs:764 durable append with double fsync |
entities.jsonl | Substrate entity store journal | entity_store.rs:99 |
substrate-edges.jsonl | Substrate edge store journal | edge_store.rs:103 |
sessions.jsonl | Memory session store journal | session_store.rs:89 |
cases.jsonl | Case store journal | case_store.rs:129 |
relations.jsonl | Relations projection journal, replayed into ProjectionState | relations.rs:233 |
witness_proofs.jsonl | Pending and witnessed seal-chain heads | witness_proofs.rs:195 |
credit-meter.jsonl | Credit meter ledger. Only when the credit meter is enabled | credit_meter.rs:260 |
session-events.jsonl | Session-plane sealed event log, one JSON line per sealed event, fsync per append | sealer.rs:122 |
sync-outbox.jsonl | Outbound sync queue | outbox.rs:32 |
sync-cursor.json | Sync replication cursor | corecrux-memory/src/sync.rs:887 |
cost/reports.jsonl | Cost-lens reports. Only when CORECRUXD_FEATURE_COST_LENS is on, the path function returns nothing otherwise, so "feature off means zero on-disk writes" | cost.rs:217 |
activity/journal.jsonl | Activity log. Only when CORECRUXD_DATA_DIR is set in the process environment, see §6.1. Best-effort: I/O errors are swallowed | activity.rs:597 |
provenance/verification-records.jsonl | Provenance verification records | provenance.rs:581 |
observations/<sanitised-id>.jsonl | Signed observation and receipt records, one file per scoped session id. Per-record payload capped by CORECRUXD_MAX_OBSERVATION_PAYLOAD_BYTES, default 1 MiB with a 64 KiB floor | observations.rs:452 |
Fact-journal compaction is operator-triggered, not automatic. FactStore::compact_journal (fact_store.rs:1787) rewrites facts.jsonl into a temp file in the same directory, fsyncs it, renames atomically, then fsyncs the parent directory. Deleted facts become value-free tombstones, the original value never reaches the rewritten journal (fact_store.rs:1898). There is no scheduled compaction.
Sessions, console, integrations, extensions
| Path | What it is | Format | Written by |
|---|---|---|---|
sessions/<session_id:32hex>.json | Session registry entry: capability plan, TTL, canonical CBOR body hex-encoded | Pretty JSON, temp file and rename | registry.rs:171 |
console/settings.json | Console onboarding state | JSON, temp file and rename | onboarding.rs:86 |
console/chunks-index.json | Console content chunk index | Pretty JSON, temp file and rename | console_index.rs:259 |
console/chunks-index.lock | Guards read-modify-write of the chunk index across concurrent requests | Empty file + flock | console_index.rs:274 |
integrations/index.json | Installed integration-pack index | JSON, atomic | crux-integrations/src/lib.rs:1351 |
integrations/audit.jsonl | Unified pack and extension audit log. Best-effort: an append failure is warn-logged and never fails the operation | JSONL | crux-integrations/src/lib.rs:1332 |
integrations/packs/<id>/<version>/manifest.json | Installed pack manifest. Path components pass through a traversal guard | JSON, atomic | crux-integrations/src/lib.rs:906 |
integrations/grants/<passport_fpr>/<pack_id>.json | Per-passport pack grant | JSON, atomic | crux-integrations/src/lib.rs:1344 |
integrations/github/credentials.json | GitHub integration credentials, owner-only permissions | JSON, temp file and rename, mode 0600 | integrations_github.rs:100 |
integrations/github/selected_repos.json | Selected repository list | JSON | integrations_github.rs:181 |
integrations/openai/credentials.json | OpenAI integration credentials, owner-only permissions | JSON, temp file and rename, mode 0600 | integrations_openai.rs:109 |
extensions/trusted-keys.json | Trusted publisher keys for extension signature verification | JSON | extension_registry.rs:76 |
extensions/registry/index.json | Verified extension registry snapshot, populated by corecruxctl extensions sync | JSON | corecruxctl/src/main.rs:1355 |
extensions/<extension_id>/extension.wasm | Downloaded WASM module, SHA-256-verified before the rename | WASM binary, temp file and rename | wasm_dispatcher.rs:302 |
studio/library/index.json | Signed template-library index, re-verified by the daemon on read | JSON | studio_library.rs:68 |
6.4 Things that are deliberately not files
These subsystems keep their state as facts in facts.jsonl, not as their own artefacts. Looking for a file is the wrong search.
- The coordination plane, announces, presence, punchcards and leases, under the entity prefix
__coord__::(coord.rs:105). Punchcards additionally appear in the substrate entity store (coord.rs:268). - Extension grants, prefix
__extension_grant__::(extension_grants.rs:12). - Session bindings, prefix
__session_binding__::(session_bindings.rs). - Legal holds, mint requests, identity links, projects and principals, all fact-backed.
- Scheduler job health, under
__sync__::<job_id>keystatus, readable throughGET /v1/facts(main.rs:1487).
Two more things that are not daemon-owned state at all:
- ExecPlan work items are a read-time projection over external
.mdfiles.work_execplans.rsreads$CRUX_EXECPLANS_ROOT/*.mdand derives state; it writes nothing to the data dir (work_execplans.rs:1130). - The console SPA is served from embedded assets, not extracted to disk. Only
CORECRUXD_CONSOLE_DEV_PATHreads from a developer directory (console.rs:527).
The practical consequence: GET /v1/facts is a general-purpose inspection tool for far more of the daemon's state than its name suggests.
6.5 Delete this and the daemon breaks
Ordered by blast radius.
| File | What breaks | Recoverable? |
|---|---|---|
facts.jsonl | Total memory loss. Every fact, coordination announce, punchcard, extension grant, session binding and legal hold is gone. The daemon starts clean and healthy and will not tell you anything is missing | Only from a backup, or from a remote if sync is configured |
shards/shard-NNNN/MANIFEST | The shard reopens with an empty catalogue, so every .ccxseg in segments/ becomes an orphan and is moved to quarantine/ on the next open (lib.rs:1479). All appended events become unreachable | The segment bytes survive in quarantine/, but there is no supported rebuild-from-segments path |
shards/shard-NNNN/segments/*.ccxseg | Data loss for those segments; MANIFEST validation fails on open | No |
passport.key | The daemon mints a new identity. Previously-signed receipts no longer verify against the advertised public key; the anonymous passport claim and every issued capability token are orphaned; stored integration credentials become undecryptable | No, a fresh key is generated silently at passport.rs:232 |
passports/<id>.key | That passport's signing identity is regenerated; tokens and receipts signed by the old key stop verifying | No |
meta/node.json | A new node_id on the next boot. Shard-map entries pointing at the old id go stale, and the replicated_commit_topology readiness gate can fail | No |
meta/routing/current | Falls back to initialising a fresh default dev shard map (shard_map.rs:126), which will not match the existing shard directories | Partially, the shardmap.v*.json files are still there and current can be hand-restored |
projections.meta.json | Projection cursors reset to zero; a full replay is required, and row counts and commit_id restart | Yes, by replay, at the cost of a full rescan |
CONTROL.json | Operator valves reset to defaults, an emergency_brake or read_only you set is silently cleared | No |
audit-export-signing.key | Previously-exported audit bundles no longer verify against the current key | No |
.install-uuid | The session-plane install identity changes; the daemon reports as a different install to any collector | No |
Safe to delete, regenerated or purely additive: LOCK, shards/*/LOCK, meta/routing/LOCK and console/chunks-index.lock, all only while the daemon is stopped; shards/*/quarantine/*; shards/*/tmp/*; console/chunks-index.json; and the fastembed model cache.
A minimal backup set, if you cannot take the whole directory: facts.jsonl, passport.key, passports/, CONTROL.json, meta/, shards/, audit-export-signing.key, integrations/, .install-uuid.
6.6 The four locks
| Lock | Path | Mechanism | Guards | Held for |
|---|---|---|---|---|
| Daemon instance lock | <data_dir>/LOCK | try_lock_exclusive: non-blocking, fails startup | The entire data dir: one corecruxd per data dir. Feeds the data_dir_lock_held readiness gate | Process lifetime |
| Shard lock | <data_dir>/shards/shard-NNNN/LOCK | try_lock_exclusive with a ten-times-5 ms retry for the deferred-fput window | One writer per shard: MANIFEST, segments, directory, projections | While the shard handle is open |
| Shard-map publish lock | <data_dir>/meta/routing/LOCK | lock_exclusive, blocking | The atomic shard-map publish protocol | One publish |
| Console index lock | <data_dir>/console/chunks-index.lock | lock_exclusive, blocking, explicitly unlocked | Read-modify-write of console/chunks-index.json across concurrent HTTP requests | One index mutation |
Two locks are in-process, not on disk: the session-plane sealer uses a Mutex (sealer.rs:117), and the audit-signing-key creator a process-wide OnceLock<Mutex<()>> (audit_signing_key.rs:120).
6.7 What grows without bound
Six artefacts have no automatic reclamation path at all. On a long-lived daemon they are the reason the disk fills, and a full disk takes the daemon out of rotation via the data_dir_capacity readiness gate, see chapter 9 §9.5.
| Artefact | Growth driver | GC path | Default |
|---|---|---|---|
facts.jsonl | Every store, delete and supersede. The journal never shrinks on its own | Ephemeral GC (ephemeral_gc.rs), plus operator-triggered compact_journal | Ephemeral GC off; compaction never automatic |
__session_binding__::* facts | One durable fact per MCP session. A stateless bridge that re-initialises per poll accumulates without bound | Ephemeral GC keeps the newest 32 per passport and collects the rest once older than 1 hour | Off by default |
__reverify_receipts__::* facts | Minted by memory_reverify | Ephemeral GC deletes when older than 30 days | Off by default |
observations/*.jsonl | Every signed observation, receipt and ledger row. One file per session id, so the file count grows with session count too | None. No rotation, no retention sweep, no archive. Only the per-record payload is capped | - |
shards/*/quarantine/ | Every crash-recovery sweep on shard open. Filenames are timestamp-prefixed so nothing is overwritten | None. No code path deletes from quarantine/. Reap it manually | - |
meta/routing/shardmap.v*.json | One immutable file per shard-map version | None | - |
integrations/audit.jsonl | Every pack and extension action | None, reads are tail-only | - |
activity/journal.jsonl | Every activity-log entry | None | Flag-gated |
cost/reports.jsonl | Every POST /v1/cost/report | None | Flag-gated |
session-events.jsonl | Every sealed session event, fsync per append | None | - |
sessions/*.json | One file per issued session | Expiry is enforced in the registry; on-disk cleanup follows the registry's removal path, not a scheduled sweep | - |
cold/*/segments/* | Projection cold spill | gc_cold_segments_dir_v1 with min_age_seconds, max_delete and dry_run, the one real segment GC in the tree | Caller-driven |
witness_proofs.jsonl, entities.jsonl, substrate-edges.jsonl, sessions.jsonl, cases.jsonl, relations.jsonl, credit-meter.jsonl, sync-outbox.jsonl | Append-only journals | None. There is no compaction equivalent to the fact journal | - |
CORECRUXD_OBS_RETENTION_DAYS archives observation sessions but is unset by default, i.e. retain forever (main.rs:1223).
The ephemeral GC, in detail
- Gate:
CORECRUXD_EPHEMERAL_GC, default off. Read once at boot, toggling requires a restart (ephemeral_gc.rs:194). - Schedule: hourly. The immediate first tick is skipped so a sweep never runs mid-replay (ephemeral_gc.rs:215).
- Scope: exactly two reserved entity prefixes, matched by name,
__reverify_receipts__::and__session_binding__::(ephemeral_gc.rs:98). Non-reserved user facts are never eligible, private or not. - Mechanism: a soft delete through the journalled
FactStore::try_delete(ephemeral_gc.rs:153). It never touches the filesystem. It appends aDeletetombstone, so the fact stays visible withdeleted = true, reversible and replay-safe. - Receipt: a non-empty sweep mints a signed receipt into
observations/__governance__::gc.jsonlcarrying only{deleted, retain_days, reason_code, swept_at, run_id}, never swept content (ephemeral_gc.rs:167). A mint failure bumps an audit-debt counter and logs at error level; it is never silent.
6.8 The capacity guard writes to your control file
CONTROL.json is not only operator-owned. On reaching the emergency free-space threshold, the background capacity guard sets valves.pauseIngest with actor = "capacity_guard" and a reason string, and persists it (main.rs:2506). It takes ownership only if the valve is currently disabled or already guard-owned (main.rs:2510); it will not stomp an operator-set pause.
If you find ingest paused with actor: "capacity_guard", look at disk free ratio, not at your own actions.
Capacity classification uses strictly-less-than comparisons against free_ratio (main.rs:2424):
| Level | Condition | Config default |
|---|---|---|
emergency | free_ratio < capacity_emergency_free_ratio | 0.10 |
critical | free_ratio < capacity_critical_free_ratio | 0.10 |
warning | free_ratio < capacity_warning_free_ratio | 0.20 |
healthy | otherwise | - |
Free space is measured with fs2::total_space and fs2::available_space on the data dir (main.rs:2436), note available space, not raw free space. A measurement failure zeroes the gauges, records the error, and clears auto_paused.
The four ratios are re-ordered after parsing so they can never be inconsistent (config.rs:1117). Raising EMERGENCY to 0.5 while leaving WARNING at its default therefore raises warning to 0.5 too, rather than producing an impossible ordering.
Sources
- crates/corecruxd/src/config.rs:834, data-dir resolution
- crates/corecruxd/src/main.rs:2156,
acquire_lock - crates/corecruxd/src/control.rs:138,
CONTROL.jsonload - crates/corecrux-storage/src/lib.rs:180,
ShardPaths::for_root - crates/corecrux-storage/src/lib.rs:1461, the companion-file sweep exemption
- crates/corecrux-memory/src/fact_store.rs:1787,
compact_journal - crates/corecrux-projections/src/runner.rs:846, cold-segment GC
- crates/corecruxd/src/ephemeral_gc.rs:98, the two reserved GC prefixes
- crates/corecruxd/src/main.rs:2506, the capacity guard writing
CONTROL.json