Our contributions to cosmos-pruner and cosmos-gc

Running a Cosmos validator is not a set-it-and-forget-it operation. Beyond uptime and signing performance, operators face a quieter but relentless pressure: disk consumption. Every block, every state transition, every IAVL tree write accumulates on disk. Left unmanaged, a busy Cosmos chain can grow its data directory past 500 GB within a single year — and Realio Network, with its multi-token staking and active RWA transaction load, sits at the higher end of that range. For validators running on commodity hardware or constrained cloud instances, this is not a footnote; it is an operational ceiling.

The anatomy of state bloat

Cosmos SDK chains store application state using the IAVL (Immutable AVL) tree — a versioned, Merkle-provable key-value structure. Every block height produces a new version of the tree. By default, nodes keep every version going back to genesis. This is by design: archive nodes need full history for indexing and proof generation. But for a validator whose job is signing the current block and serving peers, retaining 18 months of historical state is pure overhead. It consumes disk, it slows down state reads, and it makes snapshot generation slower than it needs to be.

Pruning is the standard answer: tell the node to discard old versions beyond a rolling window — typically keeping the last few hundred heights plus periodic snapshots. The Cosmos SDK ships with pruning configuration built in, and the upstream cosmos-pruner CLI tool from Notional Labs provides an offline alternative for nodes that have already accumulated years of history without pruning enabled.

Where the upstream tool fell short

The upstream cosmos-pruner works well for straightforward single-token chains. Realio’s multi-staking architecture — where RIO, RST, and DSTRX each maintain separate staking module state but share the same underlying LevelDB — exposed edge cases the upstream tool was not written to handle. Specifically, we encountered two failure modes:

• Version compaction would occasionally stall when encountering orphaned references across staking module prefixes, leaving the pruning process in an inconsistent intermediate state.
• Height calculation for snapshot retention was off-by-one in certain configurations, causing the tool to retain one fewer snapshot than intended — a subtle bug that becomes visible only when you later try to restore from snapshot.

We also found that the tool had no mechanism for the second phase of cleanup: after IAVL versions are pruned, the underlying LevelDB still holds orphaned nodes — data that is no longer reachable from any live version but has not been reclaimed. This is where cosmos-gc comes in.

What cosmos-gc actually does

Think of the IAVL tree as a persistent data structure. When you delete a version, you remove the root pointer for that version — but all the intermediate nodes that were only referenced by that version remain in LevelDB until something explicitly reclaims them. The garbage collector walks the live tree, marks every reachable node, then scans LevelDB and deletes everything unmarked. On a node that has run pruning but never GC’d, this second pass typically recovers another 20–40 GB of space that pruning alone left behind.

cosmos-gc is designed to run after cosmos-pruner — in sequence, offline, with the node stopped. Both tools operate on a copy of the data directory if you want a safety net, or directly on the live directory if you’re comfortable with the risk profile (we always run on a copy first).

How to use them

The recommended workflow is straightforward. Stop your node, take a snapshot of the data directory to a separate volume, then run cosmos-pruner against the snapshot with your desired keep-recent and keep-every values. Once pruning completes — expect this to take 30 minutes to several hours depending on database size and disk speed — run cosmos-gc against the same directory to reclaim orphaned nodes. Verify the resulting database opens cleanly by starting the node against the pruned data in a test environment, confirming it syncs a few blocks, then swap it in for production.

We run this sequence quarterly on our Realio validator. The first run on an unpruned node is the most dramatic; subsequent quarterly runs typically reclaim 15–30 GB depending on block throughput since the last pass. It keeps our storage costs predictable and our node start times fast — a leaner LevelDB compacts faster and initializes faster. Both are worth having.

If you hit issues specific to Realio’s multi-staking state model, open an issue on the cosmos-gc repo. We monitor it and we’d rather know about edge cases than have operators silently work around them. The network is better when the tooling is better for everyone running it.