Disk Usage Optimization

Efficiently managing disk space is crucial for running a Dhive validator node. Blockchain databases grow over time, influenced by factors such as block speed and transaction volume. In the case of Dhive, disk usage can reach approximately 100GB within the first two weeks of operation.

Fortunately, several configuration adjustments can significantly reduce the required disk space. Some of these changes take full effect only when applied before the initial synchronization of the node.

Indexing

If transaction queries from the specific node are not required, indexing can be disabled. To do this, modify the config.toml file as follows:

indexer = "null"

If indexing is disabled on an already synchronized node, the collected index is not removed automatically and must be deleted manually. The index is located in the database directory under data/tx_index.db/.

State-Sync Snapshots

State-sync snapshots may be disabled by default on Dhive, but it is worth verifying. To disable state-sync, modify the app.toml file:

snapshot-interval = 0

If state-sync is enabled and functioning correctly, it allows a new node to synchronize within minutes. However, note that such a node will not retain historical data.

Configuring Pruning

By default, every 500th state and the last 100 states are retained, consuming significant disk space over time. Optimizing pruning settings can help manage storage more effectively:

pruning = "custom"
pruning-keep-recent = "100"
pruning-keep-every = "0"
pruning-interval = "10"

Setting pruning-keep-recent = "0" may seem attractive but poses a risk of database corruption if dhived is unexpectedly terminated. It is advisable to retain at least a few recent states.

Logging

By default, the logging level is set to info, which generates extensive logs. While this setting is beneficial during the initial sync, reducing the log level once synchronization is stable can help manage disk usage. Modify config.toml as follows:

log_level = "warn"

Additionally, ensure proper log rotation is configured to prevent excessive log file accumulation.

Results

Below is a comparison of disk usage after two weeks on the Dhive network.

Default Configuration (90GB total disk usage):

5.3G    ./state.db
70G     ./application.db
20K     ./snapshots/metadata.db
24K     ./snapshots
9.0G    ./blockstore.db
20K     ./evidence.db
1018M   ./cs.wal
4.7G    ./tx_index.db

Optimized Configuration (17GB total disk usage):

17G     .
1.1G    ./cs.wal
946M    ./application.db
20K     ./evidence.db
9.1G    ./blockstore.db
24K     ./snapshots
20K     ./snapshots/metadata.db
5.3G    ./state.db

By applying these optimizations, the required disk space can be reduced significantly, ensuring a more efficient and cost-effective operation of a Dhive validator node.