Optimising Cronos Node Storage With VersionDB

As Cronos continues to expand and generate blocks, managing archive nodes becomes more complex due to the growing size of the databases. The current archive node has reached the size of 2.4T. This means the ongoing size expansion of the IAVL database in archive nodes. 

In order to ensure that the Cronos blockchain can continue to grow, the Cronos development team has introduced support for the VersionDB initially from v1.0.4 and has been observing its performance. Drawing inspiration from the powerful features of the RocksDB engine, VersionDB provides a robust solution to the disk-size challenge. One of its most remarkable features is the ability to seamlessly integrate with various backend databases, including the notable pebbleDB. 

While VersionDB brings a host of benefits to the table, it's vital to recognize that its suitability depends on each user’s specific use case. In this article, we aim to provide an overview of VersionDB, its advantages, and the essential considerations to help Cronos node operators to make an informed decision on whether to incorporate it into their projects. Let's dive in and explore the key aspects of VersionDB!

Benefits and Limitations of VersionDB for node operators

VersionDB brings notable advantages worth highlighting:

• Reduced database size 

  VersionDB addresses the size issue of the IAVL database by directly storing multiple versions of on-chain state key-value pairs. This eliminates the need for a Merklized tree structure like the IAVL tree and significantly reduces the size of the database.
  After migrating to VersionDB, users can then prune the IAVL tree to reclaim disk size. We performed this migration for our Cronos public nodes and observed a reduction of approximately 63% in disk size for archive nodes.

• Improved query performance

  By storing data in a more efficient manner, VersionDB allows for faster and more efficient retrieval of information from the database, which triggers a better query performance compared to the IAVL tree. 

While VersionDB offers various advantages, it's essential to be aware of its limitations and considerations: 

• Different implementations

  VersionDB has different implementations. Our current implementation is based on RocksDB v7's experimental user-defined timestamp, which stores the data in a standalone RocksDB instance. However, it currently does not support other database backends.

• Missing Root Hash and Merkle Proof Generation for Historical Data

  VersionDB does not perform root hash and Merkle proof generation for historical data. Therefore, we still need the IAVL tree for those specific use cases. VersionDB does not support 1) eth_getProof for the historical versions that are pruned in the IAVL tree, and 2) non-grpc /abci_query for the historical versions that are pruned in the IAVL tree. 

Migration Overview

Once you have decided to migrate to VersionDB, you need to consider the following key steps for the migration. More comprehensive details and instructions are included in the VersionDB Migration Guide.

1. Configuration: Add VersionDB to app.toml. The node will subscribe to the latest state changes in real time and save them to VersionDB. 

2. Migration: The migration process tries to parallelise the tasks as much as possible, and uses significant RAM memory. 

   The three main steps for the legacy state migration process are as follows:

   2a) Extract and Verify Change Sets: Extract change sets from the IAVL tree, replay change sets, and verify the app hash.

   2b) Build VersionDB

   2c) Restore and Catch Up With IAVL Tree: Rebuild the application.db from scratch, and then sync VersionDB with application.db or restore application.db with the correct snapshot.

Migration Considerations and Optimization Tips for VersionDB Migration

In our migration journey to VersionDB, we have successfully achieved a significant reduction of approximately 63% in disk size for an archive node on Cronos. At this stage, this solution currently is only recommended for archive and non-validator nodes to try. Future improvements are expected to expand its applicability.

Currently GRPC query service doesn't need to support proof generation, so Versiondb alone is enough to support the GRPC query service, there's already a --grpc-only flag for one to start a standalone GRPC query service. This means GRPC queries can be performed using VersionDB alone, simplifying the query process. Nevertheless, it's crucial to be aware that its specific database structure, in comparison to regular nodes, restricts the ability to pull the Merkle tree, and certain endpoints are reserved and remain unavailable.

Lastly, it's important to note that migration from scratch may take up to a few days, with the change set extraction being the most time-consuming step. Specifically, running tests on the testnet archive node can consume approximately 11 hours on an 8-core SSD machine, while verification of the change set files takes only a few minutes. To address this issue, we prepared a downloadable link for users, which enables you to access the files and locally verify their accuracy: [VersionDB Archive Snapshot] and [VersionDB Default Snapshot].

Alternatively, for the VersionDB Default snapshot, users can download it by either of the following commands:

• Using curl:

curl -LOJ https://snapshot.cronos.org/cronos/mainnet-snapshot/cronosmainnet_25-1-versiondb-default-latest.tar.lz4

• Using wget:

Wget https://snapshot.cronos.org/cronos/mainnet-snapshot/cronosmainnet_25-1-versiondb-default-latest.tar.lz4

For those opting to perform the migration themselves, planning for the required migration downtime in advance is advisable. Also, users can bulk up their RAM memory to avoid reading from disk. Additionally, increasing the concurrency level to 128 can further expedite the process, although this may require additional CPU resources like R6g.2xlarge.

By taking these considerations into account, users can ensure a smoother transition and enhanced performance with VersionDB.

What’s Next

Exciting times lie ahead as we continue to refine the VersionDB migration process. Stay tuned for upcoming updates and enhancements, which will further optimize its efficiency and accessibility. By remaining engaged and proactive, we can unlock new levels of performance and resource utilization on Cronos. Users’ involvement and feedback play a crucial role in shaping the future of VersionDB, and together, we'll usher in a more robust and rewarding experience on the Cronos chain.