Menu

RocksDB

RocksDB is often used as the data store for Crux’s query indices, but can also be used as a transaction log and/or document store in single node clusters.

Project Dependency

In order to use RocksDB within Crux, you must first add RocksDB as a project dependency:

  • deps.edn

  • pom.xml

pro.juxt.crux/crux-rocksdb {:mvn/version "1.18.0"}
<dependency>
    <groupId>pro.juxt.crux</groupId>
    <artifactId>crux-rocksdb</artifactId>
    <version>1.18.0</version>
</dependency>

If you’re using RocksDB and seeing out-of-memory issues, we recommend setting the environment variable MALLOC_ARENA_MAX=2 - see this issue for more details.

Using RocksDB

Replace the implementation of the desired component with crux.rocksdb/->kv-store

  • JSON

  • Clojure

  • EDN

{
  "crux/index-store": {
    "kv-store": {
      "crux/module": "crux.rocksdb/->kv-store",
      "db-dir": "/tmp/rocksdb"
    }
  },

  "crux/document-store": { ... },
  "crux/tx-log": { ... }
}
{:crux/index-store {:kv-store {:crux/module 'crux.rocksdb/->kv-store
                               :db-dir (io/file "/tmp/rocksdb")}}
 :crux/document-store {...}
 :crux/tx-log {...}}
{:crux/index-store {:kv-store {:crux/module crux.rocksdb/->kv-store
                               :db-dir "/tmp/rocksdb"}}
 :crux/document-store {...}
 :crux/tx-log {...}}

It is generally advised to use independent RocksDB instances for each component, although using a single instance for the transaction log and document store is possible. Do not share the RocksDB instance used for the index store with other components as you cannot then perform Crux version upgrades.

Dependencies

Parameters

  • db-dir (required, string/File/Path): path to RocksDB data directory

  • sync? (boolean, default false): sync to disk after every write

  • disable-wal? (boolean): disables the write-ahead log

  • db-options (RocksDB Options object): extra options to pass directly to RocksDB.

Monitoring RocksDB

To include RocksDB metrics in monitoring, override the metrics dependency:

  • JSON

  • Clojure

  • EDN

{
  "crux/index-store": {
    "kv-store": {
      "crux/module": "crux.rocksdb/->kv-store",
      "metrics": {
        "crux/module": "crux.rocksdb.metrics/->metrics"
      }
      ...
    }
  },

  "crux/document-store": { ... },
  "crux/tx-log": { ... }
}
{:crux/index-store {:kv-store {:crux/module 'crux.rocksdb/->kv-store
                               :metrics {:crux/module 'crux.rocksdb.metrics/->metrics}}
 :crux/document-store {...}
 :crux/tx-log {...}}
{:crux/index-store {:kv-store {:crux/module crux.rocksdb/->kv-store
                               :metrics {:crux/module crux.rocksdb.metrics/->metrics}}
 :crux/document-store {...}
 :crux/tx-log {...}}

Parameters

  • instance-name (string, default "rocksdb"): unique name for this instance of RocksDB, used in metrics domains

  • sample-window (duration, default 3s): sample window of statistics collector

Configuring the Block Cache

To configure the block cache used by the RocksDB instance, override the block-cache dependency. In the example below, there is a single shared cache between multiple kv-stores:

  • JSON

  • Clojure

  • EDN

{
  "crux.rocksdb/block-cache": {
    "crux/module": "crux.rocksdb/>lru-block-cache",
    "cache-size":536870912
  },
  "crux/index-store": {
    "kv-store": {
      "crux/module": "crux.rocksdb/->kv-store",
      "block-cache": "crux.rocksdb/block-cache"
      ...
    }
  },
  "crux/document-store": {
    "kv-store": {
      "crux/module": "crux.rocksdb/->kv-store",
      "block-cache": "crux.rocksdb/block-cache"
    }
  },
  "crux/tx-log": {
    "kv-store": {
      "crux/module": "crux.rocksdb/->kv-store",
      "block-cache": "crux.rocksdb/block-cache"
    }
  }
}
{:crux.rocksdb/block-cache {:crux/module 'crux.rocksdb/->lru-block-cache
			    :cache-size (* 512 1024 1024)}
 :crux/index-store {:kv-store {:crux/module 'crux.rocksdb/->kv-store
                               :block-cache :crux.rocksdb/block-cache}}
 :crux/document-store {:kv-store {:crux/module 'crux.rocksdb/->kv-store
                                  :block-cache :crux.rocksdb/block-cache}}
 :crux/tx-log {:kv-store {:crux/module 'crux.rocksdb/->kv-store
                          :block-cache :crux.rocksdb/block-cache}}}
{:crux.rocksdb/block-cache {:crux/module crux.rocksdb/->lru-block-cache
			    :cache-size 536870912}
 :crux/index-store {:kv-store {:crux/module crux.rocksdb/->kv-store
                               :block-cache :crux.rocksdb/block-cache}}
 :crux/document-store {:kv-store {:crux/module crux.rocksdb/->kv-store
                                  :block-cache :crux.rocksdb/block-cache}}
 :crux/tx-log {:kv-store {:crux/module crux.rocksdb/->kv-store
                          :block-cache :crux.rocksdb/block-cache}}}

Parameters

  • cache-size (int): Size of the cache in bytes - default size is 8Mb, although it is recommended this is set to a higher amount.