Living benchmark

Database engines, measured through a real application

Most database benchmarks measure workloads nobody runs: generate uniform rows, pick flattering queries, call the driver directly, publish a number with an X after it. The number is usually true and almost always useless, because your application does not talk to a driver.

So this one swaps the storage engine underneath a working system and changes nothing else. The data, the queries, the indexes and the code path stay exactly as they are, and every measurement below includes the real cost of query translation, serialization, the access check, the soft-delete filter, the account filter and pagination.

First comparison: MongoDB against ScyllaDB. More engines get added here as they are measured, and the rules never bend for a new one.

The reasoning behind these numbers, the two runs we threw away and what we would tell a team facing this decision: We swapped the database under a real application.

Factor from
Test kind
Scenario groups in focus

Stating ScyllaDB against MongoDB, median figures. Narrowing the groups dims rows it does not cover — nothing is removed from the page.

Run C — strict index parity, indexes present during the insert

Drop-in comparison
MongoDBDocument store

Reference engine. Every factor on this page is stated against it, across 9 measured scenarios.

ScyllaDBWide-column store
1Won8Lost0Parity
Widest margin
1 123× — Deep page, page 200
Narrowest margin
1.36× — All measurements for one inspection

Against MongoDB, on 9 scenarios, median figures. No overall factor is stated: the spread is the result.

Run C — strict index parity, indexes present during the insert — 250 006 measurement records, generated from a seeded generator so both engines hold identical rows. Factors are computed from the median figures printed in this table, stating ScyllaDB against MongoDB. All 9 scenarios are listed; the group filter dims rows, it does not remove them.
ScenarioRowsMongoDBScyllaDBFactorRelative
Medianp95Medianp95
Bulk insert, 250 000 recordsWrite throughput with six secondary indexes already in place.Writes250 00011 028 rows/s17 147 rows/sScyllaDB 1.55×
Point read by identifier, 200 requests200 sequential primary-key lookups, one round trip each.Key access20068 ms83 ms115 ms122 msMongoDB 1.69×
All measurements for one inspectionEvery child record of one parent, on an indexed foreign key.Key access4944.5 ms6.3 ms6.1 ms6.9 msMongoDB 1.36×
First sorted page, 50 rowsPage one of the collection, sorted by creation date.Pagination501.9 ms2.5 ms12 ms13 msMongoDB 6.32×
Deep page, page 200Skip 9 950 rows, return the next 50. What a numbered pager asks for.Pagination506.0 ms6.5 ms6.74 s9.63 sMongoDB 1 123×
Filtered scan on an unindexed booleanEvery record where an approval flag is false. Indexed on neither engine.Scans and filters10 00080 ms147 ms6.23 s6.56 sMongoDB 78×
Range scan on an unindexed numberOne indexed equality plus a greater-than on an unindexed measurement value.Scans and filters1652.5 ms3.7 ms4.9 ms5.7 msMongoDB 1.96×
Exact count of the whole collectionHow many records are there. Not an estimate.Whole-collection work250 00676 ms78 ms6.34 s7.93 sMongoDB 83×
Load the whole collectionEvery record, deserialized into the application.Whole-collection work250 0061.95 s2.38 s6.43 s7.15 sMongoDB 3.3×
Application
Franska Bukten — a ventilation inspection system, read through its own data layer
Dataset
250 006 measurement records, generated from a seeded generator so both engines hold identical rows
Date
2026-07-29

Method

  • Median of 10 iterations, 2 warmups discarded, p95 reported alongside
  • 200 point reads per iteration
  • Every query issued through the application's data service, query handler, adapter loader and data layer — never through the driver
  • Row counts compared across engines before any timing was reported, and they matched on every scenario
  • The whole production database migrated first: 41 collections, zero row-count mismatches

Index parity

  • Both engines: secondary indexes on _type, _besiktning_id, _system_id, _ventil_id, _room_id and _anmarkning_id
  • Verified by reading each engine's own index catalog, not by trusting the setup script
  • Present on both engines while the 250 000 rows were written
  • godkand and varde left unindexed on both sides on purpose, so the scan scenarios are genuinely unindexed work for everyone

What this run does not establish

  • That MongoDB does key lookups faster in general — the point read was sequential, and ScyllaDB crossed a network boundary MongoDB did not.
  • Anything about concurrency. Every request in this run was sequential; ScyllaDB's architecture is built to pay off under load, and load was not applied.
  • Anything about multi-node behaviour, replication or failover. One node, developer mode.
  • How well either engine could perform after being redesigned around it — that is a different test, and it is not this one.

Run B — parity indexes created after the seed

Drop-in comparison

Invalid run, published anyway. The six MongoDB indexes were created after the rows were written, so no write comparison exists for this run and the reads were served by indexes built on settled data rather than maintained through the insert. The reads are close to the valid run; the method still fails.

Run B — parity indexes created after the seed — 250 006 measurement records. Factors are computed from the median figures printed in this table, stating ScyllaDB against MongoDB. All 8 scenarios are listed; the group filter dims rows, it does not remove them.
ScenarioRowsMongoDBScyllaDBFactorRelative
Medianp95Medianp95
Point read by identifier, 200 requests200 sequential primary-key lookups, one round trip each.Key access20056 ms66 ms116 ms119 msMongoDB 2.07×
All measurements for one inspectionEvery child record of one parent, on an indexed foreign key.Key access4944.5 ms5.2 ms4.9 ms6.8 msMongoDB 1.09×
First sorted page, 50 rowsPage one of the collection, sorted by creation date.Pagination501.6 ms2.2 ms9.7 ms21 msMongoDB 6.06×
Deep page, page 200Skip 9 950 rows, return the next 50. What a numbered pager asks for.Pagination505.1 ms6.0 ms5.51 s5.62 sMongoDB 1 080×
Filtered scan on an unindexed booleanEvery record where an approval flag is false. Indexed on neither engine.Scans and filters10 00081 ms89 ms5.44 s5.54 sMongoDB 67×
Range scan on an unindexed numberOne indexed equality plus a greater-than on an unindexed measurement value.Scans and filters1652.8 ms8.4 ms4.6 ms6.3 msMongoDB 1.64×
Exact count of the whole collectionHow many records are there. Not an estimate.Whole-collection work250 00666 ms68 ms5.45 s5.47 sMongoDB 83×
Load the whole collectionEvery record, deserialized into the application.Whole-collection work250 0061.99 s2.50 s5.48 s5.56 sMongoDB 2.75×
Application
Franska Bukten — a ventilation inspection system, read through its own data layer
Dataset
250 006 measurement records
Date
2026-07-28

Method

  • Median of 10 iterations, 2 warmups discarded
  • 200 point reads per iteration
  • MongoDB seeded first, indexed second — the defect
  • No write measurement taken, because a valid one was not possible

Index parity

  • MongoDB: six secondary indexes, created after the seed
  • ScyllaDB: the same six, created with the table and present during the insert
  • Field parity, but not lifecycle parity

What this run does not establish

  • Any statement about write throughput. There is none in this run.
  • That an index created after a bulk load behaves identically to one maintained during it.

Run A — no index parity

Drop-in comparison

Invalid run, published anyway. MongoDB carried only its default identifier index while ScyllaDB carried six secondary indexes. The three ScyllaDB wins below are artefacts of that gap, not properties of either engine. The timings are real; they answer the wrong question.

Run A — no index parity — 250 006 measurement records. Factors are computed from the median figures printed in this table, stating ScyllaDB against MongoDB. All 9 scenarios are listed; the group filter dims rows, it does not remove them.
ScenarioRowsMongoDBScyllaDBFactorRelative
Medianp95Medianp95
Bulk insert, 250 000 recordsWrite throughput with six secondary indexes already in place.Writes250 00027 105 rows/s21 077 rows/sMongoDB 1.29×
Point read by identifier, 200 requests200 sequential primary-key lookups, one round trip each.Key access20051 ms61 ms114 ms124 msMongoDB 2.24×
All measurements for one inspectionEvery child record of one parent, on an indexed foreign key.Key access49488 ms93 ms5.2 ms6.5 msScyllaDB 17×
First sorted page, 50 rowsPage one of the collection, sorted by creation date.Pagination5086 ms88 ms9.4 ms18 msScyllaDB 9.15×
Deep page, page 200Skip 9 950 rows, return the next 50. What a numbered pager asks for.Pagination505.9 ms6.6 ms6.12 s20.8 sMongoDB 1 037×
Filtered scan on an unindexed booleanEvery record where an approval flag is false. Indexed on neither engine.Scans and filters10 00085 ms93 ms5.76 s7.36 sMongoDB 68×
Range scan on an unindexed numberOne indexed equality plus a greater-than on an unindexed measurement value.Scans and filters16594 ms103 ms4.8 ms8.1 msScyllaDB 20×
Exact count of the whole collectionHow many records are there. Not an estimate.Whole-collection work250 00675 ms78 ms5.61 s6.01 sMongoDB 75×
Load the whole collectionEvery record, deserialized into the application.Whole-collection work250 0062.30 s2.85 s5.63 s5.94 sMongoDB 2.45×
Application
Franska Bukten — a ventilation inspection system, read through its own data layer
Dataset
250 006 measurement records
Date
2026-07-28

Method

  • Median of 10 iterations, 2 warmups discarded
  • 200 point reads per iteration
  • Every query issued through the application's data service, not the driver

Index parity

  • MongoDB: the default identifier index only
  • ScyllaDB: six secondary indexes, created automatically by the adapter
  • Not parity — this is the defect the run exists to demonstrate

What this run does not establish

  • Nothing. No comparison in this table is valid.
  • Adding the six matching MongoDB indexes moved the inspection lookup from 88 ms to 4.5 ms and the sorted first page from 86 ms to 1.6 ms.
  • All three ScyllaDB wins disappeared once the index sets matched.

The engines

MongoDB

Document store
Storage model
BSON documents in collections, B-tree indexes, WiredTiger storage engine.
Index model
General secondary indexes on any field, created independently of the query.
Query model
Ad hoc filters, sorts and ranges on any indexed or unindexed field, decided at query time.
Driver
mongodb 6.21.0

Built for

  • Flexible document structures
  • General secondary indexes
  • Ad hoc filtering and sorting
  • Range queries and aggregation
  • Query patterns that change after launch

How it was deployed

  • Native process on 127.0.0.1:27017, reached over loopback
  • Whole machine available, no container and no virtualisation layer
  • The friendlier of the two deployments — stated because it matters

ScyllaDB

Wide-column store
Storage model
Log-structured merge trees, rows in partitions, shard-per-core architecture.
Index model
Partition and clustering keys drive access; secondary indexes exist but are a second-class path.
Query model
No OR, no OFFSET, no arbitrary sort — WHERE is restricted to the key or an indexed column, and the table is designed around the query.
Version
6.2.3-0.20250119.bff9ddde1283, CQL 3.0.8
Driver
cassandra-driver 4.9.0, consistency LOCAL_ONE, fetch size 5000

Built for

  • High write throughput
  • Predictable low latency under concurrency
  • Horizontal scaling across nodes
  • Partition-key reads
  • Data models designed around known queries

How it was deployed

  • Docker container from image scylladb/scylla:6.2
  • Running under WSL2 Ubuntu 26.04, reached through WSL port forwarding
  • Two CPU shards and 4 GB of memory, with --overprovisioned 1 --developer-mode 1
  • Crosses a network boundary MongoDB did not have to cross

PostgreSQL

RelationalNot yet measured
Storage model
Heap tables with MVCC row versions, B-tree and GIN indexes, WAL-based durability.
Index model
General secondary indexes, plus expression, partial and JSONB indexes.
Query model
Full SQL with a cost-based planner: joins, aggregates, window functions, arbitrary sorts.

Built for

  • Relational integrity and transactions
  • Complex joins and aggregation
  • Mixed read and write workloads on one node

How it was deployed

  • Not deployed yet — this entry is what the benchmark needs in order to include it.

Apache Cassandra

Wide-column storeNot yet measured
Storage model
Log-structured merge trees on the JVM, the design ScyllaDB reimplemented in C++.
Index model
Partition and clustering keys, with the same second-class secondary index path.
Query model
CQL — the same restrictions as ScyllaDB, so the same scenarios should fail the same way.

Built for

  • High write throughput
  • Multi-datacenter replication
  • Query-driven table design

How it was deployed

  • Not deployed yet — worth running precisely because CQL predicts its shape.

Couchbase

Document storeNot yet measured
Storage model
Memory-first documents with a managed cache in front of persistent storage.
Index model
Global secondary indexes maintained by a separate index service.
Query model
SQL++ over documents, plus a key-value fast path that bypasses the query engine.

Built for

  • Low-latency key-value reads from cache
  • SQL-style querying over documents
  • Scaling read and index services independently

How it was deployed

  • Not deployed yet.

CockroachDB

Distributed relationalNot yet measured
Storage model
Range-partitioned key-value storage with Raft replication under a SQL layer.
Index model
Secondary indexes, themselves distributed as ranges across the cluster.
Query model
PostgreSQL-compatible SQL with distributed execution and serializable transactions.

Built for

  • Horizontal scale without giving up SQL
  • Serializable distributed transactions
  • Surviving the loss of a node or a region

How it was deployed

  • Not deployed yet — the interesting question is what distribution costs a single-node read.

Apache CouchDB

Document storeNot yet measured
Storage model
Append-only B-tree files, one revision tree per document.
Index model
Precomputed MapReduce views, plus Mango indexes for declarative queries.
Query model
HTTP and JSON throughout; a query is a view lookup rather than a plan.

Built for

  • Multi-master replication, including to offline clients
  • Crash-only durability
  • Reading through precomputed views

How it was deployed

  • Not deployed yet.

FerretDB

Document layer over PostgreSQLNot yet measured
Storage model
Documents stored in PostgreSQL, presented over the MongoDB wire protocol.
Index model
MongoDB-style index commands translated onto PostgreSQL indexes.
Query model
MongoDB query language, translated to SQL — so the application needs no adapter at all.

Built for

  • Keeping MongoDB drivers and queries on an open-source stack
  • Reusing PostgreSQL operations, backups and tooling
  • Migrating away from a document store without rewriting queries

How it was deployed

  • Not deployed yet — the only candidate that would need no data-layer work whatsoever.
The rules

How the next engine gets tested

PostgreSQL, Cassandra, Couchbase, CockroachDB, CouchDB and FerretDB can all go through the same harness. These eight rules stay fixed, because the moment one of them bends for a single engine the whole comparison is worthless.

  1. Same application

    The real data-access path, not a driver script written for the occasion. A migration happens inside an application, and that is where its cost shows up.

  2. Same data

    The production database migrated in full, plus a generated set built from a seeded generator so every engine holds byte-identical rows.

  3. Same scenarios

    The same functional queries, expressed through the application rather than hand-tuned per engine. Each engine answers the question its own way, and the cost of its way is the result.

  4. Verify results first

    Row counts must match across engines before a single timing is reported. A fast query returning the wrong rows is not an optimization, it is an efficient defect.

  5. Equivalent indexes

    Verified from each engine's own catalog, and present while the data is written. Comparing an indexed store against an unindexed one measures your setup script.

  6. Document the environment

    Including the parts that disadvantage one side. This run's WSL deployment disadvantaged ScyllaDB, and that is stated on its card rather than left out.

  7. Separate the modes

    Drop-in results and engine-optimized results never share a table. Mixed together, the winner is whichever engine got the most engineering attention.

  8. Publish the bad runs

    Both invalid runs are on this page, labelled. They taught more about benchmarking than the valid one did, and a method nobody can see fail is a method nobody can check.