Comparative benchmark snapshot — 2026-08-27#

Same-machine results from a 16 GB Apple M1 Pro MacBook Pro (10 cores), macOS 15.5. Vev revision: b0f2700d (Optimize small resident durable transactions). The benchmark processes used GraalVM JDK 25.0.1.

Raw measurements and process output are retained in:

• 2026-08-27-comparative-queries.json
• 2026-08-27-comparative-transactions.json
• 2026-08-27-musicbrainz.json

Summary#

• The incremental resident path removes the full-DB-size cost from small replacements and retractions. On the 2,000-entity fixture, replacement falls from 14.50 to 1.94 ms, explicit retract from 15.23 to 1.93 ms, and the Ro-like transaction from 15.51 to 2.81 ms.
• On the 50-entity fixture, the Ro-like durable transaction has a 0.82 ms median and 2.46 ms p95. On 2,000 entities it has a 2.81 ms median and 7.05 ms p95. The requested 3–5 ms median target is met without changing the transaction shape or omitting provenance.
• Vev remains faster than DataScript and Datomic on all seven synthetic queries. Datalevin wins the six join/predicate queries; Vev wins q1.
• Vev wins all nine durable MusicBrainz queries against Datomic, by 1.52–4.51×, with matching row counts and fingerprints.
• In small durable transactions, Vev's median beats Datalevin strict WAL and Datomic dev on append, replacement, explicit retract, and Ro-like. On the larger fixture Datalevin is faster, while Vev is near Datomic except for retractEntity.

Deterministic query comparison#

The fixture contains 20,000 entities and 100,000 user datoms. Attribute values use seed 42 and insertion order uses seed 43 for every engine. Each query has a 250 ms warmup and five independent 500 ms measurement windows. The table shows median/p95 milliseconds per completed query.

Current releases are DataScript 1.7.8, Datalevin 0.10.7, and Datomic Peer 1.0.7705.

Vev is 1.51–5.33× faster than DataScript and 3.19–19.29× faster than Datomic on these shapes. Datalevin is 1.31–5.50× faster than Vev on the six wider queries; Vev is 1.46× faster on q1.

The historical dependency track reran DataScript 1.7.4, Datalevin 0.10.5, and Datomic 1.0.7277 with the same deterministic fixture. Its full table is in the raw JSON. Vev's two independent tracks differ by only 1.2% in geometric-mean latency, which is a useful stability check. Relative to the July snapshot, current Vev medians are within 5% on five shapes and about 10% faster on q3 and q4. Historical July numbers should not be compared directly to the new peer rows because the old harness did not seed insertion order and exposed only its median.

Small resident transactions#

Every timed call is synchronous through report return. Vev uses its resident SQLite connection and Datalevin uses strict synchronous WAL; both are durable. Datomic durable rows use Peer 1.0.7705 with the locally available dev transactor 1.0.7277. Each workload has five warmups and 40 measurements. The table shows median/p95 milliseconds.

Durable, 50 resident entities#

Durable, 2,000 resident entities#

DataScript and Datomic in-memory references are included in the JSON but not mixed into the durable ranking. For the Ro-like shape their small/large medians are 0.051/0.029 ms for DataScript and 0.165/0.076 ms for Datomic.

Vev and Datomic reports include the automatic transaction-instant datom, while DataScript and Datalevin reports do not. Consequently the effective report sizes for append/replacement/retract/retractEntity/Ro-like are 2/3/2/3/6 in Vev, 1/2/1/3/5 in DataScript and Datalevin, and 2/3/2/4/6 in Datomic. No provenance was removed or special-cased.

Incremental path and phase profile#

Forced full rebuild and incremental mode start every workload from the same database size. The median speedup is modest on the 50-entity fixture and large once rebuilding all resident datoms matters:

The phase profile confirms the original root cause. On the large fixture, full construction of db-after consumes 14.10 ms for replacement, 14.98 ms for explicit retract, and 15.16 ms for Ro-like. Incremental construction reduces those phases to 0.60, 0.80, and 1.12 ms respectively.

For large incremental replacement, the remaining profiled time is 32% SQLite commit, 27% db-after, 21% planning, and 13% derived bookkeeping. For Ro-like it is 35% db-after, 28% commit, 20% planning, and 9% derived bookkeeping. The p95 spread is mostly in durable I/O and is materially wider than the median, especially for Vev.

Large retractEntity is now the clear outlier: 15.66 ms median, with resolution accounting for 85.7% of profiled time. Its entity-expansion scan is the next general optimization target; it is no longer a db-after rebuild problem.

Durable MusicBrainz queries#

This uses the Datomic 1968–1973 sample, five warmups, and seven measurements. Vev prepares each query once. Datomic Peer and transactor are both 1.0.7277 for this established fixture. All row counts and result fingerprints match.

Architecture and consequences#

The result comes from the general small non-schema incremental path. It applies the transaction's effective datoms to immutable current/index snapshots and retains the canonical log for history/as-of. Schema-affecting and ineligible transactions continue through full construction. Durable publication still appends the canonical SQLite rows, updates derived root/novelty bookkeeping, and commits before returning the report.

This benchmark refresh changes no public API, transaction semantics, storage format, durability setting, or report ownership. Its code changes are confined to deterministic fixtures, comparison adapters, orchestration, and result artifacts. The current performance commit itself also requires no storage migration; reopen and separate-connection visibility continue to use the canonical log and committed generation.

Reproduction#

scripts/build_native_library.sh

DATALEVIN_BENCH=/path/to/datalevin/benchmarks/datascript-bench \
  bench/run_comparative_benchmarks.py --track both --runs 1

bench/run_transaction_comparison.py \
  --datomic-uri-prefix datomic:dev://localhost:4334/vev-resident-

DATOMIC_HOME=/path/to/datomic-pro-1.0.7277 \
  scripts/compare_musicbrainz_workshop.sh \
    --prepared-vev --warmup-runs 5 --measure-runs 7 --skip-kvist