Vectorized hash join lands in Rapid
Batched build and probe phases with SIMD-accelerated predicate evaluation, plus fixes to aggregate overflow handling.
Open source · GPL v2
ShannonBase keeps a columnar mirror of your InnoDB tables in memory and routes each query to whichever engine wins — no ETL, no second system, no application change. And it does AI where your data lives: a native VECTOR type with in-kernel embeddings, train-and-predict from plain SQL, local LLM generation with RAG, and a native agent runtime and system agent that complete user tasks from natural language.
Wire-compatible with MySQL 8.4. Your driver, your ORM, your dump files.
ShannonBase v0.1 · MySQL 8.4 wire · Linux x86_64 · 8 vCPU / 32 GB · TPC-H SF 0.2 · static preview · Testing Database pulled via docker pull shannondata/shannonbase:0.1-tpch
What the mirror buys you
Load a table into the in-memory column store and the optimizer starts routing scans there on its own. Changes propagate from InnoDB continuously.
ALTER TABLE lineitem SECONDARY_LOAD A native vector type with an ART index, in the same transaction as the row it describes. Embeddings generated in-process.
ORDER BY vector_distance(v, ?) Train on a table and predict from SQL. LightGBM and ONNX Runtime live in the server, so the data never moves.
CALL sys.ml_train('churn', ...) A JavaScript runtime with an in-process SQL bridge. Every write goes through human approval before it commits.
CALL sys.shannon_chat(?, @out) Architecture
Rapid is a secondary engine, not a downstream replica. Committed changes reach the column store over two channels — the redo log for durability and a DML notification path for latency — so analytical reads see transactional writes without a sync window you have to reason about.
MVCC version linking runs in the column store too, which is what lets a long analytical scan and a short transaction coexist on the same table.
TPC-H SF 0.2 · developer build (debug) · full 22-query run · 2026-08-31
| Query | InnoDB | Rapid | Speedup |
|---|---|---|---|
| Q11 — important stock identification | 56.6 s | 0.9 s | 60.9× |
| Q7 — volume shipping | 106.1 s | 2.4 s | 45.0× |
| Q20 — potential part promotion | > 150 s | 6.6 s | > 22× |
| Q17 — small-quantity-order revenue | > 150 s | 7.7 s | > 19× |
| Q5 — local supplier volume | 22.2 s | 1.9 s | 11.6× |
| Q8 — national market share | 56.4 s | 5.8 s | 9.8× |
| Q3 — shipping priority | 33.4 s | 4.9 s | 6.9× |
| Q14 — promotion effect | 15.2 s | 2.2 s | 6.9× |
| Q21 — suppliers who kept orders waiting | 107.6 s | 18.3 s | 5.9× |
Harness, dataset generation, and raw output are in heatwave-tpch. Run it against your own hardware before you believe us.
Live demos
Engine routing, vectors, in-database ML, RAG, and the agent — running live. Everything below is also on the ShannonBase Data AI channel.
News
Batched build and probe phases with SIMD-accelerated predicate evaluation, plus fixes to aggregate overflow handling.
Plan rows commit independently of the approval wait, so a pending decision no longer holds a transaction open.
How version linking works in IMCS, and the correctness bugs we found building it.
Get started