Selective Offload in ShannonBase is an intelligent query-routing mechanism that dynamically determines whether a query should execute on the primary MySQL/InnoDB engine or be offloaded to the Rapid secondary engine.

The design is directly inspired by MySQL HeatWave’s WL#15829: Selective Offload project, and extends the traditional cost-threshold-based secondary-engine selection into a precision-driven, adaptive decision system.

The core objective is simple: execute each query on the engine where it is expected to complete fastest, while avoiding unnecessary optimization overhead and maintaining predictable transactional behavior.

Execution Modes

Selective Offload operates in two distinct modes, controlled by the system variable rapid_use_dynamic_offload.

• Dynamic Offload Disabled (rapid_use_dynamic_offload = 0)
  The system falls back to the traditional MySQL cost-threshold classifier. If the secondary engine is forced, all eligible queries are routed to Rapid regardless of cost.
• Dynamic Offload Enabled (rapid_use_dynamic_offload = 1)
  ShannonBase evaluates both engines and selects the optimal execution target on a per-query basis. Queries may be offloaded to Rapid or redirected back to MySQL depending on real-time conditions.

Decision Scenarios

When dynamic offload is enabled, the decision logic operates under two complementary scenarios:

1. Static Scenario

When there is no Change Propagation (CP) lag and no queue buildup on the Rapid side, Selective Offload relies on a trained decision tree classifier. This classifier achieves more than 85% precision in predicting whether a query will execute faster on MySQL or Rapid.

• Classifier latency ranges from 20–100 μs for fast queries
• Worst-case overhead observed is ~700 μs for very large queries
• Queries with execution cost < 10 and point-select shape are excluded
• 100% of such “very fast queries” are empirically faster on MySQL

Excluding very fast queries avoids unnecessary three-phase optimization overhead that would otherwise cause performance regressions.

2. Dynamic Scenario

When Rapid experiences CP lag or execution queue buildup, Selective Offload activates dynamic feature normalization. This mechanism accounts for additional catch-up time required by Rapid and continuously re-evaluates whether it remains the optimal engine.

• Long CP lag or queue depth progressively shifts queries back to MySQL
• Lighter queries are redirected first
• Heavier analytical queries are retained on Rapid as long as beneficial

Query Lifecycle Integration

With secondary_engine = ON, Selective Offload integrates deeply into the MySQL query optimization lifecycle:

• Primary Tentative Optimization
  MySQL generates an initial InnoDB execution plan.
• secondary_engine_pre_prepare_hook
  Rapid caches MySQL plan features in the statement context if dynamic offload is enabled and the query is not “very fast”.
• Secondary Optimization
  MySQL produces a Rapid-specific plan.
• prepare_secondary_engine
  Dynamic offload combines MySQL plan features with Rapid statistics (table cardinality, dictionary encoding, projection size, queue depth).
• optimize_secondary_engine
  Final decision incorporates CP lag and runtime conditions before choosing the execution engine.

At multiple checkpoints, the query may be redirected back to MySQL or committed to Rapid execution.

Observability and Diagnostics

When dynamic offload is active, ShannonBase emits detailed MySQL error-log entries that explain routing decisions:

• High-level allow/deny decision logs with condition breakdowns
• Serialized decision-tree paths identifying exact classifier branches
• Explicit outcomes: 0 → MySQL, 1 → Rapid

These logs make Selective Offload fully transparent, debuggable, and production-safe.

Summary

Selective Offload elevates ShannonBase from a static secondary-engine architecture to a workload-aware, adaptive execution system.

By combining cost models, learned classifiers, and real-time system feedback, it ensures that each query is executed on the most appropriate engine at that moment.

• Maximizes end-to-end query performance
• Avoids regressions for latency-sensitive queries
• Gracefully adapts to Rapid load and replication lag

In essence, ShannonBase Selective Offload brings HeatWave-grade, production-proven intelligence into a fully open, extensible MySQL-compatible execution framework.