Benchmarks

Suite Architecture#

The benchmark suite (helix-benchmarks, ~500 lines) provides a framework for measuring kernel performance across all subsystems.

Dependencies#

[dependencies]
helix-hal = { path = "../hal" }
helix-core = { path = "../core" }
helix-execution = { path = "../subsystems/execution" }
helix-memory = { path = "../subsystems/memory" }
helix-dis = { path = "../subsystems/dis" }
helix-modules = { path = "../modules" }

Benchmark Runner#

Feature Flags#

[features]
verbose = []        # Print per-iteration timing
extended = []       # Run additional stress benchmarks
stress = []         # High-iteration stress tests

---

Configuration#

Benchmark Parameters#

Running Benchmarks#

# Via kernel shell
> bench

# Via make (runs in QEMU)
make bench

# Specific benchmark
> bench context_switch
> bench memory_alloc
> bench syscall_latency

---

Scheduler Benchmarks#

Context Switch Latency#

Measures the time to switch between two threads:

Thread Creation#

Measures time to create a new thread (allocate stack, initialize context, add to scheduler):

Scheduler Throughput#

Measures scheduling decisions per second with varying thread counts:

DIS Intent Scheduling#

Measures overhead of intent-based scheduling vs. simple priority scheduling:

---

Memory Benchmarks#

Page Allocation#

Slab Allocation#

Virtual Memory#

Syscall Benchmarks#

---

IPC Benchmarks#

Throughput#

---

Statistical Analysis#

The benchmark framework uses rigorous statistical methods:

Outlier Detection#

The Median Absolute Deviation (MAD) method identifies outliers:

• Compute the median of all measurements
• Compute the MAD = median(|xi - median|)
• Flag values where |xi - median| > 3 * MAD as outliers
• Report with and without outliers

Confidence Intervals#

For each metric, the framework reports:

• Mean with 95% confidence interval
• Median (robust to outliers)
• Standard deviation
• Min / Max (absolute bounds)
• P50, P90, P95, P99 (percentile distribution)

Comparison#

When comparing two benchmark runs:

Benchmark: context_switch
  Before: mean=890ns, p99=4.2us
  After:  mean=850ns, p99=3.8us
  Change: -4.5% mean, -9.5% p99
  Significant: yes (p < 0.05, Mann-Whitney U test)

---

Running Benchmarks#

From the Kernel Shell#

helix> bench
=== Helix Benchmark Suite ===

[1/8] context_switch .............. 890ns mean (1000 iters)
[2/8] thread_create ............... 8.2us mean (1000 iters)
[3/8] page_alloc_bitmap ........... 180ns mean (10000 iters)
[4/8] page_alloc_buddy ............ 95ns  mean (10000 iters)
[5/8] slab_alloc_64 ............... 28ns  mean (10000 iters)
[6/8] syscall_getpid .............. 85ns  mean (10000 iters)
[7/8] ipc_channel_roundtrip ....... 210ns mean (10000 iters)
[8/8] dis_intent_classify ......... 75ns  mean (10000 iters)

All benchmarks passed target thresholds.

From Host#

# Run all benchmarks in QEMU
make bench

# Run with extended iterations
make bench FEATURES=extended

# Run stress tests
make bench FEATURES=stress

Continuous Integration#

Benchmarks run on every PR to detect performance regressions:

1. Build the kernel with profile.bench
2. Boot in QEMU with benchmark arguments
3. Parse serial output for results
4. Compare against baseline (stored in repo)
5. Fail the build if any metric regresses > 10%