Tebako runtimes under the bench: MRI vs TruffleRuby vs JRuby, and what the JITs actually buy you

The ladder runs two kernels, warmed 3× then timed 7× inside each process, in three interleaved rounds per arm: fib34, a recursive method-call workload that favors a just-in-time (JIT) compiler, and alloc2m, two million object allocations that exercise the garbage collector (GC).

Three readings fall out of this table.

The version bump is free. Plain 4.0.6 ≈ plain 3.3.12 (0.97× on the call kernel, 1.05× on allocation — noise-class). Moving the interpreter line forward is not a performance decision either way; it is a support-window decision.

Hot code converges to the same class, whatever the JIT. YJIT on 4.0.6, TruffleRuby native, and JRuby after tier-up all land between 0.14× and 0.25× of plain MRI on fib34 — a 4–7× speedup that is the argument for JITted runtimes on long-lived, CPU-bound processes. YJIT holding its own against the GraalVM family (0.16× vs 0.14×) is the genuinely notable cell: a payload no longer needs a 600-MiB runtime to obtain most of the converged-class win.

ZJIT is real but young. On 4.0.6 it gains ~9% inside the short-horizon protocol — honest, repeatable, and nowhere near YJIT’s class yet. This matches upstream’s own posture exactly: Ruby 4.0 ships ZJIT compiled in but off by default, YJIT remains the production compiler, and ZJIT is the long game. Tebako mirrors that posture: neither JIT is enabled by default in its runtimes; both are one environment variable away (RUBY_YJIT_ENABLE=1 / RUBY_ZJIT_ENABLE=1).

Synthetic kernels favor JITs. A real Metanorma compile — the ISO/OIML r060 document to HTML, with fonts and the Java conversion pipeline included — is where that advantage is priced. The table lists clean cells only (per-cell load guards):

† oracle/truffleruby#4457 — the process produces correct output and then dies in thread teardown, in three of three runs. ‡ the crash is a deterministic illegal instruction on this workload; see the TruffleRuby feedstock.

The headline finding is that on a ~45-second real compile, YJIT makes MRI slower, not faster. It burns roughly ten CPU-seconds JIT-compiling blocks whose amortization the process exits before reaching. Ruby’s own default posture — YJIT enabled for long processes, with no claim that it helps short ones — is vindicated by a real workload rather than by a microbenchmark. The GraalVM variants, meanwhile, pay an 8–10× boot cost before they execute a line of application code; they are engines for persistent services, not for one-shot compiles.

This is the answer the packaging layer was designed around, and the bench supports the design: MRI stays the default runtime; variants are an opt-in selector; JIT enablement is a packaging-time decision. With the interp_env manifest key (merged this week in tebako#560), a payload packager whose workload is long and hot — a relaton sync service, a render farm worker — can ship RUBY_YJIT_ENABLE=1 as the payload’s declared environment, and the user can still override it locally. The packager measures; the user controls; the loader stays ignorant.

Benchmarks that only report wins are marketing. Three caveats travel with every number above:

The point of all five arms is not to crown a winner. It is to prove the platform’s claim with instruments instead of adjectives: any runtime, any payload, measured the same way, chosen at packaging time, and overridable at run time. The runtime is a line, and the packager picks the line.