Kotlin/Native under load, and what I measured wrong

Two services of mine run on Kotlin/Native, and the numbers that got them there are real:

A hundredfold faster start in a seventh of the memory. The question was whether that holds once traffic arrives. It does not, and the reason is not the one everybody names.

What load does to it

Same code, single core:

The last row is the one that decides it. A number three times worse is a number you can plan around. A hundredfold spread between identical runs is not.

It is not the garbage collector

That was my first assumption and it was wrong. Garbage collection took 1.4 % of CPU.

The profile:

Memory does not grow because collecting is expensive. It grows because allocating is heavy and a full-heap collection is too expensive to run often enough to keep up.

The single largest entry was SelectorHelper.selectionLoop at 17.1 % — Ktor's network layer on Linux poll.

The clincher was comparing two ingest paths in the same binary at comparable rates: UDP sat at 35 MiB, HTTP at 350 MiB. Tenfold, same GC, same allocator. A minimal Ktor server with one trivial route allocates roughly 13 KB of garbage per request.

Three things I measured wrong

This is the part I would want to read.

The load generator shared the machine with the service. It was competing for the same cores. Moving it to another host raised the ceiling by 400 %. Every number taken before that was a measurement of the generator.

The generator deduplicated on the wrong key. It used the window timestamp instead of the sequence number, so 88 % of frames were discarded as duplicates. What I read as throughput was timestamp diversity in my own test harness.

I tuned an allocator that no longer exists. I set environment variables for an allocator removed from Kotlin/Native in 2.2, saw the numbers move, and wrote it up as the effect of tuning. Run-to-run spread on that setup was 5–11×. The variables did nothing; I was reading noise and giving it a cause.

Generational collection, tested rather than assumed

If allocation is the problem, would collecting more often help? I forced it, at intervals, to find out.

Collecting every 50 ms pushed CPU from 55 % to 86 % and p99 from 0.75 ms to 5.75 ms. So no — not like that.

But it clarified what generational collection actually buys:

Its value is not that collection is cheap. It is that frequent collection becomes cheap, so the high-water mark never grows.

The JVM held 2000 rps in 190 MiB total with a 64 MiB heap. That is the mechanism doing the work, and Kotlin/Native does not have it.

Rules I now follow

• Put the load generator on a different machine.
• Warm up with the profile you are about to measure, not a different one.
• Six repetitions minimum, with the order of variants randomised.
• Report the median and the spread. Any effect smaller than the spread is not an effect.
• Read anonymous memory from memory.stat, not memory.current.
• Checksum the binary to prove a build flag changed anything at all.

Where it still wins

Not sustained HTTP under a memory limit. But the 20 MiB idle footprint and the hundredfold faster start are real, and they are exactly what short-lived processes want: CLI tools, agents, cron jobs, webhook receivers, scale-to-zero containers.

Both services went back to the JVM, keeping the shared code through Kotlin Multiplatform — which is the arrangement KMP is for.

The next thing worth fixing is not a garbage collector. It is poll in Ktor's network layer, which on Linux should be epoll. That is a library-level change, and a much smaller one than it sounds.