Dispatcher parallelism and the memory of a Kotlin/Native service

The previous post ended with a Ktor service holding 61 MB under load with fixedBlockPageSize=16 set, and with an unfinished sentence: the memory that is left is not heap, it is per-thread allocator pages, so the remaining lever is the number of threads. This is that measurement, and it goes further than I expected when I started writing it. Capping the dispatcher the CIO engine runs on takes the service from 67 MB to 36 MB at 200 connections. Then it turns out the cap was reaching only half of what spawns threads, and fixing the other half takes it to 16 MB on eight threads — fewer threads and a lower p99 than Go on the same machine on the same afternoon.

Nothing here is filed upstream and nothing here is a patch you can depend on. There is a section below saying exactly why, and the reason gets sharper as the numbers get better. What is usable today is the rule underneath: on Kotlin/Native, resident memory is a function of how many threads a process has run work on, and the thread count is something you can decide.

The stand

The same as the previous post: two dedicated hosts, four cores and 7.6 GiB each, Ubuntu 26.04, subject and generator on separate machines. Kotlin 2.4.10 linuxX64 release binaries, Ktor 3.5.2 with ktor-server-cio, one route returning one JSON object, k6, 30 s per run, memory bounded by a cgroup and read from it. Every arm carries the merged lock fix (KTOR-9891) and fixedBlockPageSize=16, so what follows is on top of everything the previous post recommended.

Resident memory is the thread count

/proc/<pid>/smaps under load, split by mapping:

Two things fall out. The binary is the only constant — file-backed resident memory is 5.89 to 6.08 MB in every dump taken across this work, idle or loaded, ten threads or a hundred and seven, and it is clean and shared. Everything else is anonymous, and between the two loaded points it grows at 0.555 MB per thread.

An idle process tells you nothing about a loaded one. Eight megabytes at rest and thirty-six under two hundred connections is the same binary on the same machine. The per-thread cost is paid not when a thread is created but when it first allocates — that is when the allocator fills its page cache for a size class, and it never gives those pages back. Size a container from a loaded process or you will size it four times too small.

Those threads are not busy, either. Sampling /proc/<pid>/task/*/wchan under full load, 27 to 34 of 36 threads are parked in futex_do_wait at any instant, one to nine runnable. The thread count is a cache of parked workers, which is why it multiplies with a per-thread page cache into an RSS.

Capping the engine's dispatcher

Dispatchers.IO on Kotlin/Native is this, in kotlinx-coroutines-core/native/src/Dispatchers.kt:

internal object DefaultIoScheduler : CoroutineDispatcher() {
    // 2048 is an arbitrary KMP-friendly constant
    private val unlimitedPool = newFixedThreadPoolContext(2048, "Dispatchers.IO")
    private val io = unlimitedPool.limitedParallelism(64) // Default JVM size

    override fun limitedParallelism(parallelism: Int, name: String?): CoroutineDispatcher {
        return unlimitedPool.limitedParallelism(parallelism, name)
    }
    ...
}

limitedParallelism does not shrink the pool; it caps how many of its workers may run this view's tasks at once. Note the override: a limited view is built over unlimitedPool, not over the already-64-limited io. Every view shares one set of workers, so the pool's size is the combined concurrent demand across all of them — which matters later, and cost me a wrong conclusion first.

For dispatchers in your own code this is one line and public API:

private val io = Dispatchers.IO.limitedParallelism(4)

For the CIO engine it is not, because the engine's dispatcher is internal. Measuring it meant forking ktor-server-cio and replacing three lines of posix/src/io/ktor/server/cio/internal/CoroutineUtilsNix.kt:

private val limitedIO: CoroutineDispatcher = Dispatchers.IO.limitedParallelism(4)

internal actual val Dispatchers.IOBridge: CoroutineDispatcher
    get() = limitedIO

The val matters: IOBridge is a property with a getter, so calling limitedParallelism inside it builds a new limiting dispatcher on every access — a leak, and a different program from the one you meant to measure.

What the value is worth

Eight repetitions per arm, 2 000 rps offered, 200 connections, 512 MiB, all forty runs alive at 0.00 % failed. One binary throughout: the parallelism is read from an environment variable, so the arms — including the uncapped control — differ by that variable and by nothing else.

Monotone in all four columns: less parallelism is less of everything. Against a hard-coded 8, a parallelism of 4 is better on every axis measured — memory, threads, p99 and CPU — with permutation p < 0.025 on each.

It does not cost capacity

The obvious objection is that a cap low enough to save memory is free only until the offered rate rises. Three arms up a ladder of rates, three repetitions each, 200 connections; the measure is the shortfall between what was offered and what was delivered. The rungs stop at 4 000 rps because a Go control on the same pair of machines put 8 000 rps through at 261 % CPU — four cores is 400 %, and rungs above 4 000 would have found every arm equally stuck against the CPU rather than against the dispatcher.

The uncapped arm is the only one short on all three of its runs at the top rung — 115 806 to 117 982 against 118 465 to 120 193 for a parallelism of 4 — and its median p99 there is twice the parallelism-4 arm's and three times the parallelism-8 arm's. Three runs per cell is a small sample and a 2 % shortfall is a small effect, so this is a consistent ordering across nine runs rather than a capacity figure. It is enough for the negative: the failure mode the objection describes does not appear anywhere this stand can reach.

Two honest notes. Four is not uniformly better than eight here the way it was at 2 000 rps — at the top rung its median p99 is the worse of the two, and the ordering that holds all the way up is capped against uncapped, not four against eight. And these are medians of three runs against the sweep's eight: the ladder's own 2 000 rps cell gives 31.6 MB and 5.57 ms for a parallelism of 4 where the sweep gives 28.5 MB and 5.53 ms, with the thread count the noisy column in both, 34 against 26.

The cap was reaching half of what spawns threads

The table above has a loose end. Connections are fixed at 200 there; hold them fixed and raise the rate instead, and the thread count moves anyway — 32, 33, 47, 63 for a parallelism of four as the offered rate climbs. A cap that bounds concurrent tasks at four should not permit sixty-three threads.

The answer is not in the data, it is in ktor-network. posix/src/io/ktor/network/sockets/, CIOReader.kt and CIOWriter.kt:

): WriterJob = writer(Dispatchers.IO, userChannel) {
): ReaderJob = reader(Dispatchers.IO, userChannel) {

Every accepted socket gets a read pump and a write pump on a raw, uncapped Dispatchers.IO — one module below anything ktor-server-cio can reach. The engine's limited view and the pumps' default 64-limited view share one pool of workers, as the override above shows, so capping the engine never constrained the pumps at all.

Routing all three through a single newFixedThreadPoolContext — declared in ktor-utils, the one module both depend on — changes the shape rather than the slope. Four repetitions per cell, 200 connections:

Doubling the offered rate moves a fixed pool by at most one thread. It moves the engine-only cap by fourteen and stock by twenty-eight, and stock is also the only arm that fails to deliver the offered rate — 117 058 requests of 120 000 here, 117 546 in the ladder above, at a p99 of 72 ms. Those are two different sweeps of the same cell, which is about as close as this stand reproduces.

Against Go

Same stand, same afternoon, 200 connections at 2 000 rps: Go twelve runs, the two-worker fixed pool four. Ranges rather than medians, because that is what decides whether a difference is there at all:

On threads and on latency a Kotlin/Native Ktor service is past Go here — its worst p99 run is below Go's best. I make no claim about CPU: Go's own spread on this stand covers the Kotlin range entirely, and a difference you cannot see through the noise is not a difference.

What is left is memory, and it is no longer threads — eight is fewer than Go holds. It is the non-per-thread part of the anonymous memory, about 6.7 MB, plus the 6 MB binary. No dispatcher reaches either, and I have not looked at what they are.

Ktor gives you no supported way to do this

Everything above needs a fork, and that is worth a section because it is not an oversight of one missing parameter.

The base ApplicationEngine.Configuration — the one every engine's configuration extends, in common code — already declares three public, documented properties for exactly this: connectionGroupSize, workerGroupSize and callGroupSize. Netty reads all three. Across ktor-server-cio, excluding tests, they appear zero times. Set callGroupSize on a CIO server and nothing happens and nothing says so.

The property that would have worked instead is discarded. CIOApplicationEngine builds its scope as

CoroutineScope(applicationProvider().parentCoroutineContext + engineDispatcher)

and in a CoroutineContext the right-hand operand wins, so a dispatcher supplied through the public parentCoroutineContext is overridden. The handler is not an escape either: handleRequest opens with withContext(userDispatcher), and that is the same engine dispatcher.

This is not a Kotlin/Native matter — that file is in common, so it holds for CIO on the JVM too.

And there is no laziness to configure

The other way to hold a thread count down is to make the dispatcher slower to create threads. That does not exist either. native/src/MultithreadedDispatchers.kt:

override fun dispatch(context: CoroutineContext, block: Runnable) {
    val state = tasksAndWorkersCounter.getAndUpdate { ... it + 2 }
    if (state.hasWorkers()) {
        obtainWorker().resume(block)
    } else {
        workerPool.allocate()
        // no workers are available, we must queue the task
        val result = tasksQueue.trySend(block)
        checkChannelResult(result)
    }
}

A task arriving while every worker is busy creates a thread immediately, up to the cap — no queue-depth threshold, no delay, no waiting to see whether one frees up. And workerRunLoop() is while (true) whose only exit is close(): a worker is never retired for being idle. The class takes (name, workersCount) and workersCount is its only tuning parameter. There is no core-versus-max, no keep-alive, no scheduler policy to tune.

Which makes the missing retirement the more useful half of the finding. Because a worker never goes away, the thread count is a high-water mark, not a steady state: a dispatcher that grew only under genuine pressure would still sit at the level of the day's worst spike for the rest of the day. Laziness would postpone the ratchet; only a cap bounds it.

That is the third mechanism here with that shape. The allocator's per-thread pages are filled on first use and never returned; glibc's arenas release only from their top; a worker never exits. All three make resident memory a function of the worst moment the process has lived through.

Why none of this is filed upstream

Two things are missing, and both are the kind a reviewer asks about first.

Four is this machine's core count. On four cores, "a parallelism of 4" and "a parallelism of availableProcessors" are the same arm, and nothing measured here separates a good constant from a rule. That is the difference between a number every user has to tune and a default that is right everywhere, and settling it needs a machine with a different core count, which I do not have.

The threads are not named. Kotlin/Native names exactly three threads in the process — two GC threads and a GC timer — and leaves every other thread carrying the truncated process name, so /proc cannot tell an IO worker from a Dispatchers.Default worker. A change argued from a thread count ought to be able to name the threads it removes, and this one cannot yet.

And the sharpest reason arrived with the best result. A pool that cannot grow cannot absorb a blocking call. With N fixed workers, N concurrent blocking syscalls do not slow the server down, they stop it. Go is safe from this because its runtime takes the processor away from a thread parked in a syscall and gives it to another; Kotlin/Native has nobody to take it away. This probe blocks nowhere — one route, one JSON object, no disk, no C library — so the stand cannot see that failure at all. Every number in this post comes from the half of the space where the change is free.

That is a limit of the measurement rather than a caveat about its precision, and on its own it disqualifies any of this as a default. A service that reads a file, calls into C, or resolves a name on a request path is the ordinary case, and it is the case this work excludes by construction.

What to do today

• For dispatchers you own, cap them — and prefer newFixedThreadPoolContext(n) to Dispatchers.IO.limitedParallelism(n) when you want a bound on threads rather than on concurrency, remembering that a fixed pool is a deadlock waiting for a blocking call.
• Do not expect the engine knobs to do anything on CIO. Three of them exist and are inert.
• Size containers from a loaded process. The resting footprint of a Kotlin/Native service is not a smaller version of its working footprint; here it was a fifth of it.
• Bound the connections instead, if you are on a stock Ktor. That was the previous post's recommendation and it is still the one thing that works without a fork: threads follow concurrency, and a reverse proxy with a bounded keep-alive pool turns a variable you do not control into one you do.

Limitations

linuxX64 is the only target measured, and four cores the only machine — which is also why the strongest result here is not a proposal.

One route returning one small JSON object is the only workload. A handler that blocks is precisely the case Dispatchers.IO exists for, and a cap of four would be the wrong setting for it; everything above is about a service whose work is parsing and serialising.

Every rate is one the generator held at the stated value, so no arm was run to its actual ceiling: the ladder shows which arm falls behind first, not what any of them can do.

The source quotations are from kotlinx.coroutines 1.11.0 and Ktor 3.5.2, the versions the probe resolves. The claim that CIO ignores the three engine knobs follows from those names appearing nowhere in the module, which is strong but is not the same as running a server with callGroupSize set and counting its threads. And every peak is over a 30-second run at a steady rate — nothing here says what a day-long process does, and a page cache filled on first use and never returned is exactly the kind of thing that could behave differently over hours.