# Leyden AOT cache for a plain Kotlin/JVM service: what it gives, what it costs, and the three things Spring does for you that you'll do yourself

> How to give a Ktor, http4k or any application-plugin service a Project Leyden AOT cache on JDK 25: what the cache is and is not, what it did to a real service's cold start and memory, the training, verification and packaging that Spring Boot and Quarkus hide, the same by hand in five steps, and through a Gradle plugin.

Page: https://kotlin.website/blog/leyden-aot-cache-for-a-plain-kotlin-jvm-service · Published: 2026-09-09

This is the how-to I could not find when I needed it: a Ktor service on the plain `application`
plugin, JDK 25, and the question of whether Project Leyden's AOT cache is worth an afternoon.
Spring Boot and Quarkus answer it with a flag. Everything else gets two JVM options and a list
of ways the result stops working without telling anyone. The numbers below are from a reference
service and from a stand, each with its methodology next to it; the commands are the ones the
plugin at the end runs for you, written out so you know what it hides.

## What the cache does, and what it does not

A training run of your application under `-XX:AOTCacheOutput=app.aot` (JEP 514, JDK 25) records
every class it loaded and linked, a set of heap objects — the module graph, class mirrors, the lambda
proxies it resolved — and the method profiles the JIT gathered (JEP 515). At the next start under
`-XX:AOTCache=app.aot` the JVM maps that file and skips the loading, parsing, verification and
linking of those classes, and starts with the archived objects in place. That is the whole
effect: the part of start-up that is class loading disappears, and the first request pays less
because its classes are already linked.

What the cache does not hold on JDK 25 and 26 is compiled code. AOT code compilation is still a
draft JEP, so C1 and C2 compile the same methods after the start either way — on zavarnik's Ktor
sample C2 compiled 1 171 methods cold and 1 172 with the cache, ten runs each. The profiles help the JIT choose what to
compile first, not skip it. A service is not warm from the first request, and throughput after
warm-up does not change. And the cache is tied to what it was trained on: the same jars, byte for
byte and timestamp for timestamp, in the same order, on the same JDK build, with the same modules
and agents. Change any of that and the JVM starts without the cache, prints three lines to stderr
and goes on, exit code 0.

## What it gave a Ktor service

[konekt](https://github.com/youndie/konekt) is my reference project: an eSIM operator's account
backend, Ktor CIO with Exposed and Postgres, 128 jars on the classpath, `eclipse-temurin:25-jre`,
deployed with a limit of one core and 1 GiB. Two images that differ by one layer — the cache,
65 MiB, trained inside a container of the first image against the stand's database and verified
under `-XX:AOTMode=on`, 5 329 of 5 332 application classes from the cache. Ten restarts of each,
in alternation, on the build box under the chart's limits; `docker start` to the first `200` on
`/health` measured from outside the container.

| | without the cache | with the cache |
|---|---|---|
| `docker start` → `/health`, median of 10 | 4 380 ms | **2 042 ms** |
| the same, range | 4 186–5 877 ms | 1 915–2 227 ms, one restart at 4 128 |
| first signed-in screen, median | 510 ms | **240 ms** |
| p50 / p95 of the next hundred requests | 9.3 / 89 ms | 8.8 / 97 ms |
| the cache as an image layer | — | 65 MiB |

Readiness halves, the first request halves, the hundred after it do not move. In the cluster the
pod is Ready 3 s after its container starts, with the cache and a startup probe that asks every
second; the 11 s it replaced included a readiness probe that waited five seconds before its
first ask, so only part of that gap is the cache, and the gain reached a rollout at all only
because the probe was retuned with it. The image totals `docker image ls` printed for the two
images, 602 and 685 MB, differ by more than the layer `docker history` attributes to the cache;
the layer is what the cache adds. The methodology and the CSVs are in the repository under
`docs/research/measurements-2026-09-07/aot/`.

The share of classes matters more than the milliseconds, because it is what the milliseconds
follow from. On the Ktor stand in [zavarnik](https://github.com/youndie/zavarnik) — Ktor 3.5.2 on
CIO, kotlinx.serialization, three routes, 26 jars — 2 322 of 2 322 application, Ktor, coroutines
and stdlib classes came from the cache (`-Xlog:class+load`, `source: shared objects file`), and
time to the first `200` on `/health` fell by 59–69 % over four series of twenty runs on two
collectors. A service whose start is not class loading gains less: a broker of mine on the
same JDK build went from 1 377 to 1 225 ms at the median of seven runs, eleven per cent, because
most of its start is opening its log.

Memory is the question I get asked next, so I measured it: RSS of the sample from
`/proc/<pid>/status`, five starts per variant in alternation, read at readiness and after 200
requests.

| | without the cache | with the cache |
|---|---|---|
| RSS at readiness, median of 5 | 85.0 MiB | 84.7 MiB |
| RSS after 200 requests | 122.9 MiB | 119.6 MiB |

The sample's 31 MiB cache is not visible in resident memory; the difference is smaller than the
spread of either row. What it costs is disk — the file and its layer in every image — and the
training run: after the training JVM exits a second one assembles the cache, and JEP 514 says
that sub-invocation "uses its own Java heap with the same size as the heap used for the training
run" — a 4 GB training heap needs 8 GB.

## Why Spring and Quarkus have it and you do not

Spring Boot's recipe is one property: `-Dspring.context.exit=onRefresh` makes the application
exit as soon as the context is up, so the training run ends by itself. Its documentation lists
the conditions in one paragraph: the classpath "must be the same as the one used to create it,
in the same order", extra jars only "at the end (but will not be cached)", "the timestamps of
the JARs must be preserved", and "avoid the usage of directories and `*` wildcard characters".
Quarkus runs its `@QuarkusIntegrationTest` tests as the training under
`./mvnw verify -Dquarkus.package.jar.aot.enabled=true -DskipITs=false` and packs the result. Both
frameworks own three things a plain `application` distribution does not:

1. **Training.** Something has to start the application, drive it through the hot path — the
   cache holds what the run loaded, so an idle start yields a cache that covers half the classes —
   and stop it in a way that writes the cache. `SIGTERM` and `System.exit` write it; `SIGKILL`
   does not. Then wait: the process runs a second JVM to assemble the file, and the
   application process being gone does not mean the cache is there.
2. **Verification.** A rejected cache is silent by default. The only way to know that production
   will use the cache is to start with it made mandatory (`-XX:AOTMode=on`) and count classes.
   And on JDK 25.0.0–25.0.3 and 26.0.0–26.0.1 even that is not enough: those builds never
   compare the cache with the jars, so a jar rebuilt after training runs its old classes with
   exit code 0 — [the previous post](https://kotlin.website/blog/stale-aot-cache-on-jdk25) has the logs. Something
   other than the JVM has to compare the jars.
3. **Packaging.** The JVM checks each jar's size and modification time, the classpath string, and
   the JDK down to the size of `lib/modules`. A zip loses the timestamps. A `-jre` image refuses
   a cache trained on the `-jdk` image of the same build. A `lib/*` wildcard expands in the
   filesystem's own order, which is not the same order on every container runtime.

## The same by hand, in five steps

For a service on the `application` plugin with a `/health` route on 8080. Every step is what the
plugin does; nothing here needs it.

**1. Install, and pin the jar timestamps.** The JVM compares mtimes in whole seconds against the
cache, and `distTar` and Docker `COPY` preserve them; set them to a constant now so the cache
matches wherever the tar is unpacked. The constant is Gradle's own for reproducible tars —
`TarCopyAction.CONSTANT_TIME_FOR_TAR_ENTRIES`, 2 January 1970: one day rather than zero, because
some tools read a zero mtime as no timestamp at all.

```bash
./gradlew installDist && cd build/install/app
find lib -name '*.jar' -exec touch -d @86400 {} +
```

**2. Train.** Start with `-XX:AOTCacheOutput`, wait for readiness, hit the hot path, stop with
`SIGTERM`, and wait for the process — after the application exits it runs a second JVM that
assembles the cache and prints `AOTCache creation is complete` when it is done.
`-XX:-AOTAdapterCaching` keeps machine code for the training CPU's instruction set out of the
file; it is a diagnostic flag, which is what `-XX:+UnlockDiagnosticVMOptions` is for, and a later
JDK may rename or drop it — the mainline loader already checks CPU features itself. A cache
trained on a CI runner with AVX-512 has been reported to `SIGILL` on a narrower machine, and a
cache trained without that code is accepted by a JVM started without the flags.

```bash
JAVA_OPTS="-XX:AOTCacheOutput=$PWD/lib/app.aot -XX:+UnlockDiagnosticVMOptions -XX:-AOTAdapterCaching" bin/app &
until curl -sf localhost:8080/health >/dev/null; do sleep 0.1; done
for i in $(seq 20); do curl -s localhost:8080/api/items >/dev/null; done
kill -TERM $! && wait $!
```

**3. Write down what the cache was trained against.** On a fixed JDK the JVM will refuse a
changed jar; on the six builds that skip the check this file is the only thing that will.

```bash
sha256sum lib/*.jar > lib/app.aot.jars
```

**4. Verify.** Start with the cache mandatory and class loading logged, then count. The
application classes must all come from `shared objects file`; a `source: file:` line for one of
them names what the training did not reach.

```bash
sha256sum -c --quiet lib/app.aot.jars
JAVA_OPTS="-XX:AOTCache=$PWD/lib/app.aot -XX:AOTMode=on -Xlog:class+load=info" bin/app > verify.log &
until curl -sf localhost:8080/health >/dev/null; do sleep 0.1; done; kill -TERM $!
grep -c 'source: shared objects file' verify.log; grep 'source: file:' verify.log | head
```

**5. Ship.** The start script has to add `-XX:AOTCache` only when the file is there, because the
JVM refuses that flag next to `-XX:AOTCacheOutput` and the same script is the training launcher.
Three lines before the `exec` in `bin/app`:

```bash
if [ -f "$APP_HOME/lib/app.aot" ]; then
    DEFAULT_JVM_OPTS="$DEFAULT_JVM_OPTS \"-XX:AOTCache=$APP_HOME/lib/app.aot\""
fi
```

Then `distTar`, not `distZip`, and in a Dockerfile steps 2–4 run in the runtime stage, on the
`-jre` image that will start the service — not in the `-jdk` build stage, whose cache the
runtime JVM will refuse. Steps 1–4 again after every change to any jar, on every build.

## The same through the plugin

[zavarnik](https://github.com/youndie/zavarnik) is those five steps as Gradle tasks, with the
verification made a red build. It is on the Plugin Portal, so this is the whole installation:

```kotlin
plugins {
    application
    id("io.github.youndie.zavarnik") version "0.1.0"
}

zavarnik {
    training {
        readyWhen.url("http://127.0.0.1:8080/health")
        workload {
            post("http://127.0.0.1:8080/auth/login", "application/json", """{"user":"demo"}""") {
                capture("token", "accessToken")
            }
            get("http://127.0.0.1:8080/api/home") { header("Authorization", "Bearer {{token}}") }
        }
    }
}
```

`./gradlew check` runs `aotTrain` — the installed distribution through its own start script,
timestamps pinned, `SIGTERM`, the manifest — and `aotVerify`, which fails the build with the name
of the thing that failed: a jar that changed since training, a JVM that refused the cache (with
the JVM's own reason), or a class share below 90 %. `distTar` carries the cache; `installDist`
refuses a wildcard on the classpath. For the image, every distribution's `lib/` carries
`zavarnik-runner.jar`, the same training and verification on a bare JRE with no Gradle and no
curl, so the runtime stage of the Dockerfile trains the cache on the JVM that will use it:

```dockerfile
FROM eclipse-temurin:25.0.4_7-jre
COPY --from=build /src/build/install/app /opt/app
RUN java -cp /opt/app/lib/zavarnik-runner.jar io.github.youndie.zavarnik.runner.Main train /opt/app \
 && java -cp /opt/app/lib/zavarnik-runner.jar io.github.youndie.zavarnik.runner.Main verify /opt/app
CMD ["/opt/app/bin/app"]
```

A service that cannot start on the build machine sets `training { onAssemble = false }` and trains
where it can — konekt does it in its release pipeline, inside a container of the release image on
the stand's network, and adds the cache as a layer. On `ktor { docker { } }` or Jib directly,
`jib { containerizingMode = "packaged" }` plus `jibAotTrain` and `jibAotVerify` do the same
through a container of the Jib image; that path needs a Docker daemon on the build machine.
`aotReport` prints readiness with and without the cache and, as a second table, what the JIT
still compiles afterwards — so nobody reads the first table as a warm service.

## What breaks

- **A stale cache accepted on JDK 25.0.0–25.0.3 and 26.0.0–26.0.1.** The JVM never looks at the
  jars, `-XX:AOTMode=on` exits 0, and the old classes run. Move to 25.0.4 or 26.0.2; until then
  the manifest from step 3 is the check ([the post](https://kotlin.website/blog/stale-aot-cache-on-jdk25)).
- **`-jdk` at training, `-jre` in production.** The same Temurin build, two sizes of
  `lib/modules`, cache refused: `This file is not the one used while building the AOT cache:
  '/opt/java/openjdk/lib/modules', size has changed`. Train in the image that runs.
- **A wildcard on the classpath.** `lib/*` expands in `readdir` order; Docker's overlay2 on the
  CI runner and containerd on a k0s node answered differently, and the first cache to reach a
  cluster was refused there with `The name of app classpath [1] does not match`. No check on the
  build machine can see this, because it verifies where it trained. List the jars.
- **Anything that changes the JVM's view of the classpath or modules on one side only:** an entry
  added at the front (at the end is fine), `--add-modules`, a `-javaagent` (an agent on both sides
  is fine), ZGC on one side and another collector on the other, on JDK 25 and 26 alike — the
  boundary is compressed oops, not the collector; ZGC on both sides works.
- **A killed training run.** `SIGKILL`, or a container stopped without a grace period, writes no
  cache and no error.

What the plugin's first version does not do: train on Windows (no `SIGTERM` there; production on
Windows works), ship a cache in `distZip`, make a missing cache fatal in production (the start
script runs without it, by design), or reproduce the cross-CPU `SIGILL` on its own hardware — the
adapter code is left out by default on the strength of two published reports, and a positive
control on a pair of machines is still open.

---

Source: [github.com/youndie/zavarnik](https://github.com/youndie/zavarnik) —
`experiments/ktor-readiness/`, `experiments/jit-warmup/` and `experiments/memory/` for the stand
figures with their logs, `docs/research/research-architecture.md` §1 for what the JVM checks,
with a path for every row; [github.com/youndie/konekt](https://github.com/youndie/konekt),
`docs/research/measurements-2026-09-07/aot/`, for the service. The bug this post keeps pointing
at has its own write-up:
[OpenJDK 25.0.0–25.0.3 uses a stale AOT cache without saying so](https://kotlin.website/blog/stale-aot-cache-on-jdk25).
The other way to start faster — a snapshot of the warmed-up process, JIT included — is
[CRaC on a Ktor service with HikariCP and Exposed](https://kotlin.website/blog/crac-on-a-ktor-service-with-hikari-and-exposed).