youndie@kotlin.website:~/src$ cat mostik/README.md

mostik

kmp: jvm + native · ktor · kafka · kore
An HTTP to Kafka bridge whose status code is a true statement about the record: 200 means the broker acknowledged it, 429 means it was never queued, 504 means nobody knows. Kotlin/Native and JVM from one source, the JVM build the oracle for the native one.
github/mostikall libraries

Why it exists

The usual HTTP-to-Kafka bridge answers 200 the moment the record is queued in its local client. After that it either waits for the broker without a bound, or cuts the wait and answers 503 for a record that lands anyway — and the caller, reading a failure, sends it again. mostik waits for the broker's acknowledgement under a deadline, and when it cannot know what happened it says so: 504, with "outcome": "unknown" and "retrySafe": false in the body. The set of answers is closed; no route here answers 500.

mostik is Russian for "a small bridge".

What you get

One route, POST /topics/{topic}/records. The body is the record's value, byte for byte, never parsed; Record-Key is the key; each Record-Header-<name> becomes a header named <name>.

Status What it says about the record Retry?
200 the broker acknowledged it; the body names partition and offset —
429 it was provably never queued; Retry-After says when safe
502 the producer refused it before queueing safe, though waiting will not help
503 the service is draining and did not take it safe
504 queued, then the deadline passed or delivery failed: it may or may not be written may write it twice
404, 413 topic not on the allowlist, or body over the size limit; the producer was never called no

200, 429 and 504 are each checked by reading the topic with somebody else's client, on both builds. So is the shutdown: clients publish under load while the service gets SIGTERM, and every client's ledger is compared with what the topic holds afterwards — zero disagreements in twenty rounds per build.

What it does not do is on purpose: authentication (the proxy in front does it, and MOSTIK_TOPICS is the allowlist), idempotency keys, batches, tombstones, and reading from Kafka.

Install

It ships twice from one source — a Kotlin/Native linuxX64 binary and a JVM distribution — and both are built from the repository. There is no published image yet; the Dockerfile puts the native binary on gcr.io/distroless/cc-debian13.

ci/broker/broker.sh up                                     # a broker on 127.0.0.1:19092

./gradlew :server:linkReleaseExecutableLinuxX64            # native; builds on Linux x86-64
MOSTIK_BOOTSTRAP_SERVERS=127.0.0.1:19092 MOSTIK_TOPICS=orders,payments \
  server/build/bin/linuxX64/releaseExecutable/mostik.kexe

./gradlew :distribution:installDist                       # JVM, any host with a JDK
MOSTIK_BOOTSTRAP_SERVERS=127.0.0.1:19092 MOSTIK_TOPICS=orders,payments \
  distribution/build/install/distribution/bin/distribution

Everything comes from the environment. A MOSTIK_ variable not in the table stops the start, and so does a budget that cannot hold — a drain shorter than the publish deadline plus a second, or a queue wait not shorter than the deadline — with both values named. KAFKA_* keys pass through to the producer; KAFKA_BOOTSTRAP_SERVERS and KAFKA_MAX_BLOCK_MS are refused, because mostik sets those itself and one value with two sources is a disagreement waiting to happen.

How the deadline is split

Each publish is two calls into kafkakn. First enqueue, bounded by max.block.ms (MOSTIK_QUEUE_WAIT_MS, 1 s): a failure there is provably not written, so it is 429. Then await, under what is left of MOSTIK_PUBLISH_DEADLINE_MS (5 s): a failure there is 504. The first half of that API exists because mostik asked for it. kore runs the lifecycle — the configuration schema, the three probes, and the ordered stop: announce for 5 s with readiness at 503, drain the requests in flight, then close the producer, cut after 3 s if the broker is gone.

Limitations

  • No published image. Build it, or run the binary or the JVM distribution from the repository.
  • On the native build, Ktor's CIO engine occasionally closes a connection without sending the answer the route already wrote — between once in 60 000 and once in 375 000 requests under 64 clients. The client then has no answer and must treat it like a 504. The JVM build has shown none; it reproduces with Ktor alone, in a separate repository.
  • The two builds disagree in one measured case: with the broker stopped, not paused, the JVM build answers 429 and the native one 504. It is written down rather than papered over.
  • kafkakn, kore, sborka and razves come from reposilite, not Maven Central; settings.gradle.kts declares the repository, filtered to their group.
more
kompotshildiktracyzavarnikbooblikpetich