# kafkakn

> A Kafka client for Kotlin Multiplatform: librdkafka linked statically inside the klib on Kotlin/Native, the official Java client on the JVM as the oracle. One test suite runs on both arms against one broker.

Page: https://kotlin.website/kafkakn

## Why it exists

A service compiled to a single Kotlin/Native binary has no way to talk to Kafka: the clients are
JVM libraries, and the one native client ever published for Kotlin (`com.icemachined:kafka-client`
0.2.0, October 2022) ships three native artefacts and **no JVM variant** — so it has no way to check
itself except by believing what a broker tells it. [kafkakn](https://github.com/youndie/kafkakn) is
built the other way round. The `jvm` target is not there for portability; it is there as the
**differential oracle**: one `commonTest` suite runs on both arms against one broker, and the native
arm is correct when it agrees with the reference implementation.

That asymmetry has already paid twice. The two clients put the same key on different partitions —
librdkafka hashes with CRC32, the Java producer with murmur2 — and each arm was consistent with the
broker on its own, so neither could have noticed alone. And the first published klib carried the
bindings and not the C: this repository's own binaries linked because of options on the build
machine, and a stranger's link failed with fourteen undefined symbols. It took a build that was not
this one to find it.

## What you get

- **A producer whose `send` suspends until the broker has acknowledged.** Holding a `RecordMetadata`
  *is* the acknowledgement; there is no third outcome to inspect. This follows from one measured
  fact: `rd_kafka_produce` only enqueues, a record that was never queued produces **no delivery
  report**, and a binding that counts delivery reports can lose 264 826 records of a million while
  flushing successfully. No public API here exposes a delivery-report count, and that is a build
  gate rather than a habit.
- **A consumer** that assigns and seeks, joins groups with manual commits, offers a `Flow` over
  `poll`, cooperative rebalancing, static membership, and the KIP-848 group protocol as a value
  the caller chooses.
- **An admin client**: create, delete and describe topics; describe the cluster; list and describe
  consumer groups and read their committed offsets and lag; move and delete group offsets; change a
  topic's configuration incrementally; add partitions; delete records.
- **TLS, client certificates, SASL PLAIN, SCRAM and OAUTHBEARER** with a token the caller supplies.
  Certificate trust cannot be turned off: the key that would do it is refused on both arms, because
  it exists only on the one without an oracle.
- **librdkafka and its TLS stack inside the klib.** A downstream link needs no configuration of its
  own, and `ldd` on the linked binary is the same set as on the same binary built without kafkakn —
  measured on a downstream build, not on this repository's test binary.

## Install

Published to reposilite only, never to Maven Central, with no compatibility promise. Every publish
is a version of its own, `0.1.0.<n>`, tagged `v0.1.0.<n>` at the commit it was built from and never
overwritten; the newest is the highest tag.

```kotlin
plugins {
    kotlin("multiplatform") version "2.4.20"
}

repositories {
    maven("https://reposilite.kotlin.website/snapshots") {
        content { includeGroupAndSubgroups("io.github.youndie") }
    }
    mavenCentral()
}

kotlin {
    linuxX64("native") {
        binaries.executable { entryPoint = "main" }
    }
    sourceSets.commonMain.dependencies {
        implementation("io.github.youndie.kafkakn:kafkakn-core:0.1.0.12")
    }
}
```

**The Kotlin version is part of the instructions.** A klib carries metadata that a build on another
compiler refuses, so `2.4.20` is the version this is known to work with rather than a placeholder.
Measured from an empty machine — a JDK, Gradle and nothing else — that build file gets to a record on
a topic in 106–109 s, of which 74–78 s is the Kotlin/Native toolchain downloading.

## What the native artefact requires

The same shared libraries as a binary built without it: `libc`, `libgcc_s`, `libm`, `libpthread`,
`libresolv` and the rest of the loader's usual set — linking kafkakn adds none and removes none. The
one thing it does raise is the **glibc floor, to 2.17**, which is `manylinux2014`'s, because that is
where the C bundle is built. The bundle carries a one-line local patch to librdkafka 2.13.0 for that
glibc, kept as a file in the repository and re-applied on every bump; nothing from this project is
filed upstream.

## Limitations

- `jvm` and `linuxX64` are the published targets. `linuxArm64` and `macosArm64` build for
  contributors and pass the same suite against the same broker, but are **not published and not run
  in CI**, by decision. No `macosX64`, no Windows, no `musl`.
- Reposilite only; nothing on Maven Central, and no compatibility promise between versions. What
  would change that is a condition, not a date: somebody outside this portfolio wanting the library.
- Topic metadata is planned and not yet in. OIDC token fetching is not planned — the native bundle
  has no curl, so OAUTHBEARER takes a token the caller already has.
- Schema Registry and Streams are in neither client underneath, so they are not a gap between
  kafkakn and what it wraps — and they are not here either.
- `send` does not batch for you: one call is one record and one acknowledgement. Throughput comes
  from calling it concurrently; both clients batch internally once records are in flight together.
