Moq Kit

moq-kit

Native Swift and Kotlin SDKs for Media over QUIC live streaming: connect to a relay, discover broadcasts, play catalog-described streams with sub-second latency, publish camera/microphone/screen tracks, and exchange raw data tracks. Wraps the UniFFI bindings generated from moq-ffi (Luke Curley's moq-dev/moq); targets the moq-lite protocol, not the IETF moq-transport draft.

Requirements: iOS 16+ / macOS 13+ (Xcode 16+ to build) / Android minSdk 29, compileSdk 35. A moq-lite relay to connect to. The references cover iOS and Android; macOS is declared by the Swift package but untested here.

moq-kit is an active preview: public APIs, packaging, and codec coverage can still change between releases. Pin an exact version and check signatures against the installed artifact rather than assuming a shape.

Install — iOS (SPM):

.package(url: "https://github.com/software-mansion-labs/moq-kit", exact: "<latest release>")
// product: .product(name: "MoQKit", package: "moq-kit")

Install — Android (Maven Central):

dependencies { implementation("com.swmansion.moqkit:moqkit:<latest release>") }

Fill in <latest release> from https://github.com/software-mansion-labs/moq-kit/releases. Both pins are exact on purpose — loosen the SPM one (.upToNextMinor(from:)) only if you mean to track preview releases as they land.

Platform detection

Reference files are split per platform — read only the ones for the project at hand:

• Package.swift, *.xcodeproj, or a Podfile → the *-ios.md references (Swift).
• build.gradle/build.gradle.kts with com.swmansion.moqkit → the *-android.md references (Kotlin).
• Cross-platform work (feature parity, bindings on top of both SDKs) → read both; the APIs are deliberately kept equivalent.

Mental model

• A Session owns one QUIC connection to a relay and serves both subscribing and publishing. connect() once; a session is one-shot — after close() (or a connection error) create a new Session. Swift: actor, connect() async throws. Kotlin: optionally takes a parentScope (defaults to an internal IO scope; pass viewModelScope/lifecycleScope so cancelling it tears the session down); connect() is the only suspend call.
• Subscribing: session.subscribe(prefix:) → stream of Broadcast → broadcast.catalogs() → pick playable tracks → Player(catalog:videoTrackName:audioTrackName:targetBuffering:) → play(). Rendering is platform-specific: iOS adds player.videoLayer (an AVSampleBufferDisplayLayer) to the view hierarchy; Android calls player.setSurface(surface) with a Surface you own.
• Publishing: create capture sources (CameraCapture, MicrophoneCapture, MultiCameraCapture, ScreenCapture) and start them yourself → Publisher() + addVideoTrack/addAudioTrack/addDataTrack → session.publish(path, publisher) → publisher.start(). A publisher is single-use; add all tracks before start().
• Data tracks (DataTrackEmitter → addDataTrack; consume with broadcast.subscribeTrack(name:)) bypass the media catalog — publisher and subscriber agree on the track name out of band.
• Everything is observable: session.state, publisher.state/events, player events/stats. Kotlin uses StateFlow/SharedFlow/cold Flow; Swift uses AsyncStream for session/publisher state and diagnostics, but callback subscriptions (subscribeEvents/subscribeStats) for player events/stats.

Quick start — watch

iOS:

let session = Session(url: "http://localhost:4443/anon")
try await session.connect()
let subscription = try await session.subscribe(prefix: "live")
for await broadcast in subscription.broadcasts {
    for await catalog in broadcast.catalogs() {
        let video = catalog.playableVideoTracks.first?.name
        let audio = catalog.playableAudioTracks.first?.name
        guard video != nil || audio != nil else { continue }
        let player = try await MainActor.run {
            try Player(catalog: catalog, videoTrackName: video, audioTrackName: audio)
        }
        try await player.play() // render via player.videoLayer — see playback-ios.md
    }
}

Android:

scope.launch {
    val session = Session(url = "http://localhost:4443/anon", parentScope = scope)
    session.connect()
    session.subscribe(prefix = "live").broadcasts.collect { broadcast ->
        broadcast.catalogs().collect { catalog ->
            val video = catalog.playableVideoTracks.firstOrNull()?.name
            val audio = catalog.playableAudioTracks.firstOrNull()?.name
            if (video == null && audio == null) return@collect
            val player = Player(catalog, video, audio, parentScope = scope)
            player.setSurface(surfaceView.holder.surface)
            player.play()
        }
    }
}

Minimal on purpose: the nested loops handle one broadcast at a time — spawn a Task/launch per broadcast in real code — and each catalog update should tear down the previous player before creating the new one (players hold native handles).

Quick start — go live

Same four steps on both platforms, and the order is load-bearing (rule 3):

iOS:

try await camera.start()                                   // 1. captures — publisher.start() won't do this
let publisher = try Publisher()                            // 2. every track before start()
publisher.addVideoTrack(name: "camera", source: camera, config: VideoEncoderConfig())
try await session.publish(path: "live/ios", publisher: publisher)  // 3. register
try await publisher.start()                                // 4. go live

Android:

camera.start(context, lifecycleOwner)
val publisher = Publisher()
publisher.addVideoTrack(name = "camera", source = camera, config = VideoEncoderConfig())
session.publish(path = "live/android", publisher = publisher)
publisher.start()

Captures, previews, encoder configs, and permissions are in the publishing references.

Shared concepts (both platforms)

• Catalogs: broadcast.catalogs() streams catalog updates — each element replaces the previous; the stream ending means the broadcast went away (mark it offline). It often ends by throwing rather than completing — a producer that disappears surfaces as MoqException$Mux: json: remote error: code=… out of collect, so handle the exception and the clean completion identically (both demo apps and react-native-moq do). Always select from playableVideoTracks / playableAudioTracks (device-filtered on Android and for iOS audio; iOS video playability is codec-family recognition only), not the raw lists; per-track isPlayable/unsupportedReason explain exclusions (they live on the concrete VideoTrackInfo/AudioTrackInfo, not the base TrackInfo). Video codec strings: "avc1", "hev1", "av01"; audio: "mp4a.40.2"/"aac" (Android-published tracks advertise "aac"), "opus". AV1 is playback-only — neither platform publishes it, and Apple-side decode is iPhone 15 Pro-class hardware, not a general capability.
• Broadcast paths are prefix-relative: subscribing with prefix "live" reports broadcast live/game as path: "game" — subscribe with "" to see full paths.
• Three observation channels on Player — don't mix them up: events (lifecycle milestones — player.init, playback.start, track.ready, track.switch, track.stall.start, rebuffer.end, decode.error — wire values; Kotlin matches the PlayerEventName enum; not replayed), stats (PlaybackStats at ~1 s cadence — latency, fps, bitrates, StallStats, timeToFirst, buffer depths, drops, switches), and diagnostics (typed per-frame PipelineEvent — frameDropped with DropStage/DropReason, stallStarted, latencySample, … — bounded at 256 drop-oldest — Android hands every caller the same shared flow — and never backpressures playback; for logging/telemetry only).
• Publisher states: Idle / Publishing / Stopped / Error(message) — but Android never actually enters Error, and it does not react to the session dying either: an Android publisher sits in Publishing indefinitely after its session is gone, so the session's own state is the only signal that the broadcast stopped. Wire failure UI to events: TrackStarted / TrackStopped / a per-track error (Kotlin TrackError(name, message), Swift .error(name, message)); per-track PublishedTrack.state (Idle / Starting / Active / Stopped) with PublishedTrack.stop() to end one track. session.unpublish(path) calls publisher.stop().
• TrackDelivery for raw track subscription: monotonic (default) skips late groups — right for newest-value-wins live state; arrival delivers every group in order — right for chat and anything lossless.

Critical rules

1. Sessions, publishers, and players are single-use. A Session connects once (alreadyConnected/IllegalStateException on reuse); after close() make a new one. A stopped Publisher cannot be restarted. Never restart a torn-down Player on either platform — create a fresh one.
2. Media track names are local labels, catalog names are muxer-generated. The name: passed to addVideoTrack/addAudioTrack (defaults "video"/"audio") only labels PublishedTrack and publisher events; subscribers must discover real track names from Catalog.videoTracks/audioTracks. Exception: data tracks — addDataTrack(name:) (default "data") IS the name subscribers pass to subscribeTrack(name:).
3. publisher.start() does not start capture. Start CameraCapture/MicrophoneCapture/etc. yourself, in this order: start captures → add tracks → session.publish(path, publisher) → publisher.start().
4. Gate codecs before use. VideoEncoderConfig.supportedCodecs() / AudioEncoderConfig.supportedCodecs() / .isSupported on the publish side; catalog.playableVideoTracks / playableAudioTracks / per-track isPlayable on the playback side. Unsupported configs throw (SessionError.unsupportedCodec / UnsupportedCodecException). H.264 + AAC/Opus are the best-tested paths; default to H.264. Opus wants 48 kHz — on Android keep MicrophoneCapture(sampleRate = …) equal to the encoder config; iOS has no mic rate knob and resamples internally.
5. Player startup events are not replayed. Subscribe before play(): iOS player.subscribeEvents { ... } (and retain the returned subscription — it cancels on deinit); Android collect player.events with CoroutineStart.UNDISPATCHED. On both platforms player.init fires during construction and can never be observed — the rule protects track.ready/playback.start.
6. Permissions and audio sessions are the app's job. iOS: NSCameraUsageDescription, NSMicrophoneUsageDescription, NSLocalNetworkUsageDescription (local relays), and configure AVAudioSession yourself (.playAndRecord to publish, .playback to watch) — MicrophoneCapture will not. Android: declare + request CAMERA/RECORD_AUDIO (MicrophoneCapture.start() silently no-ops without it), INTERNET, and the MediaProjection foreground-service setup for screen capture.
7. Own your references and clean up. iOS: keep strong refs to Session, BroadcastSubscription, Publisher, DataTrackEmitter, Player, and every event/stats subscription; hold the Task handles that drive for await loops (the AsyncStreams are single-consumer, too). Android: nearly everything is AutoCloseable and holds ref-counted native handles — close() each Broadcast, BroadcastSubscription, TrackSubscription, MediaTrack, AudioDataStream, and Player (leaking one keeps the broadcast alive). broadcasts, objects, frames, and audio flows allow a single collector.
8. iOS Player is @MainActor — construct and drive it on the main actor. iOS Publisher.stop() flushes encoders synchronously — call it off the main thread (Task.detached; note Publisher isn't Sendable, so under strict Swift 6 concurrency this needs nonisolated(unsafe) or a Swift 5-mode target).
9. When one session both publishes and subscribes the same prefix, it sees its own broadcast — filter it out, remembering reported paths are prefix-relative: compare against your publish path with the subscribe prefix stripped. Two Sessions in one process (one publishing, one subscribing) is also supported — and since sessions are one-shot, separate ones keep a transport failure or teardown on the publish side from taking playback down with it.
10. Latency is tunable live: targetBuffering defaults to 100 ms; player.updateTargetLatency(_:) adjusts during playback. A second consumer of the same media track (another Player, an AudioDataStream) reuses the first subscriber's upstream buffering — it can't independently lower it.
11. Custom sources must share the publisher's clock domain. A Publisher stamps all of its tracks against one epoch, set by the first frame of any track, so a source emitting its own zero-based timeline drifts against the mic or leaves video scheduled far in the future. iOS: PTS on the host clock (CMClockGetTime(CMClockGetHostTimeClock())) — the domain AVFoundation capture already uses. Android: SystemClock.elapsedRealtimeNanos() / 1_000 for PCM (what MicrophoneCapture uses) and the same clock behind the encoder surface's presentation timestamps.