GlimpseEngine · Audio Integration · V1

Live Audio Integration Guide

You publish one live video stream over SRT. We watch it, write football commentary, synthesise it, and hand you ready-to-play WAV segments with the media timestamp each one belongs to. Subtitles come with them, from the same record.

HTTPS + WebSocketSRT + MPEG-TSWAV 24 kHz monoUpdated 2026-09-13

1. Two transports

Audio delivery is a per-match choice. You declare it when you create the match; one deployment serves both, and two matches running against the same service can use different ones.

TransportShapeChoose it when
websocketYou connect to us. Audio bytes are inlined in the message; receipts go back up the same socket.You cannot expose a public HTTPS endpoint, or you would rather not run one. Start here.
callbackWe POST to an endpoint you host. The body carries a short-lived signed URL you download the WAV from.You already run a webhook receiver and want delivery to survive your client process restarting.
They carry the same payload. Every field is identical except for how the bytes arrive: audio_base64 inside the WebSocket message, versus audio_url in the callback body. Everything downstream of that — timestamps, idempotency keys, receipts — is the same, so switching later does not change how you handle a segment.

2. Credentials

You will be given two tokens over a private channel. They are not interchangeable, and neither of them is the operations token.

TokenOpens
Match AdminPOST /v1/matches, /arm, /end, and reading a match back.
Audio ReceiptThe audio WebSocket, and POST /v1/audio/receipts.

Both go in Authorization: Bearer <token>. If you are ever handed a token that also opens /v1/status or /v1/scout, that is an operations credential and it was sent to you by mistake — tell us.

3. Create the match, and declare the transport

POST/v1/matches

{
  "match_id": "your-match-001",
  "match_package": { "competition": "…", "teams": { … } },
  "output": {
    "mode": "audio",
    "languages": ["zh-CN"],
    "commentary_profile": "betting",
    "audio_transport": "websocket"
  }
}
FieldRule
mode"audio" or "subtitle". Required when output is supplied; there is no default inside that object.
languagesIn audio mode: exactly one. Two, or none, is a 400 — we will not silently pick the first one for you.
audio_transport"websocket" or "callback". No default. Omitting it in audio mode is a 400 that names the field.
audio_callback_urlCallback transport only, and then required. Must be https, must not carry credentials in the URL, and must resolve to a public address.
output.commentary_profileOptional: "betting" gives event-focused factual lines; "tv" adds objective play descriptions and relevant verified background. Neither promises continuous speech or complete event coverage. Precedence: output.commentary_profile, then match_package.commentary_profile, then the service setting (source default betting, which a deployment can override). Null or blank means omitted; surrounding whitespace and case are normalized. Other nonblank values return 400. GET /v1/matches/{match_id} returns output.effective_commentary_profile.

Creation returns the stored request settings. A later GET /v1/matches/{match_id} also returns the effective settings. For example, on a service configured to use v1 audio:

"output": {
  "mode": "audio", "languages": ["zh-CN"],
  "audio_transport": "websocket", "commentary_profile": "betting",
  "effective_mode": "v1", "effective_transport": "websocket",
  "effective_commentary_profile": "betting"
}
Read the echo back on your first integration run. effective_transport is what the engine will actually use. If it does not say what you asked for, stop there rather than waiting to find out during a match.

Output settings are fixed at creation. The match package can be updated only before arming; an explicit output profile overrides its preference.

4. Arm the match and publish

POST /v1/matches/{match_id}/arm returns your SRT ingest URL and the stream epoch for this attempt. Publish your MPEG-TS feed to it; commentary starts once we see frames.

Arm is where transport misconfiguration surfaces. A callback match is refused at arm if the deployment has no public object-storage endpoint configured; a websocket match is refused if the audio WebSocket is not enabled. Both are refused before kickoff, with the missing setting named — not silently mid-match.

5. WebSocket transport

wss://<host>/v1/matches/{match_id}/audio/stream?after_sequence=0
Authorization: Bearer <audio receipt token>

Each delivery is one self-contained JSON message. There is no metadata/bytes frame pairing to reassemble.

{
  "type": "delivery",
  "action": "upsert",
  "delivery_sequence": 41,
  "idempotency_key": "your-match-001:epoch-…:speech-acde1234:r1",
  "target_start_pts_ms": 51600,
  "expires_pts_ms": 61600,
  "duration_ms": 4500,
  "codec": "wav", "sample_rate": 24000, "channels": 1,
  "payload_bytes": 216044,
  "content_sha256": "<64 lowercase hex>",
  "audio_base64": "<the WAV bytes>",
  "commentary_text": "…",
  "emotion": "Tension"
}
Raise your client’s maximum message size before you integrate. A delivery is a whole WAV segment in base64 — roughly 64 KB per second of audio, plus a third again for base64. Most WebSocket libraries default to a 1 MiB limit, and exceeding it does not drop one message: it closes the whole connection with code 1009. The symptom is “it reconnects every so often and we lose a few lines”, which is very hard to trace back to a size setting. Set it to 32 MiB. Our own limit for a single delivery is 16 MiB.

Cancellations

"action": "cancel" withdraws a segment revision that has not been played yet. It carries no audio_base64. Drop the matching idempotency_key from your playout queue.

Reconnecting

Reconnect with after_sequence set to the highest delivery_sequence you have processed. Anything enqueued while you were away is replayed in order. Connecting with after_sequence=0 after the match was re-armed gives you the current epoch only — a previous attempt’s audio is never replayed into a new one.

1008 is fatal, not a blip. A bad token does not fail the HTTP upgrade: the connection is established and then closed with code 1008 and a reason. The same is true for an unknown match and for a malformed receipt. Treat 1008 as “fix the request” and stop; a client that reconnects on it will loop forever.

6. Callback transport

We POST to the audio_callback_url you gave at match creation. Each request carries:

HeaderMeaning
X-Eros-TimestampUnix seconds. Reject anything more than five minutes from your own clock.
X-Eros-Signaturesha256=<hex>, an HMAC over <timestamp>.<raw body bytes> with the shared secret. Compare in constant time.
Idempotency-KeySame value as the body’s idempotency_key. Retries reuse it; dedupe on it.

The body is identical to the WebSocket message except that audio_base64 is replaced by audio_url — a signed link, valid five minutes.

Download the first one for real, on day one. audio_url is built from our deployment’s public object-storage address, and if that address is not reachable from your network you will see nothing wrong on your side except empty downloads — our side reports the delivery as sent, and arm cannot detect it, because it only checks that the address is configured, not that you can reach it. Fetch one, check the length against payload_bytes and the digest against content_sha256, and tell us if it fails.
We validate your callback URL again immediately before every single POST, re-resolving DNS and pinning the address we checked. A URL that starts resolving to a private address mid-match stops being delivered to, permanently, rather than being retried.

7. Receipts — the part integrations skip, and regret

A receipt tells us what actually happened to a segment. Send exactly one per delivery.

→ {"type":"receipt","idempotency_key":"…:r1","status":"played"}
← {"type":"receipt_ack","idempotency_key":"…:r1","accepted":true,"duplicate":false,"status":"played"}

On the callback transport, send the same thing as POST /v1/audio/receipts with the Audio Receipt token.

StatusSend it when
playedThe segment was actually heard.
failedYou reached expires_pts_ms without playing it, or playback failed. Send this rather than nothing.
cancelledYou dropped it because a cancel arrived first.
Receipts are the only thing that tells us a fact was spoken. Without a played receipt, the same fact stays eligible to be narrated again — and it will be, for the rest of the match. This is the single most common integration defect, and it looks like a commentary-quality problem, not like a missing HTTP call.

Re-sending the same key with the same status is safe and answers duplicate: true. Sending the same key with a different status is refused — the first receipt is the one that counts. On the WebSocket transport that refusal closes the connection with 1008 and the reason timed audio already has a different receipt; on the callback transport it is a 400. So do not, for example, report failed at expiry and then played when a late buffer finally drains: pick one, and it is the first one that sticks. receipt_ack is what lets you tell “it landed” from “the socket died on the way” — do not treat a successful send() as confirmation; keep it queued until the ack arrives, and re-send unacked receipts after a reconnect.

8. Subtitles come with the audio

An audio match also produces subtitles, derived from the same admitted segment — same text, same timing, one duration model. You do not ask for them separately and they cannot disagree with what was spoken.

Read them the same way a subtitle-only integration does: GET /v1/matches/{match_id}/subtitles, or the subtitle WebSocket. See the subtitle guide for that surface.

9. Playing the audio

Schedule against your video clock, not arrival time. target_start_pts_ms is a media timestamp on the stream you published. Map it into your playout clock the same way you map your own video; do not start playing when the HTTP request or WebSocket message happens to arrive.
Give up at expires_pts_ms. Past that point the line is about something the viewer has moved on from. Do not shift it later to “catch up” — drop it and report failed.
Honour revisions. The idempotency_key ends in :r1, :r2…; a higher revision for the same segment replaces a lower one you have not played yet.
Never play two segments at once. We already guarantee they do not overlap on the timeline. If your queue drifts enough that they would, drop the older one.

emotion

emotion is the delivery mood of the segment — exactly one of five fixed values, present on every line, never empty. The voice in the WAV already follows it; it is set from the confirmed match situation the segment reports, not from its wording. A replay is always Calm: it is never voiced as if it were happening again.

emotionWhen
CalmRoutine play, substitutions, kick-off, replays, background information
TensionDanger or a caution: a shot, a foul, a free kick, a corner, a yellow card
NervousThe outcome is still open: a VAR check, a penalty about to be taken, a chance developing
DramaA decisive turn that is not a goal: penalty awarded, red card, big save, goal ruled out
ExcitedA goal

The same value appears on the matching subtitle line. Use it for on-screen treatment or mixing if you wish; you do not need it to play the audio. Treat any value you do not recognise as Calm.

10. Latency — read this before you integrate

Commentary is produced behind live edge by a fixed broadcast buffer. That buffer is the delay your viewers experience, and it is a deployment setting, not something a match can shorten below the pipeline’s worst case.

Practically: your player is expected to be running that far behind the source feed already. If you play our audio against a feed that is not delayed, every segment will arrive after the moment it describes and you will fail all of them at expires_pts_ms.

11. Errors

What you seeWhat it means
400 output.audio_transport must be one of …Audio mode without a declared transport. There is no default.
400 audio_callback_url must use httpsCallback transport with a non-HTTPS address. Raised at match creation, not at delivery time.
400 audio mode with two languagesAudio is exactly one language per match.
401Wrong token for that surface. Check you are not using the Match Admin token on the receipt endpoint, or vice versa.
WebSocket close 1008Rejected: bad token, unknown match, or a malformed receipt. The reason string says which. Do not reconnect on this.
WebSocket close 1009Your client’s maximum message size is too small. See section 5.
Refused at armThe deployment is not configured for the transport this match declared. The error names the setting.

12. Integration checklist

Created a match with output.mode="audio", exactly one language, and an explicit audio_transport.
Read effective_transport back and confirmed it is what you asked for.
Raised the WebSocket client’s maximum message size to 32 MiB — or, on callback, actually downloaded one audio_url and verified its length and SHA-256.
Send a receipt for every delivery, including failed for ones you could not play.
Keep receipts queued until receipt_ack, and re-send unacked ones after a reconnect.
Reconnect with after_sequence, and treat close code 1008 as fatal.
Schedule playback from target_start_pts_ms, and drop anything past expires_pts_ms.

13. Changelog

DateChange
2026-09-24Every delivery now carries emotion (§9), on both transports. Additive; schema versions unchanged. If you validate payloads strictly against a schema copy taken before this date, refresh it: the schemas do not allow unknown keys.
2026-09-13First published. Both transports are live.
2026-09-11WebSocket transport added. The deployment-wide callback address was retired: a callback match now carries its own audio_callback_url on the create request.