Realtime WebSocket API

GET /v1/realtime

Handshake and protocol

Connect to placeholder URL wss://gateway.example.com/v1/realtime?model=YOUR_MODEL_ID with a WebSocket client and Bearer gateway token. The model query is required and URL-encoded. There is no JSON body. Successful upgrade returns HTTP 101; failures may be HTTP 4xx/5xx, and upgraded connections can still receive errors or close. This site uses GA session configuration and response.output_* events. The OpenAI adapter adds upstream Beta identification only for legacy mapped model names containing -realtime-preview; that does not imply event conversion. Do not mix Beta modalities / input_audio_format / response.audio.delta with the GA fields here.

Client events and fields

Event	Common fields and purpose
session.update	session.type=realtime; output_modalities selects text or audio; instructions, tools; audio.input / audio.output
conversation.item.create	item.type=message, role=user, content containing input_text; add text input
input_audio_buffer.append	audio is Base64 raw audio bytes, not a media URL or WAV file
input_audio_buffer.commit	Commit buffered audio; manual mode still needs response.create for generation
input_audio_buffer.clear	Clear uncommitted input, not generation cancellation
response.create	response contains output modality and other model settings; differs from top-level Responses create
response.cancel	Request cancellation; confirm the server’s terminal state separately from local playback
conversation.item.truncate	Synchronize played audio position when implementing interruption; requires upstream support

Realtime client events defines the protocol; the gateway and model may not support every extension. The script disables tools and VAD and generates one turn. These are three separate messages illustrating text mode:

{"type":"session.update","session":{"type":"realtime","output_modalities":["text"],"audio":{"input":{"format":{"type":"audio/pcm","rate":24000},"turn_detection":null}},"tools":[]}}

{"type":"conversation.item.create","item":{"type":"message","role":"user","content":[{"type":"input_text","text":"Hello"}]}}

{"type":"response.create","response":{"output_modalities":["text"]}}

Server events and correlation

Event	Meaning
session.created	Read session.id and protocol type
session.updated	Effective configuration; verify format, modality, and turn_detection
input_audio_buffer.committed	Audio committed with item_id, not proof of successful transcription
conversation.item.added / done	Conversation-item lifecycle, not generation completion
response.created	Obtain response.id for the turn
response.output_text.delta / done	Text delta and text-part completion
response.output_audio.delta / done	Base64 audio delta and audio-part completion
response.output_audio_transcript.delta / done	Transcript of output audio, not audio bytes
response.done	Inspect response.status; completed, cancelled, failed, and incomplete are distinct
error / rate_limits.updated	Request error / upstream limit hints, not universal gateway limits

Correlate content by response_id, item_id, output_index, and content_index. The final response.done does not contain complete audio bytes; retain audio deltas. Audio done can also occur on unsuccessful termination, so check final status. See the server event reference for upstream structures.

Audio format and limits

The example requests PCM16LE, 24 kHz, mono through audio.input.format / audio.output.format using type: audio/pcm, rate: 24000; voice is audio.output.voice. turn_detection: null disables automatic VAD. The script validates effective configuration before sending input. Other formats, voices, and rates require model support and cannot simply be saved as PCM WAV. The backend can impose connection, stream-idle, and realtime request limits. Disconnection proves neither success nor non-execution. Retain completion state and playback position, and do not automatically resend audio. Browser temporary keys, WebRTC, SIP, and Live API are not automatically covered by this route. Source: router/relay-router.go, relay/channel/openai/adaptor.go, relay/channel/openai/relay_realtime.go, checked at 30a6d88a4. This portable Markdown page documents handshake and events without pretending they are an ordinary JSON REST operation. Runnable text and audio example · Connection module 阅读中文

Getting started

Integration guides

Application guides

Multimodal guides

Retrieval and reranking

Production and troubleshooting

Additional protocols and tasks

API reference

Realtime WebSocket API

Handshake and protocol

Client events and fields

Server events and correlation

Audio format and limits

​Handshake and protocol

​Client events and fields

​Server events and correlation

​Audio format and limits

Handshake and protocol

Client events and fields

Server events and correlation

Audio format and limits