Skip to main content
Start with a minimal request, then add failure handling and resource bounds to your application. This page provides integration guidance, not universal deployment limits, latency, or billing guarantees.

​
Timeouts and request lifecycle

Example timeout values are for demonstration. An SDK read timeout may not bound the entire operation; set an application deadline separately. Disconnection or client cancellation does not guarantee upstream work has stopped, especially for durable media tasks.

​
Concurrency and retries

Bound concurrency and queue lengths per application, tenant, and model. Return a retryable message when queues fill. One RAG workflow can involve embedding, reranking, and generation calls, so account for the whole workflow. Bound tool-call rounds and calls per round too. Examples disable automatic SDK retries to expose the result of one request. For production, see errors and retries: consider 429, transient 5xx, jittered backoff, and any gateway retries already configured. Avoid unbounded retries at the client, application queue, and gateway simultaneously. Correct invalid parameters on HTTP 400. After partial output, do not silently concatenate a second generation into the first answer.

​
Proxies and output handling

For SSE, check reverse-proxy buffering, compression, idle timeouts, and whether events arrive incrementally. Client stream: true alone cannot remove proxy buffering. Verify WebSocket upgrades and connection lifetimes. Media upload sizes must be allowed by the client, proxy, and gateway together. Validate final text completion, refusals, and content types. Check business schemas for structured output and call IDs for tools. Verify media MIME types and actual content. HTTP 200, valid JSON, and successful download are not complete business acceptance criteria.

​
Observability and data

Record an application request ID, returned X-Gateway-Request-Id (compatibility header: X-Oneapi-Request-Id), time and timezone, model, endpoint, status, first-event latency, total latency, and retry count. Responses intercepted by a front proxy may lack gateway IDs. Do not assume SDK _request_id reads these custom headers. Exclude keys, cookies, complete inputs, media Base64, and compaction ciphertext from default logs. Decide whether to retain history, tool results, and task IDs, and confirm gateway logging, upstream retention, and artifact URL expiry with the deployment operator. store: false is only a request option.

​
Real-service acceptance

  1. Discover a model and make a minimal call with the same address and token; verify the actual protocol.
  2. Test each capability the application uses: text, SSE, tools, structured output, and multimodal inputs or outputs.
  3. Exercise denied access, rate limits, timeouts, truncated streams, refusals, and failed tasks; ensure no infinite retries or duplicate publication.
  4. Evaluate representative inputs for accuracy, latency, and resource use. Record versions and routing settings for comparison after upgrades.
  5. Start with limited traffic and recoverable settings. Fall back to a verified application path instead of switching models arbitrarily.
This repository’s npm run check checks documentation and local simulated responses only. These real-service checks require a deployment and test access and have not been performed. GitHub, hosting accounts, and domain connections remain deferred to publishing. 阅读中文 Model routing · Rate limits · Observability · AI SDK