Building Robust AI Chatbot Streaming: Reconnect, Partial Responses, and Accessible Status Messages

Streaming makes an AI chatbot feel faster because the first words appear before the full response is calculated. Technically, however, this creates a distributed process: the server, model provider, proxy, browser, and user interface share state for seconds or minutes. Mobile networks switch connections, a tab moves to the background, a proxy terminates an idle connection, or a user accidentally re-sends a request. Without a clear protocol, text fragments get displayed twice, half-formed statements are marked as complete, or the same tool action is triggered twice.

A robust website chatbot therefore treats streaming as a state machine, not an animation. This guide shows how event IDs, reconnection, atomic completion, and reserved screen reader announcements work together.

A message needs a persistent identity

Assign a client-side request ID upon sending and an immutable server-side message ID. Each stream segment also receives a sequential sequence number. If the same request arrives again after a connection error, the server must not start a second independent run; instead, it should return the existing state or safely resume execution.

These identities serve different purposes: the request ID makes the write operation idempotent, the message ID identifies the result, and the sequence number orders the fragments. A timestamp alone is not enough, as parallel requests can collide or arrive out of order.

Separate transport from business state

Whether you use Server-Sent Events, Fetch Streams, or WebSockets, the business lifecycle remains the same. Model at least the states accepted, running, completed, canceled, and failed. Only an explicit completion event makes a response binding. Conversely, the end of a TCP connection does not automatically signify success.

For Server-Sent Events, the HTML standard specifies reconnection mechanisms and passing the last event ID. While useful, this feature does not replace a server-side history. The server must know which fragments belong to a message and whether a new fetch can skip already emitted sequences.

Reconnect without duplicate text

Store a limited event buffer per active message. Upon reconnecting, the client sends the last acknowledged sequence number. The server only transmits subsequent events. If the buffer has expired, it should not reply with guessed fragments, but with a snapshot of the current complete text and a new baseline sequence.

The client processes events idempotently: sequence numbers less than or equal to the last applied value are ignored. Larger gaps trigger a snapshot request. This keeps the UI accurate even if a proxy retransmits data or the browser comes back online after a short disconnect.

Partial responses must not trigger actions

Streamed text is preliminary. Links might be incomplete, a restriction may only appear in the next sentence, and structured tool arguments remain syntactically invalid until finished. Render text progressively, but enable risky actions only after complete delivery and separate validation.

This is especially true for orders, appointment bookings, updates to customer data, or sending emails. Executing a tool requires its own idempotent action ID, permission checks, and, if appropriate, visible user confirmation. Reconnecting must never trigger the same side effect again.

Treat cancellation as a real protocol event

A stop button should do more than just pause the UI. The client sends a cancellation request with the message ID; the server flags the execution run and terminates model and tool operations whenever possible. Fragments arriving later are discarded. The user interface clearly indicates that the response was canceled.

If the cancellation request fails to reach the server, processing might continue in the background. Therefore, the server should periodically check execution status as well. Cost and latency metrics should track canceled runs separately, otherwise they appear as normal errors or disappear completely from analytics.

Make errors understandable and retryable

Differentiate at least between network interruptions, timeouts, provider errors, security blocks, and validation failures. User-facing messages do not need to expose internal implementation details, but they should suggest a safe next step. "Connection lost – resuming response" is fundamentally different from "This action was not executed."

A retry button should reuse the original request ID only if the exact same run is to be resumed. For a true re-generation, a new ID is generated, ensuring the UI does not render both versions as a single merged result.

Do not flood screen readers with every token

Dynamic content must be perceptible to assistive technologies. WAI-ARIA defines Live Regions and varying urgency levels for this purpose. However, a region updated token by token with aria-live can cause hundreds of interruptions. A better approach is a visual streaming indicator paired with a separate, polite status channel.

For example, announce "Generating response", followed by a completed sentence or section at reasonable intervals, and finally "Response complete". Use aria-live="polite" for standard progress updates; assertive is reserved strictly for critical errors. Focus should remain on the input field or the user's chosen location rather than jumping with every fragment.

Set aria-busy="true" on the response container while the content is incomplete, and remove it upon atomic completion. The stop button requires a clear accessible label and full keyboard accessibility. Be sure to also test for reduced motion, page zoom, and small mobile viewports.

Targeted testing of the state machine

A happy-path test is insufficient. Automate at least these test scenarios:

• Disconnect after several fragments and resume without duplicate text.
• Deliver the same event twice and apply it only once.
• Skip a sequence number and trigger a snapshot request.
• Pause the tab, switch networks, and ensure the final complete status renders correctly.
• Cancel during tool preparation and verify no side effects occurred.
• Mark a timeout following a visible partial response as incomplete.
• Verify screen reader output frequency and focus retention behavior.

Track time to first visible segment, time to full completion, reconnect rates, duplicate or discarded sequences, and cancellation success rate. Time to first token alone can look misleadingly good, even if many responses never finish reliably.

A step-by-step rollout plan

1. Define message and event states on the server side.
2. Implement idempotent IDs and sequence numbers before building UI animations.
3. Add reconnection handling with event buffering and snapshot fallbacks.
4. Strictly decouple tool execution from preliminary response text.
5. Verify status announcements using keyboard navigation and screen readers.
6. Test edge cases under throttled and switching network connections.
7. Only then gradually enable streaming for production traffic.

Conclusion: Fast to display, definitively complete

Good streaming balances perceived speed with a reliable model of truth. Persistent IDs, ordered events, atomic completion, and safe reconnection mechanisms prevent duplicate or fragmented responses. A polite live region ensures accessibility without overwhelming screen reader users with every generated token.

As a next step, test a live chat session on an unstable mobile network. If, after an interruption and reconnect, it is unclear which message is complete and which action was actually executed, fix the protocol underlying your architecture first—not the loading spinner.

Sources

• WHATWG HTML Standard: Server-Sent Events
• MDN Web Docs: Streams API
• W3C: WAI-ARIA 1.2
• W3C WAI: Status Messages with ARIA Live Regions