Skip to main content

AI chat

A chat window in the style of an LLM app. A Hono route in the same Vite project plays the model: it writes its answer one word at a time, and the client shows the text while it grows. Stop cancels the answer halfway, and Regenerate asks again with the same history.

intermediateuseStreamstreamingcancellationserver
import { Hono } from "hono";
import { stream } from "hono/streaming";

export type ChatMessage = { role: "user" | "assistant"; content: string };

/** A fake model: a few canned answers, picked by keyword, with some variety for "Regenerate". */
const answers: Record<string, string[]> = {
cache: [
"HyperFetch keeps every response in a cache keyed by the request. Two components that read the same request share one entry, so the second one renders at once without another call.",
"Responses are cached by key. staleTime decides when data counts as old, and cacheTime decides when it is removed. You can also write to the cache yourself with client.cache.update().",
],
offline: [
"Requests made offline wait in the dispatcher queue. When the app manager reports the network again, the queue is sent in order. Give the dispatcher a storage and the queue survives a reload too.",
],
default: [
"Good question. I am a fake model running in server/app.ts, so I only know a few answers. I stream them word by word, the way a real model streams tokens, so you can see useStream render each chunk as it arrives.",
"This answer comes from a Hono route that writes one word every few dozen milliseconds. The client reads the response body as a stream and shows the text while it grows. Try asking about the cache or offline mode.",
"Streaming lets the reader start before the answer is complete. Press Stop to cancel the response: the client cancels the body, and the server sees the request abort and stops writing.",
],
};

const pickAnswer = (messages: ChatMessage[]) => {
const question =
[...messages]
.reverse()
.find((message) => message.role === "user")
?.content.toLowerCase() ?? "";
const topic = Object.keys(answers).find((key) => question.includes(key)) ?? "default";
const options = answers[topic];
return options[Math.floor(Math.random() * options.length)];
};

const app = new Hono().basePath("/api").post("/chat", async (c) => {
const { messages } = await c.req.json<{ messages: ChatMessage[] }>();
// Split into words with their spaces, so the client gets a few characters at a time.
const tokens = pickAnswer(messages).match(/\S+\s*/g) ?? [];

c.header("Content-Type", "text/plain; charset=utf-8");
return stream(c, async (body) => {
let sent = 0;
body.onAbort(() => console.log(`[chat] client stopped the stream after ${sent} of ${tokens.length} tokens`));

for (const token of tokens) {
if (body.aborted) return;
await body.write(token);
sent += 1;
// A model does not answer at a steady pace.
await body.sleep(30 + Math.random() * 70);
}
});
});

// `@hono/vite-dev-server` serves the default export inside the Vite dev server.
export default app;
Run locallynpx tiged BetterTyped/hyper-fetch/examples/react/ai-chat hyper-fetch-react-ai-chat && cd hyper-fetch-react-ai-chat && npm install && npm run dev
What it shows
  1. useStream sends a request in streaming mode and returns text that grows with every chunk.
  2. autoStart starts the stream on mount, so each answer is one component with its own stream.
  3. abort() cancels the request, or the body once it is being read. The server sees the abort and stops writing.
  4. Regenerate mounts a new answer with the same history, which starts a fresh stream.

How it works​

  1. server/app.ts handles POST /api/chat. It picks a canned answer by keyword and writes it word by word with Hono's stream(), sleeping 30 to 100 ms between words like a model that produces tokens at an uneven pace.

  2. sendChat in api.ts is an ordinary POST request with the chat history as payload and string as the response type. Nothing in it is specific to streaming.

  3. StreamingAnswer builds the request with setPayload({ messages: history }) and passes it to useStream:

    const { text, streaming, done, error, abort } = useStream(request, { autoStart: true });

    useStream sends it with the fetch adapter's streaming option. The response body comes back as a ReadableStream, and each chunk is decoded and appended to text.

  4. When done or error turns true, StreamingAnswer reports the final text to App, which stores it as a finished message and unmounts the streaming one.

  5. Stop calls abort() and keeps the text received so far. The body is canceled, the connection closes, and the server's onAbort logs how many words it sent. You can see it in the terminal that runs npm run dev.

  6. Regenerate removes the last answer and mounts a new StreamingAnswer with a new key, so it starts a new stream with its own state.

The adapter timeout covers the wait for the response headers. Once the body is streaming it can take as long as it needs.

When to use it
  • Chat and assistant UIs that call an LLM through your own API.
  • Any long text response, such as logs, reports or code generation, that people can start reading early.
  • Answers people may want to cancel to save tokens or time.

Summary

A plain POST request and useStream render an answer while the server writes it. abort() cancels the body and ends the server's work, and a new component with the same history regenerates the answer.