AI chat
A chat window in the style of an LLM app. A Hono route in the same Vite project plays the model: it writes its answer one word at a time, and the client shows the text while it grows. Stop cancels the answer halfway, and Regenerate asks again with the same history.
Opens the example in StackBlitz in a new tab. The first start installs packages and takes a few seconds.
import { Hono } from "hono";
import { stream } from "hono/streaming";
export type ChatMessage = { role: "user" | "assistant"; content: string };
/** A fake model: a few canned answers, picked by keyword, with some variety for "Regenerate". */
const answers: Record<string, string[]> = {
cache: [
"HyperFetch keeps every response in a cache keyed by the request. Two components that read the same request share one entry, so the second one renders at once without another call.",
"Responses are cached by key. staleTime decides when data counts as old, and cacheTime decides when it is removed. You can also write to the cache yourself with client.cache.update().",
],
offline: [
"Requests made offline wait in the dispatcher queue. When the app manager reports the network again, the queue is sent in order. Give the dispatcher a storage and the queue survives a reload too.",
],
default: [
"Good question. I am a fake model running in server/app.ts, so I only know a few answers. I stream them word by word, the way a real model streams tokens, so you can see useStream render each chunk as it arrives.",
"This answer comes from a Hono route that writes one word every few dozen milliseconds. The client reads the response body as a stream and shows the text while it grows. Try asking about the cache or offline mode.",
"Streaming lets the reader start before the answer is complete. Press Stop to cancel the response: the client cancels the body, and the server sees the request abort and stops writing.",
],
};
const pickAnswer = (messages: ChatMessage[]) => {
const question =
[...messages]
.reverse()
.find((message) => message.role === "user")
?.content.toLowerCase() ?? "";
const topic = Object.keys(answers).find((key) => question.includes(key)) ?? "default";
const options = answers[topic];
return options[Math.floor(Math.random() * options.length)];
};
const app = new Hono().basePath("/api").post("/chat", async (c) => {
const { messages } = await c.req.json<{ messages: ChatMessage[] }>();
// Split into words with their spaces, so the client gets a few characters at a time.
const tokens = pickAnswer(messages).match(/\S+\s*/g) ?? [];
c.header("Content-Type", "text/plain; charset=utf-8");
return stream(c, async (body) => {
let sent = 0;
body.onAbort(() => console.log(`[chat] client stopped the stream after ${sent} of ${tokens.length} tokens`));
for (const token of tokens) {
if (body.aborted) return;
await body.write(token);
sent += 1;
// A model does not answer at a steady pace.
await body.sleep(30 + Math.random() * 70);
}
});
});
// `@hono/vite-dev-server` serves the default export inside the Vite dev server.
export default app;
npx tiged BetterTyped/hyper-fetch/examples/react/ai-chat hyper-fetch-react-ai-chat && cd hyper-fetch-react-ai-chat && npm install && npm run devuseStreamsends a request in streaming mode and returnstextthat grows with every chunk.autoStartstarts the stream on mount, so each answer is one component with its own stream.abort()cancels the request, or the body once it is being read. The server sees the abort and stops writing.- Regenerate mounts a new answer with the same history, which starts a fresh stream.
How it works
-
server/app.tshandlesPOST /api/chat. It picks a canned answer by keyword and writes it word by word with Hono'sstream(), sleeping 30 to 100 ms between words like a model that produces tokens at an uneven pace. -
sendChatinapi.tsis an ordinaryPOSTrequest with the chat history as payload andstringas the response type. Nothing in it is specific to streaming. -
StreamingAnswerbuilds the request withsetPayload({ messages: history })and passes it touseStream:const { text, streaming, done, error, abort } = useStream(request, { autoStart: true });useStreamsends it with the fetch adapter'sstreamingoption. The response body comes back as aReadableStream, and each chunk is decoded and appended totext. -
When
doneorerrorturns true,StreamingAnswerreports the final text toApp, which stores it as a finished message and unmounts the streaming one. -
Stop calls
abort()and keeps the text received so far. The body is canceled, the connection closes, and the server'sonAbortlogs how many words it sent. You can see it in the terminal that runsnpm run dev. -
Regenerate removes the last answer and mounts a new
StreamingAnswerwith a newkey, so it starts a new stream with its own state.
The adapter timeout covers the wait for the response headers. Once the body is streaming it can take as long as it needs.
- Chat and assistant UIs that call an LLM through your own API.
- Any long text response, such as logs, reports or code generation, that people can start reading early.
- Answers people may want to cancel to save tokens or time.
Related
A plain POST request and useStream render an answer while the server writes it. abort() cancels the body and ends
the server's work, and a new component with the same history regenerates the answer.
