Streaming
A response that arrives slowly, rendered as it downloads. httpbin's /drip endpoint sends a few bytes at a time over
several seconds, which is the same shape as an LLM answer streamed token by token.
Opens the example in StackBlitz in a new tab. The first start installs packages and takes a few seconds.
import { useStream } from "@hyper-fetch/react";
import { useState } from "react";
import { getDrip } from "./api";
export const App = () => {
const [numbytes, setNumbytes] = useState(120);
const [duration, setDuration] = useState(6);
// `start()` sends the request in streaming mode. `text` and `chunks` grow with every chunk the server sends.
const { text, chunks, streaming, done, error, status, start, reset } = useStream(
getDrip.setQueryParams({ numbytes, duration, delay: 0 }),
);
const restart = () => {
reset();
start();
};
let state = "Idle";
if (streaming) state = "Streaming";
else if (done) state = "Done";
else if (error) state = "Failed";
return (
<main className="container">
<hgroup>
<h1>Streaming</h1>
<p>The response is rendered while it downloads, the way chat apps show an LLM answer.</p>
</hgroup>
<article>
<div className="grid">
<label>
Bytes: {numbytes}
<input
type="range"
min={20}
max={400}
step={20}
value={numbytes}
disabled={streaming}
onChange={(event) => setNumbytes(Number(event.target.value))}
/>
</label>
<label>
Duration: {duration}s
<input
type="range"
min={1}
max={15}
value={duration}
disabled={streaming}
onChange={(event) => setDuration(Number(event.target.value))}
/>
</label>
</div>
<button type="button" onClick={restart} disabled={streaming} aria-busy={streaming}>
{done || error ? "Stream again" : "Start"}
</button>
</article>
<article>
<header className="toolbar muted">
<span>
<strong>{state}</strong> · status {status ?? "-"} · {text.length} of {numbytes} bytes in {chunks.length}{" "}
chunks
</span>
</header>
<progress value={text.length} max={numbytes} />
{error && <p className="error">The stream ended early: {error.message}</p>}
<pre className="output">
{text}
{streaming && <span className="cursor" />}
</pre>
</article>
</main>
);
};
npx tiged BetterTyped/hyper-fetch/examples/react/streaming hyper-fetch-react-streaming && cd hyper-fetch-react-streaming && npm install && npm run devuseStreamsends a request in streaming mode and reads the response body chunk by chunk.textholds everything decoded so far, so the UI grows with every chunk.chunksholds the rawUint8Arraypieces, for binary data or for counting what arrived.streaminganddonetell the UI when to show a cursor and when the answer is complete.
How it works
-
api.tsdefinesgetDriplike any other request. Nothing in it is specific to streaming. -
Apppasses the request touseStream.start()sends it with thestreaming: truerequest option, so the HTTP adapter resolves with the body as aReadableStreaminstead of waiting for the whole response:const { text, chunks, streaming, done, error, start, reset } = useStream(getDrip.setQueryParams({ numbytes, duration, delay: 0 }),); -
Every chunk re-renders the component. The
<pre>showstextwith a blinking cursor whilestreamingistrue, and the progress bar compares the received length with the requested size. -
Stream again calls
reset()to clear the previous answer andstart()to send the request again. -
For an LLM API the code stays the same: point the request at the completion endpoint, set the prompt as the payload, and render
text.
- Chat and completion UIs that show an answer while it is generated.
- Long reports, logs or exports that are useful before they finish.
- Large downloads where you want the raw chunks.
Related
useStream turns a regular request into a streamed one. text grows as chunks arrive, and streaming and done drive
the loading and finished states.
