A streaming chat app

Moth DocsTutorials

A chat page in Next.js whose answers appear word by word, with the key kept on your server.

The page sends the conversation to a route on your own server, the route calls Moth with your key and streams the answer back as plain text, and the page shows it as it arrives. A key never goes into the browser, where anyone using the page could read it.

  1. Make a Next.js app

    Any recent Next.js app with the App Router works, such as a new one from npx create-next-app@latest moth-chat.

  2. Install the SDK and set the key

    Run npm install openai, then put MOTH_API_KEY=moth_your_key_here in .env.local, which Next.js reads on the server only.

  3. Add the route

    The route takes the messages, asks for a stream and passes each piece of text on as it arrives.

  4. Add the page

    The page posts the conversation and reads the stream, adding each piece to the last message.

app/api/chat/route.ts
import OpenAI from "openai";

// The key stays on the server. The browser only ever talks to this route.
const client = new OpenAI({ baseURL: "https://api.runmoth.si/v1", apiKey: process.env.MOTH_API_KEY });

export async function POST(req: Request) {
  const { messages } = await req.json();
  const stream = await client.chat.completions.create({
    model: "gemini-3.8-flash",
    messages,
    stream: true,
  });
  const encoder = new TextEncoder();
  const body = new ReadableStream({
    async start(controller) {
      try {
        for await (const chunk of stream) {
          const text = chunk.choices[0]?.delta?.content;
          if (text) controller.enqueue(encoder.encode(text));
        }
        controller.close();
      } catch (err) {
        controller.error(err);
      }
    },
  });
  return new Response(body, { headers: { "Content-Type": "text/plain; charset=utf-8" } });
}
app/page.tsx
"use client";
import { useState } from "react";

type Message = { role: "user" | "assistant"; content: string };

export default function Chat() {
  const [messages, setMessages] = useState<Message[]>([]);
  const [input, setInput] = useState("");
  const [busy, setBusy] = useState(false);

  async function send(e: React.FormEvent) {
    e.preventDefault();
    if (!input.trim() || busy) return;
    const next: Message[] = [...messages, { role: "user", content: input }];
    setMessages([...next, { role: "assistant", content: "" }]);
    setInput("");
    setBusy(true);
    try {
      const res = await fetch("/api/chat", { method: "POST", body: JSON.stringify({ messages: next }) });
      if (!res.ok || !res.body) throw new Error(`The chat route answered ${res.status}`);
      const reader = res.body.getReader();
      const decoder = new TextDecoder();
      let reply = "";
      for (;;) {
        const { done, value } = await reader.read();
        if (done) break;
        reply += decoder.decode(value, { stream: true });
        setMessages([...next, { role: "assistant", content: reply }]);
      }
    } catch (err) {
      setMessages([...next, { role: "assistant", content: String(err) }]);
    } finally {
      setBusy(false);
    }
  }

  return (
    <main style={{ maxWidth: 640, margin: "40px auto", fontFamily: "system-ui" }}>
      {messages.map((m, i) => (
        <p key={i}><b>{m.role === "user" ? "You" : "Model"}</b> {m.content}</p>
      ))}
      <form onSubmit={send}>
        <input value={input} onChange={(e) => setInput(e.target.value)} placeholder="Ask anything" style={{ width: "80%" }} />
        <button disabled={busy}>Send</button>
      </form>
    </main>
  );
}

Run npm run dev, open the page and ask something. The answer fills in as the model writes it.

Taking it further

  • Put a spend cap on the key this app uses, so a loop or a busy day can't spend more than you meant.
  • To show what each answer cost, send stream_options with include_usage and read the last chunk, as Chat and streaming shows.
  • The Vercel AI SDK gives you the same thing with ready-made hooks for the page.
  • Swap gemini-3.8-flash for any chat model ID in the catalog. The code stays the same.