ai-streaming

Wire streaming AI responses from model to browser. Uses the Vercel AI SDK — the right abstraction for Next.js AI apps. Reads the project first, wires into existing auth and model config.

Four things that silently break AI streaming

1. Wrong runtime on the route handler. Streaming requires the Edge runtime or a Node.js runtime with proper response handling. Without export const runtime = 'edge' or proper streaming setup, the response buffers entirely and "streams" all at once at the end.
2. No abort handling. If the user navigates away mid-generation, the model keeps running and spending tokens. Always pass abortSignal from an AbortController and clean up on component unmount.
3. Auth check after the stream starts. Once you call streamText, headers are sent. You can't return a 401 after streaming begins. Auth must be checked and validated before calling any SDK method.
4. Missing error boundaries on the client. If the stream errors mid-response, useChat surfaces it in the error state — but if you don't render it, the UI freezes silently with partial output.

Phase 1: Detect the Project

cat package.json | grep -E "ai|anthropic|openai|@ai-sdk"

• ai package present? → Vercel AI SDK installed, go to Phase 3
• @anthropic-ai/sdk only? → add Vercel AI SDK on top
• openai only? → add Vercel AI SDK on top
• Nothing? → install from scratch

Check for existing AI routes:

find . -name "*.ts" -path "*/api/*" | xargs grep -l "streamText\|createOpenAI\|anthropic" 2>/dev/null

Phase 2: Install

npm install ai @ai-sdk/anthropic
# or for OpenAI:
npm install ai @ai-sdk/openai

Add to .env.example:

ANTHROPIC_API_KEY=
# or
OPENAI_API_KEY=

Phase 3: The Route Handler

Create the streaming API route. Auth check must come first — before any SDK call:

// app/api/chat/route.ts
import { streamText } from 'ai';
import { anthropic } from '@ai-sdk/anthropic';

export const runtime = 'edge';  // required for true streaming

export async function POST(req: Request) {
  // 1. Auth — must happen before streamText
  const user = await getAuthUser(req);
  if (!user) {
    return Response.json({ error: 'Unauthorized' }, { status: 401 });
  }

  // 2. Credits / rate limit check (if applicable)
  if (user.credits <= 0) {
    return Response.json({ error: 'No credits remaining', upgradeUrl: '/pricing' }, { status: 402 });
  }

  // 3. Parse the request
  const { messages } = await req.json();

  // 4. Stream
  const result = streamText({
    model: anthropic('claude-sonnet-4-6'),
    messages,
    system: 'You are a helpful assistant.',
    maxTokens: 1024,
    abortSignal: req.signal,  // propagate client abort
    onFinish: async ({ usage }) => {
      // Deduct credits after completion
      await deductCredits(user.id, usage.totalTokens);
    },
  });

  return result.toDataStreamResponse();
}

For single-turn generation (not chat):

// app/api/generate/route.ts
import { streamText } from 'ai';
import { anthropic } from '@ai-sdk/anthropic';

export const runtime = 'edge';

export async function POST(req: Request) {
  const user = await getAuthUser(req);
  if (!user) return Response.json({ error: 'Unauthorized' }, { status: 401 });

  const { prompt } = await req.json();

  const result = streamText({
    model: anthropic('claude-sonnet-4-6'),
    prompt,
    abortSignal: req.signal,
  });

  return result.toDataStreamResponse();
}

Phase 4: The Client Hook

Chat interface (multi-turn):

'use client';
import { useChat } from 'ai/react';

export function Chat() {
  const { messages, input, handleInputChange, handleSubmit, isLoading, error, stop } = useChat({
    api: '/api/chat',
    onError: (err) => console.error('Chat error:', err),
  });

  return (
    <div>
      <div>
        {messages.map((m) => (
          <div key={m.id} className={m.role === 'user' ? 'text-right' : 'text-left'}>
            <p>{m.content}</p>
          </div>
        ))}
        {isLoading && <p className="text-muted animate-pulse">Thinking...</p>}
        {error && <p className="text-red-500">Something went wrong. Try again.</p>}
      </div>

      <form onSubmit={handleSubmit}>
        <input
          value={input}
          onChange={handleInputChange}
          placeholder="Ask anything..."
          disabled={isLoading}
        />
        {isLoading ? (
          <button type="button" onClick={stop}>Stop</button>
        ) : (
          <button type="submit">Send</button>
        )}
      </form>
    </div>
  );
}

Single-turn generation:

'use client';
import { useCompletion } from 'ai/react';

export function Generator() {
  const { completion, input, handleInputChange, handleSubmit, isLoading, error, stop } = useCompletion({
    api: '/api/generate',
  });

  return (
    <div>
      <form onSubmit={handleSubmit}>
        <textarea value={input} onChange={handleInputChange} />
        <button type="submit" disabled={isLoading}>Generate</button>
        {isLoading && <button type="button" onClick={stop}>Stop</button>}
      </form>

      {error && <p className="text-red-500">{error.message}</p>}
      {completion && <div className="whitespace-pre-wrap">{completion}</div>}
    </div>
  );
}

Phase 5: Streaming Loading States

Three states to handle, all distinct:

// 1. Not started — show the input
// 2. Loading (model thinking, no tokens yet) — show spinner
// 3. Streaming (tokens arriving) — show partial text

function StreamingOutput({ isLoading, completion }: { isLoading: boolean; completion: string }) {
  if (!isLoading && !completion) return null;

  return (
    <div>
      {isLoading && !completion && (
        // Model is thinking — no tokens yet
        <div className="flex gap-1">
          <span className="animate-bounce">·</span>
          <span className="animate-bounce delay-100">·</span>
          <span className="animate-bounce delay-200">·</span>
        </div>
      )}
      {completion && (
        // Tokens arriving
        <p className="whitespace-pre-wrap">
          {completion}
          {isLoading && <span className="animate-pulse">▋</span>}
        </p>
      )}
    </div>
  );
}

Phase 6: Wire Credits Into the Stream

If the project uses a credits system created with the ship-credits skill, deduct after the stream finishes — not before:

// In the route handler onFinish callback:
onFinish: async ({ usage, finishReason }) => {
  if (finishReason === 'stop' || finishReason === 'length') {
    const cost = Math.ceil(usage.totalTokens / 100);  // define your credit cost
    await db.update(users)
      .set({ credits: sql`credits - ${cost}` })
      .where(eq(users.id, user.id));
  }
},

Never deduct before the stream — if the model errors on token 1, the user lost credits for nothing.

Verify

[ ] Route handler exports runtime = 'edge'
[ ] Auth check happens before any streamText call
[ ] req.signal passed as abortSignal to streamText
[ ] useChat/useCompletion renders error state (not just isLoading)
[ ] Stop button shown when isLoading — lets user cancel
[ ] Credits deducted in onFinish, not before the stream
[ ] Partial text renders during stream (not only after complete)
[ ] Test: start generation, navigate away — confirm model call aborts in server logs
[ ] Test: trigger an error — confirm error state renders, not a frozen UI