Effect

Your server code uses Effect 4. Core chat() returns an async iterable and throws plain errors, so it does not fit in your Effect program.

@tanstack/ai-effect wraps TanStack AI for Effect. AI.chat returns an Effect Stream of chunks. Its errors are typed, and interruption stops the model request.

1. Install the packages

shell

npm i @tanstack/ai @tanstack/ai-effect @tanstack/ai-openai @tanstack/ai-react effect

shell

pnpm add @tanstack/ai @tanstack/ai-effect @tanstack/ai-openai @tanstack/ai-react effect

shell

yarn add @tanstack/ai @tanstack/ai-effect @tanstack/ai-openai @tanstack/ai-react effect

shell

bun add @tanstack/ai @tanstack/ai-effect @tanstack/ai-openai @tanstack/ai-react effect

The package needs Effect 4 (effect 4.1 or later).

2. Stream the chat from your server route

import { chatParamsFromRequest, toServerSentEventsResponse } from '@tanstack/ai'
import * as AI from '@tanstack/ai-effect'
import { openaiText } from '@tanstack/ai-openai'
import { Stream } from 'effect'

export async function POST(request: Request) {
  const params = await chatParamsFromRequest(request)

  const stream = AI.chat({
    adapter: openaiText('gpt-6.1-sol'),
    messages: params.messages,
    threadId: params.threadId,
    runId: params.runId,
  })

  return toServerSentEventsResponse(Stream.toAsyncIterable(stream))
}

What each piece does:

AI.chat takes the options of core chat(). It returns a Stream<StreamChunk, AIError>.
Stream.toAsyncIterable turns the Stream into the async iterable that toServerSentEventsResponse reads.
If the browser closes the request, the stream stops. Then AI.chat aborts the request to the model.

3. Show the chat on the client

The client does not use Effect. It reads the Server-Sent Events with useChat:

tsx

import { useState } from 'react'
import { fetchServerSentEvents, useChat } from '@tanstack/ai-react'

export function Chat() {
  const [input, setInput] = useState('')
  const { messages, sendMessage, isLoading } = useChat({
    connection: fetchServerSentEvents('/api/chat'),
    threadId: 'effect-chat',
  })

  return (
    <div>
      {messages.map((message) => (
        <p key={message.id}>
          {message.role}:{' '}
          {message.parts.map((part) =>
            part.type === 'text' ? part.content : null,
          )}
        </p>
      ))}
      <form
        onSubmit={(event) => {
          event.preventDefault()
          sendMessage(input)
          setInput('')
        }}
      >
        <input value={input} onChange={(event) => setInput(event.target.value)} />
        <button type="submit" disabled={isLoading}>
          Send
        </button>
      </form>
    </div>
  )
}

Send a message. The reply streams in token by token from your Effect server.

Get the whole reply as an Effect

If you do not need tokens as they arrive, pass stream: false. Then AI.chat returns an Effect of { text, chunks }:

import * as AI from '@tanstack/ai-effect'
import { openaiText } from '@tanstack/ai-openai'
import { Effect } from 'effect'

const program = Effect.gen(function* () {
  const { text } = yield* AI.chat({
    adapter: openaiText('gpt-6.1-sol'),
    messages: [{ role: 'user', content: 'Write a haiku about rivers.' }],
    threadId: 'haiku',
    stream: false,
  })
  return text
})

const haiku = await Effect.runPromise(program)