LanguageModel.ts overview

The LanguageModel module provides AI text generation capabilities with tool calling support.

This module offers a comprehensive interface for interacting with large language models, supporting both streaming and non-streaming text generation, structured output generation, and tool calling functionality. It provides a unified API that can be implemented by different AI providers while maintaining type safety and effect management.

Example

import { LanguageModel } from "@effect/ai"
import { Effect } from "effect"

// Basic text generation
const program = Effect.gen(function* () {
  const response = yield* LanguageModel.generateText({
    prompt: "Explain quantum computing"
  })

  console.log(response.text)

  return response
})

Example

import { LanguageModel } from "@effect/ai"
import { Effect, Schema } from "effect"

// Structured output generation
const ContactSchema = Schema.Struct({
  name: Schema.String,
  email: Schema.String
})

const extractContact = Effect.gen(function* () {
  const response = yield* LanguageModel.generateObject({
    prompt: "Extract contact: John Doe, john@example.com",
    schema: ContactSchema
  })

  return response.value
})

Since v1.0.0

Exports Grouped by Category

Constructors
- make
Context
- LanguageModel (class)
Functions
Models
Utility Types
- ExtractContext (type alias)
- ExtractError (type alias)

Constructors

make

Creates a LanguageModel service from provider-specific implementations.

This constructor takes provider-specific implementations for text generation and streaming text generation and returns a LanguageModel service.

Signature

declare const make: (params: ConstructorParams) => Effect.Effect<Service>

Since v1.0.0

Context

LanguageModel (class)

The LanguageModel service tag for dependency injection.

This tag provides access to language model functionality throughout your application, enabling text generation, streaming, and structured output capabilities.

Example

import { LanguageModel } from "@effect/ai"
import { Effect } from "effect"

const useLanguageModel = Effect.gen(function* () {
  const model = yield* LanguageModel
  const response = yield* model.generateText({
    prompt: "What is machine learning?"
  })
  return response.text
})

Signature

declare class LanguageModel

Since v1.0.0

Functions

generateObject

Generate a structured object from a schema using a language model.

Example

import { LanguageModel } from "@effect/ai"
import { Effect, Schema } from "effect"

const EventSchema = Schema.Struct({
  title: Schema.String,
  date: Schema.String,
  location: Schema.String
})

const program = Effect.gen(function* () {
  const response = yield* LanguageModel.generateObject({
    prompt: "Extract event info: Tech Conference on March 15th in San Francisco",
    schema: EventSchema,
    objectName: "event"
  })

  console.log(response.value)
  // { title: "Tech Conference", date: "March 15th", location: "San Francisco" }

  return response.value
})

Signature

declare const generateObject: <
  A,
  I extends Record<string, unknown>,
  R,
  Options extends NoExcessProperties<GenerateObjectOptions<any, A, I, R>, Options>,
  Tools extends Record<string, Tool.Any> = {}
>(
  options: Options & GenerateObjectOptions<Tools, A, I, R>
) => Effect.Effect<GenerateObjectResponse<Tools, A>, ExtractError<Options>, LanguageModel | R | ExtractContext<Options>>

Since v1.0.0

generateText

Generate text using a language model.

Example

import { LanguageModel } from "@effect/ai"
import { Effect } from "effect"

const program = Effect.gen(function* () {
  const response = yield* LanguageModel.generateText({
    prompt: "Write a haiku about programming",
    toolChoice: "none"
  })

  console.log(response.text)
  console.log(response.usage.totalTokens)

  return response
})

Signature

declare const generateText: <
  Options extends NoExcessProperties<GenerateTextOptions<any>, Options>,
  Tools extends Record<string, Tool.Any> = {}
>(
  options: Options & GenerateTextOptions<Tools>
) => Effect.Effect<GenerateTextResponse<Tools>, ExtractError<Options>, LanguageModel | ExtractContext<Options>>

Since v1.0.0

streamText

Generate text using a language model with streaming output.

Returns a stream of response parts that are emitted as soon as they are available from the model, enabling real-time text generation experiences.

Example

import { LanguageModel } from "@effect/ai"
import { Effect, Stream, Console } from "effect"

const program = LanguageModel.streamText({
  prompt: "Write a story about a space explorer"
}).pipe(
  Stream.runForEach((part) => {
    if (part.type === "text-delta") {
      return Console.log(part.delta)
    }
    return Effect.void
  })
)

Signature

declare const streamText: <
  Options extends NoExcessProperties<GenerateTextOptions<any>, Options>,
  Tools extends Record<string, Tool.Any> = {}
>(
  options: Options & GenerateTextOptions<Tools>
) => Stream.Stream<Response.StreamPart<Tools>, ExtractError<Options>, LanguageModel | ExtractContext<Options>>

Since v1.0.0

Models

ConstructorParams (interface)

Parameters required to construct a LanguageModel service.

Signature

export interface ConstructorParams {
  /**
   * A method which requests text generation from the large language model
   * provider.
   *
   * The final result is returned when the large language model provider
   * finishes text generation.
   */
  readonly generateText: (
    options: ProviderOptions
  ) => Effect.Effect<Array<Response.PartEncoded>, AiError.AiError, IdGenerator>

  /**
   * A method which requests text generation from the large language model
   * provider.
   *
   * Intermediate results are streamed from the large language model provider.
   */
  readonly streamText: (
    options: ProviderOptions
  ) => Stream.Stream<Response.StreamPartEncoded, AiError.AiError, IdGenerator>
}

Since v1.0.0

GenerateObjectOptions (interface)

Configuration options for structured object generation.

Signature

export interface GenerateObjectOptions<Tools extends Record<string, Tool.Any>, A, I extends Record<string, unknown>, R>
  extends GenerateTextOptions<Tools> {
  /**
   * The name of the structured output that should be generated. Used by some
   * large language model providers to provide additional guidance to the model.
   */
  readonly objectName?: string | undefined

  /**
   * The schema to be used to specify the structure of the object to generate.
   */
  readonly schema: Schema.Schema<A, I, R>
}

Since v1.0.0

GenerateObjectResponse (class)

Response class for structured object generation operations.

Example

import { LanguageModel } from "@effect/ai"
import { Effect, Schema } from "effect"

const UserSchema = Schema.Struct({
  name: Schema.String,
  email: Schema.String
})

const program = Effect.gen(function* () {
  const response = yield* LanguageModel.generateObject({
    prompt: "Create user: John Doe, john@example.com",
    schema: UserSchema
  })

  console.log(response.value) // { name: "John Doe", email: "john@example.com" }
  console.log(response.text) // Raw generated text

  return response.value
})

Signature

declare class GenerateObjectResponse<Tools, A> {
  constructor(value: A, content: Array<Response.Part<Tools>>)
}

Since v1.0.0

value (property)

The parsed structured object that conforms to the provided schema.

Signature

readonly value: A

GenerateTextOptions (interface)

Configuration options for text generation.

Signature

export interface GenerateTextOptions<Tools extends Record<string, Tool.Any>> {
  /**
   * The prompt input to use to generate text.
   */
  readonly prompt: Prompt.RawInput

  /**
   * A toolkit containing both the tools and the tool call handler to use to
   * augment text generation.
   */
  readonly toolkit?: Toolkit.WithHandler<Tools> | Effect.Effect<Toolkit.WithHandler<Tools>, any, any> | undefined

  /**
   * The tool choice mode for the language model.
   * - `auto` (default): The model can decide whether or not to call tools, as
   *   well as which tools to call.
   * - `required`: The model **must** call a tool but can decide which tool will
   *   be called.
   * - `none`: The model **must not** call a tool.
   * - `{ tool: <tool_name> }`: The model must call the specified tool.
   * - `{ mode?: "auto" (default) | "required", "oneOf": [<tool-names>] }`: The
   *   model is restricted to the subset of tools specified by `oneOf`. When
   *   `mode` is `"auto"` or omitted, the model can decide whether or not a tool
   *   from the allowed subset of tools can be called. When `mode` is
   *   `"required"`, the model **must** call one tool from the allowed subset of
   *   tools.
   */
  readonly toolChoice?: ToolChoice<{ [Name in keyof Tools]: Tools[Name]["name"] }[keyof Tools]> | undefined

  /**
   * The concurrency level for resolving tool calls.
   */
  readonly concurrency?: Concurrency | undefined

  /**
   * When set to `true`, tool calls requested by the large language model
   * will not be auto-resolved by the framework.
   *
   * This option is useful when:
   *   1. The user wants to include tool call definitions from an `AiToolkit`
   *      in requests to the large language model so that the model has the
   *      capability to call tools
   *   2. The user wants to control the execution of tool call resolvers
   *      instead of having the framework handle tool call resolution
   */
  readonly disableToolCallResolution?: boolean | undefined
}

Since v1.0.0

GenerateTextResponse (class)

Response class for text generation operations.

Contains the generated content and provides convenient accessors for extracting different types of response parts like text, tool calls, and usage information.

Example

import { LanguageModel } from "@effect/ai"
import { Effect } from "effect"

const program = Effect.gen(function* () {
  const response = yield* LanguageModel.generateText({
    prompt: "Explain photosynthesis"
  })

  console.log(response.text) // Generated text content
  console.log(response.finishReason) // "stop", "length", etc.
  console.log(response.usage) // Usage information

  return response
})

Signature

declare class GenerateTextResponse<Tools> {
  constructor(content: Array<Response.Part<Tools>>)
}

Since v1.0.0

content (property)

Signature

readonly content: Array<Response.Part<Tools>>

ProviderOptions (interface)

Configuration options passed along to language model provider implementations.

This interface defines the normalized options that are passed to the underlying provider implementation, regardless of the specific provider being used.

Signature

export interface ProviderOptions {
  /**
   * The prompt messages to use to generate text.
   */
  readonly prompt: Prompt.Prompt

  /**
   * The tools that the large language model will have available to provide
   * additional information which can be incorporated into its text generation.
   */
  readonly tools: ReadonlyArray<Tool.Any>

  /**
   * The format which the response should be provided in.
   *
   * If `"text"` is specified, the large language model response will be
   * returned as text.
   *
   * If `"json"` is specified, the large language model respose will be provided
   * as an JSON object that conforms to the shape of the specified schema.
   *
   * Defaults to `{ type: "text" }`.
   */
  readonly responseFormat:
    | {
        readonly type: "text"
      }
    | {
        readonly type: "json"
        readonly objectName: string
        readonly schema: Schema.Schema.Any
      }

  /**
   * The tool choice mode for the language model.
   * - `auto` (default): The model can decide whether or not to call tools, as
   *   well as which tools to call.
   * - `required`: The model **must** call a tool but can decide which tool will
   *   be called.
   * - `none`: The model **must not** call a tool.
   * - `{ tool: <tool_name> }`: The model must call the specified tool.
   * - `{ mode?: "auto" (default) | "required", "oneOf": [<tool-names>] }`: The
   *   model is restricted to the subset of tools specified by `oneOf`. When
   *   `mode` is `"auto"` or omitted, the model can decide whether or not a tool
   *   from the allowed subset of tools can be called. When `mode` is
   *   `"required"`, the model **must** call one tool from the allowed subset of
   *   tools.
   */
  readonly toolChoice: ToolChoice<any>

  /**
   * The span to use to trace interactions with the large language model.
   */
  readonly span: Span
}

Since v1.0.0

Service (interface)

The service interface for language model operations.

Defines the contract that all language model implementations must fulfill, providing text generation, structured output, and streaming capabilities.

Signature

export interface Service {
  /**
   * Generate text using the language model.
   */
  readonly generateText: <
    Options extends NoExcessProperties<GenerateTextOptions<any>, Options>,
    Tools extends Record<string, Tool.Any> = {}
  >(
    options: Options & GenerateTextOptions<Tools>
  ) => Effect.Effect<GenerateTextResponse<Tools>, ExtractError<Options>, ExtractContext<Options>>

  /**
   * Generate a structured object from a schema using the language model.
   */
  readonly generateObject: <
    A,
    I extends Record<string, unknown>,
    R,
    Options extends NoExcessProperties<GenerateObjectOptions<any, A, I, R>, Options>,
    Tools extends Record<string, Tool.Any> = {}
  >(
    options: Options & GenerateObjectOptions<Tools, A, I, R>
  ) => Effect.Effect<GenerateObjectResponse<Tools, A>, ExtractError<Options>, R | ExtractContext<Options>>

  /**
   * Generate text using the language model with streaming output.
   */
  readonly streamText: <
    Options extends NoExcessProperties<GenerateTextOptions<any>, Options>,
    Tools extends Record<string, Tool.Any> = {}
  >(
    options: Options & GenerateTextOptions<Tools>
  ) => Stream.Stream<Response.StreamPart<Tools>, ExtractError<Options>, ExtractContext<Options>>
}

Since v1.0.0

ToolChoice (type alias)

The tool choice mode for the language model.

auto (default): The model can decide whether or not to call tools, as well as which tools to call.
required: The model must call a tool but can decide which tool will be called.
none: The model must not call a tool.
{ tool: <tool_name> }: The model must call the specified tool.
{ mode?: "auto" (default) | "required", "oneOf": [<tool-names>] }: The model is restricted to the subset of tools specified by oneOf. When mode is "auto" or omitted, the model can decide whether or not a tool from the allowed subset of tools can be called. When mode is "required", the model must call one tool from the allowed subset of tools.

Signature

type ToolChoice<Tools> =
  | "auto"
  | "none"
  | "required"
  | {
      readonly tool: Tools
    }
  | {
      readonly mode?: "auto" | "required"
      readonly oneOf: ReadonlyArray<Tools>
    }

Since v1.0.0

Utility Types

ExtractContext (type alias)

Utility type that extracts the context requirements from LanguageModel options.

Automatically infers the required services based on the toolkit configuration.

Signature

type ExtractContext<Options> = Options extends {
  readonly toolkit: Toolkit.WithHandler<infer _Tools>
}
  ? Tool.Requirements<_Tools[keyof _Tools]>
  : Options extends {
        readonly toolkit: Effect.Effect<Toolkit.WithHandler<infer _Tools>, infer _E, infer _R>
      }
    ? Tool.Requirements<_Tools[keyof _Tools]> | _R
    : never

Since v1.0.0

ExtractError (type alias)

Utility type that extracts the error type from LanguageModel options.

Automatically infers the possible error types based on toolkit configuration and tool call resolution settings.

Signature

type ExtractError<Options> = Options extends {
  readonly disableToolCallResolution: true
}
  ? Options extends {
      readonly toolkit: Effect.Effect<Toolkit.WithHandler<infer _Tools>, infer _E, infer _R>
    }
    ? AiError.AiError | _E
    : AiError.AiError
  : Options extends {
        readonly toolkit: Toolkit.WithHandler<infer _Tools>
      }
    ? AiError.AiError | Tool.Failure<_Tools[keyof _Tools]>
    : Options extends {
          readonly toolkit: Effect.Effect<Toolkit.WithHandler<infer _Tools>, infer _E, infer _R>
        }
      ? AiError.AiError | Tool.Failure<_Tools[keyof _Tools]> | _E
      : AiError.AiError

Since v1.0.0