ChatOpenAI 통합

The below points apply to @langchain/openai>=0.4.5-rc.0.

OpenAI는 에이전틱 애플리케이션 구축을 지향하는 Responses API를 지원해요. 여기에는 웹 및 파일 검색을 포함한 내장 도구 제품군이 포함돼요. 또한 대화 상태 관리를 지원해 이전 메시지를 명시적으로 전달하지 않고도 대화 스레드를 계속할 수 있어요.

ChatOpenAI는 이 기능 중 하나가 사용되면 Responses API로 라우팅해요. ChatOpenAI를 인스턴스화할 때 useResponsesApi: true를 지정할 수도 있어요.

내장 도구

ChatOpenAI에 내장 도구를 장착하면 파일 컨텍스트나 웹 같은 외부 정보로 응답을 기반(grounding)시켜요. 모델에서 생성된 AIMessage는 내장 도구 호출에 대한 정보를 포함해요.

웹 검색

웹 검색을 트리거하려면 다른 도구처럼 {"type": "web_search_preview"}를 모델에 전달하세요.

llm.invoke("...", { tools: [{ type: "web_search_preview" }] });

import { ChatOpenAI } from "@langchain/openai";

const llm = new ChatOpenAI({ model: "gpt-5.4-mini" }).bindTools([
  { type: "web_search_preview" },
]);

await llm.invoke("What was a positive news story from today?");

응답은 응답 텍스트와 소스를 인용하는 OpenAI annotations를 모두 포함하는 구조화된 콘텐츠 블록을 포함한다는 점에 유의하세요. 출력 메시지에는 도구 호출의 정보도 포함돼요.

파일 검색

파일 검색을 트리거하려면 다른 도구처럼 파일 검색 도구를 모델에 전달하세요. OpenAI 관리형 벡터 저장소를 채우고 도구 정의에 벡터 저장소 ID를 포함해야 해요. 자세한 내용은 OpenAI 문서를 참고하세요.

import { ChatOpenAI } from "@langchain/openai";

const llm = new ChatOpenAI({ model: "gpt-5.4-mini" }).bindTools([
  { type: "file_search", vector_store_ids: ["vs..."] },
]);

await llm.invoke("Is deep research by OpenAI?");

웹 검색과 마찬가지로 응답에는 인용이 포함된 콘텐츠 블록이 포함되고, 내장 도구 호출의 정보도 포함돼요.

컴퓨터 사용

ChatOpenAI는 내장 컴퓨터 사용 도구를 위한 특화 모델인 computer-use-preview 모델을 지원해요. 활성화하려면 다른 도구처럼 컴퓨터 사용 도구를 전달하세요.

현재 컴퓨터 사용 도구 출력은 AIMessage.additional_kwargs.tool_outputs에 있어요. 컴퓨터 사용 도구 호출에 응답하려면 해당하는 ToolMessage를 만들 때 additional_kwargs.type: "computer_call_output"을 설정해야 해요.

자세한 내용은 OpenAI 문서를 참고하세요.

import { AIMessage, ToolMessage } from "@langchain/core/messages";
import { ChatOpenAI } from "@langchain/openai";
import * as fs from "node:fs/promises";

const findComputerCall = (message: AIMessage) => {
  const toolOutputs = message.additional_kwargs.tool_outputs as
    | { type: "computer_call"; call_id: string; action: { type: string } }[]
    | undefined;

  return toolOutputs?.find((toolOutput) => toolOutput.type === "computer_call");
};

const llm = new ChatOpenAI({ model: "computer-use-preview" })
  .bindTools([
    {
      type: "computer-preview",
      display_width: 1024,
      display_height: 768,
      environment: "browser",
    },
  ])
  .bind({ truncation: "auto" });

let message = await llm.invoke("Check the latest OpenAI news on bing.com.");
const computerCall = findComputerCall(message);

if (computerCall) {
  // Act on a computer call action
  const screenshot = await fs.readFile("./screenshot.png", {
    encoding: "base64",
  });

  message = await llm.invoke(
    [
      new ToolMessage({
        additional_kwargs: { type: "computer_call_output" },
        tool_call_id: computerCall.call_id,
        content: [
          {
            type: "computer_screenshot",
            image_url: `data:image/png;base64,${screenshot}`,
          },
        ],
      }),
    ],
    { previous_response_id: message.response_metadata["id"] }
  );
}

코드 인터프리터

ChatOpenAI는 내장 코드 인터프리터 도구를 사용해 코드의 샌드박스 생성과 실행을 지원할 수 있게 해줘요.

import { ChatOpenAI } from "@langchain/openai";

const llm = new ChatOpenAI({
  model: "o4-mini",
  useResponsesApi: true,
});

const llmWithTools = llm.bindToools([
  {
    type: "code_interpreter",
    // Creates a new container
    container: { type: "auto" }
  },
]);

const response = await llmWithTools.invoke(
  "Write and run code to answer the question: what is 3^3?"
);

위 명령은 새 컨테이너를 만든다는 점에 유의하세요. 기존 컨테이너 ID를 지정해 호출 간 컨테이너를 재사용할 수 있어요.

const tool_outputs: Record<string, any>[] = response.additional_kwargs.tool_outputs
const container_id = tool_outputs[0].container_id

const llmWithTools = llm.bindTools([
  {
    type: "code_interpreter",
    // Reuses container from the last call
    container: container_id,
  },
]);

원격 MCP

ChatOpenAI는 내장 원격 MCP 도구를 지원하며, 모델이 생성한 MCP 서버 호출이 OpenAI 서버에서 일어나도록 해줘요.

import { ChatOpenAI } from "@langchain/openai";

const llm = new ChatOpenAI({
    model: "o4-mini",
    useResponsesApi: true,
});

const llmWithMcp = llm.bindTools([
    {
        type: "mcp",
        server_label: "deepwiki",
        server_url: "https://mcp.deepwiki.com/mcp",
        require_approval: "never"
    }
]);

const response = await llmWithMcp.invoke(
    "What transport protocols does the 2025-03-26 version of the MCP spec (modelcontextprotocol/modelcontextprotocol) support?"
);

...
const llmWithMcp = llm.bindTools([
  {
    type: "mcp",
    server_label: "deepwiki",
    server_url: "https://mcp.deepwiki.com/mcp",
    require_approval: {
      always: {
        tool_names: ["read_wiki_structure"],
      },
    },
  },
]);
const response = await llmWithMcp.invoke(
    "What transport protocols does the 2025-03-26 version of the MCP spec (modelcontextprotocol/modelcontextprotocol) support?"
);

const approvals = [];
if (Array.isArray(response.additional_kwargs.tool_outputs)) {
  for (const content of response.additional_kwargs.tool_outputs) {
    if (content.type === "mcp_approval_request") {
      approvals.push({
        type: "mcp_approval_response",
        approval_request_id: content.id,
        approve: true,
      });
    }
  }
}

const nextResponse = await model.invoke(
  [
    response,
    new HumanMessage({ content: approvals }),
  ],
);

이미지 생성

ChatOpenAI는 내장 이미지 생성 도구를 통해 responses API를 사용한 다중 턴 대화의 일부로 이미지를 생성할 수 있게 해줘요.

import { ChatOpenAI } from "@langchain/openai";

const llm = new ChatOpenAI({
  model: "gpt-5.5",
  useResponsesApi: true,
});

const llmWithImageGeneration = llm.bindTools([
  {
    type: "image_generation",
    quality: "low",
  }
]);

const response = await llmWithImageGeneration.invoke(
  "Draw a random short word in green font."
)

추론 모델

o1 같은 추론 모델을 사용할 때 withStructuredOutput의 기본 메서드는 OpenAI의 내장 구조화 출력 메서드예요(withStructuredOutput에 method: "jsonSchema" 옵션을 전달하는 것과 동일). JSON 스키마는 다른 모델과 거의 동일하게 작동하지만 한 가지 중요한 주의사항이 있어요: 스키마를 정의할 때 z.optional()은 존중되지 않으므로 대신 z.nullable()을 사용해야 해요.

예시:

import * as z from "zod";
import { ChatOpenAI } from "@langchain/openai";

// Will not work
const reasoningModelSchemaOptional = z.object({
  color: z.optional(z.string()).describe("A color mentioned in the input"),
});

const reasoningModelOptionalSchema = new ChatOpenAI({
  model: "o1",
}).withStructuredOutput(reasoningModelSchemaOptional, {
  name: "extract_color",
});

await reasoningModelOptionalSchema.invoke([{
  role: "user",
  content: `I am 6'5" tall and love fruit.`
}]);

{ color: 'No color mentioned' }

그리고 z.nullable()을 사용한 예시:

import * as z from "zod";
import { ChatOpenAI } from "@langchain/openai";

// Will not work
const reasoningModelSchemaNullable = z.object({
  color: z.nullable(z.string()).describe("A color mentioned in the input"),
});

const reasoningModelNullableSchema = new ChatOpenAI({
  model: "o1",
}).withStructuredOutput(reasoningModelSchemaNullable, {
  name: "extract_color",
});

await reasoningModelNullableSchema.invoke([{
  role: "user",
  content: `I am 6'5" tall and love fruit.`
}]);

{ color: null }

프롬프트 캐싱

최신 OpenAI 모델은 입력이 특정 크기(작성 시점 기준 1024 토큰)를 넘어서면 프롬프트 일부를 자동으로 캐시하여 긴 컨텍스트가 필요한 사용 사례의 비용을 줄여요.

참고: 특정 쿼리에 캐시된 토큰 수는 AIMessage.usage_metadata에 아직 표준화되지 않았으며 대신 AIMessage.response_metadata 필드에 포함돼요.

예시는 다음과 같아요

// @lc-docs-hide-cell

const CACHED_TEXT = `## Components

LangChain provides standard, extendable interfaces and external integrations for various components useful for building with LLMs.
Some components LangChain implements, some components we rely on third-party integrations for, and others are a mix.

### Chat models

<span data-heading-keywords="chat model,chat models"></span>

Language models that use a sequence of messages as inputs and return chat messages as outputs (as opposed to using plain text).
These are generally newer models (older models are generally \`LLMs\`, see below).
Chat models support the assignment of distinct roles to conversation messages, helping to distinguish messages from the AI, users, and instructions such as system messages.

Although the underlying models are messages in, message out, the LangChain wrappers also allow these models to take a string as input.
This gives them the same interface as LLMs (and simpler to use).
When a string is passed in as input, it will be converted to a \`HumanMessage\` under the hood before being passed to the underlying model.

LangChain does not host any Chat Models, rather we rely on third party integrations.

We have some standardized parameters when constructing ChatModels:

- \`model\`: the name of the model

Chat Models also accept other parameters that are specific to that integration.

<Warning>
**Some chat models have been fine-tuned for **tool calling** and provide a dedicated API for it.**

Generally, such models are better at tool calling than non-fine-tuned models, and are recommended for use cases that require tool calling.
Please see the [tool calling section](/oss/javascript/langchain/tools) for more information.
</Warning>

For specifics on how to use chat models, see the [relevant how-to guides here](/oss/javascript/langchain/models).

#### Multimodality

Some chat models are multimodal, accepting images, audio and even video as inputs.
These are still less common, meaning model providers haven't standardized on the "best" way to define the API.
Multimodal outputs are even less common. As such, we've kept our multimodal abstractions fairly light weight
and plan to further solidify the multimodal APIs and interaction patterns as the field matures.

In LangChain, most chat models that support multimodal inputs also accept those values in OpenAI's content blocks format.
So far this is restricted to image inputs. For models like Gemini which support video and other bytes input, the APIs also support the native, model-specific representations.

For specifics on how to use multimodal models, see the [relevant how-to guides here](/oss/javascript/how-to/#multimodal).

### LLMs

<span data-heading-keywords="llm,llms"></span>

<Warning>
**Pure text-in/text-out LLMs tend to be older or lower-level. Many popular models are best used as [chat completion models](/oss/javascript/langchain/models),**

even for non-chat use cases.
</Warning>

Language models that takes a string as input and returns a string.
These are traditionally older models (newer models generally are [Chat Models](/oss/javascript/langchain/models), see above).

Although the underlying models are string in, string out, the LangChain wrappers also allow these models to take messages as input.
This gives them the same interface as [Chat Models](/oss/javascript/langchain/models).
When messages are passed in as input, they will be formatted into a string under the hood before being passed to the underlying model.

LangChain does not host any LLMs, rather we rely on third party integrations.

For specifics on how to use LLMs, see the [relevant how-to guides here](/oss/javascript/langchain/models).

### Message types

Some language models take an array of messages as input and return a message.
There are a few different types of messages.
All messages have a \`role\`, \`content\`, and \`response_metadata\` property.

The \`role\` describes WHO is saying the message.
LangChain has different message classes for different roles.

The \`content\` property describes the content of the message.
This can be a few different things:

- A string (most models deal this type of content)
- A List of objects (this is used for multi-modal input, where the object contains information about that input type and that input location)

#### HumanMessage

This represents a message from the user.

#### AIMessage

This represents a message from the model. In addition to the \`content\` property, these messages also have:

**\`response_metadata\`**

The \`response_metadata\` property contains additional metadata about the response. The data here is often specific to each model provider.
This is where information like log-probs and token usage may be stored.

**\`tool_calls\`**

These represent a decision from a language model to call a tool. They are included as part of an \`AIMessage\` output.
They can be accessed from there with the \`.tool_calls\` property.

This property returns a list of \`ToolCall\`s. A \`ToolCall\` is an object with the following arguments:

- \`name\`: The name of the tool that should be called.
- \`args\`: The arguments to that tool.
- \`id\`: The id of that tool call.

#### SystemMessage

This represents a system message, which tells the model how to behave. Not every model provider supports this.

#### ToolMessage

This represents the result of a tool call. In addition to \`role\` and \`content\`, this message has:

- a \`tool_call_id\` field which conveys the id of the call to the tool that was called to produce this result.
- an \`artifact\` field which can be used to pass along arbitrary artifacts of the tool execution which are useful to track but which should not be sent to the model.

#### (Legacy) FunctionMessage

This is a legacy message type, corresponding to OpenAI's legacy function-calling API. \`ToolMessage\` should be used instead to correspond to the updated tool-calling API.

This represents the result of a function call. In addition to \`role\` and \`content\`, this message has a \`name\` parameter which conveys the name of the function that was called to produce this result.

### Prompt templates

<span data-heading-keywords="prompt,prompttemplate,chatprompttemplate"></span>

Prompt templates help to translate user input and parameters into instructions for a language model.
This can be used to guide a model's response, helping it understand the context and generate relevant and coherent language-based output.

Prompt Templates take as input an object, where each key represents a variable in the prompt template to fill in.

Prompt Templates output a PromptValue. This PromptValue can be passed to an LLM or a ChatModel, and can also be cast to a string or an array of messages.
The reason this PromptValue exists is to make it easy to switch between strings and messages.

There are a few different types of prompt templates:

#### String PromptTemplates

These prompt templates are used to format a single string, and generally are used for simpler inputs.
For example, a common way to construct and use a PromptTemplate is as follows:

\`\`\`typescript
import { PromptTemplate } from "@langchain/core/prompts";

const promptTemplate = PromptTemplate.fromTemplate(
  "Tell me a joke about {topic}"
);

await promptTemplate.invoke({ topic: "cats" });
\`\`\`

#### ChatPromptTemplates

These prompt templates are used to format an array of messages. These "templates" consist of an array of templates themselves.
For example, a common way to construct and use a ChatPromptTemplate is as follows:

\`\`\`typescript
import { ChatPromptTemplate } from "@langchain/core/prompts";

const promptTemplate = ChatPromptTemplate.fromMessages([
  ["system", "You are a helpful assistant"],
  ["user", "Tell me a joke about {topic}"],
]);

await promptTemplate.invoke({ topic: "cats" });
\`\`\`

In the above example, this ChatPromptTemplate will construct two messages when called.
The first is a system message, that has no variables to format.
The second is a HumanMessage, and will be formatted by the \`topic\` variable the user passes in.

#### MessagesPlaceholder

<span data-heading-keywords="messagesplaceholder"></span>

This prompt template is responsible for adding an array of messages in a particular place.
In the above ChatPromptTemplate, we saw how we could format two messages, each one a string.
But what if we wanted the user to pass in an array of messages that we would slot into a particular spot?
This is how you use MessagesPlaceholder.

\`\`\`typescript
import {
  ChatPromptTemplate,
  MessagesPlaceholder,
} from "@langchain/core/prompts";
import { HumanMessage } from "@langchain/core/messages";

const promptTemplate = ChatPromptTemplate.fromMessages([
  ["system", "You are a helpful assistant"],
  new MessagesPlaceholder("msgs"),
]);

promptTemplate.invoke({ msgs: [new HumanMessage({ content: "hi!" })] });
\`\`\`

This will produce an array of two messages, the first one being a system message, and the second one being the HumanMessage we passed in.
If we had passed in 5 messages, then it would have produced 6 messages in total (the system message plus the 5 passed in).
This is useful for letting an array of messages be slotted into a particular spot.

An alternative way to accomplish the same thing without using the \`MessagesPlaceholder\` class explicitly is:

\`\`\`typescript
const promptTemplate = ChatPromptTemplate.fromMessages([
  ["system", "You are a helpful assistant"],
  ["placeholder", "{msgs}"], // <-- This is the changed part
]);
\`\`\`

For specifics on how to use prompt templates, see the [relevant how-to guides here](/oss/javascript/how-to/#prompt-templates).

### Example Selectors

One common prompting technique for achieving better performance is to include examples as part of the prompt.
This gives the language model concrete examples of how it should behave.
Sometimes these examples are hardcoded into the prompt, but for more advanced situations it may be nice to dynamically select them.
Example Selectors are classes responsible for selecting and then formatting examples into prompts.

For specifics on how to use example selectors, see the [relevant how-to guides here](/oss/javascript/how-to/#example-selectors).

### Output parsers

<span data-heading-keywords="output parser"></span>

<Note>
**The information here refers to parsers that take a text output from a model try to parse it into a more structured representation.**

More and more models are supporting function (or tool) calling, which handles this automatically.
It is recommended to use function/tool calling rather than output parsing.
See the [LangChain tools documentation](/oss/javascript/langchain/tools).

</Note>

Responsible for taking the output of a model and transforming it to a more suitable format for downstream tasks.
Useful when you are using LLMs to generate structured data, or to normalize output from chat models and LLMs.

There are two main methods an output parser must implement:

- "Get format instructions": A method which returns a string containing instructions for how the output of a language model should be formatted.
- "Parse": A method which takes in a string (assumed to be the response from a language model) and parses it into some structure.

And then one optional one:

- "Parse with prompt": A method which takes in a string (assumed to be the response from a language model) and a prompt (assumed to be the prompt that generated such a response) and parses it into some structure. The prompt is largely provided in the event the OutputParser wants to retry or fix the output in some way, and needs information from the prompt to do so.

Output parsers accept a string or \`BaseMessage\` as input and can return an arbitrary type.

LangChain has many different types of output parsers. This is a list of output parsers LangChain supports. The table below has various pieces of information:

**Name**: The name of the output parser

**Supports Streaming**: Whether the output parser supports streaming.

**Input Type**: Expected input type. Most output parsers work on both strings and messages, but some (like OpenAI Functions) need a message with specific arguments.

**Output Type**: The output type of the object returned by the parser.

**Description**: Our commentary on this output parser and when to use it.

The current date is ${new Date().toISOString()}`;

// Noop statement to hide output
void 0;

import { ChatOpenAI } from "@langchain/openai";

const modelWithCaching = new ChatOpenAI({
  model: "gpt-5.4-mini",
});

// CACHED_TEXT is some string longer than 1024 tokens
const LONG_TEXT = `You are a pirate. Always respond in pirate dialect.

Use the following as context when answering questions:

${CACHED_TEXT}`;

const longMessages = [
  {
    role: "system",
    content: LONG_TEXT,
  },
  {
    role: "user",
    content: "What types of messages are supported in LangChain?",
  },
];

const originalRes = await modelWithCaching.invoke(longMessages);

console.log("USAGE:", originalRes.response_metadata.usage);

USAGE: {
  prompt_tokens: 2624,
  completion_tokens: 263,
  total_tokens: 2887,
  prompt_tokens_details: { cached_tokens: 0 },
  completion_tokens_details: { reasoning_tokens: 0 }
}

const resWitCaching = await modelWithCaching.invoke(longMessages);

console.log("USAGE:", resWitCaching.response_metadata.usage);

USAGE: {
  prompt_tokens: 2624,
  completion_tokens: 272,
  total_tokens: 2896,
  prompt_tokens_details: { cached_tokens: 2432 },
  completion_tokens_details: { reasoning_tokens: 0 }
}

예측 출력 (Predicted output)

일부 OpenAI 모델(예: gpt-4o 및 gpt-4o-mini 시리즈)은 예측 출력(Predicted Outputs)을 지원하며, LLM의 예상 출력 중 알려진 부분을 미리 전달해 지연 시간을 줄일 수 있게 해줘요. 텍스트 편집이나 코드 편집처럼 모델 출력의 작은 부분만 변경되는 경우에 유용해요.

예시:

import { ChatOpenAI } from "@langchain/openai";

const modelWithPredictions = new ChatOpenAI({
  model: "gpt-5.4-mini",
});

const codeSample = `
/// <summary>
/// Represents a user with a first name, last name, and username.
/// </summary>
public class User
{
/// <summary>
/// Gets or sets the user's first name.
/// </summary>
public string FirstName { get; set; }

/// <summary>
/// Gets or sets the user's last name.
/// </summary>
public string LastName { get; set; }

/// <summary>
/// Gets or sets the user's username.
/// </summary>
public string Username { get; set; }
}
`;

// Can also be attached ahead of time
// using `model.bind({ prediction: {...} })`;
await modelWithPredictions.invoke(
  [
    {
      role: "user",
      content:
        "Replace the Username property with an Email property. Respond only with code, and with no markdown formatting.",
    },
    {
      role: "user",
      content: codeSample,
    },
  ],
  {
    prediction: {
      type: "content",
      content: codeSample,
    },
  }
);

AIMessage {
  "id": "chatcmpl-AQLyQKnazr7lEV7ejLTo1UqhzHDBl",
  "content": "/// <summary>\n/// Represents a user with a first name, last name, and email.\n/// </summary>\npublic class User\n{\n/// <summary>\n/// Gets or sets the user's first name.\n/// </summary>\npublic string FirstName { get; set; }\n\n/// <summary>\n/// Gets or sets the user's last name.\n/// </summary>\npublic string LastName { get; set; }\n\n/// <summary>\n/// Gets or sets the user's email.\n/// </summary>\npublic string Email { get; set; }\n}",
  "additional_kwargs": {},
  "response_metadata": {
    "tokenUsage": {
      "promptTokens": 148,
      "completionTokens": 217,
      "totalTokens": 365
    },
    "finish_reason": "stop",
    "usage": {
      "prompt_tokens": 148,
      "completion_tokens": 217,
      "total_tokens": 365,
      "prompt_tokens_details": {
        "cached_tokens": 0
      },
      "completion_tokens_details": {
        "reasoning_tokens": 0,
        "accepted_prediction_tokens": 36,
        "rejected_prediction_tokens": 116
      }
    },
    "system_fingerprint": "fp_0ba0d124f1"
  },
  "tool_calls": [],
  "invalid_tool_calls": [],
  "usage_metadata": {
    "output_tokens": 217,
    "input_tokens": 148,
    "total_tokens": 365,
    "input_token_details": {
      "cache_read": 0
    },
    "output_token_details": {
      "reasoning": 0
    }
  }
}

현재 예측은 추가 토큰으로 청구되며 이 감소된 지연 시간에 대한 대가로 사용량과 비용이 증가한다는 점에 유의하세요.

오디오 출력

일부 OpenAI 모델(예: gpt-4o-audio-preview)은 오디오 출력 생성을 지원해요. 이 예제는 그 기능을 사용하는 방법을 보여줘요:

import { ChatOpenAI } from "@langchain/openai";

const modelWithAudioOutput = new ChatOpenAI({
  model: "gpt-4o-audio-preview",
  // You may also pass these fields to `.bind` as a call argument.
  modalities: ["text", "audio"], // Specifies that the model should output audio.
  audio: {
    voice: "alloy",
    format: "wav",
  },
});

const audioOutputResult = await modelWithAudioOutput.invoke("Tell me a joke about cats.");
const castAudioContent = audioOutputResult.additional_kwargs.audio as Record<string, any>;

console.log({
  ...castAudioContent,
  data: castAudioContent.data.slice(0, 100) // Sliced for brevity
})

{
  id: 'audio_67129e9466f48190be70372922464162',
  data: 'UklGRgZ4BABXQVZFZm10IBAAAAABAAEAwF0AAIC7AAACABAATElTVBoAAABJTkZPSVNGVA4AAABMYXZmNTguMjkuMTAwAGRhdGHA',
  expires_at: 1729277092,
  transcript: "Why did the cat sit on the computer's keyboard? Because it wanted to keep an eye on the mouse!"
}

오디오 데이터가 data 필드에 반환되는 것을 볼 수 있어요. 또한 expires_at 날짜 필드가 제공되는데, 이 필드는 서버에서 오디오 응답을 더 이상 다중 턴 대화에서 사용할 수 없게 되는 날짜를 나타내요.

스트리밍 오디오 출력

OpenAI는 스트리밍 오디오 출력도 지원해요. 예시:

import { AIMessageChunk } from "@langchain/core/messages";
import { concat } from "@langchain/core/utils/stream"
import { ChatOpenAI } from "@langchain/openai";

const modelWithStreamingAudioOutput = new ChatOpenAI({
  model: "gpt-4o-audio-preview",
  modalities: ["text", "audio"],
  audio: {
    voice: "alloy",
    format: "pcm16", // Format must be `pcm16` for streaming
  },
});

const audioOutputStream = await modelWithStreamingAudioOutput.stream("Tell me a joke about cats.");
let finalAudioOutputMsg: AIMessageChunk | undefined;
for await (const chunk of audioOutputStream) {
  finalAudioOutputMsg = finalAudioOutputMsg ? concat(finalAudioOutputMsg, chunk) : chunk;
}
const castStreamedAudioContent = finalAudioOutputMsg?.additional_kwargs.audio as Record<string, any>;

console.log({
  ...castStreamedAudioContent,
  data: castStreamedAudioContent.data.slice(0, 100) // Sliced for brevity
})

{
  id: 'audio_67129e976ce081908103ba4947399a3eaudio_67129e976ce081908103ba4947399a3e',
  transcript: 'Why was the cat sitting on the computer? Because it wanted to keep an eye on the mouse!',
  index: 0,
  data: 'CgAGAAIADAAAAA0AAwAJAAcACQAJAAQABQABAAgABQAPAAAACAADAAUAAwD8/wUA+f8MAPv/CAD7/wUA///8/wUA/f8DAPj/AgD6',
  expires_at: 1729277096
}

오디오 입력

이러한 모델은 입력으로 오디오를 전달하는 것도 지원해요. 이를 위해 아래와 같이 input_audio 필드를 지정해야 해요:

import { HumanMessage } from "@langchain/core/messages";

const userInput = new HumanMessage({
  content: [{
    type: "input_audio",
    input_audio: {
      data: castAudioContent.data, // Reuse the base64 data from the first example
      format: "wav",
    },
  }]
})

// Reuse the same model instance
const userInputAudioRes = await modelWithAudioOutput.invoke([userInput]);

console.log((userInputAudioRes.additional_kwargs.audio as Record<string, any>).transcript);

That's a great joke! It's always fun to imagine why cats do the funny things they do. Keeping an eye on the "mouse" is a creatively punny way to describe it!

API 레퍼런스

모든 ChatOpenAI 기능과 구성에 대한 자세한 문서는 API 레퍼런스를 참고하세요.

더 알아보기

• 이 문서를 MCP로 연결하면 Claude, VSCode 등에서 실시간 답변을 받을 수 있어요.
• GitHub에서 이 페이지 편집하기 또는 이슈 제출하기.