OpenAI 임베딩

OpenAI 임베딩

Colab에서 이 노트북을 여는 경우라면 LlamaIndex 🦙 설치가 필요할 거예요.

%pip install llama-index-embeddings-openai

!pip install llama-index

import os


os.environ["OPENAI_API_KEY"] = "sk-..."

from llama_index.embeddings.openai import OpenAIEmbedding
from llama_index.core import Settings


embed_model = OpenAIEmbedding(embed_batch_size=10)
Settings.embed_model = embed_model

OpenAI text-embedding-3-large와 text-embedding-3-small 사용하기

openai 클라이언트를 업데이트해야 할 수도 있어요: pip install -U openai

# get API key and create embeddings
from llama_index.embeddings.openai import OpenAIEmbedding


embed_model = OpenAIEmbedding(model="text-embedding-3-large")


embeddings = embed_model.get_text_embedding(
    "Open AI new Embeddings models is great."
)

print(embeddings[:5])

[-0.011500772088766098, 0.02457442320883274, -0.01760469563305378, -0.017763426527380943, 0.029841400682926178]

print(len(embeddings))

3072

# get API key and create embeddings
from llama_index.embeddings.openai import OpenAIEmbedding


embed_model = OpenAIEmbedding(
    model="text-embedding-3-small",
)


embeddings = embed_model.get_text_embedding(
    "Open AI new Embeddings models is awesome."
)

print(len(embeddings))

1536

출력 임베딩의 차원 변경하기

주의: 최신 OpenAI 클라이언트가 설치되어 있어야 해요.

# get API key and create embeddings
from llama_index.embeddings.openai import OpenAIEmbedding




embed_model = OpenAIEmbedding(
    model="text-embedding-3-large",
    dimensions=512,
)


embeddings = embed_model.get_text_embedding(
    "Open AI new Embeddings models with different dimensions is awesome."
)
print(len(embeddings))

512

출처: 문서

더 알아보기 (Learn more)