MongoDBAtlasEmbeddingRetriever

MongoDBAtlasEmbeddingRetriever

MongoDB Atlas Document Store와 호환되는 임베딩 기반 Retriever예요.

출처: 문서

본문

MongoDBAtlasEmbeddingRetriever는 MongoDBAtlasDocumentStore와 호환되는 임베딩 기반 Retriever예요. 검색어와 문서의 임베딩을 비교해서, 그 결과에 따라 Document Store에서 검색어와 가장 관련 있는 문서들을 가져와요.

파라미터

MongoDBAtlasEmbeddingRetriever를 NLP 시스템에서 쓸 때는 검색어와 문서의 임베딩이 준비돼 있어야 해요. 인덱싱 파이프라인에는 Document Embedder를, 검색 파이프라인에는 Text Embedder를 추가하면 되죠.

query_embedding 외에도 top_k(가져올 최대 문서 수)와 검색 공간을 좁히는 filters 같은 선택 파라미터를 받아요.

더 알아보기 (Learn more)

설치

MongoDB Atlas를 Haystack에서 쓰려면 패키지를 설치해요.

pip install mongodb-atlas-haystack

단독으로 쓰기

Retriever를 돌리려면 MongoDBAtlasDocumentStore 인스턴스와 인덱싱된 문서가 필요해요.

from haystack_integrations.document_stores.mongodb_atlas import (
    MongoDBAtlasDocumentStore,
)
from haystack_integrations.components.retrievers.mongodb_atlas import (
    MongoDBAtlasEmbeddingRetriever,
)


document_store = MongoDBAtlasDocumentStore()

retriever = MongoDBAtlasEmbeddingRetriever(document_store=document_store)

# example run query
retriever.run(query_embedding=[0.1] * 384)

파이프라인에서 쓰기

이 페이지의 예제는 sentence-transformers-haystack 패키지의 Sentence Transformers 임베더를 사용해요. 예제를 실행하려면 설치하세요.

pip install sentence-transformers-haystack

from haystack import Pipeline, Document
from haystack.document_stores.types import DuplicatePolicy
from haystack.components.writers import DocumentWriter
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.components.builders import ChatPromptBuilder
from haystack.dataclasses import ChatMessage
from haystack_integrations.components.embedders.sentence_transformers import (
    SentenceTransformersDocumentEmbedder,
    SentenceTransformersTextEmbedder,
)
from haystack_integrations.document_stores.mongodb_atlas import (
    MongoDBAtlasDocumentStore,
)
from haystack_integrations.components.retrievers.mongodb_atlas import (
    MongoDBAtlasEmbeddingRetriever,
)


# Create some example documents
documents = [
    Document(content="My name is Jean and I live in Paris."),
    Document(content="My name is Mark and I live in Berlin."),
    Document(content="My name is Giorgio and I live in Rome."),
]

document_store = MongoDBAtlasDocumentStore()

# Define some more components
doc_writer = DocumentWriter(document_store=document_store, policy=DuplicatePolicy.SKIP)
doc_embedder = SentenceTransformersDocumentEmbedder(model="intfloat/e5-base-v2")
query_embedder = SentenceTransformersTextEmbedder(model="intfloat/e5-base-v2")

# Pipeline that ingests document for retrieval
ingestion_pipe = Pipeline()
ingestion_pipe.add_component(instance=doc_embedder, name="doc_embedder")
ingestion_pipe.add_component(instance=doc_writer, name="doc_writer")

ingestion_pipe.connect("doc_embedder.documents", "doc_writer.documents")
ingestion_pipe.run({"doc_embedder": {"documents": documents}})

# Build a RAG pipeline with a Retriever to get relevant documents to
# the query and an OpenAIChatGenerator interacting with LLMs using a custom prompt.
prompt_template = [
    ChatMessage.from_user(
        """
    Given these documents, answer the question.\nDocuments:
    {% for doc in documents %}
        {{ doc.content }}
    {% endfor %}

    \nQuestion: {{question}}
    \nAnswer:
    """,
    ),
]

rag_pipeline = Pipeline()
rag_pipeline.add_component(instance=query_embedder, name="query_embedder")
rag_pipeline.add_component(
    instance=MongoDBAtlasEmbeddingRetriever(document_store=document_store),
    name="retriever",
)
rag_pipeline.add_component(
    instance=ChatPromptBuilder(template=prompt_template, required_variables="*"),
    name="prompt_builder",
)
rag_pipeline.add_component(instance=OpenAIChatGenerator(), name="llm")
rag_pipeline.connect("query_embedder", "retriever.query_embedding")
rag_pipeline.connect("retriever", "prompt_builder.documents")
rag_pipeline.connect("prompt_builder.prompt", "llm.messages")

# Ask a question on the data you just added.
question = "Where does Mark live?"
result = rag_pipeline.run(
    {
        "query_embedder": {"text": question},
        "prompt_builder": {"question": question},
    },
)

# The generated reply is a ChatMessage; its text holds the answer.
print(result["llm"]["replies"][0].text)