MariaDBKeywordRetriever

MariaDBKeywordRetriever

MariaDB Document Store에서 쿼리와 일치하는 문서를 가져오는 키워드 기반 Retriever예요.

출처: MariaDBKeywordRetriever

본문

개요

MariaDBKeywordRetriever는 MariaDBDocumentStore와 호환되는 키워드 기반 Retriever예요. MariaDB에 내장된 전문(full-text) 검색으로 주어진 쿼리와 일치하는 문서를 찾아요.

query 외에도 Retriever는 top_k(가져올 최대 문서 수), 검색 공간을 좁히는 filters 같은 선택적 파라미터를 받아요.

설치

MariaDB 11.7 인스턴스를 빠르게 설정하려면 Docker를 사용하면 돼요.

docker run -d -p 3306:3306 \
 -e MARIADB_ROOT_PASSWORD=secret \
 -e MARIADB_DATABASE=haystack \
 -e MARIADB_USER=haystack \
 -e MARIADB_PASSWORD=secret \
 mariadb:11.7

시스템 라이브러리와 통합 패키지를 설치해요.

# Ubuntu / Debian
sudo apt-get install -y libmariadb-dev

pip install mariadb-haystack

사용법

단독 사용:

import os
from haystack_integrations.document_stores.mariadb import MariaDBDocumentStore
from haystack_integrations.components.retrievers.mariadb import MariaDBKeywordRetriever

os.environ["MARIADB_USER"] = "haystack"
os.environ["MARIADB_PASSWORD"] = "secret"

document_store = MariaDBDocumentStore()
retriever = MariaDBKeywordRetriever(document_store=document_store)

retriever.run(query="my search query")

RAG 파이프라인 안에서:

import os
from haystack import Document, Pipeline
from haystack.components.builders import AnswerBuilder, ChatPromptBuilder
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage
from haystack.document_stores.types import DuplicatePolicy

from haystack_integrations.document_stores.mariadb import MariaDBDocumentStore
from haystack_integrations.components.retrievers.mariadb import MariaDBKeywordRetriever

os.environ["MARIADB_USER"] = "haystack"
os.environ["MARIADB_PASSWORD"] = "secret"
os.environ["OPENAI_API_KEY"] = "your-openai-api-key"

prompt_template = [
    ChatMessage.from_user(
        """
Given these documents, answer the question.
Documents:
{% for doc in documents %}
 {{ doc.content }}
{% endfor %}

Question: {{question}}

Answer:
"""
    ),
]

document_store = MariaDBDocumentStore()

documents = [
    Document(content="There are over 7,000 languages spoken around the world today."),
    Document(
        content="Elephants have been observed to recognize themselves in mirrors."
    ),
    Document(
        content="Bioluminescent waves can be seen in the Maldives and Puerto Rico."
    ),
]

document_store.write_documents(documents=documents, policy=DuplicatePolicy.SKIP)

retriever = MariaDBKeywordRetriever(document_store=document_store)
rag_pipeline = Pipeline()
rag_pipeline.add_component(name="retriever", instance=retriever)
rag_pipeline.add_component(
    instance=ChatPromptBuilder(template=prompt_template, required_variables="*"),
    name="prompt_builder",
)
rag_pipeline.add_component(instance=OpenAIChatGenerator(), name="llm")
rag_pipeline.add_component(instance=AnswerBuilder(), name="answer_builder")
rag_pipeline.connect("retriever", "prompt_builder.documents")
rag_pipeline.connect("prompt_builder.prompt", "llm.messages")
rag_pipeline.connect("llm.replies", "answer_builder.replies")
rag_pipeline.connect("retriever", "answer_builder.documents")

question = "languages spoken around the world today"
result = rag_pipeline.run(
    {
        "retriever": {"query": question},
        "prompt_builder": {"question": question},
        "answer_builder": {"query": question},
    }
)
print(result["answer_builder"])

더 알아보기 (Learn more)