Hologres

Hologres

이번엔 Alibaba Cloud의 실시간 데이터 웨어하우스인 Hologres를 LlamaIndex의 벡터 저장소로 사용하는 방법을 배워볼게요. Hologres는 고성능 OLAP 분석과 높은 QPS의 온라인 서비스를 모두 지원하는 원스톱 데이터 웨어하우스라서, 대규모 실시간 시나리오에 어울려요. 클라우드 인스턴스가 준비되어 있다고 가정하고 진행합니다.

출처: 문서

본문

Hologres 는 고성능 OLAP 분석과 높은 QPS 온라인 서비스를 지원하는 원스톱 실시간 데이터 웨어하우스입니다.

이 노트북을 실행하려면 클라우드에서 실행 중인 Hologres 인스턴스가 필요합니다. 이 링크를 따라 인스턴스를 얻을 수 있습니다.

인스턴스를 만든 뒤에는 Hologres 콘솔로 다음 설정을 알아낼 수 있어야 합니다.

test_hologres_config = {
    "host": "<host>",
    "port": 80,
    "user": "<user>",
    "password": "<password>",
    "database": "<database>",
    "table_name": "<table_name>",
}

참고로, llama-index 설치가 필요합니다:

%pip install llama-index-vector-stores-hologres

!pip install llama-index

필요한 패키지 의존성 import (Import needed package dependencies):

from llama_index.core import (
    VectorStoreIndex,
    SimpleDirectoryReader,
    StorageContext,
)
from llama_index.vector_stores.hologres import HologresVectorStore

예제 데이터 로드 (Load some example data):

!mkdir -p 'data/paul_graham/'
!curl 'https://raw.githubusercontent.com/run-llama/llama_index/main/docs/examples/data/paul_graham/paul_graham_essay.txt' -o 'data/paul_graham/paul_graham_essay.txt'

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 75042  100 75042    0     0  31985      0  0:00:02  0:00:02 --:--:-- 31987

데이터 읽기 (Read the data):

# load documents
documents = SimpleDirectoryReader("./data/paul_graham/").load_data()
print(f"Total documents: {len(documents)}")
print(f"First document, id: {documents[0].doc_id}")
print(f"First document, hash: {documents[0].hash}")
print(
    "First document, text"
    f" ({len(documents[0].text)} characters):\n{'='*20}\n{documents[0].text[:360]} ..."
)

Total documents: 1
First document, id: 824dafc0-0aa1-4c80-b99c-33895cfc606a
First document, hash: 8430b3bdb65ee0a7853463b71e7e1e20beee3a3ce15ef3ec714919f8653b2eb9
First document, text (75014 characters):
====================




What I Worked On


February 2021


Before college the two main things I worked on, outside of school, were writing and programming. I didn't write essays. I wrote what beginning writers were supposed to write then, and probably still are: short stories. My stories were awful. They had hardly any plot, just characters with strong feelings, which I imagined ma ...

AnalyticDB Vector Store 객체 생성 (Create the AnalyticDB Vector Store object):

hologres_store = HologresVectorStore.from_param(
    host=test_hologres_config["host"],
    port=test_hologres_config["port"],
    user=test_hologres_config["user"],
    password=test_hologres_config["password"],
    database=test_hologres_config["database"],
    table_name=test_hologres_config["table_name"],
    embedding_dimension=1536,
    pre_delete_table=True,
)

문서로 인덱스 구축 (Build the Index from the Documents):

storage_context = StorageContext.from_defaults(vector_store=hologres_store)


index = VectorStoreIndex.from_documents(
    documents, storage_context=storage_context
)

인덱스로 쿼리 (Query using the index):

query_engine = index.as_query_engine()
response = query_engine.query("Why did the author choose to work on AI?")


print(response.response)

The author was inspired to work on AI due to the influence of a science fiction novel, "The Moon is a Harsh Mistress," which featured an intelligent computer named Mike, and a PBS documentary showcasing Terry Winograd's use of the SHRDLU program. These experiences led the author to believe that creating intelligent machines was an imminent reality and sparked their interest in the field of AI.

더 알아보기 (Learn more)

• Hologres 공식 문서
• Hologres 인스턴스 구매
• HologresVectorStore API 문서