5분 만에 시맨틱 검색 엔진 만들기
5분 만에 시맨틱 검색 엔진 만들기 (tutorials-basics-search-beginners)
벡터 검색 엔진이 처음이라면 이 튜토리얼이 딱이에요. 5분 안에 공상과학(SF) 소설용 시맨틱 검색 엔진을 만들어 볼 거예요. 만들어 두면 다가오는 외계인의 위협에 대해 검색 엔진에 물어보고, 잠재적인 우주 공격에 대비할 책을 추천받을 수 있게 돼요.
출처: Qdrant 공식문서
| Time: 5 - 15 min | Level: Beginner |  |
|---|
이 튜토리얼에는 두 가지 버전이 있어요.
- 이 페이지의 버전은 Qdrant Cloud를 사용해요. Qdrant Cloud의 forever free 티어(신용카드 불필요)로 클라우드에서 클러스터를 배포하고 벡터 임베딩을 생성할 거예요.
- 또는 Qdrant를 직접 실행할 수도 있어요. 이 경우 클러스터와 벡터 임베딩 인프라를 직접 관리해야 해요. 이 옵션을 선호한다면 이 튜토리얼의 로컬 배포 버전을 확인해 보세요.
개요 (Overview)
Python을 사용한다면 이 Google Colab 노트북을 사용할 수 있어요.
1. Qdrant 클러스터 만들기
아직 Qdrant 클러스터가 없다면 다음 단계를 따라 만들어 보세요.
- 이메일, Google 또는 Github 자격 증명으로 Qdrant Cloud 계정에 등록해요.
- Create a Free Cluster에서 클러스터 이름을 입력하고 원하는 클라우드 제공자와 리전을 선택해요.
- Create Free Cluster를 클릭해요.
- 메시지가 표시되면 API key를 복사해 안전한 곳에 보관해요. 다시 표시되지 않으니까요.
- Cluster Endpoint를 복사해요.
https://xxx.cloud.qdrant.io형태여야 해요.
2. 클라이언트 연결 설정하기
먼저 선호하는 프로그래밍 언어용 Qdrant Client를 설치해요.
pip install qdrant-client
npm install @qdrant/js-client-rest
cargo add qdrant-client
// build.gradle
dependencies {
implementation("io.qdrant:client:+") // specify the desired version
}
dotnet add package Qdrant.Client
go get github.com/qdrant/go-client
이 라이브러리를 사용하면 코드에서 Qdrant와 상호작용할 수 있어요.
다음으로, 엔드포인트와 API 키를 사용해 Qdrant 클러스터에 클라이언트 연결을 만들어요.
from qdrant_client import QdrantClient, models
client = QdrantClient(
url=QDRANT_URL,
api_key=QDRANT_API_KEY,
cloud_inference=True
)
const client = new QdrantClient({
url: QDRANT_URL,
apiKey: QDRANT_API_KEY,
});
let client = Qdrant::from_url(QDRANT_URL)
.api_key(QDRANT_API_KEY)
.build()?;
QdrantClient client =
new QdrantClient(
QdrantGrpcClient.newBuilder(QDRANT_URL, 6334, true)
.withApiKey(QDRANT_API_KEY)
.build());
var client = new QdrantClient(
host: QDRANT_URL,
port: 6334,
https: true,
apiKey: QDRANT_API_KEY
);
client, err := qdrant.NewClient(&qdrant.Config{
Host: QDRANT_URL,
APIKey: QDRANT_API_KEY,
UseTLS: true,
})
QDRANT_URL과 QDRANT_API_KEY를 이전 단계에서 얻은 클러스터 엔드포인트와 API 키로 바꿔요. cloud_inference=True 파라미터는 Qdrant Cloud의 inference 기능을 활성화해서, 클러스터가 자체 임베딩 인프라를 관리할 필요 없이 벡터 임베딩을 생성할 수 있게 해줘요.
3. 컬렉션 만들기
Qdrant의 모든 데이터는 컬렉션 안에 정리돼요. 책을 저장할 것이므로 my_books라는 컬렉션을 만들어 볼게요.
COLLECTION_NAME="my_books"
client.create_collection(
collection_name=COLLECTION_NAME,
vectors_config=models.VectorParams(
size=384, # Vector size is defined by used model
distance=models.Distance.COSINE,
),
)
const collectionName = "my_books";
await client.createCollection(collectionName, {
vectors: {
size: 384, // Vector size is defined by used model
distance: "Cosine",
},
});
let collection_name = "my_books";
client
.create_collection(
CreateCollectionBuilder::new(collection_name)
.vectors_config(VectorParamsBuilder::new(384, Distance::Cosine)), // Vector size is defined by used model
)
.await?;
String COLLECTION_NAME = "my_books";
client.createCollectionAsync(COLLECTION_NAME,
VectorParams.newBuilder().setDistance(Distance.Cosine).setSize(384).build()).get();
string COLLECTION_NAME = "my_books";
await client.CreateCollectionAsync(
collectionName: COLLECTION_NAME,
vectorsConfig: new VectorParams { Size = 384, Distance = Distance.Cosine }
);
collectionName := "my_books"
client.CreateCollection(context.Background(), &qdrant.CreateCollection{
CollectionName: collectionName,
VectorsConfig: qdrant.NewVectorsConfig(&qdrant.VectorParams{
Size: 384, // Vector size is defined by used model
Distance: qdrant.Distance_Cosine,
}),
})
size파라미터는 컬렉션의 벡터 차원 수를 정의해요. 384는 이 튜토리얼에서 사용하는 임베딩 모델의 출력 차원 수에 해당해요.distance파라미터는 두 지점 사이의 거리를 측정하는 데 사용되는 함수를 지정해요.
4. 클러스터에 데이터 업로드하기
데이터셋은 공상과학 소설 목록으로 구성돼 있어요. 각 항목에는 제목, 저자, 출판 연도, 간단한 설명이 있어요.
documents = [
{
"name": "The Time Machine",
"description": "A man travels through time and witnesses the evolution of humanity.",
"author": "H.G. Wells",
"year": 1895,
},
{
"name": "Ender's Game",
"description": "A young boy is trained to become a military leader in a war against an alien race.",
"author": "Orson Scott Card",
"year": 1985,
},
{
"name": "Brave New World",
"description": "A dystopian society where people are genetically engineered and conditioned to conform to a strict social hierarchy.",
"author": "Aldous Huxley",
"year": 1932,
},
{
"name": "The Hitchhiker's Guide to the Galaxy",
"description": "A comedic science fiction series following the misadventures of an unwitting human and his alien friend.",
"author": "Douglas Adams",
"year": 1979,
},
{
"name": "Dune",
"description": "A desert planet is the site of political intrigue and power struggles.",
"author": "Frank Herbert",
"year": 1965,
},
{
"name": "Foundation",
"description": "A mathematician develops a science to predict the future of humanity and works to save civilization from collapse.",
"author": "Isaac Asimov",
"year": 1951,
},
{
"name": "Snow Crash",
"description": "A futuristic world where the internet has evolved into a virtual reality metaverse.",
"author": "Neal Stephenson",
"year": 1992,
},
{
"name": "Neuromancer",
"description": "A hacker is hired to pull off a near-impossible hack and gets pulled into a web of intrigue.",
"author": "William Gibson",
"year": 1984,
},
{
"name": "The War of the Worlds",
"description": "A Martian invasion of Earth throws humanity into chaos.",
"author": "H.G. Wells",
"year": 1898,
},
{
"name": "The Hunger Games",
"description": "A dystopian society where teenagers are forced to fight to the death in a televised spectacle.",
"author": "Suzanne Collins",
"year": 2008,
},
{
"name": "The Andromeda Strain",
"description": "A deadly virus from outer space threatens to wipe out humanity.",
"author": "Michael Crichton",
"year": 1969,
},
{
"name": "The Left Hand of Darkness",
"description": "A human ambassador is sent to a planet where the inhabitants are genderless and can change gender at will.",
"author": "Ursula K. Le Guin",
"year": 1969,
},
{
"name": "The Three-Body Problem",
"description": "Humans encounter an alien civilization that lives in a dying system.",
"author": "Liu Cixin",
"year": 2008,
},
]
const documents = [
{ name: "The Time Machine", description: "A man travels through time and witnesses the evolution of humanity.", author: "H.G. Wells", year: 1895 },
{ name: "Ender's Game", description: "A young boy is trained to become a military leader in a war against an alien race.", author: "Orson Scott Card", year: 1985 },
{ name: "Brave New World", description: "A dystopian society where people are genetically engineered and conditioned to conform to a strict social hierarchy.", author: "Aldous Huxley", year: 1932 },
{ name: "The Hitchhiker's Guide to the Galaxy", description: "A comedic science fiction series following the misadventures of an unwitting human and his alien friend.", author: "Douglas Adams", year: 1979 },
{ name: "Dune", description: "A desert planet is the site of political intrigue and power struggles.", author: "Frank Herbert", year: 1965 },
{ name: "Foundation", description: "A mathematician develops a science to predict the future of humanity and works to save civilization from collapse.", author: "Isaac Asimov", year: 1951 },
{ name: "Snow Crash", description: "A futuristic world where the internet has evolved into a virtual reality metaverse.", author: "Neal Stephenson", year: 1992 },
{ name: "Neuromancer", description: "A hacker is hired to pull off a near-impossible hack and gets pulled into a web of intrigue.", author: "William Gibson", year: 1984 },
{ name: "The War of the Worlds", description: "A Martian invasion of Earth throws humanity into chaos.", author: "H.G. Wells", year: 1898 },
{ name: "The Hunger Games", description: "A dystopian society where teenagers are forced to fight to the death in a televised spectacle.", author: "Suzanne Collins", year: 2008 },
{ name: "The Andromeda Strain", description: "A deadly virus from outer space threatens to wipe out humanity.", author: "Michael Crichton", year: 1969 },
{ name: "The Left Hand of Darkness", description: "A human ambassador is sent to a planet where the inhabitants are genderless and can change gender at will.", author: "Ursula K. Le Guin", year: 1969 },
{ name: "The Three-Body Problem", description: "Humans encounter an alien civilization that lives in a dying system.", author: "Liu Cixin", year: 2008 },
];
let documents = [
("The Time Machine", "A man travels through time and witnesses the evolution of humanity.", "H.G. Wells", 1895),
("Ender's Game", "A young boy is trained to become a military leader in a war against an alien race.", "Orson Scott Card", 1985),
("Brave New World", "A dystopian society where people are genetically engineered and conditioned to conform to a strict social hierarchy.", "Aldous Huxley", 1932),
("The Hitchhiker's Guide to the Galaxy", "A comedic science fiction series following the misadventures of an unwitting human and his alien friend.", "Douglas Adams", 1979),
("Dune", "A desert planet is the site of political intrigue and power struggles.", "Frank Herbert", 1965),
("Foundation", "A mathematician develops a science to predict the future of humanity and works to save civilization from collapse.", "Isaac Asimov", 1951),
("Snow Crash", "A futuristic world where the internet has evolved into a virtual reality metaverse.", "Neal Stephenson", 1992),
("Neuromancer", "A hacker is hired to pull off a near-impossible hack and gets pulled into a web of intrigue.", "William Gibson", 1984),
("The War of the Worlds", "A Martian invasion of Earth throws humanity into chaos.", "H.G. Wells", 1898),
("The Hunger Games", "A dystopian society where teenagers are forced to fight to the death in a televised spectacle.", "Suzanne Collins", 2008),
("The Andromeda Strain", "A deadly virus from outer space threatens to wipe out humanity.", "Michael Crichton", 1969),
("The Left Hand of Darkness", "A human ambassador is sent to a planet where the inhabitants are genderless and can change gender at will.", "Ursula K. Le Guin", 1969),
("The Three-Body Problem", "Humans encounter an alien civilization that lives in a dying system.", "Liu Cixin", 2008),
];
List<Map<String, Value>> payloads = List.of(
Map.of(
"name", value("The Time Machine"),
"description", value("A man travels through time and witnesses the evolution of humanity."),
"author", value("H.G. Wells"),
"year", value(1895)),
Map.of(
"name", value("Ender's Game"),
"description",
value("A young boy is trained to become a military leader in a war against an alien race."),
"author", value("Orson Scott Card"),
"year", value(1985)),
Map.of(
"name", value("Brave New World"),
"description",
value(
"A dystopian society where people are genetically engineered and conditioned to conform to a strict social hierarchy."),
"author", value("Aldous Huxley"),
"year", value(1932)),
Map.of(
"name", value("The Hitchhiker's Guide to the Galaxy"),
"description",
value(
"A comedic science fiction series following the misadventures of an unwitting human and his alien friend."),
"author", value("Douglas Adams"),
"year", value(1979)),
Map.of(
"name", value("Dune"),
"description", value("A desert planet is the site of political intrigue and power struggles."),
"author", value("Frank Herbert"),
"year", value(1965)),
Map.of(
"name", value("Foundation"),
"description",
value(
"A mathematician develops a science to predict the future of humanity and works to save civilization from collapse."),
"author", value("Isaac Asimov"),
"year", value(1951)),
Map.of(
"name", value("Snow Crash"),
"description",
value("A futuristic world where the internet has evolved into a virtual reality metaverse."),
"author", value("Neal Stephenson"),
"year", value(1992)),
Map.of(
"name", value("Neuromancer"),
"description",
value(
"A hacker is hired to pull off a near-impossible hack and gets pulled into a web of intrigue."),
"author", value("William Gibson"),
"year", value(1984)),
Map.of(
"name", value("The War of the Worlds"),
"description", value("A Martian invasion of Earth throws humanity into chaos."),
"author", value("H.G. Wells"),
"year", value(1898)),
Map.of(
"name", value("The Hunger Games"),
"description",
value("A dystopian society where teenagers are forced to fight to the death in a televised spectacle."),
"author", value("Suzanne Collins"),
"year", value(2008)),
Map.of(
"name", value("The Andromeda Strain"),
"description", value("A deadly virus from outer space threatens to wipe out humanity."),
"author", value("Michael Crichton"),
"year", value(1969)),
Map.of(
"name", value("The Left Hand of Darkness"),
"description",
value(
"A human ambassador is sent to a planet where the inhabitants are genderless and can change gender at will."),
"author", value("Ursula K. Le Guin"),
"year", value(1969)),
Map.of(
"name", value("The Three-Body Problem"),
"description", value("Humans encounter an alien civilization that lives in a dying system."),
"author", value("Liu Cixin"),
"year", value(2008)));
var payloads = new List<Dictionary<string, Value>>
{
new() { ["name"] = "The Time Machine", ["description"] = "A man travels through time and witnesses the evolution of humanity.", ["author"] = "H.G. Wells", ["year"] = 1895 },
new() { ["name"] = "Ender's Game", ["description"] = "A young boy is trained to become a military leader in a war against an alien race.", ["author"] = "Orson Scott Card", ["year"] = 1985 },
new() { ["name"] = "Brave New World", ["description"] = "A dystopian society where people are genetically engineered and conditioned to conform to a strict social hierarchy.", ["author"] = "Aldous Huxley", ["year"] = 1932 },
new() { ["name"] = "The Hitchhiker's Guide to the Galaxy", ["description"] = "A comedic science fiction series following the misadventures of an unwitting human and his alien friend.", ["author"] = "Douglas Adams", ["year"] = 1979 },
new() { ["name"] = "Dune", ["description"] = "A desert planet is the site of political intrigue and power struggles.", ["author"] = "Frank Herbert", ["year"] = 1965 },
new() { ["name"] = "Foundation", ["description"] = "A mathematician develops a science to predict the future of humanity and works to save civilization from collapse.", ["author"] = "Isaac Asimov", ["year"] = 1951 },
new() { ["name"] = "Snow Crash", ["description"] = "A futuristic world where the internet has evolved into a virtual reality metaverse.", ["author"] = "Neal Stephenson", ["year"] = 1992 },
new() { ["name"] = "Neuromancer", ["description"] = "A hacker is hired to pull off a near-impossible hack and gets pulled into a web of intrigue.", ["author"] = "William Gibson", ["year"] = 1984 },
new() { ["name"] = "The War of the Worlds", ["description"] = "A Martian invasion of Earth throws humanity into chaos.", ["author"] = "H.G. Wells", ["year"] = 1898 },
new() { ["name"] = "The Hunger Games", ["description"] = "A dystopian society where teenagers are forced to fight to the death in a televised spectacle.", ["author"] = "Suzanne Collins", ["year"] = 2008 },
new() { ["name"] = "The Andromeda Strain", ["description"] = "A deadly virus from outer space threatens to wipe out humanity.", ["author"] = "Michael Crichton", ["year"] = 1969 },
new() { ["name"] = "The Left Hand of Darkness", ["description"] = "A human ambassador is sent to a planet where the inhabitants are genderless and can change gender at will.", ["author"] = "Ursula K. Le Guin", ["year"] = 1969 },
new() { ["name"] = "The Three-Body Problem", ["description"] = "Humans encounter an alien civilization that lives in a dying system.", ["author"] = "Liu Cixin", ["year"] = 2008 }
};
documents := []map[string]any{
{
"name": "The Time Machine",
"description": "A man travels through time and witnesses the evolution of humanity.",
"author": "H.G. Wells",
"year": 1895,
},
{
"name": "Ender's Game",
"description": "A young boy is trained to become a military leader in a war against an alien race.",
"author": "Orson Scott Card",
"year": 1985,
},
{
"name": "Brave New World",
"description": "A dystopian society where people are genetically engineered and conditioned to conform to a strict social hierarchy.",
"author": "Aldous Huxley",
"year": 1932,
},
{
"name": "The Hitchhiker's Guide to the Galaxy",
"description": "A comedic science fiction series following the misadventures of an unwitting human and his alien friend.",
"author": "Douglas Adams",
"year": 1979,
},
{
"name": "Dune",
"description": "A desert planet is the site of political intrigue and power struggles.",
"author": "Frank Herbert",
"year": 1965,
},
{
"name": "Foundation",
"description": "A mathematician develops a science to predict the future of humanity and works to save civilization from collapse.",
"author": "Isaac Asimov",
"year": 1951,
},
{
"name": "Snow Crash",
"description": "A futuristic world where the internet has evolved into a virtual reality metaverse.",
"author": "Neal Stephenson",
"year": 1992,
},
{
"name": "Neuromancer",
"description": "A hacker is hired to pull off a near-impossible hack and gets pulled into a web of intrigue.",
"author": "William Gibson",
"year": 1984,
},
{
"name": "The War of the Worlds",
"description": "A Martian invasion of Earth throws humanity into chaos.",
"author": "H.G. Wells",
"year": 1898,
},
{
"name": "The Hunger Games",
"description": "A dystopian society where teenagers are forced to fight to the death in a televised spectacle.",
"author": "Suzanne Collins",
"year": 2008,
},
{
"name": "The Andromeda Strain",
"description": "A deadly virus from outer space threatens to wipe out humanity.",
"author": "Michael Crichton",
"year": 1969,
},
{
"name": "The Left Hand of Darkness",
"description": "A human ambassador is sent to a planet where the inhabitants are genderless and can change gender at will.",
"author": "Ursula K. Le Guin",
"year": 1969,
},
{
"name": "The Three-Body Problem",
"description": "Humans encounter an alien civilization that lives in a dying system.",
"author": "Liu Cixin",
"year": 2008,
},
}
각 책을 my_books 컬렉션의 포인트(point)로 저장해요. 각 포인트는 고유 ID, 설명에서 생성된 벡터, 그리고 책의 메타데이터를 담은 페이로드로 구성돼요.
EMBEDDING_MODEL="sentence-transformers/all-minilm-l6-v2"
client.upload_points(
collection_name=COLLECTION_NAME,
points=[
models.PointStruct(
id=idx,
vector=models.Document(
text=doc["description"],
model=EMBEDDING_MODEL
),
payload=doc
)
for idx, doc in enumerate(documents)
],
)
const embeddingModel = "sentence-transformers/all-minilm-l6-v2";
const points = documents.map((doc, idx) => ({
id: idx,
vector: {
text: doc.description,
model: embeddingModel,
},
payload: doc,
}));
await client.upsert(collectionName, { points });
let embedding_model = "sentence-transformers/all-minilm-l6-v2";
let points: Vec<PointStruct> = documents
.iter()
.enumerate()
.map(|(idx, (name, description, author, year))| {
PointStruct::new(
idx as u64,
Document::new(*description, embedding_model),
[
("name", (*name).into()),
("description", (*description).into()),
("author", (*author).into()),
("year", (*year).into()),
],
)
})
.collect();
client
.upsert_points(UpsertPointsBuilder::new(collection_name, points))
.await?;
String EMBEDDING_MODEL = "sentence-transformers/all-minilm-l6-v2";
List<PointStruct> points = new ArrayList<>();
for (int idx = 0; idx < payloads.size(); idx++) {
Map<String, Value> payload = payloads.get(idx);
String description = payload.get("description").getStringValue();
PointStruct point =
PointStruct.newBuilder()
.setId(id((long) idx))
.setVectors(
vectors(
vector(
Document.newBuilder()
.setText(description)
.setModel(EMBEDDING_MODEL)
.build())))
.putAllPayload(payload)
.build();
points.add(point);
}
client.upsertAsync(COLLECTION_NAME, points).get();
string EMBEDDING_MODEL = "sentence-transformers/all-minilm-l6-v2";
var points = new List<PointStruct>();
for (ulong idx = 0; idx < (ulong)payloads.Count; idx++)
{
var payload = payloads[(int)idx];
string description = payload["description"].StringValue;
var point = new PointStruct
{
Id = idx,
Vectors = new Document
{
Text = description,
Model = EMBEDDING_MODEL
},
Payload = { payload }
};
points.Add(point);
}
await client.UpsertAsync(
collectionName: COLLECTION_NAME,
points: points
);
embeddingModel := "sentence-transformers/all-minilm-l6-v2"
points := make([]*qdrant.PointStruct, len(documents))
for idx, doc := range documents {
points[idx] = &qdrant.PointStruct{
Id: qdrant.NewIDNum(uint64(idx)),
Vectors: qdrant.NewVectorsDocument(&qdrant.Document{
Text: doc["description"].(string),
Model: embeddingModel,
}),
Payload: qdrant.NewValueMap(doc),
}
}
client.Upsert(context.Background(), &qdrant.UpsertPoints{
CollectionName: collectionName,
Points: points,
})
이 코드는 Qdrant Cloud에 sentence-transformers/all-minilm-l6-v2 임베딩 모델을 사용해 책 설명에서 벡터 임베딩을 생성하도록 지시해요. 이는 Qdrant Cloud에서 사용 가능한 무료 모델 중 하나예요. 사용 가능한 무료·유료 모델 목록은 Qdrant Cloud Console의 Cluster Detail 페이지에서 Inference 탭을 참조하세요.
5. 엔진에 쿼리하기
이제 데이터가 Qdrant에 저장됐으니, 쿼리를 실행하고 의미상 관련된 결과를 받을 수 있어요.
hits = client.query_points(
collection_name=COLLECTION_NAME,
query=models.Document(
text="alien invasion",
model=EMBEDDING_MODEL
),
limit=3,
).points
for hit in hits:
print(hit.payload, "score:", hit.score)
const queryResult = await client.query(collectionName, {
query: {
text: "alien invasion",
model: embeddingModel,
},
limit: 3,
});
for (const hit of queryResult.points) {
console.log(hit.payload, "score:", hit.score);
}
let query_result = client
.query(
QueryPointsBuilder::new(collection_name)
.query(Query::new_nearest(Document::new(
"alien invasion",
embedding_model,
)))
.limit(3)
.with_payload(true),
)
.await?;
for hit in query_result.result {
println!("{:?} score: {}", hit.payload, hit.score);
}
QueryPoints request =
QueryPoints.newBuilder()
.setCollectionName(COLLECTION_NAME)
.setQuery(
nearest(
Document.newBuilder()
.setText("alien invasion")
.setModel(EMBEDDING_MODEL)
.build()))
.setLimit(3)
.build();
var hits = client.queryAsync(request).get();
for (var hit : hits) {
System.out.println(hit.getPayloadMap() + " score: " + hit.getScore());
}
var hits = await client.QueryAsync(
collectionName: COLLECTION_NAME,
query: new Document
{
Text = "alien invasion",
Model = EMBEDDING_MODEL
},
limit: 3
);
foreach (var hit in hits)
{
Console.WriteLine($"{hit.Payload} score: {hit.Score}");
}
queryResult, err := client.Query(context.Background(), &qdrant.QueryPoints{
CollectionName: collectionName,
Query: qdrant.NewQueryDocument(&qdrant.Document{
Text: "alien invasion",
Model: embeddingModel,
}),
Limit: qdrant.PtrOf(uint64(3)),
})
for _, hit := range queryResult {
fmt.Println(hit.Payload, "score:", hit.Score)
}
이 쿼리는 동일한 임베딩 모델을 사용해 "alien invasion" 쿼리에 대한 벡터를 생성해요. 그러면 검색 엔진이 컬렉션에서 가장 유사한 세 개의 벡터를 찾고, 해당 페이로드와 유사도 점수를 반환해요.
응답 (Response):
검색 엔진은 외계인 침략과 관련된 가장 관련성 높은 세 권의 책을 반환해요. 각각에는 쿼리와의 유사도를 나타내는 점수가 할당돼요.
{'name': 'The War of the Worlds', 'description': 'A Martian invasion of Earth throws humanity into chaos.', 'author': 'H.G. Wells', 'year': 1898} score: 0.570093257022374
{'name': "The Hitchhiker's Guide to the Galaxy", 'description': 'A comedic science fiction series following the misadventures of an unwitting human and his alien friend.', 'author': 'Douglas Adams', 'year': 1979} score: 0.5040468703143637
{'name': 'The Three-Body Problem', 'description': 'Humans encounter an alien civilization that lives in a dying system.', 'author': 'Liu Cixin', 'year': 2008} score: 0.45902943411768216
쿼리 좁히기 (Narrow down the Query)
2000년대 초의 가장 최근 책은 어때요? Qdrant는 필터를 적용해 쿼리 결과를 좁힐 수 있어요. 2000년 이후에 출판된 책만 필터링하려면 페이로드의 year 필드에 필터를 적용하면 돼요.
페이로드 필드에 필터링을 적용하기 전에, 그 필드에 대한 페이로드 인덱스를 먼저 만들어요.
client.create_payload_index(
collection_name=COLLECTION_NAME,
field_name="year",
field_schema=models.PayloadSchemaType.INTEGER,
)
await client.createPayloadIndex(collectionName, {
field_name: "year",
field_schema: "integer",
});
client
.create_field_index(
CreateFieldIndexCollectionBuilder::new(collection_name, "year", FieldType::Integer)
.wait(true),
)
.await?;
client
.createPayloadIndexAsync(
COLLECTION_NAME,
"year",
PayloadSchemaType.Integer,
null,
true,
null,
null)
.get();
await client.CreatePayloadIndexAsync(
collectionName: COLLECTION_NAME,
fieldName: "year",
schemaType: PayloadSchemaType.Integer
);
client.CreateFieldIndex(context.Background(), &qdrant.CreateFieldIndexCollection{
CollectionName: collectionName,
FieldName: "year",
FieldType: qdrant.FieldType_FieldTypeInteger.Enum(),
})
프로덕션 환경에서는 인덱싱의 이점을 최대화하기 위해 데이터를 업로드하기 전에 페이로드 인덱스를 만들어 두는 게 좋아요.
이제 쿼리에 필터를 적용할 수 있어요.
hits = client.query_points(
collection_name=COLLECTION_NAME,
query=models.Document(
text="alien invasion",
model=EMBEDDING_MODEL
),
query_filter=models.Filter(
must=[models.FieldCondition(key="year", range=models.Range(gte=2000))]
),
limit=1,
).points
for hit in hits:
print(hit.payload, "score:", hit.score)
const queryResultFiltered = await client.query(collectionName, {
query: {
text: "alien invasion",
model: embeddingModel,
},
filter: {
must: [
{
key: "year",
range: {
gte: 2000,
},
},
],
},
limit: 1,
});
for (const hit of queryResultFiltered.points) {
console.log(hit.payload, "score:", hit.score);
}
let query_result_filtered = client
.query(
QueryPointsBuilder::new(collection_name)
.query(Query::new_nearest(Document::new(
"alien invasion",
embedding_model,
)))
.filter(Filter::must([Condition::range(
"year",
Range {
gte: Some(2000.0),
..Default::default()
},
)]))
.limit(1)
.with_payload(true),
)
.await?;
for hit in query_result_filtered.result {
println!("{:?} score: {}", hit.payload, hit.score);
}
QueryPoints filteredRequest =
QueryPoints.newBuilder()
.setCollectionName(COLLECTION_NAME)
.setQuery(
nearest(
Document.newBuilder()
.setText("alien invasion")
.setModel(EMBEDDING_MODEL)
.build()))
.setFilter(
Filter.newBuilder()
.addMust(range("year", Range.newBuilder().setGte(2000.0).build()))
.build())
.setLimit(1)
.build();
var filteredHits = client.queryAsync(filteredRequest).get();
for (var hit : filteredHits) {
System.out.println(hit.getPayloadMap() + " score: " + hit.getScore());
}
var filteredHits = await client.QueryAsync(
collectionName: COLLECTION_NAME,
query: new Document
{
Text = "alien invasion",
Model = EMBEDDING_MODEL
},
filter: new Filter
{
Must = { Range("year", new Qdrant.Client.Grpc.Range { Gte = 2000.0 }) }
},
limit: 1
);
foreach (var hit in filteredHits)
{
Console.WriteLine($"{hit.Payload} score: {hit.Score}");
}
queryResultFiltered, err := client.Query(context.Background(), &qdrant.QueryPoints{
CollectionName: collectionName,
Query: qdrant.NewQueryDocument(&qdrant.Document{
Text: "alien invasion",
Model: embeddingModel,
}),
Filter: &qdrant.Filter{
Must: []*qdrant.Condition{
qdrant.NewRange("year", &qdrant.Range{
Gte: qdrant.PtrOf(2000.0),
}),
},
},
Limit: qdrant.PtrOf(uint64(1)),
})
for _, hit := range queryResultFiltered {
fmt.Println(hit.Payload, "score:", hit.Score)
}
응답 (Response):
결과가 2008년의 한 건으로 좁혀졌어요.
{'name': 'The Three-Body Problem', 'description': 'Humans encounter an alien civilization that lives in a dying system.', 'author': 'Liu Cixin', 'year': 2008} score: 0.45902943411768216
다음 단계 (Next Steps)
축하해요. 여러분만의 첫 검색 엔진을 만들었어요! 믿어 주세요. Qdrant의 나머지도 그리 복잡하지 않아요. 다음 튜토리얼로는 하이브리드 검색 서비스 만들기를 시도하거나, 무료 Qdrant Essentials 코스를 들어 보세요.