Vector search workload
출처: 문서
본문
벡터 검색 워크로드
벡터 검색 워크로드는 인덱싱과 검색 작업 모두에 대해 OpenSearch의 벡터 엔진 기능을 벤치마킹해요. 다양한 벡터 검색 알고리즘, 양자화 방법, 인덱스 구성을 테스트하여 처리량, 지연 시간, 재현율 정확도 같은 성능 지표를 측정해요. 이 워크로드는 여러 데이터셋을 지원하며, 훈련된 방법과 훈련되지 않은 벡터 검색 방법을 모두 평가할 수 있어요.
이 워크로드는 HDF5 또는 BIG-ANN 형식의 데이터셋을 지원해요. 데이터셋을 다운로드하려면 이 링크를 사용해요.
지원되는 워크로드 매개변수
벡터 검색 워크로드에서 지원하는 워크로드 매개변수는 다음과 같아요.
| 이름 | 설명 |
|---|---|
target_index_name |
벡터를 추가할 인덱스의 이름이에요. |
target_field_name |
벡터를 추가할 필드의 이름이에요. 중첩된 필드를 나타내려면 '.'을 사용해요. |
target_index_body |
대상 인덱스 정의의 경로예요. |
target_index_primary_shards |
대상 인덱스의 기본 샤드예요. |
target_index_replica_shards |
대상 인덱스의 복제본 샤드예요. |
target_index_dimension |
대상 인덱스의 차원이에요. |
target_index_space_type |
대상 인덱스의 공간 유형이에요. |
target_index_bulk_size |
대상 인덱스의 벌크 크기예요. |
target_index_bulk_index_data_set_format |
벡터 데이터셋의 형식이에요. |
target_index_bulk_index_data_set_path |
인덱스에 있는 벡터 데이터셋의 경로예요. |
target_index_bulk_index_data_set_corpus |
벡터 데이터셋의 코퍼스 이름이에요. |
target_index_bulk_index_clients |
벌크 수집에 사용할 클라이언트예요. 데이터셋 크기의 약수여야 해요. |
target_index_max_num_segments |
검색을 시작하기 전에 대상 인덱스로 병합할 세그먼트 수예요. |
target_index_force_merge_timeout |
강제 병합 요청 전에 대기할 시간(초)이에요. |
hnsw_ef_search |
HNSW ef 검색 매개변수예요. |
hnsw_ef_construction |
HNSW ef 구성 매개변수예요. |
id_field_name |
인덱스에서 문서를 식별하는 데 사용할 필드의 이름이에요. |
hnsw_m |
HNSW m 매개변수예요. |
query_k |
검색에 반환할 이웃의 수예요. query_k, query_max_distance, query_min_score 중 하나만 제공할 수 있어요. |
query_max_distance |
벡터 검색에 반환할 최대 거리예요. query_k, query_max_distance, query_min_score 중 하나만 제공할 수 있어요. |
query_min_score |
벡터 검색에 반환할 최소 점수예요. query_k, query_max_distance, query_min_score 중 하나만 제공할 수 있어요. |
query_data_set_format |
쿼리에 사용되는 벡터 데이터셋의 형식이에요. query_k, query_max_distance, query_min_score 중 하나만 제공할 수 있어요. |
query_data_set_path |
쿼리에 사용되는 벡터 데이터셋의 경로예요. |
query_count |
검색 작업에 사용되는 쿼리 수예요. |
query_body |
검색 본문과 병합될 JSON 속성이에요. |
search_clients |
쿼리를 실행하는 데 사용되는 클라이언트 수예요. |
repetitions |
데이터셋이 소진될 때까지 완료되는 반복 횟수예요. 기본값은 1이에요. |
target_throughput |
각 쿼리 작업의 목표 처리량(초당 요청 수)이에요. 기본값은 10이에요. |
time_period |
벤치마크 테스트 실행에 할당된 시간(초)이에요. 기본값은 900이에요. |
테스트 절차
벡터 검색 워크로드는 다음 테스트 절차를 지원해요.
훈련 없음 테스트 절차
훈련 없음 테스트 절차는 훈련이 필요 없는 벡터 검색 인덱스를 테스트해요. 특정 엔진이나 공간 유형 지정과 같은 벡터 검색 알고리즘의 기본 구성을 메서드 정의로 정의할 수 있어요.
훈련 없음 테스트(인덱스 전용) 절차
훈련 없음 테스트(인덱스 전용) 절차는 훈련이 필요 없는 벡터 검색 인덱스를 인덱싱하는 데 사용돼요. 인덱싱 작업만 벤치마킹하려는 경우 특히 유용해요.
훈련 없음 테스트(Amazon OpenSearch Serverless)
Amazon OpenSearch Serverless용 훈련 없음 테스트 절차는 OpenSearch Serverless 벡터 검색 컬렉션에 특별히 사용돼요. 이 절차는 refresh 및 warmup 같은 작업을 포함하지 않는데, 벡터 검색 컬렉션에서 지원되지 않기 때문이에요.
강제 병합 인덱스 절차
강제 병합 인덱스 절차는 주어진 최대 세그먼트 수까지 강제 병합 작업을 수행하여 벡터 검색 인덱스를 최적화해요. 대규모 데이터셋의 경우 강제 병합은 비용이 많이 드는 작업이에요. 따라서 사용자 요구에 따라 별도의 절차를 사용하여 강제 병합 작업을 간헐적으로 트리거하는 것을 권장해요.
훈련 테스트 절차
훈련 테스트 절차는 훈련 단계가 필요한 근사 k-NN 검색 알고리즘을 벤치마킹해요. 예를 들어, Faiss Inverted File Indexing (IVF) 기법은 클러스터 벡터를 검색하기 위해 훈련 단계가 필요해요. 이 단계가 수행된 후 벤치마크는 전체 데이터셋 대신 더 적은 수의 클러스터 중심점을 검색할 수 있어요.
검색 절차
검색 절차는 이전에 인덱싱된 벡터 검색 인덱스를 벤치마킹해요. 대규모 데이터셋의 경우 로드 시간이 상당할 수 있으므로 매번 재인덱싱하지 않고 대규모 벡터 검색 인덱스를 벤치마킹하려는 경우 유용해요. 이 절차는 벡터 검색 중 콜드 스타트 문제를 피하기 위한 워밍업 작업을 포함해요.
사용자 정의 실행기
벡터 검색 워크로드에서 지원하는 사용자 정의 실행기는 warmup-knn-indices 하나뿐이에요. 이 실행기는 k-NN 인덱스를 워밍업하고 성공할 때까지 재시도해요.
워크로드 실행
벡터 검색 워크로드를 실행하려면 다음 명령어를 사용해요:
export ENDPOINT=<cluster-endpoint>
export PARAMS_FILE=<params-file-path>
opensearch-benchmark run \
--target-hosts $ENDPOINT \
--workload vectorsearch \
--workload-params ${PARAMS_FILE} \
--pipeline benchmark-only \
--kill-running-processes
{% include copy.html %}
샘플 결과
벡터 검색 워크로드를 사용하면 다음과 유사한 결과를 기대할 수 있어요.
훈련 테스트 절차
다음 예제는 훈련 테스트 절차의 결과를 제공해요:
------------------------------------------------------
_______ __ _____
/ ____(_)___ ____ _/ / / ___/_________ ________
/ /_ / / __ \/ __ `/ / \__ \/ ___/ __ \/ ___/ _ \
/ __/ / / / / / /_/ / / ___/ / /__/ /_/ / / / __/
/_/ /_/_/ /_/\__,_/_/ /____/\___/\____/_/ \___/
------------------------------------------------------
| Metric | Task | Value | Unit |
|---------------------------------------------------------------:|-------------------:|------------:|-------:|
| Cumulative indexing time of primary shards | | 0.00946667 | min |
| Min cumulative indexing time across primary shards | | 0 | min |
| Median cumulative indexing time across primary shards | | 0.00298333 | min |
| Max cumulative indexing time across primary shards | | 0.00336667 | min |
| Cumulative indexing throttle time of primary shards | | 0 | min |
| Min cumulative indexing throttle time across primary shards | | 0 | min |
| Median cumulative indexing throttle time across primary shards | | 0 | min |
| Max cumulative indexing throttle time across primary shards | | 0 | min |
| Cumulative merge time of primary shards | | 0 | min |
| Cumulative merge count of primary shards | | 0 | |
| Min cumulative merge time across primary shards | | 0 | min |
| Median cumulative merge time across primary shards | | 0 | min |
| Max cumulative merge time across primary shards | | 0 | min |
| Cumulative merge throttle time of primary shards | | 0 | min |
| Min cumulative merge throttle time across primary shards | | 0 | min |
| Median cumulative merge throttle time across primary shards | | 0 | min |
| Max cumulative merge throttle time across primary shards | | 0 | min |
| Cumulative refresh time of primary shards | | 0.00861667 | min |
| Cumulative refresh count of primary shards | | 33 | |
| Min cumulative refresh time across primary shards | | 0 | min |
| Median cumulative refresh time across primary shards | | 0.00268333 | min |
| Max cumulative refresh time across primary shards | | 0.00291667 | min |
| Cumulative flush time of primary shards | | 0.000183333 | min |
| Cumulative flush count of primary shards | | 2 | |
| Min cumulative flush time across primary shards | | 0 | min |
| Median cumulative flush time across primary shards | | 0 | min |
| Max cumulative flush time across primary shards | | 0.000183333 | min |
| Total Young Gen GC time | | 0.075 | s |
| Total Young Gen GC count | | 17 | |
| Total Old Gen GC time | | 0 | s |
| Total Old Gen GC count | | 0 | |
| Store size | | 0.00869293 | GB |
| Translog size | | 2.56114e-07 | GB |
| Heap used for segments | | 0 | MB |
| Heap used for doc values | | 0 | MB |
| Heap used for terms | | 0 | MB |
| Heap used for norms | | 0 | MB |
| Heap used for points | | 0 | MB |
| Heap used for stored fields | | 0 | MB |
| Segment count | | 9 | |
| Min Throughput | custom-vector-bulk | 25527 | docs/s |
| Mean Throughput | custom-vector-bulk | 25527 | docs/s |
| Median Throughput | custom-vector-bulk | 25527 | docs/s |
| Max Throughput | custom-vector-bulk | 25527 | docs/s |
| 50th percentile latency | custom-vector-bulk | 36.3095 | ms |
| 90th percentile latency | custom-vector-bulk | 52.2662 | ms |
| 100th percentile latency | custom-vector-bulk | 68.6513 | ms |
| 50th percentile service time | custom-vector-bulk | 36.3095 | ms |
| 90th percentile service time | custom-vector-bulk | 52.2662 | ms |
| 100th percentile service time | custom-vector-bulk | 68.6513 | ms |
| error rate | custom-vector-bulk | 0 | % |
| Min Throughput | prod-queries | 211.26 | ops/s |
| Mean Throughput | prod-queries | 213.85 | ops/s |
| Median Throughput | prod-queries | 213.48 | ops/s |
| Max Throughput | prod-queries | 216.49 | ops/s |
| 50th percentile latency | prod-queries | 3.43393 | ms |
| 90th percentile latency | prod-queries | 4.01881 | ms |
| 99th percentile latency | prod-queries | 5.56238 | ms |
| 99.9th percentile latency | prod-queries | 9.95666 | ms |
| 99.99th percentile latency | prod-queries | 39.7922 | ms |
| 100th percentile latency | prod-queries | 62.415 | ms |
| 50th percentile service time | prod-queries | 3.43405 | ms |
| 90th percentile service time | prod-queries | 4.0191 | ms |
| 99th percentile service time | prod-queries | 5.56316 | ms |
| 99.9th percentile service time | prod-queries | 9.95666 | ms |
| 99.99th percentile service time | prod-queries | 39.7922 | ms |
| 100th percentile service time | prod-queries | 62.415 | ms |
| error rate | prod-queries | 0 | % |
------------------------------------
[INFO] ✅ SUCCESS (took 119 seconds)
------------------------------------
Faiss 결과
다음 샘플 출력은 Faiss IVF 벤치마킹 절차를 사용하여 생성되었어요. 간결함을 위해 테스트는 매개변수 파일에 지정된 10,000개 대신 100개의 검색 쿼리를 사용했어요. 다른 모든 매개변수는 params/train 폴더의 매개변수와 동일해요. 첫 번째 실행은 양자화 없는 결과를 보여주고, 두 번째 실행은 스칼라 양자화, 세 번째 실행은 곱 양자화를 보여줘요. 양자화로 인해 검색 재현율이 떨어질 수 있어요.
양자화 없는 Faiss IVF/플랫 인코딩
| Metric | Task | Value | Unit |
|---------------------------------------------------------------:|-------------------------:|------------:|-------:|
| Cumulative indexing time of primary shards | | 11.7662 | min |
| Min cumulative indexing time across primary shards | | 0.000266667 | min |
| Median cumulative indexing time across primary shards | | 0.1423 | min |
| Max cumulative indexing time across primary shards | | 11.6236 | min |
| Cumulative indexing throttle time of primary shards | | 0 | min |
| Min cumulative indexing throttle time across primary shards | | 0 | min |
| Median cumulative indexing throttle time across primary shards | | 0 | min |
| Max cumulative indexing throttle time across primary shards | | 0 | min |
| Cumulative merge time of primary shards | | 1.09872 | min |
| Cumulative merge count of primary shards | | 21 | |
| Min cumulative merge time across primary shards | | 0 | min |
| Median cumulative merge time across primary shards | | 0.00045 | min |
| Max cumulative merge time across primary shards | | 1.09827 | min |
| Cumulative merge throttle time of primary shards | | 0.872417 | min |
| Min cumulative merge throttle time across primary shards | | 0 | min |
| Median cumulative merge throttle time across primary shards | | 0 | min |
| Max cumulative merge throttle time across primary shards | | 0.872417 | min |
| Cumulative refresh time of primary shards | | 0.113733 | min |
| Cumulative refresh count of primary shards | | 59 | |
| Min cumulative refresh time across primary shards | | 0.00235 | min |
| Median cumulative refresh time across primary shards | | 0.00516667 | min |
| Max cumulative refresh time across primary shards | | 0.106217 | min |
| Cumulative flush time of primary shards | | 0.01685 | min |
| Cumulative flush count of primary shards | | 8 | |
| Min cumulative flush time across primary shards | | 0 | min |
| Median cumulative flush time across primary shards | | 0.00791667 | min |
| Max cumulative flush time across primary shards | | 0.00893333 | min |
| Total Young Gen GC time | | 5.442 | s |
| Total Young Gen GC count | | 3739 | |
| Total Old Gen GC time | | 0 | s |
| Total Old Gen GC count | | 0 | |
| Store size | | 1.3545 | GB |
| Translog size | | 0.0304573 | GB |
| Heap used for segments | | 0 | MB |
| Heap used for doc values | | 0 | MB |
| Heap used for terms | | 0 | MB |
| Heap used for norms | | 0 | MB |
| Heap used for points | | 0 | MB |
| Heap used for stored fields | | 0 | MB |
| Segment count | | 14 | |
| Min Throughput | custom-vector-bulk-train | 32222.6 | docs/s |
| Mean Throughput | custom-vector-bulk-train | 32222.6 | docs/s |
| Median Throughput | custom-vector-bulk-train | 32222.6 | docs/s |
| Max Throughput | custom-vector-bulk-train | 32222.6 | docs/s |
| 50th percentile latency | custom-vector-bulk-train | 26.5199 | ms |
| 90th percentile latency | custom-vector-bulk-train | 34.9823 | ms |
| 99th percentile latency | custom-vector-bulk-train | 196.712 | ms |
| 100th percentile latency | custom-vector-bulk-train | 230.342 | ms |
| 50th percentile service time | custom-vector-bulk-train | 26.5158 | ms |
| 90th percentile service time | custom-vector-bulk-train | 34.9823 | ms |
| 99th percentile service time | custom-vector-bulk-train | 196.712 | ms |
| 100th percentile service time | custom-vector-bulk-train | 230.342 | ms |
| error rate | custom-vector-bulk-train | 0 | % |
| Min Throughput | delete-model | 10.58 | ops/s |
| Mean Throughput | delete-model | 10.58 | ops/s |
| Median Throughput | delete-model | 10.58 | ops/s |
| Max Throughput | delete-model | 10.58 | ops/s |
| 100th percentile latency | delete-model | 93.6958 | ms |
| 100th percentile service time | delete-model | 93.6958 | ms |
| error rate | delete-model | 0 | % |
| Min Throughput | train-knn-model | 0.63 | ops/s |
| Mean Throughput | train-knn-model | 0.63 | ops/s |
| Median Throughput | train-knn-model | 0.63 | ops/s |
| Max Throughput | train-knn-model | 0.63 | ops/s |
| 100th percentile latency | train-knn-model | 1577.49 | ms |
| 100th percentile service time | train-knn-model | 1577.49 | ms |
| error rate | train-knn-model | 0 | % |
| Min Throughput | custom-vector-bulk | 11055 | docs/s |
| Mean Throughput | custom-vector-bulk | 14163.8 | docs/s |
| Median Throughput | custom-vector-bulk | 12878.9 | docs/s |
| Max Throughput | custom-vector-bulk | 33841.3 | docs/s |
| 50th percentile latency | custom-vector-bulk | 81.6677 | ms |
| 90th percentile latency | custom-vector-bulk | 117.848 | ms |
| 99th percentile latency | custom-vector-bulk | 202.484 | ms |
| 99.9th percentile latency | custom-vector-bulk | 406.209 | ms |
| 99.99th percentile latency | custom-vector-bulk | 458.823 | ms |
| 100th percentile latency | custom-vector-bulk | 459.417 | ms |
| 50th percentile service time | custom-vector-bulk | 81.6621 | ms |
| 90th percentile service time | custom-vector-bulk | 117.843 | ms |
| 99th percentile service time | custom-vector-bulk | 202.294 | ms |
| 99.9th percentile service time | custom-vector-bulk | 406.209 | ms |
| 99.99th percentile service time | custom-vector-bulk | 458.823 | ms |
| 100th percentile service time | custom-vector-bulk | 459.417 | ms |
| error rate | custom-vector-bulk | 0 | % |
| Min Throughput | force-merge-segments | 0.1 | ops/s |
| Mean Throughput | force-merge-segments | 0.1 | ops/s |
| Median Throughput | force-merge-segments | 0.1 | ops/s |
| Max Throughput | force-merge-segments | 0.1 | ops/s |
| 100th percentile latency | force-merge-segments | 10017.4 | ms |
| 100th percentile service time | force-merge-segments | 10017.4 | ms |
| error rate | force-merge-segments | 0 | % |
| Min Throughput | warmup-indices | 9.63 | ops/s |
| Mean Throughput | warmup-indices | 9.63 | ops/s |
| Median Throughput | warmup-indices | 9.63 | ops/s |
| Max Throughput | warmup-indices | 9.63 | ops/s |
| 100th percentile latency | warmup-indices | 103.228 | ms |
| 100th percentile service time | warmup-indices | 103.228 | ms |
| error rate | warmup-indices | 0 | % |
| Min Throughput | prod-queries | 120.06 | ops/s |
| Mean Throughput | prod-queries | 120.06 | ops/s |
| Median Throughput | prod-queries | 120.06 | ops/s |
| Max Throughput | prod-queries | 120.06 | ops/s |
| 50th percentile latency | prod-queries | 1.75219 | ms |
| 90th percentile latency | prod-queries | 2.29527 | ms |
| 99th percentile latency | prod-queries | 50.4419 | ms |
| 100th percentile latency | prod-queries | 97.9905 | ms |
| 50th percentile service time | prod-queries | 1.75219 | ms |
| 90th percentile service time | prod-queries | 2.29527 | ms |
| 99th percentile service time | prod-queries | 50.4419 | ms |
| 100th percentile service time | prod-queries | 97.9905 | ms |
| error rate | prod-queries | 0 | % |
| Mean recall@k | prod-queries | 0.96 | |
| Mean recall@1 | prod-queries | 0.99 | |
------------------------------------
[INFO] ✅ SUCCESS (took 218 seconds)
------------------------------------
스칼라 양자화를 사용한 Faiss IVF(검색 쿼리 100개)
| Metric | Task | Value | Unit |
|---------------------------------------------------------------:|-------------------------:|------------:|-------:|
| Cumulative indexing time of primary shards | | 11.5 | min |
| Min cumulative indexing time across primary shards | | 0.000283333 | min |
| Median cumulative indexing time across primary shards | | 0.10915 | min |
| Max cumulative indexing time across primary shards | | 11.3905 | min |
| Cumulative indexing throttle time of primary shards | | 0 | min |
| Min cumulative indexing throttle time across primary shards | | 0 | min |
| Median cumulative indexing throttle time across primary shards | | 0 | min |
| Max cumulative indexing throttle time across primary shards | | 0 | min |
| Cumulative merge time of primary shards | | 1.03638 | min |
| Cumulative merge count of primary shards | | 22 | |
| Min cumulative merge time across primary shards | | 0 | min |
| Median cumulative merge time across primary shards | | 0.000266667 | min |
| Max cumulative merge time across primary shards | | 1.03612 | min |
| Cumulative merge throttle time of primary shards | | 0.798767 | min |
| Min cumulative merge throttle time across primary shards | | 0 | min |
| Median cumulative merge throttle time across primary shards | | 0 | min |
| Max cumulative merge throttle time across primary shards | | 0.798767 | min |
| Cumulative refresh time of primary shards | | 0.107117 | min |
| Cumulative refresh count of primary shards | | 61 | |
| Min cumulative refresh time across primary shards | | 0.00236667 | min |
| Median cumulative refresh time across primary shards | | 0.00543333 | min |
| Max cumulative refresh time across primary shards | | 0.0993167 | min |
| Cumulative flush time of primary shards | | 0.0193167 | min |
| Cumulative flush count of primary shards | | 9 | |
| Min cumulative flush time across primary shards | | 0 | min |
| Median cumulative flush time across primary shards | | 0.00871667 | min |
| Max cumulative flush time across primary shards | | 0.0106 | min |
| Total Young Gen GC time | | 5.267 | s |
| Total Young Gen GC count | | 3688 | |
| Total Old Gen GC time | | 0 | s |
| Total Old Gen GC count | | 0 | |
| Store size | | 1.11609 | GB |
| Translog size | | 0.0304573 | GB |
| Heap used for segments | | 0 | MB |
| Heap used for doc values | | 0 | MB |
| Heap used for terms | | 0 | MB |
| Heap used for norms | | 0 | MB |
| Heap used for points | | 0 | MB |
| Heap used for stored fields | | 0 | MB |
| Segment count | | 18 | |
| Min Throughput | custom-vector-bulk-train | 35950.5 | docs/s |
| Mean Throughput | custom-vector-bulk-train | 35950.5 | docs/s |
| Median Throughput | custom-vector-bulk-train | 35950.5 | docs/s |
| Max Throughput | custom-vector-bulk-train | 35950.5 | docs/s |
| 50th percentile latency | custom-vector-bulk-train | 22.8328 | ms |
| 90th percentile latency | custom-vector-bulk-train | 34.864 | ms |
| 99th percentile latency | custom-vector-bulk-train | 99.471 | ms |
| 100th percentile latency | custom-vector-bulk-train | 210.424 | ms |
| 50th percentile service time | custom-vector-bulk-train | 22.823 | ms |
| 90th percentile service time | custom-vector-bulk-train | 34.864 | ms |
| 99th percentile service time | custom-vector-bulk-train | 99.471 | ms |
| 100th percentile service time | custom-vector-bulk-train | 210.424 | ms |
| error rate | custom-vector-bulk-train | 0 | % |
| Min Throughput | delete-model | 8.39 | ops/s |
| Mean Throughput | delete-model | 8.39 | ops/s |
| Median Throughput | delete-model | 8.39 | ops/s |
| Max Throughput | delete-model | 8.39 | ops/s |
| 100th percentile latency | delete-model | 118.241 | ms |
| 100th percentile service time | delete-model | 118.241 | ms |
| error rate | delete-model | 0 | % |
| Min Throughput | train-knn-model | 0.64 | ops/s |
| Mean Throughput | train-knn-model | 0.64 | ops/s |
| Median Throughput | train-knn-model | 0.64 | ops/s |
| Max Throughput | train-knn-model | 0.64 | ops/s |
| 100th percentile latency | train-knn-model | 1564.44 | ms |
| 100th percentile service time | train-knn-model | 1564.44 | ms |
| error rate | train-knn-model | 0 | % |
| Min Throughput | custom-vector-bulk | 11313.1 | docs/s |
| Mean Throughput | custom-vector-bulk | 14065.7 | docs/s |
| Median Throughput | custom-vector-bulk | 12894.8 | docs/s |
| Max Throughput | custom-vector-bulk | 30050.8 | docs/s |
| 50th percentile latency | custom-vector-bulk | 81.4293 | ms |
| 90th percentile latency | custom-vector-bulk | 111.812 | ms |
| 99th percentile latency | custom-vector-bulk | 196.45 | ms |
| 99.9th percentile latency | custom-vector-bulk | 370.543 | ms |
| 99.99th percentile latency | custom-vector-bulk | 474.156 | ms |
| 100th percentile latency | custom-vector-bulk | 499.048 | ms |
| 50th percentile service time | custom-vector-bulk | 81.4235 | ms |
| 90th percentile service time | custom-vector-bulk | 111.833 | ms |
| 99th percentile service time | custom-vector-bulk | 197.125 | ms |
| 99.9th percentile service time | custom-vector-bulk | 370.543 | ms |
| 99.99th percentile service time | custom-vector-bulk | 474.156 | ms |
| 100th percentile service time | custom-vector-bulk | 499.048 | ms |
| error rate | custom-vector-bulk | 0 | % |
| Min Throughput | force-merge-segments | 0.1 | ops/s |
| Mean Throughput | force-merge-segments | 0.1 | ops/s |
| Median Throughput | force-merge-segments | 0.1 | ops/s |
| Max Throughput | force-merge-segments | 0.1 | ops/s |
| 100th percentile latency | force-merge-segments | 10015.2 | ms |
| 100th percentile service time | force-merge-segments | 10015.2 | ms |
| error rate | force-merge-segments | 0 | % |
| Min Throughput | warmup-indices | 19 | ops/s |
| Mean Throughput | warmup-indices | 19 | ops/s |
| Median Throughput | warmup-indices | 19 | ops/s |
| Max Throughput | warmup-indices | 19 | ops/s |
| 100th percentile latency | warmup-indices | 52.1685 | ms |
| 100th percentile service time | warmup-indices | 52.1685 | ms |
| error rate | warmup-indices | 0 | % |
| Min Throughput | prod-queries | 159.49 | ops/s |
| Mean Throughput | prod-queries | 159.49 | ops/s |
| Median Throughput | prod-queries | 159.49 | ops/s |
| Max Throughput | prod-queries | 159.49 | ops/s |
| 50th percentile latency | prod-queries | 1.92377 | ms |
| 90th percentile latency | prod-queries | 2.63867 | ms |
| 99th percentile latency | prod-queries | 48.513 | ms |
| 100th percentile latency | prod-queries | 90.543 | ms |
| 50th percentile service time | prod-queries | 1.92377 | ms |
| 90th percentile service time | prod-queries | 2.63867 | ms |
| 99th percentile service time | prod-queries | 48.513 | ms |
| 100th percentile service time | prod-queries | 90.543 | ms |
| error rate | prod-queries | 0 | % |
| Mean recall@k | prod-queries | 0.96 | |
| Mean recall@1 | prod-queries | 0.98 | |
------------------------------------
[INFO] ✅ SUCCESS (took 218 seconds)
------------------------------------
곱 양자화를 사용한 Faiss IVF(검색 쿼리 100개)
| Metric | Task | Value | Unit |
|---------------------------------------------------------------:|-------------------------:|------------:|-------:|
| Cumulative indexing time of primary shards | | 11.3862 | min |
| Min cumulative indexing time across primary shards | | 0.0003 | min |
| Median cumulative indexing time across primary shards | | 0.12735 | min |
| Max cumulative indexing time across primary shards | | 11.2586 | min |
| Cumulative indexing throttle time of primary shards | | 0 | min |
| Min cumulative indexing throttle time across primary shards | | 0 | min |
| Median cumulative indexing throttle time across primary shards | | 0 | min |
| Max cumulative indexing throttle time across primary shards | | 0 | min |
| Cumulative merge time of primary shards | | 1.50842 | min |
| Cumulative merge count of primary shards | | 19 | |
| Min cumulative merge time across primary shards | | 0 | min |
| Median cumulative merge time across primary shards | | 0.000233333 | min |
| Max cumulative merge time across primary shards | | 1.50818 | min |
| Cumulative merge throttle time of primary shards | | 0.58095 | min |
| Min cumulative merge throttle time across primary shards | | 0 | min |
| Median cumulative merge throttle time across primary shards | | 0 | min |
| Max cumulative merge throttle time across primary shards | | 0.58095 | min |
| Cumulative refresh time of primary shards | | 0.2059 | min |
| Cumulative refresh count of primary shards | | 61 | |
| Min cumulative refresh time across primary shards | | 0.00238333 | min |
| Median cumulative refresh time across primary shards | | 0.00526667 | min |
| Max cumulative refresh time across primary shards | | 0.19825 | min |
| Cumulative flush time of primary shards | | 0.0254667 | min |
| Cumulative flush count of primary shards | | 10 | |
| Min cumulative flush time across primary shards | | 0 | min |
| Median cumulative flush time across primary shards | | 0.0118333 | min |
| Max cumulative flush time across primary shards | | 0.0136333 | min |
| Total Young Gen GC time | | 6.477 | s |
| Total Young Gen GC count | | 3565 | |
| Total Old Gen GC time | | 0 | s |
| Total Old Gen GC count | | 0 | |
| Store size | | 0.892541 | GB |
| Translog size | | 0.0304573 | GB |
| Heap used for segments | | 0 | MB |
| Heap used for doc values | | 0 | MB |
| Heap used for terms | | 0 | MB |
| Heap used for norms | | 0 | MB |
| Heap used for points | | 0 | MB |
| Heap used for stored fields | | 0 | MB |
| Segment count | | 21 | |
| Min Throughput | custom-vector-bulk-train | 31931 | docs/s |
| Mean Throughput | custom-vector-bulk-train | 31931 | docs/s |
| Median Throughput | custom-vector-bulk-train | 31931 | docs/s |
| Max Throughput | custom-vector-bulk-train | 31931 | docs/s |
| 50th percentile latency | custom-vector-bulk-train | 25.3297 | ms |
| 90th percentile latency | custom-vector-bulk-train | 35.3864 | ms |
| 99th percentile latency | custom-vector-bulk-train | 144.372 | ms |
| 100th percentile latency | custom-vector-bulk-train | 209.37 | ms |
| 50th percentile service time | custom-vector-bulk-train | 25.3226 | ms |
| 90th percentile service time | custom-vector-bulk-train | 35.3864 | ms |
| 99th percentile service time | custom-vector-bulk-train | 144.372 | ms |
| 100th percentile service time | custom-vector-bulk-train | 209.37 | ms |
| error rate | custom-vector-bulk-train | 0 | % |
| Min Throughput | delete-model | 8.65 | ops/s |
| Mean Throughput | delete-model | 8.65 | ops/s |
| Median Throughput | delete-model | 8.65 | ops/s |
| Max Throughput | delete-model | 8.65 | ops/s |
| 100th percentile latency | delete-model | 114.725 | ms |
| 100th percentile service time | delete-model | 114.725 | ms |
| error rate | delete-model | 0 | % |
| Min Throughput | train-knn-model | 0.03 | ops/s |
| Mean Throughput | train-knn-model | 0.03 | ops/s |
| Median Throughput | train-knn-model | 0.03 | ops/s |
| Max Throughput | train-knn-model | 0.03 | ops/s |
| 100th percentile latency | train-knn-model | 37222.2 | ms |
| 100th percentile service time | train-knn-model | 37222.2 | ms |
| error rate | train-knn-model | 0 | % |
| Min Throughput | custom-vector-bulk | 10669.3 | docs/s |
| Mean Throughput | custom-vector-bulk | 14468.6 | docs/s |
| Median Throughput | custom-vector-bulk | 12496.1 | docs/s |
| Max Throughput | custom-vector-bulk | 35027.8 | docs/s |
| 50th percentile latency | custom-vector-bulk | 74.2584 | ms |
| 90th percentile latency | custom-vector-bulk | 113.426 | ms |
| 99th percentile latency | custom-vector-bulk | 293.075 | ms |
| 99.9th percentile latency | custom-vector-bulk | 1774.41 | ms |
| 99.99th percentile latency | custom-vector-bulk | 1969.99 | ms |
| 100th percentile latency | custom-vector-bulk | 1971.29 | ms |
| 50th percentile service time | custom-vector-bulk | 74.2577 | ms |
| 90th percentile service time | custom-vector-bulk | 113.477 | ms |
| 99th percentile service time | custom-vector-bulk | 292.481 | ms |
| 99.9th percentile service time | custom-vector-bulk | 1774.41 | ms |
| 99.99th percentile service time | custom-vector-bulk | 1969.99 | ms |
| 100th percentile service time | custom-vector-bulk | 1971.29 | ms |
| error rate | custom-vector-bulk | 0 | % |
| Min Throughput | force-merge-segments | 0.05 | ops/s |
| Mean Throughput | force-merge-segments | 0.05 | ops/s |
| Median Throughput | force-merge-segments | 0.05 | ops/s |
| Max Throughput | force-merge-segments | 0.05 | ops/s |
| 100th percentile latency | force-merge-segments | 20015.2 | ms |
| 100th percentile service time | force-merge-segments | 20015.2 | ms |
| error rate | force-merge-segments | 0 | % |
| Min Throughput | warmup-indices | 47.06 | ops/s |
| Mean Throughput | warmup-indices | 47.06 | ops/s |
| Median Throughput | warmup-indices | 47.06 | ops/s |
| Max Throughput | warmup-indices | 47.06 | ops/s |
| 100th percentile latency | warmup-indices | 20.6798 | ms |
| 100th percentile service time | warmup-indices | 20.6798 | ms |
| error rate | warmup-indices | 0 | % |
| Min Throughput | prod-queries | 87.76 | ops/s |
| Mean Throughput | prod-queries | 87.76 | ops/s |
| Median Throughput | prod-queries | 87.76 | ops/s |
| Max Throughput | prod-queries | 87.76 | ops/s |
| 50th percentile latency | prod-queries | 1.81677 | ms |
| 90th percentile latency | prod-queries | 2.80454 | ms |
| 99th percentile latency | prod-queries | 51.2039 | ms |
| 100th percentile latency | prod-queries | 98.2032 | ms |
| 50th percentile service time | prod-queries | 1.81677 | ms |
| 90th percentile service time | prod-queries | 2.80454 | ms |
| 99th percentile service time | prod-queries | 51.2039 | ms |
| 100th percentile service time | prod-queries | 98.2032 | ms |
| error rate | prod-queries | 0 | % |
| Mean recall@k | prod-queries | 0.62 | |
| Mean recall@1 | prod-queries | 0.52 | |
------------------------------------
[INFO] ✅ SUCCESS (took 413 seconds)
------------------------------------