mcpField
Q
qdrant-minimize-latency

qdrant-minimize-latency

github.com/qdrantData & databases26541 installsUpdated 1d ago
Repository

About qdrant-minimize-latency

qdrant-minimize-latency is a Qdrant tuning skill focused on reducing single-query latency. It covers segment sizing, in-memory indexes, query parameters, storage, and RAM management, while distinguishing latency goals from throughput optimization.

Key features

  1. 01Tune segment count for CPU cores
  2. 02Keep quantized vectors and HNSW in RAM
  3. 03Trade recall for query speed
  4. 04Use local NVMe storage
  5. 05Manage RAM pressure and background indexing

Use cases

  1. 01Reduce latency for single-query workloads
  2. 02Diagnose RAM-related latency degradation
  3. 03Tune Qdrant during peak indexing periods

Install

Copy the command and run it in a terminal.

$npx skills add https://github.com/qdrant/skills --skill qdrant-minimize-latency

After installing, type/qdrant-minimize-latencyin Claude Code to confirm it loaded.

Details

From the project README

Scaling for Query Latency

Latency of a single query is determined by the slowest component in the query execution path. It is sometimes correlated with throughput, but not always — throughput and latency are opposite tuning directions.

Low latency optimization is aimed at utilising maximum resource saturation for a single query, while throughput optimization is aimed at minimizing per-query resource usage to allow more parallel queries.

Performance Tuning for Lower Latency

  • Increase segment count to match CPU cores (default_segment_number: 16) Minimizing latency
  • Keep quantized vectors and HNSW in RAM: memory: pinned on Qdrant 1.19 or newer; on 1.18 or older, always_ram: true for quantized vectors and hnsw_config.on_disk: false for HNSW Memory tiers
  • Reduce hnsw_ef at query time (trade recall for speed) Search params
  • Use local NVMe, avoid network-attached storage

Memory Pressure and Latency

RAM is the most critical resource for latency. If working set exceeds available RAM, OS cache eviction causes severe, sustained latency degradation.

  • Vertical scale RAM first. Critical if working set >80%.
  • Use quantization: scalar (4x reduction) or binary (up to 32x reduction) Quantization
  • Move payload indexes to disk if filtering is infrequent: memory: cold on Qdrant 1.19 or newer, on_disk: true on 1.18 or older On-disk payload index
  • Set optimizer_cpu_budget to limit background optimization CPUs
  • Schedule indexing: set high indexing_threshold during peak hours

Vertical Scaling for Latency

More RAM and faster CPU directly reduce latency. See Vertical Scaling for node sizing guidelines.

What NOT to Do

  • Do not expect to optimize latency and throughput simultaneously on the same node
  • Do not use few large segments for latency-sensitive workloads (each segment takes longer to search)
  • Do not run at >90% RAM (cache eviction causes severe latency degradation that can last days)
  • Do not ignore optimizer status during performance debugging
  • Do not scale down RAM without load testing (cache eviction causes days-long latency incidents)

Comments

Markdown supported