After reducing cache-fill overhead, we inspected three Qdrant operations used by a filtered, sorted search: exact counting, checking indexed-field coverage, and retrieving an ordered page. The diagnostic found that exact counting had the largest single-client server-reported time of those operations. It also showed a large gap between server-reported timings and parallel client wall time under concurrency. Neither observation identifies the entire gateway bottleneck.

This is not an Elasticsearch-compatible or native-engine comparison. The runner sends Qdrant RPCs directly inside the container network. It bypasses the gateway, catalog, response cache and public client path. Its Python client is not the gateway’s Rust client.

Three pieces of work

The requests use a range filter over the retained 10,000-document corpus. The count is exact. The coverage probe asks whether any matching document lacks the expected indexed-field marker. The ordered request retrieves 21 full-source documents with their document IDs in ascending rank order. All results are checked, including counts, ordering, document IDs and full sources.

A separate payload-schema snapshot captured after the diagnostic found an integer index on rank and a keyword index on the coverage marker, each covering all 10,000 points. It does not support a missing-rank-index explanation for the observed count cost, but it is not contemporaneous proof of the index state during timing.

These are pooled p50 Qdrant response time fields, in milliseconds, from 400 measured calls per configuration:

RPC Run alone, one / eight clients Run in parallel, one / eight clients
Exact count 0.698 / 0.892 0.734 / 0.873
Coverage probe 0.163 / 0.298 0.180 / 0.320
Ordered page 0.136 / 0.190 0.170 / 0.263

The parallel group waits for all three operations and includes Python dispatch, JSON parsing and correctness validation. Its median trial p50/p95 was 1.023 / 1.161 ms with one client and 4.426 / 41.780 ms with eight clients. Per-RPC client wall time ends when the response body is received; it has a different boundary from both group time and the server’s reported field.

Do not add parallel quantiles or subtract these independently aggregated timings to estimate gateway overhead. Network transit, client scheduling, executor work, validation and server queueing can contribute at different boundaries. This experiment does not isolate their individual costs or prove count is the sole bottleneck. The full summary retains standalone and parallel group quantiles plus each RPC’s client and server-reported quantiles.

Resource context and reproducibility

All 4,800 measured RPC responses across 3,200 groups passed checks. Four trials rotate the operation order at each concurrency level. Connections are persistent per worker and RPC. The isolated Python generator has two CPUs and 256 MiB, outside the server budget; its execution model can influence group wall time.

Qdrant 1.19.2 retained 1.25 CPUs and 3,328 MiB. The collection was green at both sampled boundaries, with 10,000 points and two segments. Search-pool telemetry reported high-I/O mode and two/eight pool budgets at both boundaries. Across the whole capture, Qdrant was throttled in 13 of 32 scheduling periods (40.6%), with 1.107 seconds of reported throttled time. Other recorded services had no throttled periods. These whole-run counters do not assign the long parallel tail to a particular request or exclude client effects.

Use the prepared harness at revision 47113fc with its retained corpus and resource settings. Copy the launcher and RPC runner into benchmark/shared-cache/, then run from the repository root:

python3 benchmark/shared-cache/backend_diagnostic.py --prefix backend-decomposition

The launcher requires the recorded Python image digest to be available, checks the prepared corpus and resource limits, and refuses to overwrite evidence. It creates a restricted temporary runner without changing the engines. The manifest records the image provenance, exact runner hashes, raw samples and resource snapshots. The two diagnostic scripts were uncommitted at execution; their exact copies are retained. The gateway image was built from 47113fc, but the gateway was not in this measured request path. Only ephemeral container IDs were removed from public records.

The next useful question is which changes reduce complete request latency while preserving exact counts and result correctness. These measurements help choose experiments; they are not evidence of a new production speedup.