Compatibility and semantic differences

Compatibility and semantic differences

Supported requests cover CRUD, mappings, bulk ingestion, common index lifecycle calls, multi-search, cursor pagination, basic lexical search, payload filters, source projection, count, complete filter-only aggregations, health, and durable aliases. Request bodies must be valid UTF-8; malformed byte sequences are rejected before JSON or NDJSON parsing instead of being stored as Unicode replacement characters. Index creation enforces the static name rules shared by Elasticsearch 9.5.5 and OpenSearch 3.9: lowercase UTF-8 names of at most 255 bytes, excluding their reserved punctuation, the leading _/-/+ forms, and the exact names . and ... Invalid names return HTTP 400 with invalid_index_name_exception before catalogue or Qdrant work. Dot-prefixed names other than . and .. remain accepted; the gateway does not emulate engine-specific deprecation warnings, hidden-index settings, or system-index ownership. Creating an index that already exists returns HTTP 400 with resource_already_exists_exception before Qdrant work; it does not replace the stored mapping. Names that collide only after Qdrant collection normalization remain a distinct gateway validation error. PUT /<index>/_mapping merges new non-text properties into the existing mapping and creates Qdrant payload indexes for supported filterable types; identical existing definitions are idempotent, while type changes and new text fields are rejected. New text fields require a new index because the pinned Qdrant 1.15 schema cannot add their named sparse vectors to an existing collection. Both POST and PUT bulk requests are accepted. Bulk update and delete actions require an explicit _id; every action line must name exactly one supported action with object metadata and string _index/_id values. Malformed actions fail before any item is executed instead of assigning an unrelated generated ID or partially applying the request. Standalone and bulk index operations check the authoritative document store so a new document returns HTTP 201 with result: "created", while replacing an existing document returns HTTP 200 with result: "updated". Qdrant 1.15.5 reports an absent single point as HTTP 404 rather than a successful null result; that specific response is treated as a missing document, while a missing collection remains an upstream 502. Document HEAD requests preserve the same distinction: an absent point is 404, but transport and storage failures are 502 rather than false “not found” results. HEAD responses are bodyless and include the Elasticsearch product header. Contiguous bulk index writes remain one Qdrant upsert batch and use one batched existence lookup rather than a read per item. Successful bulk create, update, and delete items preserve the operation’s result metadata and report the concrete backing index when addressed through an alias. Standalone and bulk create operations first check for an existing document and return a 409 conflict rather than overwriting it; they are serialized within one gateway process but are not an atomic cross-replica compare-and-insert. Index existence checks likewise describe the preflight state rather than providing atomic compare-and-write semantics. The compatibility analyzer at GET /_qdrant_gateway/compatibility is local-only and reports request counts; no telemetry leaves the process.

Search requests with size: 0 are accepted for aggregation-only use and return no hits; the gateway uses a one-item internal scroll because Qdrant rejects a zero scroll limit. from and size must be non-negative integers, size cannot exceed MAX_PAGE_SIZE, and their combined result window cannot exceed 10,000. Cursor requests require an array with one scalar value per explicit sort field and cannot combine search_after with a non-zero from. Invalid pagination fails before index lookup or Qdrant access rather than silently using defaults or truncating the page. Native filter-only searches issue an exact Qdrant count alongside the bounded hit retrieval, so hits.total is not truncated to from + size. Ranked text searches derive lower-bound totals from their candidate window. Filter-only post-filter searches and complete gateway scans compute totals over all matching pages; concurrent writes can change what a scan observes. Boolean and non-negative integer track_total_hits modes follow the OpenSearch/Elasticsearch response shape: false omits hits.total and skips the otherwise eligible exact-count request, while a numeric threshold caps the reported value and changes the relation to gte when the threshold is exceeded. Numeric thresholds currently shape the response but do not terminate Qdrant’s exact count early. The _search and _msearch endpoints also honor OpenSearch’s rest_total_hits_as_int=true query parameter: ordinary totals become integers and disabled totals use the native -1 sentinel. Lowercase true and false are accepted; as in OpenSearch 3.9.0, an empty or valueless parameter enables the mode. Other values and the native-incompatible combination of integer formatting with a numeric tracking threshold fail before Qdrant work. Integer formatting cannot express the gateway’s usual eq/gte accuracy relation for bounded text searches. Strict track_total_hits parsing intentionally excludes OpenSearch’s undocumented coercion of strings, fractional numbers, and negative values. The extra count can add an upstream round trip to filter-only search latency. Supported filter-only aggregations scan every matching Qdrant page independently of the returned hit count. Ranked text and pattern queries with aggregations fail explicitly.

Search _source filtering accepts booleans, a single field pattern, an array of patterns, or an include/includes and exclude/excludes object. Dotted paths and */? wildcards preserve the nested object shape rather than flattening selected fields, and the same parser is used for collapse inner hits. Invalid types, mixed-type arrays, and unknown options fail before index lookup or Qdrant work. Filtering shapes the response after the gateway has hydrated each candidate’s complete source; it reduces returned data but does not reduce source-read bandwidth or memory use. Non-empty body docvalue_fields requests and the URL query parameter are rejected before index lookup or Qdrant work because the gateway cannot return native column-store values in each hit’s fields object; an empty body array remains an accepted no-op. Non-empty script_fields objects are rejected at the same boundary because executing no script and omitting its computed per-hit fields value would violate the response contract; an empty object remains a no-op.

Partial doc updates use read-merge-write semantics. Inner objects are merged recursively; arrays and scalar values replace the existing value. Identical merged documents return result: "noop" without an upstream write by default; detect_noop: false forces the write. A missing document can be created explicitly with either doc_as_upsert: true or a separate upsert object; the separate form writes the upsert source rather than the partial doc, while an existing document still receives doc. Both standalone and bulk update paths share this behavior. In the default embedded-source mode, non-text updates refresh the complete stored _source and all mapped filter/sort payload fields in one payload write; this preserves untouched source fields and keeps filtering consistent with document reads. This path transfers the merged source even when the patch is small. Updates are not an atomic compare-and-write, Elasticsearch nested query semantics remain unsupported, scripts remain unsupported, and version counters remain approximate.

Aliases currently map one alias to one concrete index. CRUD, search, count, mapping, and multi-search requests resolve that concrete index before selecting a Qdrant collection or deriving a deterministic point ID. Alias inspection is Elasticsearch-shaped and keyed by the concrete index. Alias action batches are fully validated before a single SQLite transaction applies them, so a malformed later action cannot leave an earlier routing change behind. Missing or empty action lists return the HTTP 400 illegal_argument_exception shape independently observed from Elasticsearch 9.5.5 and OpenSearch 3.9.0; the gateway no longer acknowledges an empty catalogue transaction. Both engines also preserve the original target when an add action attaches the same alias to a second index. Because the gateway cannot represent that multi-index state, it rejects such a final topology before changing the catalogue instead of silently replacing the original target. An atomic batch that removes the old target and adds the new one remains supported in either action order. A batch in which every supported removal misses returns HTTP 404 with aliases_not_found_exception, including when the alias exists on a different index. A mixed batch still applies successful actions and returns HTTP 200. Elasticsearch 9.5.5 adds errors and per-action results for that partial failure, while OpenSearch 3.9.0 returns only acknowledged; the gateway currently uses the simpler OpenSearch-shaped success body and does not expose per-action alias results. Native alias filters, routing fields, write-index/hidden flags, remove must_exist, and unknown action options are rejected before catalogue access instead of being acknowledged and discarded. Multi-index aliases, filtered aliases, write-index selection, routing, hidden aliases, and Qdrant-native alias switching remain unsupported.

Qdrant’s server-side BM25 uses sparse vectors and its own tokenization/model implementation. Results should be evaluated by relevant document IDs and product metrics, not numerical _score equality with Elasticsearch. match_phrase currently uses the same native lexical representation and is therefore not a Lucene positional phrase guarantee.

Filter-only should clauses and numeric or percentage minimum_should_match are supported when they can be represented as Qdrant filter alternatives. Scalar and array forms of must_not are accepted, and nested filter-only bools preserve their clause boundaries when negated or used as alternatives. As in Elasticsearch, a bool containing must or filter defaults minimum_should_match to zero; without either required clause, at least one should remains required. Optional filter-only clauses preserve the matching document set but do not boost Qdrant scores. dis_max, match_bool_prefix, and fuzzy-shaped requests are accepted through the gateway’s lexical field path; their ranking and typo/prefix behavior are intentionally approximate because Qdrant’s sparse model is not Lucene. Basic response sorting accepts a field name, a direct order such as {"price":"asc"}, or the standard options object {"price":{"order":"desc"}}; _score defaults to descending and other fields default to ascending. Unsupported sort options and invalid directions are rejected instead of silently changing the order. Filter-only field sorting and search_after visit every matching page and retain the best from + size hits, so point order cannot hide a later winner. Ranked text searches still sort a bounded candidate window. Cursors do not freeze the data: use stable data and a unique explicit tie-breaker for repeatable pages. Per-search min_score, terminate_after, and timeout controls are rejected before index lookup or Qdrant work. They change hit membership, totals, early-termination metadata, or partial-result behavior, and Elasticsearch 9.5.5 and OpenSearch 3.9.0 do not report identical totals for the same terminated search. Search highlight requests are also rejected before backend work: Elasticsearch 8.15.0 and OpenSearch 3.9.0 both return per-hit snippets for the same basic request, while the gateway cannot reproduce analyzer-aware offsets or fragment selection and must not silently omit them. explain: true now fails at the same boundary instead of omitting each hit’s requested _explanation; explain: false remains a no-op. Elasticsearch 8.15.0 and OpenSearch 3.9.0 both returned a nested per-hit score explanation in the focused native probe, but the recorded values and component trees differed, reinforcing that Qdrant scores cannot be labeled as Lucene internals. profile: true is likewise rejected before backend work instead of omitting the requested query and collector timing trees; explicit false remains a no-op. The focused native probe found both engines returned profiling data, but their observed collector names and response sizes differed, so the gateway does not manufacture a shared Lucene-shaped profile for Qdrant execution. Search version: true and seq_no_primary_term: true are also rejected before backend work; returning the gateway’s approximate write metadata would incorrectly present it as a native concurrency token. Explicit false remains a no-op. A focused Elasticsearch 8.15.0 and OpenSearch 3.9.0 probe observed _version: 2, _seq_no: 1, and _primary_term: 1 on each native hit after one update. The gateway requires actual JSON booleans for these controls rather than adopting the engines’ observed string coercion. Scripts, scroll, and analyzer settings that would materially change semantics remain unsupported. Every rejection identifies the request feature and a safe alternative where one exists.

Field collapse accepts one field and one standard inner_hits definition with name, size, _source, and _id in fields. The response uses the requested inner-hit name, caps its size at MAX_PAGE_SIZE, and applies its source projection independently of the top-level projection. Unsupported collapse options, multiple inner-hit definitions, and inner-hit sorting are rejected before Qdrant work. Collapse follows the candidate scope of its query; complete filter scans retain all group members, so collapse memory is proportional to matching documents. Collapse with search_after is rejected. Ranked text collapse remains limited to retrieved candidates.

exists queries over mapped payload fields exclude missing values, JSON null, empty arrays, and arrays containing only null; empty strings and arrays with at least one non-null value count as present. This matches the ordinary Elasticsearch boundary, but not mapping-dependent cases such as index: false, ignore_above, or malformed values ignored by an analyzer. Server-side exists filtering is limited to fields mirrored into the searchable Qdrant payload; post_filter evaluates the hydrated _source.

For application mappings that expose a conventional .keyword multi-field name, exact filters, range and exists predicates, gateway-side patterns, sorting, collapse, and terms facets resolve that suffix to the base payload/source field. This makes client-generated requests such as term: {"brand.keyword":"Acme"} consistent across native and hydrated-source paths. It does not create a second analyzed representation or emulate custom multi-field analyzers: the base field must still be suitable for the requested operation.

Pattern queries

regexp, prefix, and wildcard clauses are evaluated by the gateway using Rust’s regular-expression engine after candidate documents are read from Qdrant. Positive pattern clauses can be combined with ordinary Qdrant-translated filters in bool.must or bool.filter; pattern clauses in should and must_not are rejected rather than approximated. Pattern-only search and count requests scan the collection in bounded scroll pages, so their totals are exact for the stable collection snapshot seen by the scan, but they are useful only for modest catalog and identifier searches and should not be used as a substitute for a dedicated indexed pattern field at very large scale. Wildcards preserve the position of Elasticsearch/OpenSearch-style * and ? operators, including leading, trailing, and consecutive stars. Gateway-side matching uses stored source strings, so prefix and wildcard case behavior on analyzed text fields can differ from a native engine’s normalized term dictionary; use a normalized keyword field when case behavior is part of the contract. Regular expressions use Rust/RE2-style syntax and reject invalid expressions with a structured 400 response.

Search responses also accept post_filter for simple term, terms, numeric range, exists, match-all, and bool clauses. The complete post_filter tree is validated before index lookup or Qdrant work; unsupported query types, unknown bool options, ambiguous multi-field clauses, and malformed range options return a structured 400 instead of silently producing an empty result set. The filter is applied to hydrated hits after the main query. Min, max, terms, filter, keyed filters, and supported calendar histograms reduce every query-matched page before hit-only post-filtering, collapse, or cursor selection. Scalar and array-valued source fields use the same any-element matching rule for term, terms, range, prefix, wildcard, and regexp predicates, keeping gateway-side evaluation aligned with native Qdrant filters. Arrays of objects still require Elasticsearch nested semantics and are unsupported. Aggregation memory uses constant metric state and at most 10,000 distinct buckets per aggregation, with at most 100 aggregations per request. Exceeding a bound fails instead of returning partial results. Calendar histograms accept day, week, month, and year intervals, IANA timezones, and explicit min_doc_count: 1; unsupported formats, bounds, offsets, and empty-bucket filling fail. Timezone rules come from the compiled chrono-tz dependency. Scans are not point-in-time snapshots. In document-projection mode, aggregation fields must be supported mapped payload fields; reductions use search-projection values. See the comparison fixture.

Explicit sparse embeddings

New indices can map a top-level field as sparse_vector. Optional method: {name: seismic, engine: lucene} is accepted for OpenSearch request compatibility; Qdrant performs exact sparse dot-product search, not SEISMIC. The optional engine may be omitted. Other method keys, the native engine and pruning options are rejected. Named vectors are provisioned at index creation; adding sparse fields through _mapping requires a new index.

Documents supply a token-weight object, for example {"v":{"65536":1.0}}. Keys must be canonical decimal unsigned 32-bit IDs; weights must remain positive and finite after float32 conversion. IDs are never hashed, folded or narrowed. Missing/null fields omit the vector; an empty document vector is allowed. Malformed vectors fail before any source or search write in that write batch. Bulk index operations isolate invalid sparse documents as per-item 400 errors, allowing adjacent valid items to proceed. Partial updates rebuild sparse vectors when the sparse field changes, including with document projection enabled.

{"query":{"neural_sparse":{"v":{"query_tokens":{"65536":1.0}}}}} supports one mapped field as the sole top-level query. Inference (query_text, model_id), method_parameters, bool composition, sort, search_after, collapse, post_filter, min_score and aggregations are rejected. from/size retain gateway limits; reported total hits are a lower bound. Query token objects must not be empty. Scalar predicates (including exists, term, range and nested bool filters), filter/metric aggregations on sparse fields or their token subpaths are rejected for search and count; sparse vectors are not scalar payload fields. The OpenSearch Lucene default-path comparison in examples/explicit-sparse verifies full-width token IDs; it does not reproduce SEISMIC approximation bugs.

Shared metadata and consistency

SQLite remains the default single-replica catalog. The optional Redis/Valkey catalog shares mappings and single-index aliases across replicas, with explicit recovery after uncertain administrative writes. It does not supply atomic document updates or snapshot pagination. Unsupported pit search bodies and URL parameters if_seq_no, if_primary_term, version, version_type, retry_on_conflict, op_type, routing, and scroll return a structured 400 rather than being ignored.

Ordered numeric searches

Single-field numeric sorts can use Qdrant’s ordered scroll when every matching point carries a gateway-generated scalar-type marker. This avoids downloading all matching documents to return one page. Multiple sort fields, missing or array values, integers outside the exact floating-point comparison range, separate document projections, and ties crossing the requested page boundary retain the complete scan. The search_after predicate excludes the cursor value; exact totals still describe all query matches before pagination.

New writes carry versioned eligibility metadata in the reserved _qeg_fast_fields payload field. New collections and mapping updates index that field; old points without markers safely use the existing scan. Application mappings cannot define this reserved field. No reindex is required for correctness. Neither ordered scroll nor the fallback is a point-in-time snapshot during concurrent writes. Qdrant’s ordered-scroll API provides the backend ordering; the gateway checks compatibility before using it.

Exact keyword terms and simple keyword term / numeric range filter aggregations can also use Qdrant facet/count indexes after checking the matching payloads’ eligibility markers. Facets request exact counts and up to 10,001 values, preserving the existing 10,000-bucket error and sum_other_doc_count. Other reducers and unsafe or legacy values retain streaming evaluation. The facet API counts each distinct value once per document, including multivalued keywords.

An older mapping that already owns _qeg_fast_fields is treated as user data: its values and index type are preserved, and fast paths are disabled for that index. New mappings cannot introduce that name. These optimizations add no cursor sessions, result cache, or Redis/Valkey dependency.