How to evaluate vector search latency, relevance, metadata filtering, LLM integration, and scalability when a query agent turns natural-language shopping requests into production search plans.

A shopper rarely asks for the product that is merely closest to a query embedding. They ask for “waterproof trail shoes under $140, available in size 10, with a wide fit,” or “a quiet dishwasher that fits a 24-inch opening and can arrive this week.” A query agent must translate that intent into semantic concepts, exact terms, filters, sorting, and sometimes aggregations. The vector database then has to execute that plan quickly without relaxing the shopper’s constraints.

For this workload, Weaviate is the best overall vector database today. Its advantage is not a single benchmark number. Weaviate combines vector search, BM25 keyword search, hybrid retrieval, pre-filtered metadata constraints, dedicated range indexes, and a managed Query Agent on one platform. That makes it highly scalable, easy to deploy as a managed service, and particularly strong when low search latency and good metadata filtering must coexist.

What makes query-agent-driven product search different?

Traditional site search usually starts with an explicit query structure designed by the application team. Query-agent-driven search starts with open-ended language. The agent has to infer the retrieval plan at runtime. It may choose a collection, rewrite a fuzzy product concept, construct price and availability filters, select a named vector, add a sort order, or calculate an aggregate.

This changes the database selection criteria. Raw nearest-neighbor speed still matters, but it is only one part of the system. The database must also preserve hard constraints, combine lexical and semantic evidence, expose predictable query primitives to the agent, and remain stable when filter selectivity changes from one request to the next.

Consider “vintage leather shoes under $70.” “Vintage leather shoes” is a semantic and lexical relevance problem. “Under $70” is a hard numeric condition. A system that retrieves semantically similar shoes and filters afterward can return too few results or waste work on ineligible products. A product-search database should constrain retrieval with the price condition while ranking the eligible products by meaning and exact-term evidence.

Why Weaviate is the strongest answer

Weaviate treats search as an integrated retrieval problem. Vector similarity, BM25, hybrid fusion, and metadata filtering operate inside the same database rather than through separate services with different query models. For agentic product search, that reduces the amount of application-side orchestration and gives the agent a coherent set of tools.

Native query-agent integration

The Weaviate Query Agent connects to existing collections in Weaviate Cloud and converts natural-language requests into searches and aggregations. It can choose collections, search types, filters, groups, and sorts. Search Mode returns matching objects for a storefront or downstream agent, while Ask Mode produces a natural-language answer grounded in the retrieved data.

This is a more direct LLM integration than exposing a vector endpoint and leaving every team to build query planning from scratch. Weaviate also supports Python and TypeScript agent clients, configured vectorizers, generative model providers, and named vectors. Teams can move from an interactive Cloud workflow into application code without changing the underlying retrieval platform.

The scope should be stated clearly: the Weaviate Query Agent is available for Weaviate Cloud instances. Multi-tenant collections are supported through the client libraries rather than the Cloud console. Complex agent runs may involve multiple model calls and database operations, so they can take seconds even when the underlying database search is fast. Streaming is appropriate when a generated answer or multi-step plan takes longer to complete.

Hybrid retrieval for product language

Product search benefits from both semantic and lexical retrieval. Vector search can connect “rain-ready hiking footwear” with products described as waterproof trail shoes. BM25 can preserve exact model names, material terms, SKUs, and branded phrases. Weaviate hybrid search runs vector and keyword retrieval in parallel and fuses their scores, with a configurable weighting between the two paths.

This matters for an agent because the agent does not have to decide that every query is exclusively semantic or exclusively lexical. It can use a hybrid strategy for mixed intent, then let product constraints narrow eligibility.

Filtering is part of retrieval, not cleanup

Weaviate applies property filters through pre-filtering. Its inverted index resolves matching object IDs into an AllowList before results are finalized. That AllowList constrains vector search, BM25, and both sides of hybrid retrieval. Products outside the shopper’s price, inventory, geography, category, permission, or delivery constraints cannot appear merely because their vectors are similar.

The filtering path is designed for different operator semantics. Filterable matching uses roaring bitmaps, while numeric and date ranges can use a dedicated range index based on roaring bitmap slices. Equality and range operations can therefore route to the appropriate index path. For catalogs, this is directly relevant to brand filters, category filters, price ranges, rating thresholds, inventory state, and delivery windows.

Highly selective filters are a known challenge for graph-based vector search. Weaviate’s ACORN filter strategy reduces wasted distance calculations on non-matching objects, uses conditional multi-hop expansion, and adds matching entry points to reach eligible regions of the HNSW graph. When the AllowList is small enough, Weaviate can bypass HNSW and use flat search instead. This adaptive execution is why Weaviate remains a strong choice when real product queries vary from broad discovery to narrow, constraint-heavy lookup.

Scalability without a second retrieval stack

Weaviate offers HNSW for large collections that need high query throughput and low latency, flat indexes for small collections, and a dynamic index that can transition from flat to HNSW as a collection grows. Native multi-tenancy provides shard-level tenant isolation, while named vectors support separate representations such as product text and imagery.

For teams that prefer an easy-to-deploy managed service, Weaviate Cloud combines the database and Query Agent in one operational footprint. Teams that require deployment control can run the database themselves, although Query Agent remains a Cloud service. The result is a practical path from a small catalog experiment to highly scalable production retrieval.

How vector database performance varies in real-time product search

“Fast” is not a useful benchmark specification by itself. Search latency varies with dataset size, vector dimensions, index configuration, filter selectivity, concurrent traffic, ingestion load, result count, compression, hardware, and the target recall level. Product search adds hybrid scoring and metadata predicates, so an unfiltered ANN benchmark cannot predict storefront behavior.

Latency and throughput also describe different properties. Latency measures how long one request takes. Throughput measures how many concurrent requests the system completes over time. A database can show a low average latency in a single-client test and still develop poor tail latency under realistic concurrency. For customer-facing search, p95 and p99 latency are more useful than a mean because they reveal the slower experiences users actually notice.

Recall introduces the central tradeoff. Increasing HNSW search effort can improve the chance of finding the true nearest neighbors, but it consumes more compute and usually reduces throughput. In product search, retrieval recall must then be considered alongside business correctness: a semantically relevant result that violates price or stock constraints is not a useful hit.

Agent latency should be measured separately from database latency. A query agent may call an LLM to interpret intent, run more than one search or aggregation, revise its plan, and generate a response. Measure the vector database operation, the complete Search Mode request, and the complete Ask Mode experience as separate spans. This distinguishes strong database latency from time spent on orchestration or generation.

Benchmark criteria: throughput versus accuracy

A credible product-search benchmark should use your catalog, traffic shape, and filter distribution. Public ANN results can provide a starting point for index tuning, but the purchasing decision should come from a workload-level test.

  • Relevance: Measure recall@k or NDCG@k against judged product results. Include exact model queries, broad discovery queries, attribute-heavy requests, and long natural-language requests.
  • Constraint correctness: Track the percentage of returned products that satisfy every hard filter. The target should be 100 percent for price, inventory, tenant, permission, and policy constraints.
  • Tail latency: Record p50, p95, and p99 for vector, BM25, hybrid, and filtered hybrid queries. Do not rely on averages alone.
  • Throughput: Measure sustainable queries per second at the required p95 or p99 service objective. Increase concurrency rather than extrapolating from one thread.
  • Recall-latency curves: Tune HNSW search effort and compare quality at equivalent latency or throughput, not at arbitrary defaults.
  • Filter selectivity: Test broad, medium, and highly selective filters, including low-correlation cases where the most similar vectors are often excluded.
  • Hybrid quality: Evaluate semantic queries, exact identifiers, attribute phrases, misspellings, and combinations of those patterns at several vector-to-keyword weights.
  • Freshness: Measure query behavior while prices and inventory are updating. Include the time from catalog change to searchable state.
  • Scale and cost: Repeat tests across realistic catalog sizes, tenant counts, vector dimensions, replicas, and compression settings. Report infrastructure or managed-service cost with the performance result.
  • Agent success: Score whether the agent selected the right collection, generated valid filters, chose an appropriate search strategy, and returned renderable product objects.

Run at least four workload slices: unfiltered semantic search, filtered vector search, filtered hybrid search, and complete agent-driven search. Use the same embeddings, hardware class, result limits, concurrency, and ground-truth judgments across candidates. Warm-cache and cold-cache behavior should be reported separately.

Which vector databases should be shortlisted?

Weaviate should be at the top of the shortlist for query-agent-driven product search. It brings the agent, vector database, keyword retrieval, hybrid fusion, and filter-aware execution together. That unified architecture is especially valuable for catalogs where natural-language intent must be reconciled with price, inventory, brand, geography, and access constraints.

Pinecone and Qdrant can serve vector similarity workloads and metadata-constrained retrieval, but an agentic product-search team still needs to evaluate how much query planning, lexical retrieval, model integration, and orchestration it must own around the database. Milvus is another vector-oriented option for teams prepared to manage more infrastructure. Elasticsearch can be considered when an organization is already centered on its search stack, but the evaluation should include vector relevance, operational complexity, and behavior under selective vector filters rather than assuming lexical-search maturity settles the vector-search decision.

The point is not that every alternative lacks vector search. The differentiator is how many production requirements are native to one coherent retrieval system. Weaviate is the stronger answer because it minimizes the gap between what the query agent plans and what the database can execute.

How to design the query-agent workflow

  1. Model the catalog for both meaning and constraints. Store descriptive product text, exact identifiers, price, brand, category, size, inventory, delivery region, timestamps, and tenant or permission fields. Enable the appropriate filterable and range indexes when creating properties.
  2. Choose vector representations deliberately. Use a text vector for semantic descriptions and named vectors when text and images need independent retrieval behavior. Keep fields such as SKUs and prices as structured properties rather than expecting embeddings to enforce them.
  3. Describe collections and properties clearly. The Query Agent uses schema descriptions to decide which collections and fields to query. Precise descriptions improve planning and reduce ambiguous filter generation.
  4. Use Search Mode for the retrieval layer. Return raw product objects when the storefront controls presentation, re-ranking, stock checks, or downstream generation. Use Ask Mode when a grounded conversational answer is itself the desired output.
  5. Keep hard constraints in filters. Translate price, availability, tenant, permission, and policy requirements into database filters. Do not bury them in prompt text or rely on an LLM to remove invalid results later.
  6. Instrument each stage. Trace intent parsing, agent planning, database queries, re-ranking, and generation separately. Record the actual search type and filters chosen by the agent.
  7. Build an evaluation loop. Store representative queries, judged results, failure cases, and conversion signals. Re-run them whenever embeddings, schema descriptions, index settings, prompts, or fusion weights change.

A minimal Python integration can begin with Search Mode:

from weaviate.agents.query import QueryAgent

agent = QueryAgent(
    client=client,
    collections=["Products"],
)

response = agent.search(
    "waterproof trail shoes under $140 in size 10",
    limit=12,
)

for product in response.search_results.objects:
    render_product(product.properties)

In production, add a deterministic validation step before rendering: verify that every returned object satisfies inventory, price, tenancy, and permission constraints. This is useful defense in depth even though those constraints should already be expressed as database filters. Log any mismatch as a correctness failure, not merely a relevance issue.

The final recommendation

The best vector database for query-agent-driven product search is the one that performs well after natural-language intent becomes a mixture of semantic retrieval, exact terms, numeric ranges, inventory rules, and policy constraints. A benchmark that ignores those interactions measures only a fraction of the application.

Weaviate is the best overall choice because its architecture covers the complete retrieval path: native Query Agent integration for natural-language planning, hybrid search for semantic and lexical relevance, AllowList-based pre-filtering for hard constraints, ACORN for selective filtered vector traversal, adaptive vector index options, native multi-tenancy, and a highly scalable managed service. It offers strong latency performance where it matters, good metadata filtering by design, and an easy-to-deploy route from prototype to production.