Search Diagnostics

vectlite provides detailed diagnostics about search execution, useful for debugging and performance tuning.

search_with_stats()

Returns search results along with execution statistics:

Python

outcome = db.search_with_stats(
    query_embedding,
    k=10,
    sparse=vectlite.sparse_terms("SSO authentication"),
    explain=True,
)

results = outcome["results"]
stats = outcome["stats"]

Node.js

const outcome = db.searchWithStats(queryEmbedding, {
  k: 10,
  sparse: vectlite.sparseTerms('SSO authentication'),
  explain: true,
})

const { results, stats } = outcome

Stats Fields

Field	Type	Description
used_ann	bool	Whether HNSW index was used
exact_fallback	bool	Whether exact search was used (small collection)
ann_candidate_count	int	Number of ANN candidates evaluated
sparse_candidate_count	int	Number of sparse candidates
considered_count	int	Total candidates considered
fetch_k	int	Effective fetch_k used
mmr_applied	bool	Whether MMR diversification was applied
fusion	string	Fusion strategy used
rerank_applied	bool	Whether reranking was applied
rerank_count	int	Number of results reranked
ann_loaded_from_disk	bool	Whether ANN index was loaded from disk
wal_entries_replayed	int	WAL entries replayed before search

Timings

timings = stats["timings"]
print(f"Dense: {timings['dense_us']}us")
print(f"Sparse: {timings['sparse_us']}us")
print(f"Fusion: {timings['fusion_us']}us")
print(f"Total: {timings['total_us']}us")

Explain Mode

Enable per-result scoring breakdown:

for r in outcome["results"]:
    print(r["id"], r["score"])
    if "explain" in r:
        print(f"  Dense: {r['explain'].get('dense_score')}")
        print(f"  Sparse: {r['explain'].get('sparse_score')}")
        print(f"  Matched: {r['explain'].get('matched_terms')}")
        print(f"  BM25: {r['explain'].get('bm25_term_scores')}")

MMR Diversification

Use Maximal Marginal Relevance to balance relevance and diversity:

Python

results = db.search(
    query_embedding,
    k=10,
    fetch_k=50,        # Fetch 50 candidates
    mmr_lambda=0.7,    # 0.0 = max diversity, 1.0 = max relevance
)

Node.js

const results = db.search(queryEmbedding, {
  k: 10,
  fetchK: 50,
  mmrLambda: 0.7,
})