Skip to main content

Changelog

All notable changes to vectlite are listed below. The format is based on Keep a Changelog, and this project follows Semantic Versioning while the public API stabilizes.

Mirrored from vectlite/CHANGELOG.md on GitHub. The GitHub copy is the source of truth.

[0.9.2] — 2026-05-12​

Fixed​

  • Node search shorthand — Database.search* methods now accept db.search(query, k) in addition to db.search(query, options) and db.search({ query, ...options }). The shorthand is the same on searchWithStats, searchAsync, and searchWithStatsAsync.
  • Kotlin builds — no longer force Gradle to locate or provision a JDK 17 toolchain. The binding now compiles with the host JDK while still emitting Java 17-compatible bytecode, which removes the need for a specific local toolchain install.

[0.9.1] — 2026-05-12​

Fixed​

  • Python quantization introspection — db.is_quantized() is now a method (was a property in 0.9.0), matching the rest of the quantization helper API. The type stubs now also include db.quantization_method.
  • Property vs method clarity — passive database metadata (db.metric, db.dimension, db.read_only, db.path, db.wal_path) is explicitly documented as properties.
  • Scalar quantization recall — quantized search now ranks candidates with dequantized metric-aware scores before exact float32 rescoring, avoiding large recall loss from raw u8 dot-product bias.
  • Rescoring budget — quantization rescore_multiplier now directly controls the rescoring budget (k × rescore_multiplier, capped at collection size) instead of being hidden by an internal 100-candidate floor.
  • Documentation — quantization docs now clarify that memory savings apply to the in-memory candidate index, not the .vdb disk footprint (float32 vectors are retained for exact rescoring).
  • Node quantization API — Database wrapper now exposes quantization and multi-vector quantization methods that were previously only available on the private _native handle. The 0.9.0 Node bindings appeared to support these methods but did not wire them through the public surface.
  • Node alternative search syntax — db.search() accepts both db.search(query, options) and db.search({ query, ...options }). Same support on searchWithStats, searchAsync, and searchWithStatsAsync.
  • Product quantization defaults — invalid num_sub_vectors settings are now catchable errors instead of panics. When num_sub_vectors is omitted, Python and Node pick a dimension-compatible default.
  • Valid PQ partition helpers — new db.valid_num_sub_vectors() (Python) / db.validNumSubVectors() (Node) return the legal values for the current dimension. Quantization method names are parsed case-insensitively, with "pq" documented as the primary PQ alias.
  • All-zeros query rejected — search with an all-zeros query vector now raises VectLiteError for cosine and dot-product metrics instead of silently returning arbitrary results with score 0. Euclidean and Manhattan metrics are unaffected since distance from the origin is well-defined.
  • Strict dimension matching — search rejects query vectors whose dimension does not match the database dimension, returning a DimensionMismatch error. Previously, undersized queries were silently truncated via Matryoshka logic. Users must now pass truncate_dim / truncateDim explicitly to opt into prefix search.
  • Store.close() — now available in Python, Node, Swift, and Kotlin bindings for symmetry with Database.close(). The method is a no-op (the store holds no open file handles) but prevents AttributeError / missing-method surprises.
  • HNSW sidecar filenames — no longer use triple-dot filenames (c.vdb.ann...hnsw.data). Empty namespace and vector-name components now produce _ sentinels (e.g. c.vdb.ann._._.hnsw.data). Existing triple-dot files are orphaned and the ANN index rebuilds automatically on next use.
  • Swift / Kotlin "pq" alias — enableQuantization() now accepts "pq", matching Python and Node. The error message lists the alias.
  • build-xcframework.sh — sets MACOSX_DEPLOYMENT_TARGET=12.0 and IPHONEOS_DEPLOYMENT_TARGET=15.0 when building the XCFramework, preventing linker warnings on older OS versions.
  • Kotlin build.gradle.kts — declares jvmToolchain(17) so Gradle auto-provisions a compatible JDK, fixing build failures when JAVA_HOME points to JDK 25+. (Superseded in 0.9.2 — Kotlin now compiles with the host JDK and emits Java 17-compatible bytecode without forcing a toolchain.)

Upgrade notes​

  • If your code accessed db.is_quantized as a property in Python, add parentheses: db.is_quantized().
  • If you relied on undersized query vectors being silently truncated, pass truncate_dim (Python) / truncateDim (Node) explicitly.
  • If you guarded against Store.close() being missing, you can drop the guard — it now exists on every binding.

[0.9.0] — 2026-05-11​

Added​

  • Optional OpenTelemetry tracing for search operations (Python and Node.js).
    • configure_opentelemetry() (Python) / configureOpenTelemetry() (Node) enables span-based tracing.
    • Each search call creates a vectlite.search span with db.system, db.operation.name, and search-specific attributes (k, namespace, fusion, result counts, timings).
    • opentelemetry-api (Python) / @opentelemetry/api (Node) is loaded lazily — never a required runtime dependency.
    • Supports custom tracers, custom tracer names, and explicit disable via False / { enabled: false }.
  • Experimental Swift and Kotlin bindings via a shared UniFFI layer.
    • New bindings/uniffi crate with a UDL interface for the core database, store, search, metadata, indexes, TTL, quantization, bulk ingest, backup/restore, and transactions.
    • Swift package in bindings/swift with a UniFFI-generated wrapper, VectLiteFFI.xcframework, an XCFramework build script, and smoke tests.
    • Kotlin/JVM Gradle package in bindings/kotlin compiling the generated UniFFI Kotlin source, loading the native library through JNA, and running smoke tests against the Rust FFI library.

See: Observability guide, Installation — Swift / Kotlin.

[0.1.17] — 2026-05-11​

Added​

  • TTL / Expiry — records can automatically expire after a time-to-live.
    • db.set_ttl(id, ttl_secs) sets a TTL on an existing record; db.clear_ttl(id) removes it.
    • ttl parameter on insert() / upsert() (Python, Node) and transaction writes.
    • Expired records are transparently filtered from get(), list(), count(), and search() at read time.
    • compact() garbage-collects expired records permanently.
    • expires_at field returned in record output (epoch seconds, or null / None).
    • WAL SetTtl operation (tag 4) with snapshot persistence.
  • Cursor-based pagination — efficient iteration over large collections without offset overhead.
    • Rust core: Database::list_cursor(namespace, filter, limit, after) returns (Vec<Record>, Option<String>).
    • Python: db.list_cursor(namespace, filter, limit, cursor) returns (list[Record], str | None).
    • Node: db.listCursor({ namespace, filter, limit, cursor }) returns { records, cursor }.
    • Respects TTL filtering and metadata filters.
  • Async Node API — non-blocking versions of heavy operations.
    • db.searchAsync(), db.searchWithStatsAsync(), db.flushAsync(), db.compactAsync(), db.bulkIngestAsync().
    • Backed by napi::Task (runs on libuv threadpool), no tokio dependency.
  • LangChain integration — vectlite.langchain.VectLiteVectorStore implements the LangChain VectorStore protocol.
    • add_texts(), add_documents(), similarity_search(), similarity_search_with_score(), similarity_search_by_vector(), delete(), from_texts().
    • Requires langchain-core >= 0.2 (optional dependency).
  • LlamaIndex integration — vectlite.llamaindex.VectLiteVectorStore implements the LlamaIndex VectorStore protocol.
    • add(), delete(), query() methods compatible with VectorStoreIndex and StorageContext.
    • Requires llama-index-core >= 0.10 (optional dependency).
  • Built-in embedding providers — vectlite.embedders module with ready-to-use factory functions.
    • embedders.openai(), embedders.cohere(), embedders.voyage(), embedders.fastembed(), embedders.sentence_transformer(), embedders.ollama().
    • Each returns a Callable[[str], list[float]] compatible with upsert_text() and search_text().
    • All providers lazy-import their SDK (zero hard dependencies).
  • ONNX cross-encoder reranker — rerankers.onnx_cross_encoder() for zero-PyTorch reranking.
    • Uses onnxruntime + tokenizers for lightweight cross-encoder inference.
    • Auto-downloads models from HuggingFace Hub; same RerankHook interface as cross_encoder().
  • Rich CLI — full command-line interface via vectlite command or python -m vectlite.
    • Subcommands: stats, count, list, dump, search, compact, verify, bench, import-jsonl, import-csv.
  • Schema validation — optional typed metadata schemas with clear error messages.
    • schema.Schema({"price": "number", "tags": "array<string>"}) defines field types.
    • Types: string, number, integer, boolean, null, any, array, array<T>, object, nested objects.
    • schema.validated(db, s) wraps a database to auto-validate on every write.
    • Schemas persist in .vdb.schema.json sidecar files via save() / load().
    • strict=True rejects unknown fields.

See: TTL / Expiry, Built-in Embedders, CLI, Schema Validation, LangChain, LlamaIndex.

[0.1.16] — 2026-05-11​

Added​

  • Payload indexes — create keyword and numeric indexes on metadata fields to accelerate filtered queries 10-100x on large collections.
    • db.create_index(field, type) creates an index ("keyword" for string equality / $in, "numeric" for range queries $gt / $gte / $lt / $lte).
    • db.drop_index(field) removes an index.
    • db.list_indexes() returns all active indexes.
    • Indexes are automatically used by search(), count(), and list() to narrow candidates before full filter evaluation.
    • AND filters intersect index results; OR filters union when all sub-filters are indexed.
    • Indexes are incrementally maintained on upsert(), delete(), and update_metadata().
    • Index definitions persist across close/reopen in a .vdb.pidx sidecar file; data is rebuilt from records on open.
    • Sidecar files are included in backup() operations.

See: Payload Indexes guide.

[0.1.15] — 2026-05-11​

Added​

  • Partial metadata updates — new update_metadata() method that merges a patch into an existing record's metadata without re-writing the vector or rebuilding indexes.
    • Keys present in the patch overwrite existing keys; keys not in the patch remain untouched.
    • Skips all index rebuilds (ANN, sparse, quantized, multi-vector) when a WAL batch contains only metadata updates.
    • Returns true if the record was found and updated, false if the id does not exist.
    • Works with namespaces via update_metadata_in_namespace() (Rust) or namespace parameter (Python/Node).
  • Python binding: db.update_metadata(id, metadata, namespace=None).
  • Node binding: db.updateMetadata(id, metadata, { namespace }).
  • New WAL operation (UpdateMetadata, tag 3) with full serialization/deserialization support.

[0.1.14] — 2026-05-11​

Added​

  • Multiple distance metrics — databases can be created with cosine (default), euclidean (L2), dotproduct (inner product), or manhattan (L1) distance metrics.
    • The metric is persisted in the database file and automatically loaded on reopen. Older databases (format version ≤ 5) default to cosine.
    • Aliases accepted: l2 for euclidean, dot / ip / inner_product / dot_product for dotproduct, l1 for manhattan.
    • Scores are normalized so that higher is always better across all metrics; distance metrics (euclidean, manhattan) are negated.
  • SIMD-accelerated scoring via the simsimd crate for cosine, euclidean (L2), and dot product distance computations, with automatic scalar fallbacks.
    • Manhattan distance uses a scalar implementation (simsimd does not provide L1).
  • Python binding: vectlite.open(path, metric="euclidean") and db.metric property.
  • Node binding: vectlite.open(path, { metric: 'euclidean' }) and db.metric property.
  • HNSW indexes now use metric-specific distance functions (DistCosine, DistL2, DistDot, DistL1 from hnsw_rs).

Changed​

  • Binary format bumped to version 6 to store the distance metric byte after the dimension field.
  • All internal cosine similarity calls replaced with DistanceMetric::score(), enabling metric-aware scoring throughout search, MMR, MaxSim, and record similarity computations.
  • The simsimd crate (v6.5) is now a dependency of vectlite-core.

See: Distance Metrics guide.

[0.1.13] — 2026-05-11​

Added​

  • Multi-vector / late interaction (ColBERT-style) search with per-document token-level embeddings and MaxSim scoring:
    • Storage of N token vectors per document in named multi-vector spaces (e.g. "colbert", "colpali").
    • MaxSim scoring: for each query token, find the maximum cosine similarity against all document tokens, then sum across query tokens.
    • 2-bit quantization for ColBERTv2-style token compression (~16x memory reduction), using per-dimension quartile boundaries.
    • Quantized multi-vector search uses a 2-stage pipeline: fast 2-bit approximate MaxSim candidate selection followed by exact float32 rescoring.
  • Multi-vector quantization parameters persist in .vdb.mvquant.<space> sidecar files and auto-load on database open.
  • Quantized multi-vector indexes automatically rebuild on inserts, upserts, and bulk ingestion.
  • Python binding: db.upsert_multi_vectors(), db.search_multi_vector(), db.enable_multi_vector_quantization(), db.disable_multi_vector_quantization(), db.is_multi_vector_quantized().
  • Node binding: db.upsertMultiVectors(), db.searchMultiVector(), db.enableMultiVectorQuantization(), db.disableMultiVectorQuantization(), db.isMultiVectorQuantized().

Changed​

  • Binary format bumped to version 5 to support multi_vectors field on Record.
  • Mutation methods (insert_many, upsert_many, apply_operations, bulk_ingest, apply_wal_batch) now rebuild multi-vector quantized indexes alongside regular quantized indexes.
  • All three open methods now auto-load multi-vector quantization from sidecar files.

See: Multi-Vector / ColBERT guide.

[0.1.12] — 2026-05-10​

Added​

  • Vector quantization with three strategies for trading memory for search speed:
    • Scalar quantization (int8) — 4x memory reduction with minimal recall loss.
    • Binary quantization — 32x memory reduction using Hamming distance filtering, best for normalized embeddings.
    • Product quantization (PQ) — configurable compression via k-means sub-vector clustering for very large datasets.
  • All quantization methods use a 2-stage pipeline: fast quantized candidate selection followed by exact float32 cosine rescoring.
  • Quantization parameters (calibration ranges, PQ codebooks) persist in a .vdb.quant sidecar file and auto-load on database open.
  • Quantized indexes automatically rebuild on inserts, upserts, and bulk ingestion.
  • Python binding: db.enable_quantization(method, ...), db.disable_quantization(), db.is_quantized, db.quantization_method.
  • Node binding: db.enableQuantization(method, optionsJson), db.disableQuantization(), db.isQuantized, db.quantizationMethod.

Changed​

  • hybrid_search_internal() now uses quantized candidates as an alternative to HNSW when quantization is enabled.

See: Vector Quantization guide.

[0.1.11] — 2026-04-01​

Added​

  • Python and Node bindings now expose explicit database lifecycle controls with close() and context-manager-safe close semantics on Python Database objects.
  • Both bindings now support query-free record scanning with list(...), filtered / scoped record counts, and bulk removal by metadata filter.
  • Open calls now support lock wait timeouts across both read-write and read-only entry points (lock_timeout in Python, lockTimeout in Node).

Changed​

  • The Node package version is now aligned with the workspace release version again so npm and PyPI releases can be cut from the same source state with matching 0.1.11 tags.

Fixed​

  • close() now propagates persistence failures instead of silently swallowing WAL compaction errors before releasing the lock.
  • Closed databases now fail consistently on public result-bearing operations instead of sometimes behaving like empty databases.
  • Invalid lock-timeout inputs such as negative values or NaN now raise normal vectlite validation errors instead of risking a panic in Rust.
  • The Node wrapper / types and Python stubs are now kept in sync with the runtime surface for close, filtered count, list, deleteByFilter / delete_by_filter, and lock-timeout options.

[0.1.10] — 2026-03-31​

Added​

  • The repository README and Python package README now document bulk_ingest(), batch record formats, and a fuller database methods reference including maintenance and diagnostics APIs.

Changed​

  • Python sparse-query parameters now raise a clearer TypeError when callers pass a string instead of the dict[str, float] returned by vectlite.sparse_terms().
  • Dimension mismatch errors now explain how to recover after changing embedding models by deleting the existing .vdb file or creating a new database path.
  • insert_many(), upsert_many(), and transaction commits now defer index rebuilds until the end of the batch, removing the rebuild-per-operation cost from bulk writes.
  • Internal WAL batch application now skips sparse index rebuilds when an operation does not touch sparse terms.

Fixed​

  • Upserts that replace a previously sparse record with a record that has no sparse terms now rebuild sparse search state correctly instead of leaving stale sparse candidates behind.
  • Sparse-only searches no longer fall back to returning zero-score full-scan results when no sparse candidates match.

[0.1.8] — 2026-03-30​

Fixed​

  • Node 0.1.8 keeps the staged Windows prebuilt in place during the prebuilt-loader smoke test, avoiding an EPERM cleanup failure on GitHub Actions and allowing npm publication to complete.

[0.1.7] — 2026-03-30​

Fixed​

  • Node 0.1.7 is the clean npm release that ships both the async text-embedder support and the Windows prebuilt-loader cleanup fix from the correct tagged commit.

[0.1.6] — 2026-03-30​

Fixed​

  • The Node prebuilt-loader smoke test now cleans up safely on Windows, so the cross-platform npm publish workflow can complete instead of failing on EPERM during test cleanup.

[0.1.5] — 2026-03-30​

Fixed​

  • Node upsertText(), searchText(), and searchTextWithStats() now support async embedding functions that return a Promise, matching the documented usage.

[0.1.4] — 2026-03-30​

Added​

  • Initial Node binding in bindings/node with a native napi-rs addon, JavaScript wrapper, TypeScript declarations, and smoke tests for CRUD, collections, and text helpers.
  • Node prebuilt-binary support for macOS x64 / arm64, Linux x64 (glibc), and Windows x64, with a source-build fallback on unsupported targets.
  • GitHub Actions workflow for npm trusted publishing, with tag-to-package-version validation and package tarball checks before publish.
  • Contribution guide, project code of conduct, pull request template, issue templates, and maintainer notes.

Changed​

  • The main CI workflow now runs Node smoke tests on Linux, macOS, and Windows in addition to the Rust and Python checks.
  • Python and Node package releases now use separate tag namespaces (py-v* and node-v*) so Node-only releases do not trigger PyPI publication.

[0.1.3] — 2026-03-30​

Changed​

  • GitHub Actions workflows now use Node 24 native action versions for checkout, Python setup, and artifact upload / download.
  • The GitHub repository README now leads with a fuller product overview, install guidance, quick start, and feature map.
  • The Python package README now reflects the broader surface area of the published package, including collections, snapshots, analyzers, rerankers, and diagnostics.

[0.1.2] — 2026-03-30​

Added​

  • GitHub Actions CI for Rust formatting and tests plus Python install, test, and packaging validation across Linux, macOS, and Windows.
  • Dedicated GitHub Actions release flows for TestPyPI staging and PyPI publishing with repository secrets.
  • Project changelog with versioned release notes in the repository root.

[0.1.1] — 2026-03-30​

Added​

  • First public PyPI release of vectlite as an embedded Python package.
  • Cross-platform GitHub Actions workflows for CI, wheel builds, TestPyPI publishing, and PyPI publishing.
  • Crash-safe persistence with a snapshot, write-ahead log, and persisted ANN sidecars in the Rust core.
  • Dense ANN, sparse BM25-style retrieval, hybrid dense + sparse fusion, MMR diversification, and RRF fusion.
  • Python transactions, namespaces, named vectors, rerank hooks, built-in rerankers, and search diagnostics.
  • Richer metadata value support across the Rust core and Python binding, including None, lists, and dictionaries.

Fixed​

  • Duplicate inserts now raise a dedicated error instead of silently behaving like upserts.
  • open() in the Python binding now raises VectLiteError for vectlite-specific failures.
  • The Python package now exposes __version__.
  • Package metadata now includes project URLs for the repository, issues, and changelog.