Skip to main content

Changelog

All notable changes to vectlite are listed below. The format is based on Keep a Changelog, and this project follows Semantic Versioning while the public API stabilizes.

Mirrored from vectlite/CHANGELOG.md on GitHub. The GitHub copy is the source of truth.

[0.13.0] — 2026-05-19​

Startup latency​

Opening a non-trivial database used to do four O(N)-or-worse passes over every record on every open():

  1. rebuild_sparse_index() — iterate all records, BM25 posting lists.
  2. try_load_ann_from_disk() → rebuild_ann() fallback.
  3. rebuild_payload_indexes() (when indexes existed).
  4. Implicit O(N) walks via the BTreeMap to rebuild quantization codes.

This release replaces (1) and (4) with on-disk sidecars, gates (3) on having actual definitions, and skips them entirely for empty databases. The dominant remaining cost is then just the bulk read of the .vdb file itself, which is now done through a 256 KiB BufReader.

  • .sparse sidecar — the BM25 sparse index is written at every flush() / compact() and loaded directly at open(). Dense-only databases (no record carries any sparse content) skip the index path entirely. New format SPRS1.
  • .col.vector sidecar — the contiguous VectorArena is persisted at every compact and reloaded on open in one bulk read, which gives the columnar fast path a warm start without iterating records. New format VCOL1. On little-endian hosts (x86_64 / aarch64) the load is a single read_exact into a pre-sized buffer + a reinterpret — essentially memcpy speed.
  • startup_load_indexes() orchestrator prefers sidecars, falls back to in-memory rebuild only when the WAL replay changed the record set (so the sidecar would be stale).
  • replay_wal fast path when the WAL file is missing or zero-byte, and a 256 KiB BufReader for non-trivial replays.
  • Empty-database fast path — open() on a freshly-created database now skips every rebuild call.

Changed​

  • bulk_ingest_array now writes the entire vectors_flat slice into the arena via a single extend_from_slice (one memcpy on LE hosts) instead of N small slice.to_vec() allocations. The arena is left in a clean state so the next compact() writes the columnar sidecar straight away — i.e. the first ingest already pays for the next open to be fast.

The Record public API is unchanged (Record.vector: Vec<f32> is kept for compat). True format-v8 with arena-backed Records — where the per-record Vec<f32> goes away entirely — is the next milestone.

Follow-ups (not in this release)​

  • Format .vdb v8 with arena-backed Records.
  • Payload index data sidecar (.pidx.data).
  • HNSW segment construction parallelised across segments during bulk_ingest.

[0.12.0] — 2026-05-19​

Performance — Python / Node ingestion path​

This release fixes the two big "leaks" between the fast Rust core (0.11) and the bindings, plus adds two architectural pieces that make vectlite top-tier for bulk ingestion.

  • Critical fix: single-record routing — db.insert() / db.upsert() in both Python and Node previously routed to the core insert_many(once(record)) path, which always triggered a full HNSW rebuild + persist per record. They now route to the new Database::insert_record / upsert_record (incremental WAL-batch path), so streaming-style writes finally use the 0.11 incremental graph updates. Single-insert throughput goes from ~150 vec/s back to the multi-thousand vec/s the core can actually deliver.
  • Zero-copy bulk ingest from native arrays.
    • Python: db.bulk_ingest_array(ids, vectors, metadata=None, ...) takes a numpy.ndarray[float32] of shape (N, D). No per-record dict parsing, no per-element Python→Rust object crossing. 10–30× faster than bulk_ingest(list_of_dicts) on large batches.
    • Node: db.bulkIngestArray(ids, vectorsFlat, dim, options?) takes a Float32Array. The vector data is not JSON-serialised between JS and Rust — napi-rs gives the Rust side a reference into the underlying ArrayBuffer. The old bulkIngest(records) path is preserved for compat but should not be used for hot paths.
  • GIL release on Python write paths — insert, upsert, insert_many, upsert_many, bulk_ingest, and bulk_ingest_array all wrap the underlying Rust ingest call in py.allow_threads(|| { ... }). Other Python threads can now make progress during ingestion.

HNSW segmenting (LSM-tree style)​

AnnIndex is now composed of one or more AnnSegments. New inserts land in the "active" segment until it hits IndexConfig.segment_size_threshold (default 50_000), at which point a new active segment is started. Search queries every segment and merges top-K with a distance-sorted dedup.

  • Per-insert HNSW cost stays roughly bounded (O(log segment_size) instead of O(log total)), so streaming throughput doesn't degrade as the corpus grows.
  • Sets up future parallel segment construction during bulk ingest.
  • New Database::ann_segment_count(namespace, vector_name) accessor for observability (also exposed to Python / Node).

Manifest format bumped to ANN3 (tracks per-segment keys and filenames). ANN1 and ANN2 manifests remain readable (treated as 1-segment indexes).

New core API​

  • Database::insert_record(record) / upsert_record(record) — incremental WAL-batch path. Entry point bindings should use for db.insert().
  • Database::bulk_ingest_array(ids, vectors_flat, dim, metadata, ...) — high-throughput bulk entry taking a flat &[f32] buffer plus a parallel Vec<String> of ids.
  • Database::ann_segment_count.
  • IndexConfig.segment_size_threshold (default 50_000).

Binding additions​

Python (vectlite):

  • db.bulk_ingest_array(ids, vectors, metadata=None, ...) — NumPy zero-copy bulk ingest.
  • db.ann_segment_count(namespace=None, vector_name=None).
  • db.bulk_ingest, db.set_index_config, db.index_config(), db.bulk_ingest_array all accept / expose segment_size_threshold.
  • All write paths now release the GIL.
  • New runtime dep: numpy>=1.23 in the Python package metadata.

Node (vectlite):

  • db.bulkIngestArray(ids, vectorsFlat, dim, options?) — Float32Array zero-copy bulk ingest.
  • db.annSegmentCount(namespace?, vectorName?).
  • bulkIngest, bulkIngestAsync, setIndexConfig, indexConfig, bulkIngestArray all accept / expose segmentSizeThreshold.
  • New TS types: BulkIngestArrayOptions, IndexConfig / SetIndexConfigOptions extended with segmentSizeThreshold.

See: Zero-copy bulk ingest guide, HNSW tuning.

[0.11.0] — 2026-05-18​

Performance​

This release rewrites the single-record ingestion hot path that bottlenecked streaming workloads at ~150 vec/s. Expected throughput is now 10–50× higher out of the box on a typical SSD, and another 5–10× on top of that when the new WAL sync_mode is relaxed.

  • Incremental HNSW insertion — insert / upsert no longer rebuild the entire HNSW graph(s) on each call. The new vector is appended directly into the existing graph via hnsw_rs::Hnsw::insert (or parallel_insert for larger batches). Per-record cost drops from O(N log N) to O(log N). A full rebuild only happens when an op deletes or replaces an existing key — see HNSW tombstoning below for the delete fix.
  • Lazy persistence of the ANN graph — persist_ann_to_disk is no longer called on every insert. Instead an ann_dirty flag is raised and the HNSW sidecar files are dumped at flush / compact / close time. The WAL still gives per-record durability, so on a crash the ANN is rebuilt from records on the next open().
  • Lazy rebuild of quantized indexes — when quantization is enabled, the in-memory PQ / scalar / binary codebook is dropped on the first post-build insert and rebuilt at the next flush. Searches between inserts and flush transparently fall back to HNSW, never returning stale candidates.
  • Cached WAL writer — a single BufWriter<File> is now held for the lifetime of the database session. Previous behaviour opened and closed the WAL file on every insert, paying the open(2) syscall per record.
  • WAL sync_mode knob (WalSyncMode) — choose PerOp (default, fsync per insert), EveryN(n) (fsync every N inserts — amortises the fsync tax) or OnFlush (fsync only at flush / compact / close). Configurable via Database::set_wal_sync_mode and exposed in the bindings. Tradeoff: relaxing the mode trades a bounded amount of recently-acked data on crash for a 5–10× throughput multiplier on macOS APFS.
  • HNSW tombstoning — delete no longer triggers a full HNSW rebuild. The deleted record's origin_id is marked in a per-index tombstone set and silently skipped during search. A rebuild happens automatically at compact() time, or whenever the tombstone ratio crosses IndexConfig.tombstone_rebuild_pct (default 30).
  • Contiguous dense-vector arena (VectorArena) — the default dense vector is now mirrored into a single global contiguous Vec<f32> arena. Per-record vectors are appended at insert time (no extra allocations beyond the amortised arena growth), and the arena is lazily rebuilt after a delete. Call Database::prepare_for_scan() to materialise it ahead of a heavy brute-force / rescoring workload. Search-path integration is incremental: the arena is currently exposed for callers and used as the cache-friendly storage layer; wiring it into the default collect_results scan is in progress in a follow-up.
  • ANN manifest format bumped to ANN2 to persist the per-graph insertion-order keys alongside the HNSW sidecar files. The old ANN1 format is still readable. Required for the incremental-insert path to survive a reopen.

Added​

  • WalSyncMode enum (Rust core) with PerOp, EveryN(usize), OnFlush. Default remains PerOp for safety.
  • Database::set_wal_sync_mode, Database::wal_sync_mode (Rust core).
  • IndexConfig.tombstone_rebuild_pct (default 30) — once a graph accumulates this percentage of tombstones it is rebuilt at the next compact().
  • Database::tombstone_stats() returning the per-graph live / tombstoned counts.

Bindings​

Python (vectlite):

  • db.set_wal_sync_mode(mode, n=None) — mode is "per_op", "every_n" (pass n), or "on_flush".
  • db.wal_sync_mode() returns {"mode": "per_op"} | {"mode": "every_n", "n": 64} | {"mode": "on_flush"}.
  • db.tombstone_stats() returns (live, dead).
  • db.prepare_for_scan() materialises the contiguous arena up front.
  • db.vector_arena_len() returns the arena size or None.
  • db.bulk_ingest(..., tombstone_rebuild_pct=N) and db.set_index_config(..., tombstone_rebuild_pct=N) accept the new HNSW knob. db.index_config() includes a tombstone_rebuild_pct key in the returned dict.

Node (vectlite):

  • db.setWalSyncMode(mode, n) — same semantics.
  • db.walSyncMode() returns { mode, n? }.
  • db.tombstoneStats() returns { live, dead }.
  • db.prepareForScan(), db.vectorArenaLen().
  • bulkIngest, setIndexConfig, indexConfig, and bulkIngestAsync accept / return tombstoneRebuildPct.
  • TypeScript types updated: WalSyncMode, WalSyncModeInfo, TombstoneStats.

UniFFI (Swift / Kotlin) plumbing is still pending — open an issue if you need it for a binding outside Python / Node.

See: Throughput tuning guide.

[0.10.0] — 2026-05-16​

Performance​

  • bulk_ingest is now ~10–20× faster on default settings. Two optimisations:
    • HNSW graph construction now uses parallel_insert (Rayon-backed) when the dataset is large enough (≥ parallel_insert_threshold, default 256 vectors). The dominant cost — distance calculations during graph neighbour selection — is now multi-threaded.
    • WAL writes during bulk_ingest are coalesced into a single fsync at the very end instead of fsync-per-batch. This removes the fsync-per-batch tax that dominates ingestion latency on macOS and modern SSDs.
  • Synthetic 5k × 384 cosine benchmark on M-class macOS: ingest throughput improved from ~47 vec/s (article-run baseline) to ~917 vec/s out of the box, and up to ~1782 vec/s with the fast preset (lower recall).

Added​

  • New IndexConfig struct (core Rust) exposing the HNSW tuning knobs m, ef_construction, ef_search, and parallel_insert_threshold. Includes IndexConfig::high_recall() and IndexConfig::fast() presets.
  • Database::bulk_ingest_with_config(records, batch_size, Option<IndexConfig>) for one-shot tuned ingestion.
  • Database::set_index_config(IndexConfig) and Database::set_ef_search(Option<usize>) to retune an existing database. Changing m / ef_construction triggers a full ANN rebuild; changing only ef_search is free (query-time only).
  • Python: db.bulk_ingest(records, batch_size=..., m=..., ef_construction=..., ef_search=..., parallel_insert_threshold=...).
  • Python: db.set_index_config(...), db.set_ef_search(...), db.index_config().
  • Node: db.bulkIngest(records, { m, efConstruction, efSearch, parallelInsertThreshold }) and matching bulkIngestAsync options.
  • Node: db.indexConfig(), db.setEfSearch(...), db.setIndexConfig({ ... }).
  • UniFFI (Swift / Kotlin): new IndexConfigResult dictionary plus bulkIngestTuned, indexConfig, setEfSearch, setIndexConfig on the Database interface.

See: HNSW tuning guide.

[0.9.3] — 2026-05-13​

Fixed​

  • UniFFI rescore default parity — Swift/Kotlin scalar and product quantization rescore_multiplier defaults now match the core Rust default (10×) instead of being hard-coded to 4×. This fixes the recall regression where Swift scalar quantization scored ~0.66 vs Python's ~0.855. Binary quantization was already correct at 10×.
  • UniFFI "int8" alias — enableQuantization() now accepts "int8" as an alias for "scalar", matching Python and Node behaviour.

[0.9.2] — 2026-05-12​

Fixed​

  • Node search shorthand — Database.search* methods now accept db.search(query, k) in addition to db.search(query, options) and db.search({ query, ...options }). The shorthand is the same on searchWithStats, searchAsync, and searchWithStatsAsync.
  • Kotlin builds — no longer force Gradle to locate or provision a JDK 17 toolchain. The binding now compiles with the host JDK while still emitting Java 17-compatible bytecode, which removes the need for a specific local toolchain install.

[0.9.1] — 2026-05-12​

Fixed​

  • Python quantization introspection — db.is_quantized() is now a method (was a property in 0.9.0), matching the rest of the quantization helper API. The type stubs now also include db.quantization_method.
  • Property vs method clarity — passive database metadata (db.metric, db.dimension, db.read_only, db.path, db.wal_path) is explicitly documented as properties.
  • Scalar quantization recall — quantized search now ranks candidates with dequantized metric-aware scores before exact float32 rescoring, avoiding large recall loss from raw u8 dot-product bias.
  • Rescoring budget — quantization rescore_multiplier now directly controls the rescoring budget (k × rescore_multiplier, capped at collection size) instead of being hidden by an internal 100-candidate floor.
  • Documentation — quantization docs now clarify that memory savings apply to the in-memory candidate index, not the .vdb disk footprint (float32 vectors are retained for exact rescoring).
  • Node quantization API — Database wrapper now exposes quantization and multi-vector quantization methods that were previously only available on the private _native handle. The 0.9.0 Node bindings appeared to support these methods but did not wire them through the public surface.
  • Node alternative search syntax — db.search() accepts both db.search(query, options) and db.search({ query, ...options }). Same support on searchWithStats, searchAsync, and searchWithStatsAsync.
  • Product quantization defaults — invalid num_sub_vectors settings are now catchable errors instead of panics. When num_sub_vectors is omitted, Python and Node pick a dimension-compatible default.
  • Valid PQ partition helpers — new db.valid_num_sub_vectors() (Python) / db.validNumSubVectors() (Node) return the legal values for the current dimension. Quantization method names are parsed case-insensitively, with "pq" documented as the primary PQ alias.
  • All-zeros query rejected — search with an all-zeros query vector now raises VectLiteError for cosine and dot-product metrics instead of silently returning arbitrary results with score 0. Euclidean and Manhattan metrics are unaffected since distance from the origin is well-defined.
  • Strict dimension matching — search rejects query vectors whose dimension does not match the database dimension, returning a DimensionMismatch error. Previously, undersized queries were silently truncated via Matryoshka logic. Users must now pass truncate_dim / truncateDim explicitly to opt into prefix search.
  • Store.close() — now available in Python, Node, Swift, and Kotlin bindings for symmetry with Database.close(). The method is a no-op (the store holds no open file handles) but prevents AttributeError / missing-method surprises.
  • HNSW sidecar filenames — no longer use triple-dot filenames (c.vdb.ann...hnsw.data). Empty namespace and vector-name components now produce _ sentinels (e.g. c.vdb.ann._._.hnsw.data). Existing triple-dot files are orphaned and the ANN index rebuilds automatically on next use.
  • Swift / Kotlin "pq" alias — enableQuantization() now accepts "pq", matching Python and Node. The error message lists the alias.
  • build-xcframework.sh — sets MACOSX_DEPLOYMENT_TARGET=12.0 and IPHONEOS_DEPLOYMENT_TARGET=15.0 when building the XCFramework, preventing linker warnings on older OS versions.
  • Kotlin build.gradle.kts — declares jvmToolchain(17) so Gradle auto-provisions a compatible JDK, fixing build failures when JAVA_HOME points to JDK 25+. (Superseded in 0.9.2 — Kotlin now compiles with the host JDK and emits Java 17-compatible bytecode without forcing a toolchain.)

Upgrade notes​

  • If your code accessed db.is_quantized as a property in Python, add parentheses: db.is_quantized().
  • If you relied on undersized query vectors being silently truncated, pass truncate_dim (Python) / truncateDim (Node) explicitly.
  • If you guarded against Store.close() being missing, you can drop the guard — it now exists on every binding.

[0.9.0] — 2026-05-11​

Added​

  • Optional OpenTelemetry tracing for search operations (Python and Node.js).
    • configure_opentelemetry() (Python) / configureOpenTelemetry() (Node) enables span-based tracing.
    • Each search call creates a vectlite.search span with db.system, db.operation.name, and search-specific attributes (k, namespace, fusion, result counts, timings).
    • opentelemetry-api (Python) / @opentelemetry/api (Node) is loaded lazily — never a required runtime dependency.
    • Supports custom tracers, custom tracer names, and explicit disable via False / { enabled: false }.
  • Experimental Swift and Kotlin bindings via a shared UniFFI layer.
    • New bindings/uniffi crate with a UDL interface for the core database, store, search, metadata, indexes, TTL, quantization, bulk ingest, backup/restore, and transactions.
    • Swift package in bindings/swift with a UniFFI-generated wrapper, VectLiteFFI.xcframework, an XCFramework build script, and smoke tests.
    • Kotlin/JVM Gradle package in bindings/kotlin compiling the generated UniFFI Kotlin source, loading the native library through JNA, and running smoke tests against the Rust FFI library.

See: Observability guide, Installation — Swift / Kotlin.

[0.1.17] — 2026-05-11​

Added​

  • TTL / Expiry — records can automatically expire after a time-to-live.
    • db.set_ttl(id, ttl_secs) sets a TTL on an existing record; db.clear_ttl(id) removes it.
    • ttl parameter on insert() / upsert() (Python, Node) and transaction writes.
    • Expired records are transparently filtered from get(), list(), count(), and search() at read time.
    • compact() garbage-collects expired records permanently.
    • expires_at field returned in record output (epoch seconds, or null / None).
    • WAL SetTtl operation (tag 4) with snapshot persistence.
  • Cursor-based pagination — efficient iteration over large collections without offset overhead.
    • Rust core: Database::list_cursor(namespace, filter, limit, after) returns (Vec<Record>, Option<String>).
    • Python: db.list_cursor(namespace, filter, limit, cursor) returns (list[Record], str | None).
    • Node: db.listCursor({ namespace, filter, limit, cursor }) returns { records, cursor }.
    • Respects TTL filtering and metadata filters.
  • Async Node API — non-blocking versions of heavy operations.
    • db.searchAsync(), db.searchWithStatsAsync(), db.flushAsync(), db.compactAsync(), db.bulkIngestAsync().
    • Backed by napi::Task (runs on libuv threadpool), no tokio dependency.
  • LangChain integration — vectlite.langchain.VectLiteVectorStore implements the LangChain VectorStore protocol.
    • add_texts(), add_documents(), similarity_search(), similarity_search_with_score(), similarity_search_by_vector(), delete(), from_texts().
    • Requires langchain-core >= 0.2 (optional dependency).
  • LlamaIndex integration — vectlite.llamaindex.VectLiteVectorStore implements the LlamaIndex VectorStore protocol.
    • add(), delete(), query() methods compatible with VectorStoreIndex and StorageContext.
    • Requires llama-index-core >= 0.10 (optional dependency).
  • Built-in embedding providers — vectlite.embedders module with ready-to-use factory functions.
    • embedders.openai(), embedders.cohere(), embedders.voyage(), embedders.fastembed(), embedders.sentence_transformer(), embedders.ollama().
    • Each returns a Callable[[str], list[float]] compatible with upsert_text() and search_text().
    • All providers lazy-import their SDK (zero hard dependencies).
  • ONNX cross-encoder reranker — rerankers.onnx_cross_encoder() for zero-PyTorch reranking.
    • Uses onnxruntime + tokenizers for lightweight cross-encoder inference.
    • Auto-downloads models from HuggingFace Hub; same RerankHook interface as cross_encoder().
  • Rich CLI — full command-line interface via vectlite command or python -m vectlite.
    • Subcommands: stats, count, list, dump, search, compact, verify, bench, import-jsonl, import-csv.
  • Schema validation — optional typed metadata schemas with clear error messages.
    • schema.Schema({"price": "number", "tags": "array<string>"}) defines field types.
    • Types: string, number, integer, boolean, null, any, array, array<T>, object, nested objects.
    • schema.validated(db, s) wraps a database to auto-validate on every write.
    • Schemas persist in .vdb.schema.json sidecar files via save() / load().
    • strict=True rejects unknown fields.

See: TTL / Expiry, Built-in Embedders, CLI, Schema Validation, LangChain, LlamaIndex.

[0.1.16] — 2026-05-11​

Added​

  • Payload indexes — create keyword and numeric indexes on metadata fields to accelerate filtered queries 10-100x on large collections.
    • db.create_index(field, type) creates an index ("keyword" for string equality / $in, "numeric" for range queries $gt / $gte / $lt / $lte).
    • db.drop_index(field) removes an index.
    • db.list_indexes() returns all active indexes.
    • Indexes are automatically used by search(), count(), and list() to narrow candidates before full filter evaluation.
    • AND filters intersect index results; OR filters union when all sub-filters are indexed.
    • Indexes are incrementally maintained on upsert(), delete(), and update_metadata().
    • Index definitions persist across close/reopen in a .vdb.pidx sidecar file; data is rebuilt from records on open.
    • Sidecar files are included in backup() operations.

See: Payload Indexes guide.

[0.1.15] — 2026-05-11​

Added​

  • Partial metadata updates — new update_metadata() method that merges a patch into an existing record's metadata without re-writing the vector or rebuilding indexes.
    • Keys present in the patch overwrite existing keys; keys not in the patch remain untouched.
    • Skips all index rebuilds (ANN, sparse, quantized, multi-vector) when a WAL batch contains only metadata updates.
    • Returns true if the record was found and updated, false if the id does not exist.
    • Works with namespaces via update_metadata_in_namespace() (Rust) or namespace parameter (Python/Node).
  • Python binding: db.update_metadata(id, metadata, namespace=None).
  • Node binding: db.updateMetadata(id, metadata, { namespace }).
  • New WAL operation (UpdateMetadata, tag 3) with full serialization/deserialization support.

[0.1.14] — 2026-05-11​

Added​

  • Multiple distance metrics — databases can be created with cosine (default), euclidean (L2), dotproduct (inner product), or manhattan (L1) distance metrics.
    • The metric is persisted in the database file and automatically loaded on reopen. Older databases (format version ≤ 5) default to cosine.
    • Aliases accepted: l2 for euclidean, dot / ip / inner_product / dot_product for dotproduct, l1 for manhattan.
    • Scores are normalized so that higher is always better across all metrics; distance metrics (euclidean, manhattan) are negated.
  • SIMD-accelerated scoring via the simsimd crate for cosine, euclidean (L2), and dot product distance computations, with automatic scalar fallbacks.
    • Manhattan distance uses a scalar implementation (simsimd does not provide L1).
  • Python binding: vectlite.open(path, metric="euclidean") and db.metric property.
  • Node binding: vectlite.open(path, { metric: 'euclidean' }) and db.metric property.
  • HNSW indexes now use metric-specific distance functions (DistCosine, DistL2, DistDot, DistL1 from hnsw_rs).

Changed​

  • Binary format bumped to version 6 to store the distance metric byte after the dimension field.
  • All internal cosine similarity calls replaced with DistanceMetric::score(), enabling metric-aware scoring throughout search, MMR, MaxSim, and record similarity computations.
  • The simsimd crate (v6.5) is now a dependency of vectlite-core.

See: Distance Metrics guide.

[0.1.13] — 2026-05-11​

Added​

  • Multi-vector / late interaction (ColBERT-style) search with per-document token-level embeddings and MaxSim scoring:
    • Storage of N token vectors per document in named multi-vector spaces (e.g. "colbert", "colpali").
    • MaxSim scoring: for each query token, find the maximum cosine similarity against all document tokens, then sum across query tokens.
    • 2-bit quantization for ColBERTv2-style token compression (~16x memory reduction), using per-dimension quartile boundaries.
    • Quantized multi-vector search uses a 2-stage pipeline: fast 2-bit approximate MaxSim candidate selection followed by exact float32 rescoring.
  • Multi-vector quantization parameters persist in .vdb.mvquant.<space> sidecar files and auto-load on database open.
  • Quantized multi-vector indexes automatically rebuild on inserts, upserts, and bulk ingestion.
  • Python binding: db.upsert_multi_vectors(), db.search_multi_vector(), db.enable_multi_vector_quantization(), db.disable_multi_vector_quantization(), db.is_multi_vector_quantized().
  • Node binding: db.upsertMultiVectors(), db.searchMultiVector(), db.enableMultiVectorQuantization(), db.disableMultiVectorQuantization(), db.isMultiVectorQuantized().

Changed​

  • Binary format bumped to version 5 to support multi_vectors field on Record.
  • Mutation methods (insert_many, upsert_many, apply_operations, bulk_ingest, apply_wal_batch) now rebuild multi-vector quantized indexes alongside regular quantized indexes.
  • All three open methods now auto-load multi-vector quantization from sidecar files.

See: Multi-Vector / ColBERT guide.

[0.1.12] — 2026-05-10​

Added​

  • Vector quantization with three strategies for trading memory for search speed:
    • Scalar quantization (int8) — 4x memory reduction with minimal recall loss.
    • Binary quantization — 32x memory reduction using Hamming distance filtering, best for normalized embeddings.
    • Product quantization (PQ) — configurable compression via k-means sub-vector clustering for very large datasets.
  • All quantization methods use a 2-stage pipeline: fast quantized candidate selection followed by exact float32 cosine rescoring.
  • Quantization parameters (calibration ranges, PQ codebooks) persist in a .vdb.quant sidecar file and auto-load on database open.
  • Quantized indexes automatically rebuild on inserts, upserts, and bulk ingestion.
  • Python binding: db.enable_quantization(method, ...), db.disable_quantization(), db.is_quantized, db.quantization_method.
  • Node binding: db.enableQuantization(method, optionsJson), db.disableQuantization(), db.isQuantized, db.quantizationMethod.

Changed​

  • hybrid_search_internal() now uses quantized candidates as an alternative to HNSW when quantization is enabled.

See: Vector Quantization guide.

[0.1.11] — 2026-04-01​

Added​

  • Python and Node bindings now expose explicit database lifecycle controls with close() and context-manager-safe close semantics on Python Database objects.
  • Both bindings now support query-free record scanning with list(...), filtered / scoped record counts, and bulk removal by metadata filter.
  • Open calls now support lock wait timeouts across both read-write and read-only entry points (lock_timeout in Python, lockTimeout in Node).

Changed​

  • The Node package version is now aligned with the workspace release version again so npm and PyPI releases can be cut from the same source state with matching 0.1.11 tags.

Fixed​

  • close() now propagates persistence failures instead of silently swallowing WAL compaction errors before releasing the lock.
  • Closed databases now fail consistently on public result-bearing operations instead of sometimes behaving like empty databases.
  • Invalid lock-timeout inputs such as negative values or NaN now raise normal vectlite validation errors instead of risking a panic in Rust.
  • The Node wrapper / types and Python stubs are now kept in sync with the runtime surface for close, filtered count, list, deleteByFilter / delete_by_filter, and lock-timeout options.

[0.1.10] — 2026-03-31​

Added​

  • The repository README and Python package README now document bulk_ingest(), batch record formats, and a fuller database methods reference including maintenance and diagnostics APIs.

Changed​

  • Python sparse-query parameters now raise a clearer TypeError when callers pass a string instead of the dict[str, float] returned by vectlite.sparse_terms().
  • Dimension mismatch errors now explain how to recover after changing embedding models by deleting the existing .vdb file or creating a new database path.
  • insert_many(), upsert_many(), and transaction commits now defer index rebuilds until the end of the batch, removing the rebuild-per-operation cost from bulk writes.
  • Internal WAL batch application now skips sparse index rebuilds when an operation does not touch sparse terms.

Fixed​

  • Upserts that replace a previously sparse record with a record that has no sparse terms now rebuild sparse search state correctly instead of leaving stale sparse candidates behind.
  • Sparse-only searches no longer fall back to returning zero-score full-scan results when no sparse candidates match.

[0.1.8] — 2026-03-30​

Fixed​

  • Node 0.1.8 keeps the staged Windows prebuilt in place during the prebuilt-loader smoke test, avoiding an EPERM cleanup failure on GitHub Actions and allowing npm publication to complete.

[0.1.7] — 2026-03-30​

Fixed​

  • Node 0.1.7 is the clean npm release that ships both the async text-embedder support and the Windows prebuilt-loader cleanup fix from the correct tagged commit.

[0.1.6] — 2026-03-30​

Fixed​

  • The Node prebuilt-loader smoke test now cleans up safely on Windows, so the cross-platform npm publish workflow can complete instead of failing on EPERM during test cleanup.

[0.1.5] — 2026-03-30​

Fixed​

  • Node upsertText(), searchText(), and searchTextWithStats() now support async embedding functions that return a Promise, matching the documented usage.

[0.1.4] — 2026-03-30​

Added​

  • Initial Node binding in bindings/node with a native napi-rs addon, JavaScript wrapper, TypeScript declarations, and smoke tests for CRUD, collections, and text helpers.
  • Node prebuilt-binary support for macOS x64 / arm64, Linux x64 (glibc), and Windows x64, with a source-build fallback on unsupported targets.
  • GitHub Actions workflow for npm trusted publishing, with tag-to-package-version validation and package tarball checks before publish.
  • Contribution guide, project code of conduct, pull request template, issue templates, and maintainer notes.

Changed​

  • The main CI workflow now runs Node smoke tests on Linux, macOS, and Windows in addition to the Rust and Python checks.
  • Python and Node package releases now use separate tag namespaces (py-v* and node-v*) so Node-only releases do not trigger PyPI publication.

[0.1.3] — 2026-03-30​

Changed​

  • GitHub Actions workflows now use Node 24 native action versions for checkout, Python setup, and artifact upload / download.
  • The GitHub repository README now leads with a fuller product overview, install guidance, quick start, and feature map.
  • The Python package README now reflects the broader surface area of the published package, including collections, snapshots, analyzers, rerankers, and diagnostics.

[0.1.2] — 2026-03-30​

Added​

  • GitHub Actions CI for Rust formatting and tests plus Python install, test, and packaging validation across Linux, macOS, and Windows.
  • Dedicated GitHub Actions release flows for TestPyPI staging and PyPI publishing with repository secrets.
  • Project changelog with versioned release notes in the repository root.

[0.1.1] — 2026-03-30​

Added​

  • First public PyPI release of vectlite as an embedded Python package.
  • Cross-platform GitHub Actions workflows for CI, wheel builds, TestPyPI publishing, and PyPI publishing.
  • Crash-safe persistence with a snapshot, write-ahead log, and persisted ANN sidecars in the Rust core.
  • Dense ANN, sparse BM25-style retrieval, hybrid dense + sparse fusion, MMR diversification, and RRF fusion.
  • Python transactions, namespaces, named vectors, rerank hooks, built-in rerankers, and search diagnostics.
  • Richer metadata value support across the Rust core and Python binding, including None, lists, and dictionaries.

Fixed​

  • Duplicate inserts now raise a dedicated error instead of silently behaving like upserts.
  • open() in the Python binding now raises VectLiteError for vectlite-specific failures.
  • The Python package now exposes __version__.
  • Package metadata now includes project URLs for the repository, issues, and changelog.