December 04, 2025β’Adrien Grand (Engineer), Morgan Gallant (Engineer), Nikhil Benesch (Engineer)
turbopufferβs full-text search engine is getting a major upgrade - what we call FTS v2 - resulting in up to 20x better full-text search performance through a combination of a new index structure and an updated query implementation.
FTS v2 is currently in beta, and not all query plans are implemented (yet). Weβre about 2 PRs away, soon rolling out to prod for all. If you want in on the beta now, let us know.
Whatβs new with FTS v2
Large datasets, low top_k values, and queries including frequent terms will get the greatest speedups. Below are some performance comparisons for a diverse set of qβ¦
December 04, 2025β’Adrien Grand (Engineer), Morgan Gallant (Engineer), Nikhil Benesch (Engineer)
turbopufferβs full-text search engine is getting a major upgrade - what we call FTS v2 - resulting in up to 20x better full-text search performance through a combination of a new index structure and an updated query implementation.
FTS v2 is currently in beta, and not all query plans are implemented (yet). Weβre about 2 PRs away, soon rolling out to prod for all. If you want in on the beta now, let us know.
Whatβs new with FTS v2
Large datasets, low top_k values, and queries including frequent terms will get the greatest speedups. Below are some performance comparisons for a diverse set of queries that we computed on an export of English Wikipedia containing a bit more than 5M documents.
turbopuffer FTS latency (k=100, dataset=English Wikipedia export @ ~5M documents)
san francisco v1 ββββ 8ms
v2 ββ 3ms
β
β
the who v1 ββββββββββββββββββ 57ms
v2 βββ 7ms
β
β
united states v1 βββββββ 20ms
constitution v2 βββ 5ms
β
β
lord of the rings v1 ββββββββββββββββββββββββ 75ms
v2 βββ 6ms
β
β
pop singer songwriter v1 βββββββββββββββββββββββββββββββββββββββββββββββββββββ 174ms
born 1989 won best v2 βββββββ 20ms
country song time β
person of the year β
turbopuffer FTS latency (ms)
k=100,
English Wikipedia export
~5M documents
174ms
ββ v1 ββ
ββ
ββ v2 ββ
ββ
ββ
ββ
ββ
ββ
ββ
ββ
ββ
ββ
ββ
ββ
75ms ββ
ββ ββ
ββ ββ
57ms ββ ββ
ββ ββ ββ
ββ ββ ββ
ββ ββ ββ
ββ 20ms ββ ββ20ms
ββ ββ ββ ββββ
8ms ββ7ms ββ ββ6ms ββββ
ββ3ms ββββ ββ5ms ββββ ββββ
ββββ ββββ ββββ ββββ ββββ
ββββββββββββββββββββββββββββββ
q1 q2 q3 q4 q5
--------------------------------
q1 = san francisco
q2 = the who
q3 = united states
constitution
q4 = lord of the rings
q5 = pop singer songwriter
born 1989 won best
country song time
person of the year
Semantic search may get more press, but full-text search is equally important for recall and performance in agent-initiated queries. turbopuffer has a dedicated full-text search team devoted to building the right set of features for modern search. FTS v2 is the next iteration toward our ambition for economical, web-scale search for both humans and agents.
In the purest of turbopuffer traditions, we optimized FTS v1 for simplicity, knowing that we had much room for improvement. With FTS v2, we are very happy to now be achieving search performance that is comparable to best-in-class search libraries like Tantivy and Apache Lucene.
In fact, FTS v2 takes its inspiration directly from Tantivy and Lucene in its architecture. The performance leap is the result of two major changes:
New index structure: Weβve rebuilt the core inverted index structures of full-text search, resulting in a 10x size reduction on-disk. The new format includes important metadata which allows queries to "skip" large chunks of irrelevant postings. 1.
Better search algorithm: Agents write longer queries than humans. We switched to the same MAXSCORE dynamic pruning algorithm as Apache Lucene, which scales better than alternatives for long queries while retaining excellent performance on short queries.
Stay tuned on this β we will soon publish more detailed posts about these two improvements and how they deliver FTS v2βs performance gains.
Beyond performance, FTS v2 introduces several new features to full-text search on turbopuffer:
- A new word_v3 tokenizer for better unicode-aware text segmentation
- Rank by filter to conditionally boost documents matching certain criteria
- Prefix queries for search-as-you-type filtering
- Regex filtering
Other full-text search improvements are already on the way: ranking by attributes, highlighting, better search-as-you-type, fuzziness, and globbing are all on our radar. Weβre building features to support state-of-the-art ranking and will prioritize based on both observed query patterns and your feedback.
turbopuffer
We host 1T+ documents, handle writes at 10M+ writes/s, and serve 10k+ queries/s. We are ready for far more. We hope youβll trust us with your queries.