How Segment Merging Affects Search Performance

If you’ve worked with Lucene for any length of time, you’ve probably heard about segments. You know that Lucene writes documents to immutable segments, that too many segments slow down searches, and that merges consolidate them. But how exactly does merging affect search performance? And why does the same process that improves query latency sometimes cause severe slowdowns?

This post explores the relationship between segment merging and search performance: the mechanics of how merges work, why they’re necessary, the tradeoffs involved, and how to tune them for your workload.

The problem: too many segments

Lucene uses an append only architecture. When you index a new document, it goes into a new segment (after the next flush). When you update a document, Lucene writes the updated version to a new segment and marks the old one as deleted. When you delete a document, it gets a deletion marker in its existing segment rather than being removed outright.

This design is simple and reliable, but it creates a problem: segments accumulate. With a 1 second refresh interval and continuous indexing, you can easily generate tens of thousands of segments per day on a single shard if nothing consolidates them.

Searching across many segments is expensive for a few reasons:

• Per-segment overhead. Each segment must be opened and searched independently, with results combined afterward. Searching 1,000 small segments is significantly slower than searching 10 segments of equivalent total size, even though the total amount of data is the same.
• File handle consumption. Each segment maintains multiple files on disk. A segment with several indexed fields can easily have dozens of files. A thousand segments multiplies that fast, and on systems with low file descriptor limits this becomes a real operational problem, not just a performance one.
• Deleted document overhead. Deleted documents still occupy disk space and still get touched by per-document data structures until the segments containing them are merged away.
• Resource fragmentation. Each segment, regardless of its size, consumes some baseline amount of memory and file handle overhead just for bookkeeping.

Segment merging solves all of these problems by consolidating multiple smaller segments into fewer, larger ones.

How merging works

Lucene’s IndexWriter automatically orchestrates merging based on a configured merge policy. The default in modern Lucene is TieredMergePolicy, which tries to merge segments of roughly equal size rather than just merging whatever segments happen to be adjacent.

When a merge occurs, Lucene:

1. Reads documents from the selected source segments.
2. Applies all pending deletions, dropping the marked documents instead of copying them forward.
3. Writes a new, consolidated segment containing only the surviving documents.
4. Deletes the old segments only after the new segment is fully written and committed, so a crash mid merge can’t corrupt the index.

The policy decides which segments to merge based on a few conditions:

• Segment count per tier. Once a size tier accumulates more segments than TieredMergePolicy’s segmentsPerTier setting allows (10.0 by default in most recent versions, though older releases used different defaults), a merge gets scheduled for that tier.
• Deleted document ratio. When a segment’s percentage of deleted documents crosses deletesPctAllowed, Lucene prioritizes merging it to reclaim that wasted space, even if it wouldn’t otherwise be a natural merge candidate based on size alone.
• Explicit calls. You can also force a merge directly via IndexWriter.forceMerge(), or through whatever equivalent API your search platform exposes on top of Lucene.

TieredMergePolicy prefers merging similarly sized segments because this keeps the total amount of merge work bounded as the index grows, rather than letting it blow up.

The dual nature of merge impact

Here’s the part that trips people up: segment merging affects search performance in two opposite directions at once.

The upside: faster searches

Fewer segments mean faster searches, full stop. This is the entire reason merges exist. Consolidating many small segments into one larger segment means:

• Query execution has fewer segments to open and search through.
• Combining results across segments costs less, since there are fewer of them.
• Per-segment metadata overhead goes away for every segment that gets merged out of existence.
• Deleted documents are physically dropped, so there’s less dead data sitting around for queries to wade through.

This is why a freshly merged index so often shows a noticeable drop in query latency compared to the same index before the merge ran.

The downside: merge induced latency

Merges aren’t free. They burn CPU rewriting documents into the new segment, I/O reading from the source segments and writing the merged one, and memory for buffering along the way.

While a merge is running:

• Search latency can spike, because the merge is competing with queries for CPU and I/O.
• Indexing throughput can drop, since the merge is consuming write bandwidth that would otherwise go to flushing new segments.
• Memory pressure goes up, from the buffers the merge needs while it works.

This gets worse when multiple merges run at once, sometimes called a merge storm. A misconfigured merge policy that lets too many merges trigger simultaneously, often right after a bulk indexing job, can genuinely degrade a production system.

So the tradeoff is this: you need merges to keep search fast over time, but the merges themselves can slow search down while they’re actually running.

Tuning merges for your workload

The way to manage this tradeoff is to understand your workload’s read/write balance and configure the merge policy around it.

TieredMergePolicy knobs worth knowing

• segmentsPerTier: how many segments of a given size tier are allowed before a merge triggers. Lower values mean more aggressive merging, fewer segments, and generally better query latency, at the cost of more background merge work. This is a good dial to turn down for query heavy, low indexing-rate workloads.
• maxMergeAtOnce: how many segments can be merged together in a single merge operation. This effectively controls merge parallelism and how large any one merge job gets.
• floorSegmentMB (sometimes seen as floorSegmentSize): segments smaller than this are treated as if they were this size for merge selection purposes. This stops you from accumulating a long tail of tiny segments that never get prioritized for merging.
• deletesPctAllowed: the maximum percentage of deleted documents Lucene will tolerate in a segment before forcing it into a merge regardless of size. Valid range is 20 to 50, with the exact default varying a bit by Lucene version.

Strategies by workload shape

Read heavy workloads (lots of search traffic, relatively few writes):

• Use a lower segmentsPerTier to keep the segment count down.
• Consider running forceMerge during off peak hours to consolidate down to a small number of segments.
• Be careful with forceMerge on an index that’s still actively being written to. It’s expensive, and the benefit decays quickly once new segments start flushing again.

Write heavy workloads (high indexing throughput):

• Use a higher segmentsPerTier so merges fire less often and don’t compete as much with indexing.
• Accept that query latency may run a bit higher during peak indexing, since segment count will naturally stay higher.
• If you can, schedule any aggressive consolidation for quieter traffic windows.

Time series data:

• There’s ongoing work in the Lucene project on a time aware merge policy that groups segments into time windows so that merges never mix old and new data together. As of this writing it’s still in development rather than something you can just drop in, so if your workload is heavily time partitioned, it’s worth keeping an eye on but not yet something to plan around.

Vector search:

• Indices using HNSW based vector fields have extra merge overhead, because the graph structure itself has to be rebuilt during a merge, not just the postings. Keeping segment counts low matters even more here, since rebuilding HNSW graphs across many small segments adds up fast.

A word on forceMerge

forceMerge is powerful and a little dangerous. It can merge segments all the way down to a single one if you let it, which gives you the best possible query performance, but the operation itself is heavy on CPU and I/O the entire time it runs. Treat it as a maintenance window tool rather than something you run casually against a live, actively-indexed system.

What to monitor

A few signals are worth keeping an eye on if you suspect merging is affecting your search performance, in either direction:

• Segment count over time. A segment count that keeps climbing usually means merges aren’t keeping pace with how fast new segments are being flushed.
• Merge time and merge queue depth. Long merge times point to I/O bottlenecks; a growing queue means merges are falling behind.
• Deleted document ratio per segment. High delete ratios mean wasted query effort sitting around until the next merge clears it out.
• Search latency specifically during merge windows. If your p99 latency spikes line up with merge activity, that’s your contention right there.

Common pitfalls

A few patterns show up again and again in practice:

• Too many tiny segments. Usually caused by a floorSegmentMB that’s set too low, or refresh intervals that are too aggressive for the write rate. Shows up as high query latency and, in bad cases, file handle exhaustion.
• Merge storms. A batch of segments all becomes merge eligible at once, often right after a bulk load finishes, and several merges kick off in parallel and saturate I/O.
• Under merging. segmentsPerTier set too high for the workload. Segments pile up faster than they’re consolidated, and query latency degrades gradually until someone notices.
• Over merging. segmentsPerTier set too low. The system spends more time merging than indexing, throughput suffers, and merge I/O can end up competing with queries at the worst possible moments.

Wrapping up

Segment merging isn’t some background detail you can safely ignore. It’s a first order performance concern that touches both query latency and indexing throughput directly. The append only design that makes Lucene simple and crash safe just moves the complexity somewhere else, and that somewhere else is the merge process.

The right configuration depends entirely on your workload. Read heavy systems generally want more aggressive merging and fewer segments sitting around. Write heavy systems need a looser policy so indexing doesn’t get starved. Vector search and other specialized setups bring their own considerations on top of that.

Keep an eye on your segment counts, watch what your merge policy is actually doing under load, and remember the goal isn’t to eliminate merges entirely, it’s to manage them so they improve search performance instead of quietly working against it.