Package org.apache.lucene.index

Class BufferedUpdatesStream

java.lang.Object

org.apache.lucene.index.BufferedUpdatesStream

All Implemented Interfaces:

Accountable

final class BufferedUpdatesStream
extends java.lang.Object
implements Accountable

Tracks the stream of FrozenBufferedUpdates. When DocumentsWriterPerThread flushes, its buffered deletes and updates are appended to this stream and immediately resolved (to actual docIDs, per segment) using the indexing thread that triggered the flush for concurrency. When a merge kicks off, we sync to ensure all resolving packets complete. We also apply to all segments when NRT reader is pulled, commit/close is called, or when too many deletes or updates are buffered and must be flushed (by RAM usage or by count). Each packet is assigned a generation, and each flushed or merged segment is also assigned a generation, so we can track which BufferedDeletes packets to apply to any given segment.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
(package private) static class	BufferedUpdatesStream.ApplyDeletesResult
private static class	BufferedUpdatesStream.FinishedSegments	Tracks the contiguous range of packets that have finished resolving.
(package private) static class	BufferedUpdatesStream.SegmentState	Holds all per-segment internal state used while resolving deletions.

Field Summary

Fields

Modifier and Type	Field	Description
private java.util.concurrent.atomic.AtomicLong	bytesUsed
private BufferedUpdatesStream.FinishedSegments	finishedSegments
private InfoStream	infoStream
private long	nextGen
private java.util.concurrent.atomic.AtomicInteger	numTerms
private java.util.Set<FrozenBufferedUpdates>	updates

Fields inherited from interface org.apache.lucene.util.Accountable

NULL_ACCOUNTABLE

Constructor Summary

Constructors

Constructor	Description
BufferedUpdatesStream(InfoStream infoStream)

Method Summary

Modifier and Type	Method	Description
(package private) boolean	any()
private boolean	checkDeleteStats()
(package private) void	clear()	Only used by IW.rollback
(package private) void	finished(FrozenBufferedUpdates packet)	Called by indexing threads once they are fully done resolving all deletes for the provided delGen.
(package private) void	finishedSegment(long delGen)
(package private) long	getCompletedDelGen()	All frozen packets up to and including this del gen are guaranteed to be finished.
(package private) long	getNextGen()
(package private) int	getPendingUpdatesCount()
(package private) int	numTerms()
(package private) long	push(FrozenBufferedUpdates packet)
long	ramBytesUsed()	Return the memory usage of this object in bytes.
(package private) boolean	stillRunning(long delGen)	Returns true if this delGen is still running.
private void	waitApply(java.util.Set<FrozenBufferedUpdates> waitFor, IndexWriter writer)
(package private) void	waitApplyAll(IndexWriter writer)	Waits for all in-flight packets, which are already being resolved concurrently by indexing threads, to finish.
(package private) void	waitApplyForMerge(java.util.List<SegmentCommitInfo> mergeInfos, IndexWriter writer)	Waits only for those in-flight packets that apply to these merge segments.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.lucene.util.Accountable

getChildResources

Field Detail

updates

private final java.util.Set<FrozenBufferedUpdates> updates

nextGen

private long nextGen

finishedSegments

private final BufferedUpdatesStream.FinishedSegments finishedSegments

infoStream

private final InfoStream infoStream

bytesUsed

private final java.util.concurrent.atomic.AtomicLong bytesUsed

numTerms

private final java.util.concurrent.atomic.AtomicInteger numTerms

Constructor Detail

BufferedUpdatesStream

BufferedUpdatesStream(InfoStream infoStream)

Method Detail

push

long push(FrozenBufferedUpdates packet)

getPendingUpdatesCount

int getPendingUpdatesCount()

clear

void clear()

Only used by IW.rollback

any

boolean any()

numTerms

int numTerms()

ramBytesUsed

public long ramBytesUsed()

Description copied from interface: Accountable

Return the memory usage of this object in bytes. Negative values are illegal.

Specified by:

ramBytesUsed in interface Accountable

waitApplyAll

void waitApplyAll(IndexWriter writer)
           throws java.io.IOException

Waits for all in-flight packets, which are already being resolved concurrently by indexing threads, to finish. Returns true if there were any new deletes or updates. This is called for refresh, commit.

Throws:

java.io.IOException

stillRunning

boolean stillRunning(long delGen)

Returns true if this delGen is still running.

finishedSegment

void finishedSegment(long delGen)

finished

void finished(FrozenBufferedUpdates packet)

Called by indexing threads once they are fully done resolving all deletes for the provided delGen. We track the completed delGens and record the maximum delGen for which all prior delGens, inclusive, are completed, so that it's safe for doc values updates to apply and write.

getCompletedDelGen

long getCompletedDelGen()

All frozen packets up to and including this del gen are guaranteed to be finished.

waitApplyForMerge

void waitApplyForMerge(java.util.List<SegmentCommitInfo> mergeInfos,
                       IndexWriter writer)
                throws java.io.IOException

Waits only for those in-flight packets that apply to these merge segments. This is called when a merge needs to finish and must ensure all deletes to the merging segments are resolved.

Throws:

java.io.IOException

waitApply

private void waitApply(java.util.Set<FrozenBufferedUpdates> waitFor,
                       IndexWriter writer)
                throws java.io.IOException

Throws:

java.io.IOException

getNextGen

long getNextGen()

checkDeleteStats

private boolean checkDeleteStats()