org.apache.lucene.search
Class CachingCollector

java.lang.Object
  extended by org.apache.lucene.search.Collector
      extended by org.apache.lucene.search.CachingCollector

public abstract class CachingCollector
extends Collector

Caches all docs, and optionally also scores, coming from a search, and is then able to replay them to another collector. You specify the max RAM this class may use. Once the collection is done, call isCached(). If this returns true, you can use replay(org.apache.lucene.search.Collector) against a new collector. If it returns false, this means too much RAM was required and you must instead re-run the original search.

NOTE: this class consumes 4 (or 8 bytes, if scoring is cached) per collected document. If the result set is large this can easily be a very substantial amount of RAM!

NOTE: this class caches at least 128 documents before checking RAM limits.

See the Lucene contrib/grouping module for more details including a full code example.

WARNING: This API is experimental and might change in incompatible ways in the next release.

Field Summary
protected  int base
           
protected  List<int[]> cachedDocs
           
protected  List<org.apache.lucene.search.CachingCollector.SegStart> cachedSegs
           
protected  int[] curDocs
           
protected  int lastDocBase
           
protected  int maxDocsToCache
           
protected  Collector other
           
protected  int upto
           
 
Method Summary
 boolean acceptsDocsOutOfOrder()
          Return true if this collector does not require the matching docIDs to be delivered in int sort order (smallest to largest) to Collector.collect(int).
static CachingCollector create(boolean acceptDocsOutOfOrder, boolean cacheScores, double maxRAMMB)
          Creates a CachingCollector which does not wrap another collector.
static CachingCollector create(Collector other, boolean cacheScores, double maxRAMMB)
          Create a new CachingCollector that wraps the given collector and caches documents and scores up to the specified RAM threshold.
static CachingCollector create(Collector other, boolean cacheScores, int maxDocsToCache)
          Create a new CachingCollector that wraps the given collector and caches documents and scores up to the specified max docs threshold.
 boolean isCached()
           
abstract  void replay(Collector other)
          Replays the cached doc IDs (and scores) to the given Collector.
 void setNextReader(IndexReader reader, int docBase)
          Called before collecting from each IndexReader.
 
Methods inherited from class org.apache.lucene.search.Collector
collect, setScorer
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

other

protected final Collector other

maxDocsToCache

protected final int maxDocsToCache

cachedSegs

protected final List<org.apache.lucene.search.CachingCollector.SegStart> cachedSegs

cachedDocs

protected final List<int[]> cachedDocs

curDocs

protected int[] curDocs

upto

protected int upto

base

protected int base

lastDocBase

protected int lastDocBase
Method Detail

create

public static CachingCollector create(boolean acceptDocsOutOfOrder,
                                      boolean cacheScores,
                                      double maxRAMMB)
Creates a CachingCollector which does not wrap another collector. The cached documents and scores can later be replayed.

Parameters:
acceptDocsOutOfOrder - whether documents are allowed to be collected out-of-order

create

public static CachingCollector create(Collector other,
                                      boolean cacheScores,
                                      double maxRAMMB)
Create a new CachingCollector that wraps the given collector and caches documents and scores up to the specified RAM threshold.

Parameters:
other - the Collector to wrap and delegate calls to.
cacheScores - whether to cache scores in addition to document IDs. Note that this increases the RAM consumed per doc
maxRAMMB - the maximum RAM in MB to consume for caching the documents and scores. If the collector exceeds the threshold, no documents and scores are cached.

create

public static CachingCollector create(Collector other,
                                      boolean cacheScores,
                                      int maxDocsToCache)
Create a new CachingCollector that wraps the given collector and caches documents and scores up to the specified max docs threshold.

Parameters:
other - the Collector to wrap and delegate calls to.
cacheScores - whether to cache scores in addition to document IDs. Note that this increases the RAM consumed per doc
maxDocsToCache - the maximum number of documents for caching the documents and possible the scores. If the collector exceeds the threshold, no documents and scores are cached.

acceptsDocsOutOfOrder

public boolean acceptsDocsOutOfOrder()
Description copied from class: Collector
Return true if this collector does not require the matching docIDs to be delivered in int sort order (smallest to largest) to Collector.collect(int).

Most Lucene Query implementations will visit matching docIDs in order. However, some queries (currently limited to certain cases of BooleanQuery) can achieve faster searching if the Collector allows them to deliver the docIDs out of order.

Many collectors don't mind getting docIDs out of order, so it's important to return true here.

Specified by:
acceptsDocsOutOfOrder in class Collector

isCached

public boolean isCached()

setNextReader

public void setNextReader(IndexReader reader,
                          int docBase)
                   throws IOException
Description copied from class: Collector
Called before collecting from each IndexReader. All doc ids in Collector.collect(int) will correspond to reader. Add docBase to the current IndexReaders internal document id to re-base ids in Collector.collect(int).

Specified by:
setNextReader in class Collector
Parameters:
reader - next IndexReader
Throws:
IOException

replay

public abstract void replay(Collector other)
                     throws IOException
Replays the cached doc IDs (and scores) to the given Collector. If this instance does not cache scores, then Scorer is not set on other.setScorer as well as scores are not replayed.

Throws:
IllegalStateException - if this collector is not cached (i.e., if the RAM limits were too low for the number of documents + scores to cache).
IllegalArgumentException - if the given Collect's does not support out-of-order collection, while the collector passed to the ctor does.
IOException