Class QualityBenchmark

java.lang.Object
org.apache.lucene.benchmark.quality.QualityBenchmark

public class QualityBenchmark extends Object
Main entry point for running a quality benchmark.

There are two main configurations for running a quality benchmark:

  • Against existing judgements.
  • For submission (e.g. for a contest).
The first configuration requires a non null Judge. The second configuration requires a non null SubmissionLogger.
  • Field Details

    • qualityQueries

      protected QualityQuery[] qualityQueries
      Quality Queries that this quality benchmark would execute.
    • qqParser

      protected QualityQueryParser qqParser
      Parser for turning QualityQueries into Lucene Queries.
    • searcher

      protected IndexSearcher searcher
      Index to be searched.
    • docNameField

      protected String docNameField
      index field to extract doc name for each search result; used for judging the results.
  • Constructor Details

    • QualityBenchmark

      public QualityBenchmark(QualityQuery[] qqs, QualityQueryParser qqParser, IndexSearcher searcher, String docNameField)
      Create a QualityBenchmark.
      Parameters:
      qqs - quality queries to run.
      qqParser - parser for turning QualityQueries into Lucene Queries.
      searcher - index to be searched.
      docNameField - name of field containing the document name. This allows to extract the doc name for search results, and is important for judging the results.
  • Method Details

    • execute

      public QualityStats[] execute(Judge judge, SubmissionReport submitRep, PrintWriter qualityLog) throws Exception
      Run the quality benchmark.
      Parameters:
      judge - the judge that can tell if a certain result doc is relevant for a certain quality query. If null, no judgements would be made. Usually null for a submission run.
      submitRep - submission report is created if non null.
      qualityLog - If not null, quality run data would be printed for each query.
      Returns:
      QualityStats of each quality query that was executed.
      Throws:
      Exception - if quality benchmark failed to run.
    • getMaxQueries

      public int getMaxQueries()
      Returns:
      the maximum number of quality queries to run. Useful at debugging.
    • setMaxQueries

      public void setMaxQueries(int maxQueries)
      Set the maximum number of quality queries to run. Useful at debugging.
    • getMaxResults

      public int getMaxResults()
      Returns:
      the maximum number of results to collect for each quality query.
    • setMaxResults

      public void setMaxResults(int maxResults)
      set the maximum number of results to collect for each quality query.