Package org.opensearch.index.rankeval

Class MeanReciprocalRank

java.lang.Object
org.opensearch.index.rankeval.MeanReciprocalRank
All Implemented Interfaces:
NamedWriteable, Writeable, org.opensearch.core.xcontent.ToXContent, org.opensearch.core.xcontent.ToXContentObject, EvaluationMetric
public class MeanReciprocalRank extends Object implements EvaluationMetric
Metric implementing Mean Reciprocal Rank (https://en.wikipedia.org/wiki/Mean_reciprocal_rank).
By default documents with a rating equal or bigger than 1 are considered to be "relevant" for the reciprocal rank calculation. This value can be changes using the relevant_rating_threshold` parameter.

  • Field Details

  • Constructor Details

    • MeanReciprocalRank

      public MeanReciprocalRank()

    • MeanReciprocalRank

      public MeanReciprocalRank(int relevantRatingThreshold, int k)
      Metric implementing Mean Reciprocal Rank (https://en.wikipedia.org/wiki/Mean_reciprocal_rank).
      Parameters:
      relevantRatingThreshold - the rating value that a document needs to be regarded as "relevant". Defaults to 1.
      k - the search window size all request use.

  • Method Details

    • writeTo

      public void writeTo(StreamOutput out) throws IOException
      Specified by:
      writeTo in interface Writeable
      Throws:
      IOException

    • forcedSearchSize

      public OptionalInt forcedSearchSize()
      Description copied from interface: EvaluationMetric
      Metrics can define a size of the search hits windows they want to retrieve by overwriting this method. The default implementation returns an empty optional.
      Specified by:
      forcedSearchSize in interface EvaluationMetric
      Returns:
      the number of search hits this metrics requests

    • getWriteableName

      public String getWriteableName()
      Specified by:
      getWriteableName in interface NamedWriteable

    • getRelevantRatingThreshold

      public int getRelevantRatingThreshold()
      Return the rating threshold above which ratings are considered to be "relevant".

    • evaluate

      public EvalQueryQuality evaluate(String taskId, SearchHit[] hits, List<RatedDocument> ratedDocs)
      Compute ReciprocalRank based on provided relevant document IDs.
      Specified by:
      evaluate in interface EvaluationMetric
      Parameters:
      taskId - an identifier of the query for which the search ranking is evaluated
      hits - the search result hits
      ratedDocs - the documents that contain the document rating for this query case
      Returns:
      an EvalQueryQuality instance that contains the metric score with respect to the provided search hits and ratings

    • fromXContent

      public static MeanReciprocalRank fromXContent(org.opensearch.core.xcontent.XContentParser parser)

    • toXContent

      public org.opensearch.core.xcontent.XContentBuilder toXContent(org.opensearch.core.xcontent.XContentBuilder builder, org.opensearch.core.xcontent.ToXContent.Params params) throws IOException
      Specified by:
      toXContent in interface org.opensearch.core.xcontent.ToXContent
      Throws:
      IOException

    • equals

      public final boolean equals(Object obj)
      Overrides:
      equals in class Object

    • hashCode

      public final int hashCode()
      Overrides:
      hashCode in class Object