Package org.opensearch.index.rankeval

Interface EvaluationMetric

All Superinterfaces:

NamedWriteable, org.opensearch.core.xcontent.ToXContent, org.opensearch.core.xcontent.ToXContentObject, Writeable

All Known Implementing Classes:

DiscountedCumulativeGain, ExpectedReciprocalRank, MeanReciprocalRank, PrecisionAtK, RecallAtK

public interface EvaluationMetric
extends org.opensearch.core.xcontent.ToXContentObject, NamedWriteable

Implementations of EvaluationMetric need to provide a way to compute the quality metric for a result list returned by some search (@link SearchHits) and a list of rated documents.

Nested Class Summary

Nested classes/interfaces inherited from interface org.opensearch.core.xcontent.ToXContent

org.opensearch.core.xcontent.ToXContent.DelegatingMapParams, org.opensearch.core.xcontent.ToXContent.MapParams, org.opensearch.core.xcontent.ToXContent.Params

Nested classes/interfaces inherited from interface org.opensearch.common.io.stream.Writeable

Writeable.Reader<V extends Object>, Writeable.WriteableRegistry, Writeable.Writer<V extends Object>

Field Summary

Fields inherited from interface org.opensearch.core.xcontent.ToXContent

EMPTY_PARAMS

Method Summary

Modifier and Type	Method	Description
default double	combine(Collection<EvalQueryQuality> partialResults)	Combine several EvalQueryQuality results into the overall evaluation score.
EvalQueryQuality	evaluate(String taskId, SearchHit[] hits, List<RatedDocument> ratedDocs)	Evaluates a single ranking evaluation case.
static List<org.opensearch.index.rankeval.RatedDocument.DocumentKey>	filterUnratedDocuments(List<RatedSearchHit> ratedHits)	Filter RatedSearchHits that do not have a rating.
default OptionalInt	forcedSearchSize()	Metrics can define a size of the search hits windows they want to retrieve by overwriting this method.
static List<RatedSearchHit>	joinHitsWithRatings(SearchHit[] hits, List<RatedDocument> ratedDocs)	Joins hits with rated documents using the joint _index/_id document key.

Methods inherited from interface org.opensearch.common.io.stream.NamedWriteable

getWriteableName

Methods inherited from interface org.opensearch.core.xcontent.ToXContent

toXContent

Methods inherited from interface org.opensearch.core.xcontent.ToXContentObject

isFragment

Methods inherited from interface org.opensearch.common.io.stream.Writeable

writeTo

Method Details

evaluate

EvalQueryQuality evaluate(String taskId,
 SearchHit[] hits,
 List<RatedDocument> ratedDocs)

Evaluates a single ranking evaluation case.

Parameters:

taskId - an identifier of the query for which the search ranking is evaluated

hits - the search result hits

ratedDocs - the documents that contain the document rating for this query case

Returns:

an EvalQueryQuality instance that contains the metric score with respect to the provided search hits and ratings

joinHitsWithRatings

static List<RatedSearchHit> joinHitsWithRatings(SearchHit[] hits,
 List<RatedDocument> ratedDocs)

Joins hits with rated documents using the joint _index/_id document key.

filterUnratedDocuments

static List<org.opensearch.index.rankeval.RatedDocument.DocumentKey> filterUnratedDocuments(List<RatedSearchHit> ratedHits)

Filter RatedSearchHits that do not have a rating.

combine

default double combine(Collection<EvalQueryQuality> partialResults)

Combine several EvalQueryQuality results into the overall evaluation score. This defaults to averaging over the partial results, but can be overwritten to obtain a different behavior.

forcedSearchSize

default OptionalInt forcedSearchSize()

Metrics can define a size of the search hits windows they want to retrieve by overwriting this method. The default implementation returns an empty optional.

Returns:

the number of search hits this metrics requests