nl.vpro.elasticsearch.highlevel

Class HighLevelElasticSearchIterator<T>

java.lang.Object

nl.vpro.elasticsearch.highlevel.HighLevelElasticSearchIterator<T>

All Implemented Interfaces:

AutoCloseable, Iterator<T>, nl.vpro.elasticsearch.ElasticSearchIteratorInterface<T>, nl.vpro.elasticsearch.ElasticSearchIteratorMXBean, HighLevelElasticSearchIteratorMXBean, CloseableIterator<T>, CountedIterator<T>

public class HighLevelElasticSearchIterator<T>
extends Object
implements nl.vpro.elasticsearch.ElasticSearchIteratorInterface<T>, HighLevelElasticSearchIteratorMXBean

A wrapper around the Elastic Search scroll interface, to expose it as a simple Iterator

        try (ElasticSearchIterator<SearchHit> i = ElasticSearchIterator.searchHits(client)) {
        SearchSourceBuilder search = i.prepareSearchSource("pageupdates-publish");
        // fill your request here

        i.start() // optional

        i.forEachRemaining((node) -> {
             String url = node.get("url").textValue();
             if (i.getCount() % 1000 == 0) {
                 log.info("{}: {}", i.getCount(), url);

             }
         });
         }

 

Since:

2.22

Author:

Michiel Meeuwissen

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	HighLevelElasticSearchIterator.Builder<T>

Nested classes/interfaces inherited from interface nl.vpro.elasticsearch.ElasticSearchIteratorInterface

nl.vpro.elasticsearch.ElasticSearchIteratorInterface.TotalRelation

Nested classes/interfaces inherited from interface nl.vpro.util.CloseableIterator

CloseableIterator.WrappedCloseableIterator<S>, CloseableIterator.WrappedPeekingCloseableIterator<S>

Field Summary

Fields inherited from interface nl.vpro.elasticsearch.ElasticSearchIteratorInterface

SCROLL_IDS

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	HighLevelElasticSearchIterator(@NonNull org.elasticsearch.client.RestHighLevelClient client, Function<org.elasticsearch.search.SearchHit,T> adapt, Class<T> adaptTo, Duration scrollContext, String beanName, org.meeuw.math.windowed.WindowedEventRate rateMeasurerer, List<String> routingIds, org.elasticsearch.client.RequestOptions requestOptions, Boolean requestVersion)

Method Summary

Modifier and Type	Method and Description
static <T> Function<org.elasticsearch.search.SearchHit,T>	adapterTo(Class<T> clazz)
void	close()
protected void	findNext()
protected boolean	firstBatch()
float	getFraction()
org.elasticsearch.action.search.SearchResponse	getResponse()
@NonNull Optional<Long>	getSize()
Optional<nl.vpro.elasticsearch.ElasticSearchIteratorInterface.TotalRelation>	getSizeQualifier()
double	getSpeed()
boolean	hasNext()
T	next()
org.elasticsearch.search.builder.SearchSourceBuilder	prepareSearchSource(String... indices)
protected void	readResponse()
static HighLevelElasticSearchIterator<org.elasticsearch.search.SearchHit>	searchHits(org.elasticsearch.client.RestHighLevelClient client)
static HighLevelElasticSearchIterator.Builder<org.elasticsearch.search.SearchHit>	searchHitsBuilder(org.elasticsearch.client.RestHighLevelClient client)
static HighLevelElasticSearchIterator<com.fasterxml.jackson.databind.JsonNode>	sources(org.elasticsearch.client.RestHighLevelClient client)
void	start() Retrieves the first batch of results.
String	toString()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface nl.vpro.elasticsearch.ElasticSearchIteratorInterface

getCount, getETA, getETADate, getETASeconds, getScrollIds, getStart, getStartDate, getTotalSizeLong

Methods inherited from interface nl.vpro.util.CountedIterator

getTotalSize, of, of, of, of, peeking, peeking, spliterator, stream

Methods inherited from interface nl.vpro.util.CloseableIterator

closeQuietly, empty, of, peeking

Methods inherited from interface java.util.Iterator

forEachRemaining, remove

Constructor Detail

HighLevelElasticSearchIterator

protected HighLevelElasticSearchIterator(@NonNull
                                         @NonNull org.elasticsearch.client.RestHighLevelClient client,
                                         Function<org.elasticsearch.search.SearchHit,T> adapt,
                                         Class<T> adaptTo,
                                         Duration scrollContext,
                                         String beanName,
                                         org.meeuw.math.windowed.WindowedEventRate rateMeasurerer,
                                         List<String> routingIds,
                                         org.elasticsearch.client.RequestOptions requestOptions,
                                         Boolean requestVersion)

Method Detail

searchHitsBuilder

public static HighLevelElasticSearchIterator.Builder<org.elasticsearch.search.SearchHit> searchHitsBuilder(org.elasticsearch.client.RestHighLevelClient client)

searchHits

public static HighLevelElasticSearchIterator<org.elasticsearch.search.SearchHit> searchHits(org.elasticsearch.client.RestHighLevelClient client)

sources

public static HighLevelElasticSearchIterator<com.fasterxml.jackson.databind.JsonNode> sources(org.elasticsearch.client.RestHighLevelClient client)

adapterTo

public static <T> Function<org.elasticsearch.search.SearchHit,T> adapterTo(Class<T> clazz)

prepareSearchSource

public org.elasticsearch.search.builder.SearchSourceBuilder prepareSearchSource(String... indices)

hasNext

public boolean hasNext()

Specified by:

hasNext in interface Iterator<T>

start

public void start()

Retrieves the first batch of results. From now on you can't modified the query any more. This will also happen implicitely if you don't call this, but sometimes it make the code clearer when doing it explicitely.

findNext

protected void findNext()

getFraction

public float getFraction()

Specified by:

getFraction in interface nl.vpro.elasticsearch.ElasticSearchIteratorMXBean

firstBatch

protected boolean firstBatch()

readResponse

protected void readResponse()

next

public T next()

Specified by:

next in interface Iterator<T>

getSize

public @NonNull Optional<Long> getSize()

Specified by:

getSize in interface CountedIterator<T>

getSizeQualifier

public Optional<nl.vpro.elasticsearch.ElasticSearchIteratorInterface.TotalRelation> getSizeQualifier()

Specified by:

getSizeQualifier in interface nl.vpro.elasticsearch.ElasticSearchIteratorInterface<T>

getResponse

public org.elasticsearch.action.search.SearchResponse getResponse()

toString

public String toString()

Overrides:

toString in class Object

close

public void close()

Specified by:

close in interface AutoCloseable

Specified by:

close in interface CountedIterator<T>

getSpeed

public double getSpeed()

Specified by:

getSpeed in interface nl.vpro.elasticsearch.ElasticSearchIteratorInterface<T>

Specified by:

getSpeed in interface nl.vpro.elasticsearch.ElasticSearchIteratorMXBean