Package org.apache.pinot.segment.local.segment.creator.impl.inv.geospatial

Class OffHeapH3IndexCreator

java.lang.Object

org.apache.pinot.segment.local.segment.creator.impl.inv.geospatial.BaseH3IndexCreator

org.apache.pinot.segment.local.segment.creator.impl.inv.geospatial.OffHeapH3IndexCreator

All Implemented Interfaces:

Closeable, AutoCloseable, GeoSpatialIndexCreator

public class OffHeapH3IndexCreator
extends BaseH3IndexCreator

H3 index creator that uses off-heap memory.

The posting lists (map from H3 id to doc ids) are initially stored in a TreeMap, then flushed into a file for every 100,000 entries. After all the documents are added, we read all the posting lists from the file and merge them using a priority queue to calculate the final posting lists.

Off-heap creator uses less heap memory, but is more expensive on computation and needs flush data to disk which can slow down the creation because of the IO latency. Use off-heap creator in the environment where there is limited heap memory or garbage collection can cause performance issue (e.g. index creation at loading time on Pinot Server).

Field Summary

Fields inherited from class org.apache.pinot.segment.local.segment.creator.impl.inv.geospatial.BaseH3IndexCreator

HEADER_LENGTH, VERSION

Constructor Summary

Constructors

Constructor	Description
OffHeapH3IndexCreator(File indexDir, String columnName, H3IndexResolution resolution)

Method Summary

Modifier and Type	Method	Description
void	add(org.locationtech.jts.geom.Geometry geometry)
void	close()
void	seal()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

OffHeapH3IndexCreator

public OffHeapH3IndexCreator(File indexDir,
                             String columnName,
                             H3IndexResolution resolution)
                      throws IOException

Throws:

IOException

Method Detail

add

public void add(org.locationtech.jts.geom.Geometry geometry)
         throws IOException

Specified by:

add in interface GeoSpatialIndexCreator

Overrides:

add in class BaseH3IndexCreator

Throws:

IOException

seal

public void seal()
          throws IOException

Throws:

IOException

close

public void close()
           throws IOException

Specified by:

close in interface AutoCloseable

Specified by:

close in interface Closeable

Overrides:

close in class BaseH3IndexCreator

Throws:

IOException