org.apache.hudi.timeline.service.handlers.marker

Class MarkerCreationDispatchingRunnable

java.lang.Object

org.apache.hudi.timeline.service.handlers.marker.MarkerCreationDispatchingRunnable

All Implemented Interfaces:

Runnable

public class MarkerCreationDispatchingRunnable
extends Object
implements Runnable

A runnable that performs periodic, batched creation of markers for write operations.

Field Summary

Fields

Modifier and Type	Field and Description
static org.slf4j.Logger	LOG

Constructor Summary

Constructors

Constructor and Description
MarkerCreationDispatchingRunnable(Map<String,MarkerDirState> markerDirStateMap, ExecutorService executorService)

Method Summary

Modifier and Type	Method and Description
void	run() Dispatches the marker creation requests that can be process to a worker thread of batch processing the requests.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

public static final org.slf4j.Logger LOG

Constructor Detail

MarkerCreationDispatchingRunnable

public MarkerCreationDispatchingRunnable(Map<String,MarkerDirState> markerDirStateMap,
                                         ExecutorService executorService)

Method Detail

run

public void run()

Dispatches the marker creation requests that can be process to a worker thread of batch processing the requests. For each marker directory, goes through the following steps: (1) find the next available file index for writing. If no file index is available, skip the processing of this marker directory; (2) fetch the pending marker creation requests for this marker directory. If there is no request, skip this marker directory; (3) put the marker directory, marker dir state, list of requests futures, and the file index to a MarkerDirRequestContext instance and add the instance to the request context list. If the request context list is not empty, spins up a worker thread, MarkerCreationBatchingRunnable, and pass all the request context to the thread for batch processing. The thread is responsible for responding to the request futures directly.

Specified by:

run in interface Runnable