Package org.apache.druid.segment.realtime.firehose

Class EventReceiverFirehoseFactory.EventReceiverFirehose

java.lang.Object

org.apache.druid.segment.realtime.firehose.EventReceiverFirehoseFactory.EventReceiverFirehose

All Implemented Interfaces:

Closeable, AutoCloseable, org.apache.druid.data.input.Firehose, ChatHandler, EventReceiverFirehoseMetric

Enclosing class:

EventReceiverFirehoseFactory

public class EventReceiverFirehoseFactory.EventReceiverFirehose
extends Object
implements ChatHandler, org.apache.druid.data.input.Firehose, EventReceiverFirehoseMetric

Apart from adhering to Firehose contract regarding concurrency, this class has two methods that might be called concurrently with any other methods and each other, from arbitrary number of threads: addAll(java.io.InputStream, javax.servlet.http.HttpServletRequest) and shutdown(java.lang.String, javax.servlet.http.HttpServletRequest). Concurrent data flow: in addAll(java.io.InputStream, javax.servlet.http.HttpServletRequest) (can be called concurrently with any other methods and other calls to addAll(java.io.InputStream, javax.servlet.http.HttpServletRequest)) rows are pushed into buffer. The single Firehose "consumer" thread calls hasMore() and nextRow(), where rows are taken out from the other end of the buffer queue. This class creates and manages one thread (delayedCloseExecutor) for calling close() asynchronously in response to a shutdown(java.lang.String, javax.servlet.http.HttpServletRequest) request, or after this Firehose has been idle (no calls to addAll(java.io.InputStream, javax.servlet.http.HttpServletRequest)) for EventReceiverFirehoseFactory.maxIdleTimeMillis.

Method Summary

Modifier and Type	Method	Description
javax.ws.rs.core.Response	addAll(InputStream in, javax.servlet.http.HttpServletRequest req)	This method might be called concurrently from multiple threads, if multiple requests arrive to the server at the same time (possibly exact duplicates).
void	close()	This method is synchronized because it might be called concurrently from multiple threads: from delayedCloseExecutor, and from the thread that creates and uses the Firehose object.
long	getBytesReceived()	Return the number of bytes received by the firehose.
int	getCapacity()	Return the capacity of the buffer.
int	getCurrentBufferSize()	Return the current number of InputRow that are stored in the buffer.
boolean	hasMore()
org.apache.druid.data.input.InputRow	nextRow()
javax.ws.rs.core.Response	shutdown(String shutoffTimeMillis, javax.servlet.http.HttpServletRequest req)	This method might be called concurrently from multiple threads, if multiple shutdown requests arrive at the same time.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.druid.data.input.Firehose

nextRowWithRaw

Method Detail

addAll

public javax.ws.rs.core.Response addAll(InputStream in,
                                        @Context
                                        javax.servlet.http.HttpServletRequest req)
                                 throws com.fasterxml.jackson.core.JsonProcessingException

This method might be called concurrently from multiple threads, if multiple requests arrive to the server at the same time (possibly exact duplicates). Concurrency is controlled in checkProducerSequence(javax.servlet.http.HttpServletRequest, java.lang.String, com.fasterxml.jackson.databind.ObjectMapper), where only requests with "X-Firehose-Producer-Seq" number greater than the max "X-Firehose-Producer-Seq" in previously arrived requests are allowed to proceed. After that check requests don't synchronize with each other and therefore if two large batches are sent with little interval, the events from the batches might be mixed up in buffer (if two addRows(Iterable) are executed concurrently).

Throws:

com.fasterxml.jackson.core.JsonProcessingException

hasMore

public boolean hasMore()

Specified by:

hasMore in interface org.apache.druid.data.input.Firehose

nextRow

@Nullable
public org.apache.druid.data.input.InputRow nextRow()

Specified by:

nextRow in interface org.apache.druid.data.input.Firehose

getCurrentBufferSize

public int getCurrentBufferSize()

Description copied from interface: EventReceiverFirehoseMetric

Return the current number of InputRow that are stored in the buffer.

Specified by:

getCurrentBufferSize in interface EventReceiverFirehoseMetric

getCapacity

public int getCapacity()

Description copied from interface: EventReceiverFirehoseMetric

Return the capacity of the buffer.

Specified by:

getCapacity in interface EventReceiverFirehoseMetric

getBytesReceived

public long getBytesReceived()

Description copied from interface: EventReceiverFirehoseMetric

Return the number of bytes received by the firehose.

Specified by:

getBytesReceived in interface EventReceiverFirehoseMetric

close

public void close()

This method is synchronized because it might be called concurrently from multiple threads: from delayedCloseExecutor, and from the thread that creates and uses the Firehose object.

Specified by:

close in interface AutoCloseable

Specified by:

close in interface Closeable

Specified by:

close in interface org.apache.druid.data.input.Firehose

shutdown

public javax.ws.rs.core.Response shutdown(String shutoffTimeMillis,
                                          @Context
                                          javax.servlet.http.HttpServletRequest req)

This method might be called concurrently from multiple threads, if multiple shutdown requests arrive at the same time. No attempts are made to synchronize such requests, or prioritize them a-la "latest shutdown time wins" or "soonest shutdown time wins". delayedCloseExecutor's logic (see createDelayedCloseExecutor()) is indifferent to shutdown times jumping in arbitrary directions. But once a shutdown request is made, it can't be cancelled entirely, the shutdown time could only be rescheduled with a new request.