com.amazonaws.services.kinesisanalytics.flink.connectors.producer.impl

Class FirehoseProducer<O extends FirehoseProducer.UserRecordResult,R extends com.amazonaws.services.kinesisfirehose.model.Record>

java.lang.Object

com.amazonaws.services.kinesisanalytics.flink.connectors.producer.impl.FirehoseProducer<O,R>

All Implemented Interfaces:

IProducer<O,R>

@ThreadSafe
public class FirehoseProducer<O extends FirehoseProducer.UserRecordResult,R extends com.amazonaws.services.kinesisfirehose.model.Record>
extends Object
implements IProducer<O,R>

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	FirehoseProducer.UserRecordResult

Constructor Summary

Constructors

Constructor and Description
FirehoseProducer(String deliveryStream, com.amazonaws.services.kinesisfirehose.AmazonKinesisFirehose firehoseClient, Properties configProps)

Method Summary

Modifier and Type	Method and Description
com.google.common.util.concurrent.ListenableFuture<O>	addUserRecord(R record) This method should send data to an specific destination.
com.google.common.util.concurrent.ListenableFuture<O>	addUserRecord(R record, long timeoutInMillis) This method is responsible for taking a lock adding a Record into the producerBuffer, in case the producerBuffer is full waits releasing the lock for the given bufferFullWaitTimeoutInMillis.
void	destroy() Make sure that any pending scheduled thread terminates before closing as well as cleans the producerBuffer pool, allowing GC to collect.
void	flush() This method instructs the flusher thread to perform a flush on the buffer without waiting for completion.
void	flushSync() This method instructs the flusher thread to perform the flush on the buffer and wait for the completion.
int	getOutstandingRecordsCount() Should return the number of outstanding records if the producer implements buffering.
boolean	isDestroyed() Returns whether the producer has been destroyed or not
boolean	isFlushFailed() A flag representing whether the flush has failed or not.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

FirehoseProducer

public FirehoseProducer(String deliveryStream,
                        com.amazonaws.services.kinesisfirehose.AmazonKinesisFirehose firehoseClient,
                        Properties configProps)

Method Detail

addUserRecord

public com.google.common.util.concurrent.ListenableFuture<O> addUserRecord(R record)
                                                                    throws Exception

Description copied from interface: IProducer

This method should send data to an specific destination.

Specified by:

addUserRecord in interface IProducer<O extends FirehoseProducer.UserRecordResult,R extends com.amazonaws.services.kinesisfirehose.model.Record>

Parameters:

record - the type of data to be sent

Returns:

a ListenableFuture with the result for the operation.

Throws:

Exception

addUserRecord

public com.google.common.util.concurrent.ListenableFuture<O> addUserRecord(R record,
                                                                           long timeoutInMillis)
                                                                    throws TimeoutExpiredException,
                                                                           InterruptedException

This method is responsible for taking a lock adding a Record into the producerBuffer, in case the producerBuffer is full waits releasing the lock for the given bufferFullWaitTimeoutInMillis. There are cases where the producerBuffer cannot be flushed then this method keeps waiting until the given operation timeout passed as timeoutInMillis

Specified by:

addUserRecord in interface IProducer<O extends FirehoseProducer.UserRecordResult,R extends com.amazonaws.services.kinesisfirehose.model.Record>

Parameters:

record - the type of data to be buffered

timeoutInMillis - the operation timeout in case the record cannot be added into the producerBuffer.

Returns:

Throws:

TimeoutExpiredException - if the operation got stuck and is not able to proceed.

InterruptedException - if any thread interrupted the current thread before or while the current thread was waiting for a notification. The interrupted status of the current thread is cleared when this exception is thrown.

destroy

public void destroy()
             throws Exception

Make sure that any pending scheduled thread terminates before closing as well as cleans the producerBuffer pool, allowing GC to collect.

Specified by:

destroy in interface IProducer<O extends FirehoseProducer.UserRecordResult,R extends com.amazonaws.services.kinesisfirehose.model.Record>

Throws:

Exception

isDestroyed

public boolean isDestroyed()

Description copied from interface: IProducer

Returns whether the producer has been destroyed or not

Specified by:

isDestroyed in interface IProducer<O extends FirehoseProducer.UserRecordResult,R extends com.amazonaws.services.kinesisfirehose.model.Record>

Returns:

getOutstandingRecordsCount

public int getOutstandingRecordsCount()

Description copied from interface: IProducer

Should return the number of outstanding records if the producer implements buffering.

Specified by:

getOutstandingRecordsCount in interface IProducer<O extends FirehoseProducer.UserRecordResult,R extends com.amazonaws.services.kinesisfirehose.model.Record>

Returns:

an integer with the number of outstanding records.

isFlushFailed

public boolean isFlushFailed()

Description copied from interface: IProducer

A flag representing whether the flush has failed or not.

Specified by:

isFlushFailed in interface IProducer<O extends FirehoseProducer.UserRecordResult,R extends com.amazonaws.services.kinesisfirehose.model.Record>

Returns:

boolean representing whether the success of failure of flush buffer operation.

flush

public void flush()

This method instructs the flusher thread to perform a flush on the buffer without waiting for completion.

This implementation does not guarantee the whole buffer is flushed or if the flusher thread has completed the flush or not. In order to flush all records and wait until completion, use {@link #flushSync()}

Specified by:

flush in interface IProducer<O extends FirehoseProducer.UserRecordResult,R extends com.amazonaws.services.kinesisfirehose.model.Record>

flushSync

public void flushSync()

This method instructs the flusher thread to perform the flush on the buffer and wait for the completion.

This implementation is useful once there is a need to guarantee the buffer is flushed before making further progress. i.e. Shutting down the producer. i.e. Taking synchronous snapshots.

The caller needs to make sure to assert the status of isFlushFailed() in order guarantee whether the flush has successfully completed or not.

Specified by:

flushSync in interface IProducer<O extends FirehoseProducer.UserRecordResult,R extends com.amazonaws.services.kinesisfirehose.model.Record>