Package com.bakdata.kafka

Class KafkaStreamsApplication

  • All Implemented Interfaces:
    java.lang.AutoCloseable, java.lang.Runnable
    public abstract class KafkaStreamsApplication
extends KafkaApplication
implements java.lang.AutoCloseable

    The base class of the entry point of the streaming application.

    This class provides common configuration options e.g. KafkaApplication.brokers, productive for streaming application. Hereby it automatically populates the passed in command line arguments with matching environment arguments EnvironmentArgumentsParser. To implement your streaming application inherit from this class and add your custom options. Call startApplication(KafkaStreamsApplication, String[]) with a fresh instance of your class from your main.

    • Field Detail

      • inputTopics

        protected java.util.List<java.lang.String> inputTopics

      • errorTopic

        protected java.lang.String errorTopic

      • extraInputTopics

        protected java.util.Map<java.lang.String,​java.lang.String> extraInputTopics

    • Constructor Detail

      • KafkaStreamsApplication

        public KafkaStreamsApplication()

    • Method Detail

      • startApplication

        protected static void startApplication​(KafkaStreamsApplication app,
                                       java.lang.String[] args)

        This methods needs to be called in the executable custom application class inheriting from KafkaStreamsApplication.

        Parameters:
        app - An instance of the custom application class.
        args - Arguments passed in by the custom application class.

      • run

        public void run()
        Specified by:
        run in interface java.lang.Runnable

      • close

        public void close()
        Specified by:
        close in interface java.lang.AutoCloseable

      • buildTopology

        public abstract void buildTopology​(org.apache.kafka.streams.StreamsBuilder builder)

      • getUniqueAppId

        public abstract java.lang.String getUniqueAppId()
        This must be set to a unique value for every application interacting with your kafka cluster to ensure internal state encapsulation. Could be set to: className-inputTopic-outputTopic

      • createTopology

        public org.apache.kafka.streams.Topology createTopology()

      • getInputTopic

        public java.lang.String getInputTopic()

      • getInputTopic

        protected java.lang.String getInputTopic​(java.lang.String role)
        Get extra input topic for a specified role
        Parameters:
        role - role of input topic specified in CLI argument
        Returns:
        topic name

      • getUncaughtExceptionHandler

        protected java.util.Optional<org.apache.kafka.streams.errors.StreamsUncaughtExceptionHandler> getUncaughtExceptionHandler()
        Create an StreamsUncaughtExceptionHandler to use for Kafka Streams. Will not be configured if Optional.empty() is returned.
        Returns:
        Optional.empty() by default.
        See Also:
        KafkaStreams.setUncaughtExceptionHandler(StreamsUncaughtExceptionHandler)

      • createKafkaProperties

        protected java.util.Properties createKafkaProperties()

        This method should give a default configuration to run your streaming application with.

        To add a custom configuration please add a similar method to your custom application class: 
        
   protected Properties createKafkaProperties() {
       # Try to always use the kafka properties from the super class as base Map
       Properties kafkaConfig = super.createKafkaProperties();
       kafkaConfig.put(StreamsConfig.DEFAULT_KEY_SERDE_CLASS_CONFIG, GenericAvroSerde.class);
       kafkaConfig.put(StreamsConfig.DEFAULT_VALUE_SERDE_CLASS_CONFIG, GenericAvroSerde.class);
       return kafkaConfig;
   }
        Specified by:
        createKafkaProperties in class KafkaApplication
        Returns:
        Returns a default Kafka Streams configuration Properties

      • runStreamsApplication

        protected void runStreamsApplication()

      • closeResources

        protected void closeResources()
        Method to close resources outside of KafkaStreams. Will be called by default on close() and on transitioning to KafkaStreams.State.ERROR.

      • getStateListener

        protected org.apache.kafka.streams.KafkaStreams.StateListener getStateListener()
        Create a KafkaStreams.StateListener to use for Kafka Streams. Will not be configured if null is returned.
        Returns:
        KafkaStreams.StateListener that calls closeResources() on transition to KafkaStreams.State.ERROR.
        See Also:
        KafkaStreams.setStateListener(StateListener)

      • runCleanUp

        protected void runCleanUp()
        This method resets the offset for all input topics and deletes internal topics, application state, and optionally the output and error topic.
        Specified by:
        runCleanUp in class KafkaApplication

      • cleanUpRun

        protected void cleanUpRun​(CleanUpRunner cleanUpRunner)

      • getInputTopics

        public java.util.List<java.lang.String> getInputTopics()

      • getErrorTopic

        public java.lang.String getErrorTopic()

      • getExtraInputTopics

        public java.util.Map<java.lang.String,​java.lang.String> getExtraInputTopics()

      • isProductive

        public boolean isProductive()

      • isDeleteOutputTopic

        public boolean isDeleteOutputTopic()

      • getStreams

        public org.apache.kafka.streams.KafkaStreams getStreams()

      • setInputTopics

        public void setInputTopics​(java.util.List<java.lang.String> inputTopics)

      • setErrorTopic

        public void setErrorTopic​(java.lang.String errorTopic)

      • setExtraInputTopics

        public void setExtraInputTopics​(java.util.Map<java.lang.String,​java.lang.String> extraInputTopics)

      • setProductive

        public void setProductive​(boolean productive)

      • setDeleteOutputTopic

        public void setDeleteOutputTopic​(boolean deleteOutputTopic)

      • setStreams

        public void setStreams​(org.apache.kafka.streams.KafkaStreams streams)