Package com.github.charithe.kafka

Class KafkaHelper

java.lang.Object
com.github.charithe.kafka.KafkaHelper
public class KafkaHelper extends Object

  • Nested Class Summary

    Nested Classes
    Modifier and Type
    Class
    Description
    static class 
    KafkaHelper.RecordConsumer<K,V>
     

  • Method Summary

    Modifier and Type
    Method
    Description
    <K, V> com.google.common.util.concurrent.ListenableFuture<List<org.apache.kafka.clients.consumer.ConsumerRecord<K,V>>>
    consume(String topic, org.apache.kafka.clients.consumer.KafkaConsumer<K,V> consumer, int numMessagesToConsume)
    Attempt to consume the specified number of messages
    Properties
    consumerConfig()
    Get the consumer configuration (with auto-commit enabled)
    Properties
    consumerConfig(boolean enableAutoCommit)
    Get the consumer configuration
    com.google.common.util.concurrent.ListenableFuture<List<String>>
    consumeStrings(String topic, int numMessagesToConsume)
    Consume specified number of string messages
    org.apache.kafka.clients.consumer.KafkaConsumer<byte[],byte[]>
    createByteConsumer()
    Create a consumer that reads bytes
    org.apache.kafka.clients.consumer.KafkaConsumer<byte[],byte[]>
    createByteConsumer(Properties overrideConfig)
    Create a consumer that reads bytes
    org.apache.kafka.clients.producer.KafkaProducer<byte[],byte[]>
    createByteProducer()
    Create a producer that writes byte keys and values
    org.apache.kafka.clients.producer.KafkaProducer<byte[],byte[]>
    createByteProducer(Properties overrideConfig)
    Create a producer that writes byte keys and values
    <K, V> org.apache.kafka.clients.consumer.KafkaConsumer<K,V>
    createConsumer(org.apache.kafka.common.serialization.Deserializer<K> keyDeserializer, org.apache.kafka.common.serialization.Deserializer<V> valueDeserializer, Properties overrideConfig)
    Create a consumer that can read from this broker
    static KafkaHelper
    createFor(EphemeralKafkaBroker broker)
     
    <K, V> org.apache.kafka.clients.producer.KafkaProducer<K,V>
    createProducer(org.apache.kafka.common.serialization.Serializer<K> keySerializer, org.apache.kafka.common.serialization.Serializer<V> valueSerializer, Properties overrideConfig)
    Create a producer that can write to this broker
    org.apache.kafka.clients.consumer.KafkaConsumer<String,String>
    createStringConsumer()
    Create a consumer that reads strings
    org.apache.kafka.clients.consumer.KafkaConsumer<String,String>
    createStringConsumer(Properties overrideConfig)
    Create a consumer that reads strings
    org.apache.kafka.clients.producer.KafkaProducer<String,String>
    createStringProducer()
    Create a producer that writes String keys and values
    org.apache.kafka.clients.producer.KafkaProducer<String,String>
    createStringProducer(Properties overrideConfig)
    Create a producer that writes String keys and values
    int
    kafkaPort()
    Get the broker listener port
    <K, V> void
    produce(String topic, org.apache.kafka.clients.producer.KafkaProducer<K,V> producer, Map<K,V> data)
    Produce data to the specified topic
    Properties
    producerConfig()
    Get the producer configuration
    void
    produceStrings(String topic, String... values)
    Convenience method to produce a set of strings to the specified topic
    String
    zookeeperConnectionString()
    Get the zookeeper connection string
    int
    zookeeperPort()
    Get the zookeeper port

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Method Details

    • createFor

      public static KafkaHelper createFor(EphemeralKafkaBroker broker)

    • producerConfig

      public Properties producerConfig()
      Get the producer configuration
      Returns:
      Properties

    • consumerConfig

      public Properties consumerConfig()
      Get the consumer configuration (with auto-commit enabled)
      Returns:
      Properties

    • consumerConfig

      public Properties consumerConfig(boolean enableAutoCommit)
      Get the consumer configuration
      Parameters:
      enableAutoCommit - Enable auto-commit
      Returns:
      Properties

    • zookeeperConnectionString

      public String zookeeperConnectionString()
      Get the zookeeper connection string
      Returns:
      zookeeper connection string or IllegalStateException if the broker is not running

    • zookeeperPort

      public int zookeeperPort()
      Get the zookeeper port
      Returns:
      zookeeper port of or IllegalStateException if the broker is not running

    • kafkaPort

      public int kafkaPort()
      Get the broker listener port
      Returns:
      broker listener port or IllegalStateException if the broker is not running

    • createProducer

      public <K, V> org.apache.kafka.clients.producer.KafkaProducer<K,V> createProducer(org.apache.kafka.common.serialization.Serializer<K> keySerializer, org.apache.kafka.common.serialization.Serializer<V> valueSerializer, Properties overrideConfig)
      Create a producer that can write to this broker
      Type Parameters:
      K - Type of Key
      V - Type of Value
      Parameters:
      keySerializer - Key serializer class
      valueSerializer - Valuer serializer class
      overrideConfig - Producer config to override. Pass null if there aren't any.
      Returns:
      KafkaProducer

    • createStringProducer

      public org.apache.kafka.clients.producer.KafkaProducer<String,String> createStringProducer()
      Create a producer that writes String keys and values
      Returns:
      KafkaProducer

    • createStringProducer

      public org.apache.kafka.clients.producer.KafkaProducer<String,String> createStringProducer(Properties overrideConfig)
      Create a producer that writes String keys and values
      Parameters:
      overrideConfig - Producer config to override
      Returns:
      KafkaProducer

    • createByteProducer

      public org.apache.kafka.clients.producer.KafkaProducer<byte[],byte[]> createByteProducer()
      Create a producer that writes byte keys and values
      Returns:
      KafkaProducer

    • createByteProducer

      public org.apache.kafka.clients.producer.KafkaProducer<byte[],byte[]> createByteProducer(Properties overrideConfig)
      Create a producer that writes byte keys and values
      Parameters:
      overrideConfig - Producer config to override
      Returns:
      KafkaProducer

    • produce

      public <K, V> void produce(String topic, org.apache.kafka.clients.producer.KafkaProducer<K,V> producer, Map<K,V> data)
      Produce data to the specified topic
      Type Parameters:
      K - Type of key
      V - Type of value
      Parameters:
      topic - Topic to produce to
      producer - Producer to use
      data - Data to produce

    • produceStrings

      public void produceStrings(String topic, String... values)
      Convenience method to produce a set of strings to the specified topic
      Parameters:
      topic - Topic to produce to
      values - Values produce

    • createConsumer

      public <K, V> org.apache.kafka.clients.consumer.KafkaConsumer<K,V> createConsumer(org.apache.kafka.common.serialization.Deserializer<K> keyDeserializer, org.apache.kafka.common.serialization.Deserializer<V> valueDeserializer, Properties overrideConfig)
      Create a consumer that can read from this broker
      Type Parameters:
      K - Type of Key
      V - Type of Value
      Parameters:
      keyDeserializer - Key deserializer
      valueDeserializer - Value deserializer
      overrideConfig - Consumer config to override. Pass null if there aren't any
      Returns:
      KafkaConsumer

    • createStringConsumer

      public org.apache.kafka.clients.consumer.KafkaConsumer<String,String> createStringConsumer()
      Create a consumer that reads strings
      Returns:
      KafkaConsumer

    • createStringConsumer

      public org.apache.kafka.clients.consumer.KafkaConsumer<String,String> createStringConsumer(Properties overrideConfig)
      Create a consumer that reads strings
      Parameters:
      overrideConfig - Consumer config to override
      Returns:
      KafkaConsumer

    • createByteConsumer

      public org.apache.kafka.clients.consumer.KafkaConsumer<byte[],byte[]> createByteConsumer()
      Create a consumer that reads bytes
      Returns:
      KafkaConsumer

    • createByteConsumer

      public org.apache.kafka.clients.consumer.KafkaConsumer<byte[],byte[]> createByteConsumer(Properties overrideConfig)
      Create a consumer that reads bytes
      Parameters:
      overrideConfig - Consumer config to override
      Returns:
      KafkaConsumer

    • consume

      public <K, V> com.google.common.util.concurrent.ListenableFuture<List<org.apache.kafka.clients.consumer.ConsumerRecord<K,V>>> consume(String topic, org.apache.kafka.clients.consumer.KafkaConsumer<K,V> consumer, int numMessagesToConsume)
      Attempt to consume the specified number of messages
      Type Parameters:
      K - Type of Key
      V - Type of Value
      Parameters:
      topic - Topic to consume
      consumer - Consumer to use
      numMessagesToConsume - Number of messages to consume
      Returns:
      ListenableFuture

    • consumeStrings

      public com.google.common.util.concurrent.ListenableFuture<List<String>> consumeStrings(String topic, int numMessagesToConsume)
      Consume specified number of string messages
      Parameters:
      topic - Topic to consume from
      numMessagesToConsume - Number of messages to consume
      Returns:
      ListenableFuture