Package org.apache.pulsar.io.kafka

Class KafkaBytesSource

  • All Implemented Interfaces:
    java.lang.AutoCloseable, org.apache.pulsar.io.core.Source<java.nio.ByteBuffer>
    public class KafkaBytesSource
extends KafkaAbstractSource<java.nio.ByteBuffer>
    Kafka Source that transfers the data from Kafka to Pulsar and sets the Schema type properly. We use the key and the value deserializer in order to decide the type of Schema to be set on the topic on Pulsar. In case of KafkaAvroDeserializer we use the Schema Registry to download the schema and apply it to the topic. Please refer to getSchemaFromDeserializerAndAdaptConfiguration(String, Properties, boolean) for the list of supported Deserializers. If you set StringDeserializer for the key then we use the raw key as key for the Pulsar message. If you set another Deserializer for the key we use the KeyValue schema type in Pulsar with the SEPARATED encoding. This way the Key is stored in the Pulsar key, encoded as base64 string and with a Schema, the Value of the message is stored in the Pulsar value with a Schema. This way there is a one-to-one mapping between Kafka key/value pair and the Pulsar data model.