Package org.apache.beam.examples.complete.kafkatopubsub.kafka.consumer

Class SslConsumerFactoryFn

  • All Implemented Interfaces:
    java.io.Serializable, org.apache.beam.sdk.transforms.ProcessFunction<java.util.Map<java.lang.String,​java.lang.Object>,​org.apache.kafka.clients.consumer.Consumer<byte[],​byte[]>>, org.apache.beam.sdk.transforms.SerializableFunction<java.util.Map<java.lang.String,​java.lang.Object>,​org.apache.kafka.clients.consumer.Consumer<byte[],​byte[]>>
    public class SslConsumerFactoryFn
extends java.lang.Object
implements org.apache.beam.sdk.transforms.SerializableFunction<java.util.Map<java.lang.String,​java.lang.Object>,​org.apache.kafka.clients.consumer.Consumer<byte[],​byte[]>>
    Class to create Kafka Consumer with configured SSL.
    See Also:
    Serialized Form

    • Constructor Summary

      Constructors 
      Constructor Description
      SslConsumerFactoryFn​(java.util.Map<java.lang.String,​java.lang.String> sslConfig)  

    • Method Summary

      All Methods Static Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      org.apache.kafka.clients.consumer.Consumer<byte[],​byte[]> apply​(java.util.Map<java.lang.String,​java.lang.Object> config)  
      static void getGcsFileAsLocal​(java.lang.String gcsFilePath, java.lang.String outputFilePath)
      Reads a file from GCS and writes it locally.

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Constructor Detail

      • SslConsumerFactoryFn

        public SslConsumerFactoryFn​(java.util.Map<java.lang.String,​java.lang.String> sslConfig)

    • Method Detail

      • apply

        public org.apache.kafka.clients.consumer.Consumer<byte[],​byte[]> apply​(java.util.Map<java.lang.String,​java.lang.Object> config)
        Specified by:
        apply in interface org.apache.beam.sdk.transforms.ProcessFunction<java.util.Map<java.lang.String,​java.lang.Object>,​org.apache.kafka.clients.consumer.Consumer<byte[],​byte[]>>
        Specified by:
        apply in interface org.apache.beam.sdk.transforms.SerializableFunction<java.util.Map<java.lang.String,​java.lang.Object>,​org.apache.kafka.clients.consumer.Consumer<byte[],​byte[]>>

      • getGcsFileAsLocal

        public static void getGcsFileAsLocal​(java.lang.String gcsFilePath,
                                     java.lang.String outputFilePath)
                              throws java.io.IOException
        Reads a file from GCS and writes it locally.
        Parameters:
        gcsFilePath - path to file in GCS in format "gs://your-bucket/path/to/file"
        outputFilePath - path where to save file locally
        Throws:
        java.io.IOException - thrown if not able to read or write file