org.apache.beam.examples.complete.kafkatopubsub.kafka.consumer

Class SslConsumerFactoryFn

java.lang.Object

org.apache.beam.examples.complete.kafkatopubsub.kafka.consumer.SslConsumerFactoryFn

All Implemented Interfaces:

java.io.Serializable, org.apache.beam.sdk.transforms.ProcessFunction<java.util.Map<java.lang.String,java.lang.Object>,org.apache.kafka.clients.consumer.Consumer<byte[],byte[]>>, org.apache.beam.sdk.transforms.SerializableFunction<java.util.Map<java.lang.String,java.lang.Object>,org.apache.kafka.clients.consumer.Consumer<byte[],byte[]>>

public class SslConsumerFactoryFn
extends java.lang.Object
implements org.apache.beam.sdk.transforms.SerializableFunction<java.util.Map<java.lang.String,java.lang.Object>,org.apache.kafka.clients.consumer.Consumer<byte[],byte[]>>

Class to create Kafka Consumer with configured SSL.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
SslConsumerFactoryFn(java.util.Map<java.lang.String,java.lang.String> sslConfig)

Method Summary

Modifier and Type	Method and Description
org.apache.kafka.clients.consumer.Consumer<byte[],byte[]>	apply(java.util.Map<java.lang.String,java.lang.Object> config)
static void	getGcsFileAsLocal(java.lang.String gcsFilePath, java.lang.String outputFilePath) Reads a file from GCS and writes it locally.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SslConsumerFactoryFn

public SslConsumerFactoryFn(java.util.Map<java.lang.String,java.lang.String> sslConfig)

Method Detail

apply

public org.apache.kafka.clients.consumer.Consumer<byte[],byte[]> apply(java.util.Map<java.lang.String,java.lang.Object> config)

Specified by:

apply in interface org.apache.beam.sdk.transforms.ProcessFunction<java.util.Map<java.lang.String,java.lang.Object>,org.apache.kafka.clients.consumer.Consumer<byte[],byte[]>>

Specified by:

apply in interface org.apache.beam.sdk.transforms.SerializableFunction<java.util.Map<java.lang.String,java.lang.Object>,org.apache.kafka.clients.consumer.Consumer<byte[],byte[]>>

getGcsFileAsLocal

public static void getGcsFileAsLocal(java.lang.String gcsFilePath,
                                     java.lang.String outputFilePath)
                              throws java.io.IOException

Reads a file from GCS and writes it locally.

Parameters:

gcsFilePath - path to file in GCS in format "gs://your-bucket/path/to/file"

outputFilePath - path where to save file locally

Throws:

java.io.IOException - thrown if not able to read or write file