Package org.apache.beam.sdk.extensions.protobuf

Class ProtoByteUtils

  • public class ProtoByteUtils
extends java.lang.Object
    Utility class for working with Protocol Buffer (Proto) data. This class provides methods to retrieve Beam Schemas from Proto messages, convert Proto bytes to Beam Rows, and vice versa. It also includes utilities for handling Protocol Buffer schemas and related file operations.

    Users can utilize the methods in this class to facilitate the integration of Proto data processing within Apache Beam pipelines, allowing for the seamless transformation of Proto messages to Beam Rows and vice versa.

    • Constructor Summary

      Constructors 
      Constructor Description
      ProtoByteUtils()  

    • Method Summary

      All Methods Static Methods Concrete Methods 
      Modifier and Type Method Description
      static org.apache.beam.sdk.schemas.Schema getBeamSchemaFromProto​(java.lang.String fileDescriptorPath, java.lang.String messageName)
      Retrieves a Beam Schema from a Protocol Buffer message.
      static org.apache.beam.sdk.schemas.Schema getBeamSchemaFromProtoSchema​(java.lang.String schemaString, java.lang.String messageName)
      Parses the given Protocol Buffers schema string, retrieves the Descriptor for the specified message name, and constructs a Beam Schema from it.
      static org.apache.beam.sdk.transforms.SerializableFunction<byte[],​org.apache.beam.sdk.values.Row> getProtoBytesToRowFromSchemaFunction​(java.lang.String schemaString, java.lang.String messageName)  
      static org.apache.beam.sdk.transforms.SerializableFunction<byte[],​org.apache.beam.sdk.values.Row> getProtoBytesToRowFunction​(java.lang.String fileDescriptorPath, java.lang.String messageName)  
      static org.apache.beam.sdk.transforms.SerializableFunction<org.apache.beam.sdk.values.Row,​byte[]> getRowToProtoBytes​(java.lang.String fileDescriptorPath, java.lang.String messageName)  
      static org.apache.beam.sdk.transforms.SerializableFunction<org.apache.beam.sdk.values.Row,​byte[]> getRowToProtoBytesFromSchema​(java.lang.String schemaString, java.lang.String messageName)  

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Constructor Detail

      • ProtoByteUtils

        public ProtoByteUtils()

    • Method Detail

      • getBeamSchemaFromProto

        public static org.apache.beam.sdk.schemas.Schema getBeamSchemaFromProto​(java.lang.String fileDescriptorPath,
                                                                        java.lang.String messageName)
        Retrieves a Beam Schema from a Protocol Buffer message.
        Parameters:
        fileDescriptorPath - The path to the File Descriptor Set file.
        messageName - The name of the Protocol Buffer message.
        Returns:
        The Beam Schema representing the Protocol Buffer message.

      • getBeamSchemaFromProtoSchema

        public static org.apache.beam.sdk.schemas.Schema getBeamSchemaFromProtoSchema​(java.lang.String schemaString,
                                                                              java.lang.String messageName)
        Parses the given Protocol Buffers schema string, retrieves the Descriptor for the specified message name, and constructs a Beam Schema from it.
        Parameters:
        schemaString - The Protocol Buffers schema string.
        messageName - The name of the message type for which the Beam Schema is desired.
        Returns:
        The Beam Schema constructed from the specified Protocol Buffers schema.
        Throws:
        java.lang.RuntimeException - If there is an error during parsing, descriptor retrieval, or schema construction.

      • getProtoBytesToRowFromSchemaFunction

        public static org.apache.beam.sdk.transforms.SerializableFunction<byte[],​org.apache.beam.sdk.values.Row> getProtoBytesToRowFromSchemaFunction​(java.lang.String schemaString,
                                                                                                                                                    java.lang.String messageName)

      • getProtoBytesToRowFunction

        public static org.apache.beam.sdk.transforms.SerializableFunction<byte[],​org.apache.beam.sdk.values.Row> getProtoBytesToRowFunction​(java.lang.String fileDescriptorPath,
                                                                                                                                          java.lang.String messageName)

      • getRowToProtoBytesFromSchema

        public static org.apache.beam.sdk.transforms.SerializableFunction<org.apache.beam.sdk.values.Row,​byte[]> getRowToProtoBytesFromSchema​(java.lang.String schemaString,
                                                                                                                                            java.lang.String messageName)

      • getRowToProtoBytes

        public static org.apache.beam.sdk.transforms.SerializableFunction<org.apache.beam.sdk.values.Row,​byte[]> getRowToProtoBytes​(java.lang.String fileDescriptorPath,
                                                                                                                                  java.lang.String messageName)