Package org.apache.parquet.cli.util

Class Schemas

  • public class Schemas
extends Object

    • Constructor Summary

      Constructors 
      Constructor Description
      Schemas()  

    • Method Summary

      All Methods Static Methods Concrete Methods 
      Modifier and Type Method Description
      static org.apache.avro.Schema.Field copy​(org.apache.avro.Schema.Field field)
      Creates a new field with the same name, schema, doc, and default value as the incoming schema.
      static org.apache.avro.Schema fromAvro​(InputStream in)  
      static org.apache.avro.Schema fromAvsc​(InputStream in)  
      static org.apache.avro.Schema fromJSON​(String name, InputStream in)  
      static org.apache.avro.Schema fromParquet​(org.apache.hadoop.conf.Configuration conf, URI location)  
      static org.apache.avro.Schema merge​(Iterable<org.apache.avro.Schema> schemas)
      Merges Schema instances if they are compatible.
      static org.apache.avro.Schema merge​(org.apache.avro.Schema left, org.apache.avro.Schema right)
      Merges two Schema instances if they are compatible.
      static org.apache.avro.Schema mergeOrUnion​(Iterable<org.apache.avro.Schema> schemas)
      Merges Schema instances and creates a union of schemas if any are incompatible.
      static boolean nullOk​(org.apache.avro.Schema schema)
      Returns whether null is allowed by the schema.

    • Constructor Detail

      • Schemas

        public Schemas()

    • Method Detail

      • fromParquet

        public static org.apache.avro.Schema fromParquet​(org.apache.hadoop.conf.Configuration conf,
                                                 URI location)
                                          throws IOException
        Throws:
        IOException

      • nullOk

        public static boolean nullOk​(org.apache.avro.Schema schema)
        Returns whether null is allowed by the schema.
        Parameters:
        schema - a Schema
        Returns:
        true if schema allows the value to be null

      • merge

        public static org.apache.avro.Schema merge​(Iterable<org.apache.avro.Schema> schemas)
        Merges Schema instances if they are compatible.

        Schemas are incompatible if:

        • The Schema.Type does not match.
        • For record schemas, the record name does not match
        • For enum schemas, the enum name does not match

        Map value, array element, and record field types types will use unions if necessary, and union schemas are merged recursively.

        Parameters:
        schemas - a set of Schema instances to merge
        Returns:
        a merged Schema
        Throws:
        IllegalStateException - if the schemas are not compatible

      • mergeOrUnion

        public static org.apache.avro.Schema mergeOrUnion​(Iterable<org.apache.avro.Schema> schemas)
        Merges Schema instances and creates a union of schemas if any are incompatible.

        Schemas are incompatible if:

        • The Schema.Type does not match.
        • For record schemas, the record name does not match
        • For enum schemas, the enum name does not match

        Map value, array element, and record field types types will use unions if necessary, and union schemas are merged recursively.

        Parameters:
        schemas - a set of Schema instances to merge
        Returns:
        a combined Schema

      • merge

        public static org.apache.avro.Schema merge​(org.apache.avro.Schema left,
                                           org.apache.avro.Schema right)
        Merges two Schema instances if they are compatible.

        Two schemas are incompatible if:

        • The Schema.Type does not match.
        • For record schemas, the record name does not match
        • For enum schemas, the enum name does not match

        Map value and array element types will use unions if necessary, and union schemas are merged recursively.

        Parameters:
        left - a Schema
        right - a Schema
        Returns:
        a merged Schema
        Throws:
        IllegalStateException - if the schemas are not compatible

      • copy

        public static org.apache.avro.Schema.Field copy​(org.apache.avro.Schema.Field field)
        Creates a new field with the same name, schema, doc, and default value as the incoming schema.

        Fields cannot be used in more than one record (not Immutable?).

        Parameters:
        field - an Avro schema field
        Returns:
        a copy of the field