com.google.cloud.spark.bigquery

Class SchemaConverters

java.lang.Object

com.google.cloud.spark.bigquery.SchemaConverters

public class SchemaConverters
extends Object

Method Summary

Modifier and Type	Method and Description
org.apache.spark.sql.catalyst.InternalRow	convertToInternalRow(com.google.cloud.bigquery.Schema schema, List<String> namesInOrder, org.apache.avro.generic.GenericRecord record, Optional<org.apache.spark.sql.types.StructType> userProvidedSchema)
protected com.google.cloud.bigquery.Field	createBigQueryColumn(org.apache.spark.sql.types.StructField sparkField, int depth) Converts a single StructField to a BigQuery Field (column).
static SchemaConverters	from(SchemaConvertersConfiguration configuration)
static Optional<String>	getDescriptionOrCommentOfField(org.apache.spark.sql.types.StructField field, Optional<SupportedCustomDataType> supportedCustomDataTypeOptional)
com.google.cloud.bigquery.Schema	getSchemaWithPseudoColumns(com.google.cloud.bigquery.TableInfo tableInfo) Retrieves and returns BigQuery Schema from TableInfo.
com.google.cloud.bigquery.Schema	toBigQuerySchema(org.apache.spark.sql.types.StructType sparkSchema) Spark ==> BigQuery Schema Converter utils:
protected com.google.cloud.bigquery.LegacySQLTypeName	toBigQueryType(org.apache.spark.sql.types.DataType elementType, org.apache.spark.sql.types.Metadata metadata)
org.apache.spark.sql.types.StructType	toSpark(com.google.cloud.bigquery.Schema schema) Convert a BigQuery schema to a Spark schema

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

from

public static SchemaConverters from(SchemaConvertersConfiguration configuration)

toSpark

public org.apache.spark.sql.types.StructType toSpark(com.google.cloud.bigquery.Schema schema)

Convert a BigQuery schema to a Spark schema

getSchemaWithPseudoColumns

public com.google.cloud.bigquery.Schema getSchemaWithPseudoColumns(com.google.cloud.bigquery.TableInfo tableInfo)

Retrieves and returns BigQuery Schema from TableInfo. If the table support pseudo columns, they are added to schema before schema is returned to the caller.

convertToInternalRow

public org.apache.spark.sql.catalyst.InternalRow convertToInternalRow(com.google.cloud.bigquery.Schema schema,
                                                                      List<String> namesInOrder,
                                                                      org.apache.avro.generic.GenericRecord record,
                                                                      Optional<org.apache.spark.sql.types.StructType> userProvidedSchema)

toBigQuerySchema

public com.google.cloud.bigquery.Schema toBigQuerySchema(org.apache.spark.sql.types.StructType sparkSchema)

Spark ==> BigQuery Schema Converter utils:

createBigQueryColumn

protected com.google.cloud.bigquery.Field createBigQueryColumn(org.apache.spark.sql.types.StructField sparkField,
                                                               int depth)

Converts a single StructField to a BigQuery Field (column).

getDescriptionOrCommentOfField

public static Optional<String> getDescriptionOrCommentOfField(org.apache.spark.sql.types.StructField field,
                                                              Optional<SupportedCustomDataType> supportedCustomDataTypeOptional)

toBigQueryType

protected com.google.cloud.bigquery.LegacySQLTypeName toBigQueryType(org.apache.spark.sql.types.DataType elementType,
                                                                     org.apache.spark.sql.types.Metadata metadata)