| Package | Description |
|---|---|
| com.google.cloud.spark.bigquery | |
| com.google.cloud.spark.bigquery.write.context |
| Modifier and Type | Method and Description |
|---|---|
static SchemaConvertersConfiguration |
SchemaConvertersConfiguration.createDefault() |
static SchemaConvertersConfiguration |
SchemaConvertersConfiguration.from(SparkBigQueryConfig config) |
static SchemaConvertersConfiguration |
SchemaConvertersConfiguration.of(boolean allowMapTypeConversion) |
| Modifier and Type | Method and Description |
|---|---|
static ReadRowsResponseToInternalRowIteratorConverter |
ReadRowsResponseToInternalRowIteratorConverter.avro(com.google.cloud.bigquery.Schema bqSchema,
List<String> columnsInOrder,
String rawAvroSchema,
Optional<org.apache.spark.sql.types.StructType> userProvidedSchema,
Optional<BigQueryStorageReadRowsTracer> bigQueryStorageReadRowsTracer,
SchemaConvertersConfiguration schemaConvertersConfiguration,
com.google.cloud.bigquery.storage.v1.ReadSession.TableReadOptions.ResponseCompressionCodec responseCompressionCodec) |
static SchemaConverters |
SchemaConverters.from(SchemaConvertersConfiguration configuration) |
| Constructor and Description |
|---|
Avro(com.google.cloud.bigquery.Schema bqSchema,
List<String> columnsInOrder,
String rawAvroSchema,
com.google.common.base.Optional<org.apache.spark.sql.types.StructType> userProvidedSchema,
com.google.common.base.Optional<BigQueryStorageReadRowsTracer> bigQueryStorageReadRowsTracer,
SchemaConvertersConfiguration schemaConvertersConfiguration,
com.google.cloud.bigquery.storage.v1.ReadSession.TableReadOptions.ResponseCompressionCodec responseCompressionCodec) |
AvroBinaryIterator(com.google.cloud.bigquery.Schema bqSchema,
List<String> columnsInOrder,
org.apache.avro.Schema schema,
com.google.cloud.bigquery.storage.v1.ReadRowsResponse response,
Optional<org.apache.spark.sql.types.StructType> userProvidedSchema,
Optional<BigQueryStorageReadRowsTracer> bigQueryStorageReadRowsTracer,
SchemaConvertersConfiguration schemaConvertersConfiguration,
com.google.cloud.bigquery.storage.v1.ReadSession.TableReadOptions.ResponseCompressionCodec responseCompressionCodec)
An iterator for scanning over rows serialized in Avro format
|
| Constructor and Description |
|---|
BigQueryDirectDataSourceWriterContext(BigQueryClient bigQueryClient,
BigQueryClientFactory bigQueryWriteClientFactory,
com.google.cloud.bigquery.TableId destinationTableId,
String writeUUID,
org.apache.spark.sql.SaveMode saveMode,
org.apache.spark.sql.types.StructType sparkSchema,
com.google.api.gax.retrying.RetrySettings bigqueryDataWriterHelperRetrySettings,
com.google.common.base.Optional<String> traceId,
boolean enableModeCheckForSchemaFields,
com.google.common.collect.ImmutableMap<String,String> tableLabels,
SchemaConvertersConfiguration schemaConvertersConfiguration,
Optional<String> destinationTableKmsKeyName,
boolean writeAtLeastOnce,
PartitionOverwriteMode overwriteMode,
org.apache.spark.SparkContext sparkContext) |
Copyright © 2024. All rights reserved.