Serialized Form (Spark BigQuery Connector Common Library 0.39.1 API)

Package com.google.cloud.spark.bigquery

Class com.google.cloud.spark.bigquery.ReadRowsResponseToInternalRowIteratorConverter.Arrow extends Object implements Serializable

Serialized Fields

columnsInOrder
```
List<E> columnsInOrder
```

arrowSchema

com.google.protobuf.ByteString arrowSchema

userProvidedSchema

com.google.common.base.Optional<T> userProvidedSchema

bigQueryStorageReadRowsTracer

com.google.common.base.Optional<T> bigQueryStorageReadRowsTracer

responseCompressionCodec

com.google.cloud.bigquery.storage.v1.ReadSession.TableReadOptions.ResponseCompressionCodec responseCompressionCodec

Class com.google.cloud.spark.bigquery.ReadRowsResponseToInternalRowIteratorConverter.Avro extends Object implements Serializable

Serialized Fields

bqSchema

com.google.cloud.bigquery.Schema bqSchema

columnsInOrder
```
List<E> columnsInOrder
```
rawAvroSchema
```
String rawAvroSchema
```

userProvidedSchema

com.google.common.base.Optional<T> userProvidedSchema

bigQueryStorageReadRowsTracer

com.google.common.base.Optional<T> bigQueryStorageReadRowsTracer

schemaConvertersConfiguration

SchemaConvertersConfiguration schemaConvertersConfiguration

responseCompressionCodec

com.google.cloud.bigquery.storage.v1.ReadSession.TableReadOptions.ResponseCompressionCodec responseCompressionCodec

Class com.google.cloud.spark.bigquery.SchemaConvertersConfiguration extends Object implements Serializable
- Serialized Fields
  - allowMapTypeConversion
```
boolean allowMapTypeConversion
```

Class com.google.cloud.spark.bigquery.SparkBigQueryConfig extends Object implements Serializable

Serialized Fields

partitionOverwriteModeValue

PartitionOverwriteMode partitionOverwriteModeValue

tableId

com.google.cloud.bigquery.TableId tableId

query

com.google.common.base.Optional<T> query

parentProjectId
```
String parentProjectId
```

useParentProjectForMetadataOperations

boolean useParentProjectForMetadataOperations

accessTokenProviderFQCN

com.google.common.base.Optional<T> accessTokenProviderFQCN

accessTokenProviderConfig

com.google.common.base.Optional<T> accessTokenProviderConfig

loggedInUserName
```
String loggedInUserName
```
loggedInUserGroups
```
Set<E> loggedInUserGroups
```

impersonationServiceAccount

com.google.common.base.Optional<T> impersonationServiceAccount

impersonationServiceAccountsForUsers

com.google.common.base.Optional<T> impersonationServiceAccountsForUsers

impersonationServiceAccountsForGroups

com.google.common.base.Optional<T> impersonationServiceAccountsForGroups

credentialsKey

com.google.common.base.Optional<T> credentialsKey

credentialsFile

com.google.common.base.Optional<T> credentialsFile

accessToken

com.google.common.base.Optional<T> accessToken

filter

com.google.common.base.Optional<T> filter

schema

com.google.common.base.Optional<T> schema

maxParallelism
```
Integer maxParallelism
```
preferredMinParallelism
```
Integer preferredMinParallelism
```
defaultParallelism
```
int defaultParallelism
```

temporaryGcsBucket

com.google.common.base.Optional<T> temporaryGcsBucket

persistentGcsBucket

com.google.common.base.Optional<T> persistentGcsBucket

persistentGcsPath

com.google.common.base.Optional<T> persistentGcsPath

intermediateFormat

SparkBigQueryConfig.IntermediateFormat intermediateFormat

readDataFormat

com.google.cloud.bigquery.storage.v1.DataFormat readDataFormat

combinePushedDownFilters
```
boolean combinePushedDownFilters
```
viewsEnabled
```
boolean viewsEnabled
```

materializationProject

com.google.common.base.Optional<T> materializationProject

materializationDataset

com.google.common.base.Optional<T> materializationDataset

partitionField

com.google.common.base.Optional<T> partitionField

partitionExpirationMs
```
Long partitionExpirationMs
```

partitionRequireFilter

com.google.common.base.Optional<T> partitionRequireFilter

partitionType

com.google.common.base.Optional<T> partitionType

partitionRangeStart

com.google.common.base.Optional<T> partitionRangeStart

partitionRangeEnd

com.google.common.base.Optional<T> partitionRangeEnd

partitionRangeInterval

com.google.common.base.Optional<T> partitionRangeInterval

clusteredFields

com.google.common.base.Optional<T> clusteredFields

createDisposition

com.google.common.base.Optional<T> createDisposition

optimizedEmptyProjection
```
boolean optimizedEmptyProjection
```
useAvroLogicalTypes
```
boolean useAvroLogicalTypes
```
decimalTargetTypes
```
List<E> decimalTargetTypes
```
loadSchemaUpdateOptions
```
List<E> loadSchemaUpdateOptions
```

materializationExpirationTimeInMinutes

int materializationExpirationTimeInMinutes

maxReadRowsRetries
```
int maxReadRowsRetries
```
pushAllFilters
```
boolean pushAllFilters
```
enableModeCheckForSchemaFields
```
boolean enableModeCheckForSchemaFields
```

encodedCreateReadSessionRequest

com.google.common.base.Optional<T> encodedCreateReadSessionRequest

bigQueryStorageGrpcEndpoint

com.google.common.base.Optional<T> bigQueryStorageGrpcEndpoint

bigQueryHttpEndpoint

com.google.common.base.Optional<T> bigQueryHttpEndpoint

numBackgroundThreadsPerStream
```
int numBackgroundThreadsPerStream
```
numPrebufferReadRowsResponses
```
int numPrebufferReadRowsResponses
```
numStreamsPerPartition
```
int numStreamsPerPartition
```
channelPoolSize
```
int channelPoolSize
```

flowControlWindowBytes

com.google.common.base.Optional<T> flowControlWindowBytes

enableReadSessionCaching
```
boolean enableReadSessionCaching
```
readSessionCacheDurationMins
```
long readSessionCacheDurationMins
```
snapshotTimeMillis
```
Long snapshotTimeMillis
```

sparkBigQueryProxyAndHttpConfig

SparkBigQueryProxyAndHttpConfig sparkBigQueryProxyAndHttpConfig

arrowCompressionCodec

com.google.cloud.bigquery.storage.v1.ArrowSerializationOptions.CompressionCodec arrowCompressionCodec

responseCompressionCodec

com.google.cloud.bigquery.storage.v1.ReadSession.TableReadOptions.ResponseCompressionCodec responseCompressionCodec

writeMethod

SparkBigQueryConfig.WriteMethod writeMethod

writeAtLeastOnce
```
boolean writeAtLeastOnce
```
cacheExpirationTimeInMinutes
```
int cacheExpirationTimeInMinutes
```

traceId

com.google.common.base.Optional<T> traceId

bigQueryJobLabels
```
Map<K,V> bigQueryJobLabels
```
bigQueryTableLabels
```
Map<K,V> bigQueryTableLabels
```

createReadSessionTimeoutInSeconds

com.google.common.base.Optional<T> createReadSessionTimeoutInSeconds

queryJobPriority

com.google.cloud.bigquery.QueryJobConfiguration.Priority queryJobPriority

destinationTableKmsKeyName

com.google.common.base.Optional<T> destinationTableKmsKeyName

allowMapTypeConversion
```
boolean allowMapTypeConversion
```
bigQueryJobTimeoutInMinutes
```
long bigQueryJobTimeoutInMinutes
```
gpn
```
com.google.common.base.Optional<T> gpn
```

Class com.google.cloud.spark.bigquery.SparkBigQueryProxyAndHttpConfig extends Object implements Serializable

Serialized Fields

proxyUri

com.google.common.base.Optional<T> proxyUri

proxyUsername

com.google.common.base.Optional<T> proxyUsername

proxyPassword

com.google.common.base.Optional<T> proxyPassword

httpMaxRetry

com.google.common.base.Optional<T> httpMaxRetry

httpConnectTimeout

com.google.common.base.Optional<T> httpConnectTimeout

httpReadTimeout

com.google.common.base.Optional<T> httpReadTimeout

Package com.google.cloud.spark.bigquery.direct
- Class com.google.cloud.spark.bigquery.direct.BigQueryPartition extends Object implements Serializable
  - Serialized Fields
    - stream
```
String stream
```
    - index
```
int index
```
Package com.google.cloud.spark.bigquery.events
- Class com.google.cloud.spark.bigquery.events.BigQueryJobCompletedEvent extends Object implements Serializable
  - Serialized Fields
    - jobInfo
```
com.google.cloud.bigquery.JobInfo jobInfo
```
- Class com.google.cloud.spark.bigquery.events.LoadJobCompletedEvent extends BigQueryJobCompletedEvent implements Serializable
- Class com.google.cloud.spark.bigquery.events.QueryJobCompletedEvent extends BigQueryJobCompletedEvent implements Serializable
Package com.google.cloud.spark.bigquery.metrics
- Class com.google.cloud.spark.bigquery.metrics.SparkBigQueryReadSessionMetrics extends org.apache.spark.scheduler.SparkListener implements Serializable
  - Serialized Fields
    - bytesReadAccumulator
```
org.apache.spark.util.LongAccumulator bytesReadAccumulator
```
    - rowsReadAccumulator
```
org.apache.spark.util.LongAccumulator rowsReadAccumulator
```
    - scanTimeAccumulator
```
org.apache.spark.util.LongAccumulator scanTimeAccumulator
```
    - parseTimeAccumulator
```
org.apache.spark.util.LongAccumulator parseTimeAccumulator
```
    - sessionId
```
String sessionId
```
    - sparkSession
```
org.apache.spark.sql.SparkSession sparkSession
```
    - timestamp
```
long timestamp
```
    - readDataFormat
```
com.google.cloud.bigquery.storage.v1.DataFormat readDataFormat
```
    - dataOrigin
```
DataOrigin dataOrigin
```
    - numReadStreams
```
long numReadStreams
```
- Class com.google.cloud.spark.bigquery.metrics.SparkMetricsSource extends Object implements Serializable
Package com.google.cloud.spark.bigquery.write
- Class com.google.cloud.spark.bigquery.write.DataSourceWriterContextPartitionHandler extends Object implements Serializable
  - Serialized Fields
    - dataWriterContextFactory
```
DataWriterContextFactory<T> dataWriterContextFactory
```
    - epoch
```
long epoch
```
    - taskId
```
long taskId
```
Package com.google.cloud.spark.bigquery.write.context
- Class com.google.cloud.spark.bigquery.write.context.BigQueryDirectDataWriterContextFactory extends Object implements Serializable
  - Serialized Fields
    - writeClientFactory
```
BigQueryClientFactory writeClientFactory
```
    - tablePath
```
String tablePath
```
    - sparkSchema
```
org.apache.spark.sql.types.StructType sparkSchema
```
    - protoSchema
```
com.google.cloud.bigquery.storage.v1.ProtoSchema protoSchema
```
    - ignoreInputs
```
boolean ignoreInputs
```
    - bigqueryDataWriterHelperRetrySettings
```
com.google.api.gax.retrying.RetrySettings bigqueryDataWriterHelperRetrySettings
```
    - traceId
```
com.google.common.base.Optional<T> traceId
```
    - writeAtLeastOnce
```
boolean writeAtLeastOnce
```
- Class com.google.cloud.spark.bigquery.write.context.BigQueryDirectWriterCommitMessageContext extends Object implements Serializable
  
  serialVersionUID:
  
  -1562914502592461805L
  - Serialized Fields
    - writeStreamName
```
String writeStreamName
```
    - partitionId
```
int partitionId
```
    - taskId
```
long taskId
```
    - epochId
```
long epochId
```
    - tablePath
```
String tablePath
```
    - rowCount
```
long rowCount
```
    - bytesWritten
```
long bytesWritten
```
Package org.apache.spark.sql
- Class org.apache.spark.sql.AnalysisException extends Exception implements Serializable
  - Serialized Fields
    - message
```
String message
```
    - line
```
scala.Option<A> line
```
    - startPosition
```
scala.Option<A> startPosition
```
    - cause
```
scala.Option<A> cause
```
    - errorClass
```
scala.Option<A> errorClass
```
    - messageParameters
```
String[] messageParameters
```
- Class org.apache.spark.sql.AnalysisException$ extends Object implements Serializable
  - Serialization Methods
    - writeReplace
```
private Object writeReplace()
```
- Class org.apache.spark.sql.Dataset extends Object implements Serializable
  - Serialized Fields
    - org$apache$spark$sql$Dataset$$resolvedEnc
```
org.apache.spark.sql.catalyst.encoders.ExpressionEncoder<T> org$apache$spark$sql$Dataset$$resolvedEnc
```
    - rdd
```
org.apache.spark.rdd.RDD<T> rdd
```
    - org$apache$spark$sql$Dataset$$id
```
long org$apache$spark$sql$Dataset$$id
```
    - exprEnc
```
org.apache.spark.sql.catalyst.encoders.ExpressionEncoder<T> exprEnc
```
    - bitmap$0
```
byte bitmap$0
```
- Class org.apache.spark.sql.Dataset$ extends Object implements Serializable
  - Serialization Methods
    - writeReplace
```
private Object writeReplace()
```
- Class org.apache.spark.sql.DatasetHolder extends Object implements Serializable
  - Serialized Fields
    - ds
```
org.apache.spark.sql.Dataset<T> ds
```
- Class org.apache.spark.sql.DatasetHolder$ extends Object implements Serializable
  - Serialization Methods
    - writeReplace
```
private Object writeReplace()
```
- Class org.apache.spark.sql.ForeachWriter extends Object implements Serializable
- Class org.apache.spark.sql.KeyValueGroupedDataset extends Object implements Serializable
  - Serialized Fields
    - dataAttributes
```
scala.collection.immutable.Seq<A> dataAttributes
```
    - groupingAttributes
```
scala.collection.immutable.Seq<A> groupingAttributes
```
    - kExprEnc
```
org.apache.spark.sql.catalyst.encoders.ExpressionEncoder<T> kExprEnc
```
    - vExprEnc
```
org.apache.spark.sql.catalyst.encoders.ExpressionEncoder<T> vExprEnc
```
- Class org.apache.spark.sql.ObservationListener extends Object implements Serializable
  - Serialized Fields
    - observation
```
org.apache.spark.sql.Observation observation
```
- Class org.apache.spark.sql.ObservationListener$ extends scala.runtime.AbstractFunction1<org.apache.spark.sql.Observation,org.apache.spark.sql.ObservationListener> implements Serializable
  - Serialization Methods
    - writeReplace
```
private Object writeReplace()
```
- Class org.apache.spark.sql.RelationalGroupedDataset.PivotType extends Object implements Serializable
  - Serialized Fields
    - pivotCol
```
org.apache.spark.sql.catalyst.expressions.Expression pivotCol
```
    - values
```
scala.collection.immutable.Seq<A> values
```
- Class org.apache.spark.sql.RelationalGroupedDataset$PivotType$ extends scala.runtime.AbstractFunction2<org.apache.spark.sql.catalyst.expressions.Expression,scala.collection.immutable.Seq<org.apache.spark.sql.catalyst.expressions.Expression>,org.apache.spark.sql.RelationalGroupedDataset.PivotType> implements Serializable
  - Serialization Methods
    - writeReplace
```
private Object writeReplace()
```
- Class org.apache.spark.sql.Row$ extends Object implements Serializable
  - Serialization Methods
    - writeReplace
```
private Object writeReplace()
```
- Class org.apache.spark.sql.SparkSession extends Object implements Serializable
  - Serialized Fields
    - implicits$module
```
org.apache.spark.sql.SparkSession$implicits$ implicits$module
```
    - org$apache$spark$sql$SparkSession$$creationSite
```
org.apache.spark.util.CallSite org$apache$spark$sql$SparkSession$$creationSite
```
    - sessionUUID
```
String sessionUUID
```
- Class org.apache.spark.sql.SparkSession$ extends Object implements Serializable
  - Serialization Methods
    - writeReplace
```
private Object writeReplace()
```
- Class org.apache.spark.sql.SparkSession$implicits$ extends org.apache.spark.sql.SQLImplicits implements Serializable
- Class org.apache.spark.sql.SQLContext extends Object implements Serializable
  - Serialized Fields
    - implicits$module
```
org.apache.spark.sql.SQLContext$implicits$ implicits$module
```
    - sparkSession
```
org.apache.spark.sql.SparkSession sparkSession
```
- Class org.apache.spark.sql.SQLContext$ extends Object implements Serializable
  - Serialization Methods
    - writeReplace
```
private Object writeReplace()
```
- Class org.apache.spark.sql.SQLContext$implicits$ extends org.apache.spark.sql.SQLImplicits implements Serializable

Serialized Form

Package com.google.cloud.spark.bigquery

Class com.google.cloud.spark.bigquery.ReadRowsResponseToInternalRowIteratorConverter.Arrow extends Object implements Serializable

Serialized Fields

columnsInOrder

arrowSchema

userProvidedSchema

bigQueryStorageReadRowsTracer

responseCompressionCodec

Class com.google.cloud.spark.bigquery.ReadRowsResponseToInternalRowIteratorConverter.Avro extends Object implements Serializable

Serialized Fields

bqSchema

columnsInOrder

rawAvroSchema

userProvidedSchema

bigQueryStorageReadRowsTracer

schemaConvertersConfiguration

responseCompressionCodec

Class com.google.cloud.spark.bigquery.SchemaConvertersConfiguration extends Object implements Serializable

Serialized Fields

allowMapTypeConversion

Class com.google.cloud.spark.bigquery.SparkBigQueryConfig extends Object implements Serializable

Serialized Fields

partitionOverwriteModeValue

tableId

query

parentProjectId

useParentProjectForMetadataOperations

accessTokenProviderFQCN

accessTokenProviderConfig

loggedInUserName

loggedInUserGroups

impersonationServiceAccount

impersonationServiceAccountsForUsers

impersonationServiceAccountsForGroups

credentialsKey

credentialsFile

accessToken

filter

schema

maxParallelism

preferredMinParallelism

defaultParallelism

temporaryGcsBucket

persistentGcsBucket

persistentGcsPath

intermediateFormat

readDataFormat

combinePushedDownFilters

viewsEnabled

materializationProject

materializationDataset

partitionField

partitionExpirationMs

partitionRequireFilter

partitionType

partitionRangeStart

partitionRangeEnd

partitionRangeInterval

clusteredFields

createDisposition

optimizedEmptyProjection

useAvroLogicalTypes

decimalTargetTypes

loadSchemaUpdateOptions

materializationExpirationTimeInMinutes

maxReadRowsRetries

pushAllFilters

enableModeCheckForSchemaFields

encodedCreateReadSessionRequest

bigQueryStorageGrpcEndpoint

bigQueryHttpEndpoint

numBackgroundThreadsPerStream

numPrebufferReadRowsResponses

numStreamsPerPartition

channelPoolSize

flowControlWindowBytes

enableReadSessionCaching

readSessionCacheDurationMins

snapshotTimeMillis