| Package | Description |
|---|---|
| com.google.cloud.spark.bigquery.write | |
| com.google.cloud.spark.bigquery.write.context |
| Modifier and Type | Method and Description |
|---|---|
Iterator<WriterCommitMessageContext> |
DataSourceWriterContextPartitionHandler.call(Integer partitionId,
Iterator<org.apache.spark.sql.Row> rowIterator) |
| Modifier and Type | Class and Description |
|---|---|
class |
BigQueryDirectWriterCommitMessageContext |
| Modifier and Type | Method and Description |
|---|---|
WriterCommitMessageContext |
BigQueryDirectDataWriterContext.commit() |
WriterCommitMessageContext |
NoOpDataWriterContext.commit() |
WriterCommitMessageContext |
DataWriterContext.commit() |
| Modifier and Type | Method and Description |
|---|---|
void |
BigQueryDirectDataSourceWriterContext.abort(WriterCommitMessageContext[] messages)
If not in WritingMode IGNORE_INPUTS, the BigQuery Storage Write API WriteClient is shut down.
|
void |
DataSourceWriterContext.abort(WriterCommitMessageContext[] messages) |
void |
BigQueryIndirectDataSourceWriterContext.abort(WriterCommitMessageContext[] messages) |
void |
BigQueryDirectDataSourceWriterContext.commit(WriterCommitMessageContext[] messages)
This function will determine, based on the WritingMode: if in IGNORE_INPUTS mode, no work is to
be done; otherwise all streams will be batch committed using the BigQuery Storage Write API,
and then: if in OVERWRITE mode, the overwriteDestinationWithTemporary function from
BigQueryClient will be called to replace the destination table with all the data from the
temporary table; if in OVERWRITE mode with dynamic partitions enabled,
overwriteDestinationWithTemporaryDynamicPartitons from BigQueryClient will be called to replace
the required partitions;if in ALL_ELSE mode no more work needs to be done.
|
void |
DataSourceWriterContext.commit(WriterCommitMessageContext[] messages) |
void |
BigQueryIndirectDataSourceWriterContext.commit(WriterCommitMessageContext[] messages) |
void |
BigQueryDirectDataSourceWriterContext.onDataWriterCommit(WriterCommitMessageContext message) |
default void |
DataSourceWriterContext.onDataWriterCommit(WriterCommitMessageContext message) |
Copyright © 2024. All rights reserved.