com.microsoft.azure.documentdb.bulkexecutor

Class DocumentBulkExecutor

java.lang.Object

com.microsoft.azure.documentdb.bulkexecutor.DocumentBulkExecutor

All Implemented Interfaces:

AutoCloseable

public class DocumentBulkExecutor
extends Object
implements AutoCloseable

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	DocumentBulkExecutor.Builder

Method Summary

Modifier and Type	Method and Description
static DocumentBulkExecutor.Builder	builder() Creates a new DocumentBulkExecutor.Builder instance
void	close() Releases any internal resources.
BulkDeleteResponse	deleteAll(List<org.apache.commons.lang3.tuple.Pair<String,String>> pkIdPairsToDelete) Executes a bulk delete in the Azure Cosmos DB database service.
BulkImportResponse	importAll(Collection<String> documents, boolean isUpsert, boolean disableAutomaticIdGeneration, Integer maxConcurrencyPerPartitionRange) Executes a bulk import in the Azure Cosmos DB database service.
BulkImportResponse	importAll(Collection<String> documents, boolean isUpsert, boolean disableAutomaticIdGeneration, Integer maxConcurrencyPerPartitionRange, Integer effectiveMaxMiniBatchImportSize, Integer numberOfParallelTasks, Integer writeThroughputBudgetPerCosmosPartition)
BulkUpdateResponse	mergeAll(Collection<com.microsoft.azure.documentdb.Document> patchDocuments, Integer maxConcurrencyPerPartitionRange) Executes a bulk update in the Azure Cosmos DB database service with given set of patch documents.
BulkUpdateResponse	updateAll(Collection<UpdateItem> updateItems, Integer maxConcurrencyPerPartitionRange) Executes a bulk update in the Azure Cosmos DB database service.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

builder

public static DocumentBulkExecutor.Builder builder()

Creates a new DocumentBulkExecutor.Builder instance

Returns:

an instance of DocumentBulkExecutor.Builder

close

public void close()

Releases any internal resources. It is responsibility of the caller to close DocumentClient.

Specified by:

close in interface AutoCloseable

importAll

public BulkImportResponse importAll(Collection<String> documents,
                                    boolean isUpsert,
                                    boolean disableAutomaticIdGeneration,
                                    Integer maxConcurrencyPerPartitionRange)
                             throws com.microsoft.azure.documentdb.DocumentClientException

Executes a bulk import in the Azure Cosmos DB database service.

 
 ConnectionPolicy connectionPolicy = new ConnectionPolicy();
 RetryOptions retryOptions = new RetryOptions();
 
 // Set client's retry options high for initialization
 retryOptions.setMaxRetryWaitTimeInSeconds(120);
 retryOptions.setMaxRetryAttemptsOnThrottledRequests(100);
 connectionPolicy.setRetryOptions(retryOptions);
 connectionPolicy.setMaxPoolSize(1000);

 DocumentClient client = new DocumentClient(HOST, MASTER_KEY, connectionPolicy, null);

 String collectionLink = String.format("/dbs/%s/colls/%s", "mydb", "mycol");
 DocumentCollection collection = client.readCollection(collectionLink, null).getResource();

 DocumentBulkExecutor executor = DocumentBulkExecutor.builder().from(client, collection,
     collection.getPartitionKey(), collectionOfferThroughput).build();

 // Set retries to 0 to pass control to bulk executor
 client.getConnectionPolicy().getRetryOptions().setMaxRetryWaitTimeInSeconds(0);
 client.getConnectionPolicy().getRetryOptions().setMaxRetryAttemptsOnThrottledRequests(0);
 
 for(int i = 0; i < 10; i++) {
   List<String> documents = documentSource.getMoreDocuments();

   BulkImportResponse bulkImportResponse = executor.importAll(documents, false, true, 40);

   // Validate that all documents inserted to ensure no failure.
   if (bulkImportResponse.getNumberOfDocumentsImported() < documents.size()) {
      for(Exception e: bulkImportResponse.getErrors()) {
          // Validate why there were some failures.
          e.printStackTrace();
      }
      break;
   }
 }

 executor.close();
 client.close();
 
 

Parameters:

documents - specifies the collection of JSON-serialized documents to import

isUpsert - indicates whether a document in the supplied collection needs to be overwritten if the id already exists

disableAutomaticIdGeneration - indicates whether the id has to be automatically generated for a document if absent in the supplied collection

maxConcurrencyPerPartitionRange - specifies the maximum degree of concurrency per partition key range (default value is 20 if set to null)

Returns:

an instance of BulkImportResponse

Throws:

com.microsoft.azure.documentdb.DocumentClientException - if any failure happens

importAll

public BulkImportResponse importAll(Collection<String> documents,
                                    boolean isUpsert,
                                    boolean disableAutomaticIdGeneration,
                                    Integer maxConcurrencyPerPartitionRange,
                                    Integer effectiveMaxMiniBatchImportSize,
                                    Integer numberOfParallelTasks,
                                    Integer writeThroughputBudgetPerCosmosPartition)
                             throws com.microsoft.azure.documentdb.DocumentClientException

Parameters:

documents - specifies the collection of JSON-serialized documents to import

isUpsert - indicates whether a document in the supplied collection needs to be overwritten if the id already exists

disableAutomaticIdGeneration - indicates whether the id has to be automatically generated for a document if absent in the supplied collection

maxConcurrencyPerPartitionRange - specifies the maximum degree of concurrency per partition key range (default value is 20 if set to null)

effectiveMaxMiniBatchImportSize - specifies the maximum size of mini batch imports

numberOfParallelTasks - specifies the number of parallel ingestion tasks

writeThroughputBudgetPerCosmosPartition - specifies the user provided RU Budget that is split for each cosmos b physical partition

Returns:

an instance of BulkImportResponse

Throws:

com.microsoft.azure.documentdb.DocumentClientException - if any failure happens

updateAll

public BulkUpdateResponse updateAll(Collection<UpdateItem> updateItems,
                                    Integer maxConcurrencyPerPartitionRange)
                             throws com.microsoft.azure.documentdb.DocumentClientException

Executes a bulk update in the Azure Cosmos DB database service.

Parameters:

updateItems - specifies the collection of update items each of which comprises the list of field update operations to be performed on a document identified by an id and partition key value.

maxConcurrencyPerPartitionRange - specifies the maximum degree of concurrency per partition key range (default value is 20 if set to null)

Returns:

an instance of BulkUpdateResponse

Throws:

com.microsoft.azure.documentdb.DocumentClientException - if any failure happens

mergeAll

public BulkUpdateResponse mergeAll(Collection<com.microsoft.azure.documentdb.Document> patchDocuments,
                                   Integer maxConcurrencyPerPartitionRange)
                            throws com.microsoft.azure.documentdb.DocumentClientException

Executes a bulk update in the Azure Cosmos DB database service with given set of patch documents.

Parameters:

patchDocuments - which are documents comprising id, partition key values and fields to set with the corresponding values

maxConcurrencyPerPartitionRange - specifies the maximum degree of concurrency per partition key range (default value is 20 if set to null)

Returns:

an instance of BulkUpdateResponse

Throws:

com.microsoft.azure.documentdb.DocumentClientException - if any failure happens

deleteAll

public BulkDeleteResponse deleteAll(List<org.apache.commons.lang3.tuple.Pair<String,String>> pkIdPairsToDelete)
                             throws com.microsoft.azure.documentdb.DocumentClientException

Executes a bulk delete in the Azure Cosmos DB database service.

Parameters:

pkIdPairsToDelete - List of pairs of partition key and id values of documents to delete

Returns:

an instance of BulkDeleteResponse

Throws:

com.microsoft.azure.documentdb.DocumentClientException - if any failure happens