A B C D E F G H I J K L M N O P Q R S T U V W Z

A

ack() - Method in interface org.apache.gobblin.ack.Ackable

Acknowledge this entity as a success.

ack() - Method in class org.apache.gobblin.ack.BasicAckableForTesting

ack() - Method in class org.apache.gobblin.stream.StreamEntity

Ackable - Interface in org.apache.gobblin.ack

An interface for entities that can be acked

acked - Variable in class org.apache.gobblin.ack.BasicAckableForTesting

addAll(Properties) - Method in class org.apache.gobblin.configuration.ImmutableWorkUnitState

addAll(State) - Method in class org.apache.gobblin.configuration.State

Populates this instance with properties of the other instance.

addAll(Properties) - Method in class org.apache.gobblin.configuration.State

Populates this instance with values of a Properties instance.

addAll(Properties) - Method in class org.apache.gobblin.source.workunit.ImmutableExtract

addAll(State) - Method in class org.apache.gobblin.source.workunit.ImmutableExtract

addAll(Properties) - Method in class org.apache.gobblin.source.workunit.ImmutableWorkUnit

addAll(State) - Method in class org.apache.gobblin.source.workunit.ImmutableWorkUnit

addAllIfNotExist(Properties) - Method in class org.apache.gobblin.configuration.ImmutableWorkUnitState

addAllIfNotExist(State) - Method in class org.apache.gobblin.configuration.State

Add properties in a State instance that are not in the current instance.

addAllIfNotExist(Properties) - Method in class org.apache.gobblin.configuration.State

Add properties in a Properties instance that are not in the current instance.

addAllIfNotExist(Properties) - Method in class org.apache.gobblin.source.workunit.ImmutableExtract

addAllIfNotExist(State) - Method in class org.apache.gobblin.source.workunit.ImmutableExtract

addAllIfNotExist(Properties) - Method in class org.apache.gobblin.source.workunit.ImmutableWorkUnit

addAllIfNotExist(State) - Method in class org.apache.gobblin.source.workunit.ImmutableWorkUnit

addCallBack(Ackable) - Method in class org.apache.gobblin.stream.StreamEntity

addDeltaField(String...) - Method in class org.apache.gobblin.source.workunit.Extract

Deprecated.

It is recommended to add delta fields in WorkUnit instead of Extract.

addDeltaField(String...) - Method in class org.apache.gobblin.source.workunit.ImmutableExtract

addFinalConstructState(String, State) - Method in class org.apache.gobblin.configuration.WorkUnitState

Adds all properties from State to this WorkUnitState.

addMetadata(String, String) - Method in class org.apache.gobblin.dataset.DatasetDescriptor

addPrimaryKey(String...) - Method in class org.apache.gobblin.source.workunit.Extract

Deprecated.

@deprecated It is recommended to add primary keys in WorkUnit instead of Extract.

addPrimaryKey(String...) - Method in class org.apache.gobblin.source.workunit.ImmutableExtract

addSpec(V) - Method in interface org.apache.gobblin.runtime.api.SpecProducer

Add a Spec for execution on SpecExecutor.

addStep(CommitStep) - Method in class org.apache.gobblin.commit.CommitSequence.Builder

Add a CommitStep to the commit sequence.

addWorkUnit(WorkUnit) - Method in class org.apache.gobblin.source.workunit.MultiWorkUnit

Add a single WorkUnit.

addWorkUnits(Collection<WorkUnit>) - Method in class org.apache.gobblin.source.workunit.MultiWorkUnit

Add a collection of WorkUnits.

ADMIN_SERVER_ENABLED_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

ADMIN_SERVER_FACTORY_CLASS_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

The name of the class with the admin interface.

ADMIN_SERVER_HIDE_JOBS_WITHOUT_TASKS_BY_DEFAULT_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

ADMIN_SERVER_HOST_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

ADMIN_SERVER_PORT_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

ADMIN_SERVER_REFRESH_INTERVAL_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

AdminWebServerFactory - Interface in org.apache.gobblin.runtime.api

A factory interface for AdminWebServer.

ALERT_EMAIL_ENABLED_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

Configuration properties for email settings.

Alias - Annotation Type in org.apache.gobblin.annotation

Denotes that a class has an alias.

Alias - Annotation Type in org.apache.gobblin.typedconfig

Alpha - Annotation Type in org.apache.gobblin.annotation

Denotes an API that is still under active development and can undergo significant changes.

appendToListProp(String, String) - Method in class org.apache.gobblin.configuration.ImmutableWorkUnitState

appendToListProp(String, String) - Method in class org.apache.gobblin.configuration.State

Appends the input value to a list property that can be retrieved with State.getPropAsList(java.lang.String).

appendToListProp(String, String) - Method in class org.apache.gobblin.source.workunit.ImmutableExtract

appendToListProp(String, String) - Method in class org.apache.gobblin.source.workunit.ImmutableWorkUnit

appendToSetProp(String, String) - Method in class org.apache.gobblin.configuration.State

Appends the input value to a set property that can be retrieved with State.getPropAsSet(java.lang.String).

apply(Void) - Method in class org.apache.gobblin.testing.AssertWithBackoff.EqualsCheck

assertEquals(Function<Void, T>, T, String) - Method in class org.apache.gobblin.testing.AssertWithBackoff

A convenience method for AssertWithBackoff.assertTrue(Predicate, String) to keep checking until a certain value until it becomes equal to an expected value.

assertTrue(Predicate<Void>, String) - Method in class org.apache.gobblin.testing.AssertWithBackoff

Performs a check for a condition with a back-off.

assertTrue(Predicate<Void>, long, String, Logger, double, long) - Static method in class org.apache.gobblin.testing.AssertWithBackoff

Performs a check for a condition with a back-off.

AssertWithBackoff - Class in org.apache.gobblin.testing

A helper class to perform a check for a condition with a back-off.

AssertWithBackoff() - Constructor for class org.apache.gobblin.testing.AssertWithBackoff

AssertWithBackoff.EqualsCheck<T> - Class in org.apache.gobblin.testing

ATTRS_PATH_IN_CONFIG - Static variable in class org.apache.gobblin.service.ServiceConfigKeys

AUTHENTICATOR_PASSWORD - Static variable in class org.apache.gobblin.password.EncryptedPasswordAuthenticator

AUTHENTICATOR_USERNAME - Static variable in class org.apache.gobblin.password.EncryptedPasswordAuthenticator

AVG_RECORD_SIZE - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

AVRO_SCHEMA_CHECK_STRATEGY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

Configuration related to avro schema check strategy

AVRO_SCHEMA_CHECK_STRATEGY_DEFAULT - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

AZKABAN_EXEC_ID - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

AZKABAN_EXECUTION_DAYS_LIST - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

AZKABAN_EXECUTION_TIME_RANGE - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

Azkaban properties.

AZKABAN_FLOW_ID - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

AZKABAN_FLOW_URL - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

AZKABAN_JOB_EXEC_URL - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

AZKABAN_JOB_ID - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

AZKABAN_JOB_URL - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

AZKABAN_PROJECT_NAME - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

AZKABAN_SERVER_NAME - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

AZKABAN_URL - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

AZKABAN_WEBSERVERHOST - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

B

backoffActualHighWatermark() - Method in class org.apache.gobblin.configuration.ImmutableWorkUnitState

backoffActualHighWatermark() - Method in class org.apache.gobblin.configuration.WorkUnitState

Backoff the actual high watermark to the low watermark returned by WorkUnit.getLowWatermark().

backoffFactor(double) - Method in class org.apache.gobblin.testing.AssertWithBackoff

Set the number to multiple the sleep after condition failure

BasicAckableForTesting - Class in org.apache.gobblin.ack

An ackable used for testing that just counts the times it is acked and nacked.

BasicAckableForTesting() - Constructor for class org.apache.gobblin.ack.BasicAckableForTesting

BasicTestControlMessage<T> - Class in org.apache.gobblin.runtime

A basic ControlMessage used for testing.

BasicTestControlMessage() - Constructor for class org.apache.gobblin.runtime.BasicTestControlMessage

BasicWorkUnitStream - Class in org.apache.gobblin.source.workunit

A basic implementation of WorkUnitStream.

BasicWorkUnitStream.Builder - Class in org.apache.gobblin.source.workunit

beginStep(Class<T>) - Method in class org.apache.gobblin.commit.CommitSequence.Builder

Build a CommitStep.

Beta - Annotation Type in org.apache.gobblin.annotation

Denotes an API that is still under development and can change but the changes should be relatively minor.

bindSharedResourceAtScope(SharedResourceFactory<T, K, S>, K, S, T) - Method in interface org.apache.gobblin.broker.iface.SharedResourcesBroker

Bind an instance to the input factory, key, and scope.

BRANCH - Static variable in class org.apache.gobblin.dataset.DatasetConstants

Common metadata

branch - Variable in class org.apache.gobblin.writer.DataWriterBuilder

branches - Variable in class org.apache.gobblin.writer.DataWriterBuilder

BrokerConstants - Class in org.apache.gobblin.broker

Constants used for SharedResourcesBroker.

BrokerConstants() - Constructor for class org.apache.gobblin.broker.BrokerConstants

BROKERS - Static variable in class org.apache.gobblin.dataset.DatasetConstants

Kafka metadata

build() - Method in interface org.apache.gobblin.broker.iface.SubscopedBrokerBuilder

build() - Method in class org.apache.gobblin.commit.CommitSequence.Builder

build() - Method in class org.apache.gobblin.commit.CommitStepBase.Builder

build() - Method in class org.apache.gobblin.source.workunit.BasicWorkUnitStream.Builder

build() - Method in class org.apache.gobblin.writer.DataWriterBuilder

Build a DataWriter.

buildClone() - Method in class org.apache.gobblin.runtime.BasicTestControlMessage

buildClone() - Method in class org.apache.gobblin.stream.FlushControlMessage

buildClone() - Method in class org.apache.gobblin.stream.MetadataUpdateControlMessage

buildClone() - Method in class org.apache.gobblin.stream.RecordEnvelope

buildClone() - Method in class org.apache.gobblin.stream.StreamEntity

buildCredentialStore(Map<String, Object>) - Method in interface org.apache.gobblin.crypto.CredentialStoreProvider

Build a credential store based on the passed in configuration parameters - EncryptionConfigParser can be used to help parse known keys out of the config bag.

Builder() - Constructor for class org.apache.gobblin.commit.CommitSequence.Builder

Builder() - Constructor for class org.apache.gobblin.commit.CommitStepBase.Builder

Builder(CommitSequence.Builder) - Constructor for class org.apache.gobblin.commit.CommitStepBase.Builder

Builder(Iterator<WorkUnit>) - Constructor for class org.apache.gobblin.source.workunit.BasicWorkUnitStream.Builder

Builder(List<WorkUnit>) - Constructor for class org.apache.gobblin.source.workunit.BasicWorkUnitStream.Builder

builderWithInput(GlobalMetadata<SI>, Optional<SO>) - Static method in class org.apache.gobblin.metadata.GlobalMetadata

Builder that takes in an input to use as a base.

buildStreamCryptoProvider(String, Map<String, Object>) - Method in interface org.apache.gobblin.crypto.EncryptionProvider

Build a StreamEncryptor with the given configuration parameters.

bytesWritten() - Method in interface org.apache.gobblin.writer.DataWriter

Get the number of bytes written.

C

calculatePercentCompletion(Watermark, Watermark) - Method in interface org.apache.gobblin.source.extractor.Watermark

This method must return a value from [0, 100].

canBeSkipped() - Method in class org.apache.gobblin.publisher.DataPublisher

Return true if the current publisher can be skipped.

cancel(boolean) - Method in class org.apache.gobblin.util.CompletedFuture

cancelJob(URI, Properties) - Method in interface org.apache.gobblin.runtime.api.SpecProducer

Cancel the job execution identified by jobURI

Capability - Class in org.apache.gobblin.capability

Represents a set of functionality a job-creator can ask for.

Capability() - Constructor for class org.apache.gobblin.capability.Capability

CapabilityAware - Interface in org.apache.gobblin.capability

Describes an object that is aware of the capabilities it supports.

changedSpecs() - Method in interface org.apache.gobblin.runtime.api.SpecConsumer

List of newly changed Specs for execution on SpecExecutor.

CheckpointableWatermark - Interface in gobblin.source.extractor

Shim layer for org.apache.gobblin.source.extractor.CheckpointableWatermark

CheckpointableWatermark - Interface in org.apache.gobblin.source.extractor

Watermark that is Comparable and Checkpointable

CheckpointableWatermarkState - Class in org.apache.gobblin.runtime

Making CheckpointableWatermark look like State so it can be stored in a Gobblin state store.

CheckpointableWatermarkState(CheckpointableWatermark, Gson) - Constructor for class org.apache.gobblin.runtime.CheckpointableWatermarkState

Create a CheckpointableWatermarkState object from a CheckpointableWatermark

CheckpointableWatermarkState() - Constructor for class org.apache.gobblin.runtime.CheckpointableWatermarkState

Needed for reflection based construction.

childRecord(DO) - Method in class org.apache.gobblin.stream.RecordEnvelope.ForkRecordBuilder

Create a new child RecordEnvelope with the specified record.

ClassAliasResolver<T> - Class in org.apache.gobblin.util

A class that scans through all classes in the classpath annotated with Alias and is a subType of subTypeOf constructor argument.

ClassAliasResolver(Class<T>) - Constructor for class org.apache.gobblin.util.ClassAliasResolver

CLEAN_ERR_DIR - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

cleanup() - Method in interface org.apache.gobblin.writer.DataWriter

Cleanup context/resources.

CLEANUP_OLD_JOBS_DATA - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

CLEANUP_STAGING_DATA_BY_INITIALIZER - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

CLEANUP_STAGING_DATA_PER_TASK - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

close() - Method in class org.apache.gobblin.ack.HierarchicalAckable

Indicates no more children will be created.

close() - Method in interface org.apache.gobblin.broker.iface.SharedResourcesBroker

Close all resources at this and descendant scopes, meaning Closeables will be closed and Services will be shut down.

close() - Method in class org.apache.gobblin.converter.Converter

close() - Method in class org.apache.gobblin.converter.initializer.NoopConverterInitializer

close() - Method in interface org.apache.gobblin.initializer.Initializer

Removed checked exception.

close() - Method in class org.apache.gobblin.initializer.NoopInitializer

close() - Method in class org.apache.gobblin.qualitychecker.row.RowLevelPolicy

close() - Method in class org.apache.gobblin.stream.ControlMessageInjector

close() - Method in class org.apache.gobblin.stream.StreamEntity.ForkCloner

close() - Method in class org.apache.gobblin.stream.StreamEntity.ForkedEntityBuilder

close() - Method in class org.apache.gobblin.writer.initializer.NoopWriterInitializer

CombinedWorkUnitAndDatasetState - Class in org.apache.gobblin.configuration

A class that encapsulates WorkUnitState and DatasetStates.

CombinedWorkUnitAndDatasetState() - Constructor for class org.apache.gobblin.configuration.CombinedWorkUnitAndDatasetState

CombinedWorkUnitAndDatasetStateFunctional - Interface in org.apache.gobblin.configuration

A FunctionalInterface to return CombinedWorkUnitAndDatasetState.

commit(Spec) - Method in interface org.apache.gobblin.runtime.api.SpecConsumer

A commit method to allow the consumer to checkpoint state on successful consumption of a Spec.

commit() - Method in interface org.apache.gobblin.writer.DataWriter

Commit the data written.

CommitSequence - Class in org.apache.gobblin.commit

A sequence of CommitSteps that should be executed atomically.

CommitSequence.Builder - Class in org.apache.gobblin.commit

CommitSequenceStore - Interface in org.apache.gobblin.commit

A store for CommitSequences.

CommitStep - Interface in org.apache.gobblin.commit

A step during committing a dataset that should be executed atomically with other steps under exactly-once semantics.

CommitStepBase - Class in org.apache.gobblin.commit

A base implementation of CommitStep.

CommitStepBase(CommitStepBase.Builder<? extends CommitStepBase.Builder<?>>) - Constructor for class org.apache.gobblin.commit.CommitStepBase

CommitStepBase.Builder<T extends CommitStepBase.Builder<?>> - Class in org.apache.gobblin.commit

CommitStepException - Exception in org.apache.gobblin.commit

CommitStepException(String, Throwable) - Constructor for exception org.apache.gobblin.commit.CommitStepException

CommitStepException(String) - Constructor for exception org.apache.gobblin.commit.CommitStepException

commitWatermarks(Iterable<CheckpointableWatermark>) - Method in interface org.apache.gobblin.writer.WatermarkStorage

Commit a batch of watermarks to storage.

commonDatasetRoot() - Method in interface org.apache.gobblin.dataset.DatasetsFinder

Deprecated.

commonDatasetRoot() - Method in class org.apache.gobblin.dataset.test.StaticDatasetsFinderForTesting

COMPACTION_ESTIMATOR - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

COMPACTION_PREFIX - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

Configuration properties related to Compaction Suite

COMPACTION_PRIORITIZATION_PREFIX - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

COMPACTION_PRIORITIZER_ALIAS - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

COMPACTION_SUITE_FACTORY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

ComparableWatermark - Interface in org.apache.gobblin.source.extractor

Watermark that is also Comparable.

compare(URNIdentified, URNIdentified) - Method in class org.apache.gobblin.dataset.comparators.URNLexicographicalComparator

compare(URNIdentified, String) - Method in class org.apache.gobblin.dataset.comparators.URNLexicographicalComparator

Compare against a raw URN.

compare(String, URNIdentified) - Method in class org.apache.gobblin.dataset.comparators.URNLexicographicalComparator

Compare against a raw URN.

COMPILATION_SUCCESSFUL - Static variable in class org.apache.gobblin.service.ServiceConfigKeys

CompletedFuture<T> - Class in org.apache.gobblin.util

CompletedFuture(T, Throwable) - Constructor for class org.apache.gobblin.util.CompletedFuture

computeRetrySleep(long, double, long, long) - Static method in class org.apache.gobblin.testing.AssertWithBackoff

Computes a new sleep for a retry of a condition check.

CONFIG_BASED_PREFIX - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

Configuration related to ConfigStore based copy/retention

CONFIG_MANAGEMENT_STORE_ENABLED - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

CONFIG_MANAGEMENT_STORE_URI - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

Config store properties

CONFIG_RUNTIME_PREFIX - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

ConfigurationException - Exception in org.apache.gobblin.configuration

ConfigurationException(String, Exception) - Constructor for exception org.apache.gobblin.configuration.ConfigurationException

ConfigurationKeys - Class in org.apache.gobblin.configuration

A central place for all Gobblin configuration property keys.

ConfigurationKeys() - Constructor for class org.apache.gobblin.configuration.ConfigurationKeys

ConfigView<S extends ScopeType<S>,K extends SharedResourceKey> - Interface in org.apache.gobblin.broker.iface

Provides a view of a Config specific to a SharedResourceKey and factory.

CONNECTION_URL - Static variable in class org.apache.gobblin.dataset.DatasetConstants

JDBC metadata

constraint(Field, Object) - Static method in class org.apache.gobblin.typedconfig.ConstraintUtil

ConstraintUtil - Class in org.apache.gobblin.typedconfig

Util class for handling constraint annotation supports: IntRange, LongRange, EnumOption, StringRegex

constructClass() - Method in enum org.apache.gobblin.Constructs

Constructs - Enum in org.apache.gobblin

Enumeration of available gobblin constructs.

consumeRecordStream(RecordStreamWithMetadata<D, S>) - Method in interface org.apache.gobblin.records.RecordStreamConsumer

Subscribe to the input RecordStreamWithMetadata.

contains(String) - Method in class org.apache.gobblin.configuration.State

Check if a property is set.

contains(String) - Method in class org.apache.gobblin.configuration.WorkUnitState

contains(String) - Method in class org.apache.gobblin.source.workunit.WorkUnit

ControlMessage<D> - Class in org.apache.gobblin.stream

A StreamEntity used to send control messages through the ingestion pipeline.

ControlMessage() - Constructor for class org.apache.gobblin.stream.ControlMessage

ControlMessageHandler - Interface in org.apache.gobblin.records

Handles ControlMessages in the record stream.

ControlMessageHandler.Noop - Class in org.apache.gobblin.records

ControlMessageInjector<SI,DI> - Class in org.apache.gobblin.stream

A RecordStreamProcessor that inspects an input record and outputs control messages before, after, or around the input record

ControlMessageInjector() - Constructor for class org.apache.gobblin.stream.ControlMessageInjector

Converter<SI,SO,DI,DO> - Class in org.apache.gobblin.converter

An interface for classes that implement data transformations, e.g., data type conversions, schema projections, data manipulations, data filtering, etc.

Converter() - Constructor for class org.apache.gobblin.converter.Converter

CONVERTER_AVRO_BINARY_CHARSET - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

CONVERTER_AVRO_DATE_FORMAT - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

Converter configuration properties.

CONVERTER_AVRO_DATE_TIMEZONE - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

CONVERTER_AVRO_EXTRACTOR_FIELD_PATH - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

CONVERTER_AVRO_FIELD_PICK_FIELDS - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

CONVERTER_AVRO_JDBC_ENTRY_FIELDS_PAIRS - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

CONVERTER_AVRO_MAX_CONVERSION_FAILURES - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

CONVERTER_AVRO_SCHEMA_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

CONVERTER_AVRO_TIME_FORMAT - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

CONVERTER_AVRO_TIMESTAMP_FORMAT - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

CONVERTER_CLASSES_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

CONVERTER_CSV_TO_JSON_DELIMITER - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

CONVERTER_CSV_TO_JSON_ENCLOSEDCHAR - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

CONVERTER_FILTER_FIELD_NAME - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

CONVERTER_FILTER_FIELD_VALUE - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

CONVERTER_IGNORE_FIELDS - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

CONVERTER_IS_EPOCH_TIME_IN_SECONDS - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

CONVERTER_SKIP_FAILED_RECORD - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

CONVERTER_STRING_FILTER_PATTERN - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

CONVERTER_STRING_SPLITTER_DELIMITER - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

CONVERTER_STRING_SPLITTER_SHOULD_TRIM_RESULTS - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

ConverterInitializer - Interface in org.apache.gobblin.converter.initializer

ConverterInitializer is being invoked at driver which means it will only invoked once per converter.

convertJsonToWatermark(JsonElement, Class<T>) - Static method in class org.apache.gobblin.source.extractor.WatermarkSerializerHelper

Converts a JsonElement into the specified class type using the Gson.fromJson(JsonElement, Class) method.

convertPath(Path, String, RecordCountProvider) - Method in class org.apache.gobblin.util.RecordCountProvider

Convert a Path from another RecordCountProvider so that it can be used in RecordCountProvider.getRecordCount(Path) of this RecordCountProvider.

convertRecord(SO, DI, WorkUnitState) - Method in class org.apache.gobblin.converter.Converter

Convert an input data record to an Iterable of output records conforming to the output schema of Converter.convertSchema(SI, org.apache.gobblin.configuration.WorkUnitState).

convertRecordEnvelope(SO, RecordEnvelope<DI>, WorkUnitState) - Method in class org.apache.gobblin.converter.Converter

Converts a RecordEnvelope.

convertSchema(SI, WorkUnitState) - Method in class org.apache.gobblin.converter.Converter

Convert an input schema to an output schema.

convertWatermarkToJson(Watermark) - Static method in class org.apache.gobblin.source.extractor.WatermarkSerializerHelper

Converts a Watermark to a JsonElement using the Gson.toJsonTree(Object) method.

copy() - Method in class org.apache.gobblin.dataset.DatasetDescriptor

copy() - Method in class org.apache.gobblin.dataset.Descriptor

copy() - Method in class org.apache.gobblin.dataset.PartitionDescriptor

copy() - Method in interface org.apache.gobblin.fork.Copyable

Make a new copy of this instance.

copy(Object) - Static method in class org.apache.gobblin.fork.CopyHelper

Copy this object if needed.

copy() - Method in class org.apache.gobblin.metadata.GlobalMetadata

COPY_EXPECTED_SCHEMA - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

COPY_SOURCE_FILESET_WU_GENERATOR_CLASS - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

Configuration properties used by the CopySource.

Copyable<T> - Interface in org.apache.gobblin.fork

An interface for classes that supports making copies of their instances.

CopyHelper - Class in org.apache.gobblin.fork

A helper class to copy things that may or may not be Copyable.

CopyHelper() - Constructor for class org.apache.gobblin.fork.CopyHelper

CopyNotSupportedException - Exception in org.apache.gobblin.fork

A type of Exceptions thrown when copying is not supported.

CopyNotSupportedException(String) - Constructor for exception org.apache.gobblin.fork.CopyNotSupportedException

copyOf(WorkUnit) - Static method in class org.apache.gobblin.source.workunit.WorkUnit

Factory method.

copyWithNewDataset(DatasetDescriptor) - Method in class org.apache.gobblin.dataset.PartitionDescriptor

create(Extract) - Static method in class org.apache.gobblin.source.workunit.WorkUnit

Factory method.

create(Extract, WatermarkInterval) - Static method in class org.apache.gobblin.source.workunit.WorkUnit

Factory method.

create() - Static method in class org.apache.gobblin.testing.AssertWithBackoff

Creates a new instance

create(Gson, TypeToken<R>) - Method in class org.apache.gobblin.util.io.GsonInterfaceAdapter

createEmpty() - Static method in class org.apache.gobblin.source.workunit.MultiWorkUnit

Create a new empty MultiWorkUnit instance.

createEmpty() - Static method in class org.apache.gobblin.source.workunit.WorkUnit

Factory method.

createExtract(Extract.TableType, String, String) - Method in class org.apache.gobblin.configuration.SourceState

Deprecated.

createInstance(Properties, URI) - Method in interface org.apache.gobblin.runtime.api.AdminWebServerFactory

Creates a new AdminWebServer instance

createResolver(Config) - Method in interface org.apache.gobblin.dataset.DatasetResolverFactory

Deprecated.

Create a DatasetResolver instance

createResolver(Config) - Method in interface org.apache.gobblin.dataset.DescriptorResolverFactory

createResolver(Config) - Method in class org.apache.gobblin.dataset.HiveToHdfsDatasetResolverFactory

createResource(SharedResourcesBroker<S>, ScopedConfigView<S, K>) - Method in interface org.apache.gobblin.broker.iface.SharedResourceFactory

Requests an object for the provided SharedResourceKey, with the provided configuration.

createWorkUnit(Extract) - Method in class org.apache.gobblin.configuration.SourceState

Deprecated.

Properties in SourceState should not added to a WorkUnit. Having each WorkUnit contain a copy of SourceState is a waste of memory. Use WorkUnit.create(Extract).

CredentialStore - Interface in org.apache.gobblin.crypto

Interface for a simple CredentialStore that simply has a set of byte-encoded keys.

CredentialStoreProvider - Interface in org.apache.gobblin.crypto

Represents a factory object that can build CredentialStores based on a set of config.

D

DAG_STORE_KEY_SEPARATION_CHARACTER - Static variable in class org.apache.gobblin.service.ServiceConfigKeys

DATA_MOVEMENT_AUTHORIZER_CLASS - Static variable in class org.apache.gobblin.service.ServiceConfigKeys

DATA_PUBLISH_WRITER_METADATA_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

Metadata configuration PUBLISH_WRITER_METADATA_KEY: Whether or not to publish writer-generated metadata PUBLISH_WRITER_METADATA_MERGER_NAME_KEY: Class to use to merge writer-generated metadata.

DATA_PUBLISH_WRITER_METADATA_MERGER_NAME_DEFAULT - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

Metadata configuration properties used internally

DATA_PUBLISH_WRITER_METADATA_MERGER_NAME_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DATA_PUBLISHER_APPEND_EXTRACT_TO_FINAL_DIR - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DATA_PUBLISHER_CAN_BE_SKIPPED - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DATA_PUBLISHER_DATASET_DIR - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

Dynamically configured Publisher properties used internally

DATA_PUBLISHER_FILE_SYSTEM_URI - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DATA_PUBLISHER_FINAL_DIR - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DATA_PUBLISHER_FINAL_DIR_GROUP - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

Deprecated.

DATA_PUBLISHER_FINAL_NAME - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DATA_PUBLISHER_METADATA_OUTPUT_DIR - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DATA_PUBLISHER_METADATA_OUTPUT_FILE - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DATA_PUBLISHER_METADATA_STR - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DATA_PUBLISHER_OUTPUT_DIR_GROUP - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DATA_PUBLISHER_OVERWRITE_ENABLED - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DATA_PUBLISHER_PERMISSIONS - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DATA_PUBLISHER_PREFIX - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

Configuration properties for the data publisher.

DATA_PUBLISHER_REPLACE_FINAL_DIR - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DATA_PUBLISHER_TYPE - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

Deprecated.

Use ConfigurationKeys.TASK_DATA_PUBLISHER_TYPE and ConfigurationKeys.JOB_DATA_PUBLISHER_TYPE.

DataConversionException - Exception in org.apache.gobblin.converter

A type of Exception thrown when there's anything wrong with data conversion.

DataConversionException(Throwable) - Constructor for exception org.apache.gobblin.converter.DataConversionException

DataConversionException(String, Throwable) - Constructor for exception org.apache.gobblin.converter.DataConversionException

DataConversionException(String) - Constructor for exception org.apache.gobblin.converter.DataConversionException

DataPublisher - Class in org.apache.gobblin.publisher

Defines how to publish data and its corresponding metadata.

DataPublisher(State) - Constructor for class org.apache.gobblin.publisher.DataPublisher

DataRecordException - Exception in org.apache.gobblin.source.extractor

DataRecordException(String, Exception) - Constructor for exception org.apache.gobblin.source.extractor.DataRecordException

DataRecordException(String) - Constructor for exception org.apache.gobblin.source.extractor.DataRecordException

Dataset - Interface in org.apache.gobblin.dataset

Interface representing a dataset.

DATASET_BASE_INPUT_PATH_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DATASET_BASE_OUTPUT_PATH_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DATASET_COMBINE_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DATASET_COMMIT_THREADS - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

Only applicable if ConfigurationKeys.PARALLELIZE_DATASET_COMMIT is true.

DATASET_DESTINATION_PATH - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DATASET_STATE_STORE_PREFIX - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DATASET_STATE_STORE_TYPE_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DATASET_SUBPATHS_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DATASET_URN_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

Dataset-related configuration properties;

DatasetConstants - Class in org.apache.gobblin.dataset

DatasetConstants() - Constructor for class org.apache.gobblin.dataset.DatasetConstants

DatasetDescriptor - Class in org.apache.gobblin.dataset

A Descriptor identifies and provides metadata to describe a dataset

DatasetDescriptor(String, String) - Constructor for class org.apache.gobblin.dataset.DatasetDescriptor

Deprecated.

use DatasetDescriptor.DatasetDescriptor(String, URI, String) to provide storage system url.

DatasetDescriptor(String, URI, String) - Constructor for class org.apache.gobblin.dataset.DatasetDescriptor

DatasetDescriptor(DatasetDescriptor) - Constructor for class org.apache.gobblin.dataset.DatasetDescriptor

Deprecated.

use DatasetDescriptor.copy()

DatasetResolver - Interface in org.apache.gobblin.dataset

Deprecated.

use the more general DescriptorResolver

DatasetResolverFactory - Interface in org.apache.gobblin.dataset

Deprecated.

use DescriptorResolverFactory as DatasetResolver is deprecated with DescriptorResolver

datasetRoot() - Method in interface org.apache.gobblin.dataset.FileSystemDataset

DatasetsFinder<T extends Dataset> - Interface in org.apache.gobblin.dataset

Finds Datasets in the file system.

datasetURN() - Method in interface org.apache.gobblin.dataset.Dataset

Deprecated.

use Dataset.getUrn()

datasetURN() - Method in class org.apache.gobblin.dataset.test.SimpleDatasetForTesting

datasetURN() - Method in class org.apache.gobblin.dataset.test.SimplePartitionableDatasetForTesting

DATASETURN_STATESTORE_NAME_PARSER - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DataWriter<D> - Interface in org.apache.gobblin.writer

An interface for data writers

DataWriterBuilder<S,D> - Class in org.apache.gobblin.writer

A builder class for DataWriter.

DataWriterBuilder() - Constructor for class org.apache.gobblin.writer.DataWriterBuilder

decodeInputStream(InputStream) - Method in interface org.apache.gobblin.codec.StreamCodec

Wrap a bytestream and return a new stream that will decode any bytes from origStream when read from.

Decorator - Interface in org.apache.gobblin.util

Interface for decorator patterns.

DecoratorUtils - Class in org.apache.gobblin.util

Utilities for classes implementing the decorator interface.

DecoratorUtils() - Constructor for class org.apache.gobblin.util.DecoratorUtils

decryptPassword(String) - Method in class org.apache.gobblin.password.PasswordManager

Decrypt an encrypted password.

Default - Annotation Type in org.apache.gobblin.typedconfig

DEFAULT_ADMIN_SERVER_FACTORY_CLASS - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_ADMIN_SERVER_HIDE_JOBS_WITHOUT_TASKS_BY_DEFAULT - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_ADMIN_SERVER_HOST - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_ADMIN_SERVER_PORT - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_ADMIN_SERVER_REFRESH_INTERVAL - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_AUTHENTICATOR_CLASS - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

Optional property to specify a default Authenticator class for a job

DEFAULT_BUFFER_SIZE - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_CHARSET_ENCODING - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_CLEAN_ERR_DIR - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_CLEANUP_OLD_JOBS_DATA - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_CLEANUP_STAGING_DATA_PER_TASK - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_COLUMN_NAME_CASE - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_COMPACTION_SUITE_FACTORY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_CONFIG_MANAGEMENT_STORE_ENABLED - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_CONN_TIMEOUT - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_CONVERTER_AVRO_MAX_CONVERSION_FAILURES - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_CONVERTER_CSV_TO_JSON_ENCLOSEDCHAR - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_CONVERTER_STRING_SPLITTER_SHOULD_TRIM_RESULTS - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_DATA_PUBLISHER_APPEND_EXTRACT_TO_FINAL_DIR - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_DATA_PUBLISHER_CAN_BE_SKIPPED - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_DATA_PUBLISHER_TYPE - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_DATASET_COMMIT_THREADS - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_DATASET_URN - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_DEFLATE_LEVEL - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_DYNAMIC_CONFIG_GENERATOR_CLASS_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_EMAIL_HOST - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_ENABLE_DELIMITED_IDENTIFIER - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_ENABLE_MR_SPECULATIVE_EXECUTION - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_ENCRYPT_USE_STRONG_ENCRYPTOR - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_EVENT_METADATA_GENERATOR_CLASS_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_EVENTS_REPORTING_KAFKA_REPORTER_CLASS - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_EXTRACT_ID_TIME_ZONE - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_EXTRACT_IS_FULL - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_EXTRACT_LIMIT_ENABLED - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_FAILURE_REPORTING_FILE_ENABLED - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_FILEBASED_REPORT_STATUS_ON_COUNT - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_FLOW_CONCURRENCY_ALLOWED - Static variable in class org.apache.gobblin.service.ServiceConfigKeys

DEFAULT_FORCE_LEADER - Static variable in class org.apache.gobblin.service.ServiceConfigKeys

DEFAULT_FORK_BRANCH_NAME - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_FORK_CLOSE_WRITER_ON_COMPLETION - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_FORK_FINISHED_CHECK_INTERVAL - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_FORK_MAX_WAIT_MININUTES - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_FORK_OPERATOR_CLASS - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_FORK_RECORD_QUEUE_CAPACITY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_FORK_RECORD_QUEUE_TIMEOUT - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_FORK_RECORD_QUEUE_TIMEOUT_UNIT - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_FS_PROXY_AUTH_METHOD - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_GOBBLIN_FLOW_SLA_TIME_UNIT - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_GOBBLIN_JOB_EVENT_REPORTING_FAILURE_FATAL - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_GOBBLIN_JOB_METRIC_REPORTING_FAILURE_FATAL - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_GOBBLIN_JOB_START_SLA_TIME - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_GOBBLIN_JOB_START_SLA_TIME_UNIT - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_GOBBLIN_SERVICE_FLOW_CATALOG_LOCAL_COMMIT - Static variable in class org.apache.gobblin.service.ServiceConfigKeys

DEFAULT_GOBBLIN_SERVICE_FLOWCOMPILER_CLASS - Static variable in class org.apache.gobblin.service.ServiceConfigKeys

Directly use canonical class name here to avoid introducing additional dependency here.

DEFAULT_GOBBLIN_TASK_EVENT_REPORTING_FAILURE_FATAL - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_GOBBLIN_TASK_METRIC_REPORTING_FAILURE_FATAL - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_GROUP_OWNERSHIP_SERVICE - Static variable in class org.apache.gobblin.service.ServiceConfigKeys

DEFAULT_HIVE_REG_PUBLISHER_CLASS - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_JOB_COMMIT_POLICY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_JOB_CONFIG_FILE_EXTENSIONS - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_JOB_CONFIG_FILE_MONITOR_POLLING_INTERVAL - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_JOB_EXECUTOR_THREAD_POOL_SIZE - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_JOB_HISTORY_STORE_JDBC_DRIVER - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_JOB_HISTORY_STORE_PASSWORD - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_JOB_HISTORY_STORE_USER - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_JOB_MAX_FAILURES - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_JOB_RETRIGGERING_ENABLED - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_JOB_STATE_IN_STATE_STORE - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_KAFKA_SOURCE_AVG_FETCH_TIME_CAP - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_KAFKA_SOURCE_SHARE_CONSUMER_CLIENT - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_LOW_WATERMARK_BACKUP_SECONDS - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_MAX_NUMBER_OF_PARTITIONS - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_MAX_TASK_RETRIES - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_METRIC_TIMER_WINDOW_SIZE_IN_MINUTES - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_METRICS_ENABLED - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_METRICS_FILE_SUFFIX - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_METRICS_REPORT_INTERVAL - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_METRICS_REPORTING_FILE_ENABLED - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_METRICS_REPORTING_GRAPHITE_EVENTS_ENABLED - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_METRICS_REPORTING_GRAPHITE_EVENTS_VALUE_AS_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_METRICS_REPORTING_GRAPHITE_METRICS_ENABLED - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_METRICS_REPORTING_GRAPHITE_PORT - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_METRICS_REPORTING_GRAPHITE_PREFIX - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_METRICS_REPORTING_GRAPHITE_SENDING_TYPE - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_METRICS_REPORTING_INFLUXDB_EVENTS_ENABLED - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_METRICS_REPORTING_INFLUXDB_METRICS_ENABLED - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_METRICS_REPORTING_INFLUXDB_SENDING_TYPE - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_METRICS_REPORTING_JMX_ENABLED - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_METRICS_REPORTING_KAFKA_ENABLED - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_METRICS_REPORTING_KAFKA_FORMAT - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_METRICS_REPORTING_KAFKA_REPORTER_CLASS - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_METRICS_REPORTING_KAFKA_USE_SCHEMA_REGISTRY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_MR_JOB_MAX_MAPPERS - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_MR_REPORT_METRICS_AS_COUNTERS - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_NODE_SECURITY - Static variable in class org.apache.gobblin.service.ServiceConfigKeys

DEFAULT_NUMBER_OF_MASTER_PASSWORDS - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_OVERWRITE_CONFIGS_IN_STATESTORE - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_PARALLELIZE_DATASET_COMMIT - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_PROGRESS_REPORTING_THRESHOLD - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_PUBLISH_DATA_AT_JOB_LEVEL - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_QUALITY_CHECKER_TIMEZONE - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_QUEUED_TASK_TIME_MAX_AGE - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_QUEUED_TASK_TIME_MAX_SIZE - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_REPORT_JOB_PROGRESS - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_REPORTER_KEY_SIZE - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_REST_SERVER_HOST - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_REST_SERVER_PORT - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_ROW_LEVEL_ERR_FILE_RECORDS_PER_TASK - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_SCHEDULER_WAIT_FOR_JOB_COMPLETION - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_SCHEMA_FILENAME - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_SCHEMA_IN_SOURCE_DIR - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_SERVICE_POLICY - Static variable in class org.apache.gobblin.service.ServiceConfigKeys

DEFAULT_SHOULD_FS_PROXY_AS_USER - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_SOURCE_EARLY_STOP_ENABLED - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_SOURCE_FETCH_SIZE - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_SOURCE_FILEBASED_FS_PRIOR_SNAPSHOT_REQUIRED - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_SOURCE_QUERYBASED_IS_METADATA_COLUMN_CHECK_ENABLED - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_SOURCE_QUERYBASED_JDBC_RESULTSET_FETCH_SIZE - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_SOURCE_QUERYBASED_PROMOTE_UNSIGNED_INT_TO_BIGINT - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_SOURCE_QUERYBASED_RESET_EMPTY_PARTITION_WATERMARK - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_SOURCE_QUERYBASED_WATERMARK_PREDICATE_SYMBOL - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_SOURCE_TIMEZONE - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_SPEC_EXECUTOR - Static variable in class org.apache.gobblin.service.ServiceConfigKeys

DEFAULT_STATE_STORE_COMPRESSED_VALUES - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_STATE_STORE_DB_CONN_MIN_EVICTABLE_IDLE_TIME - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_STATE_STORE_DB_JDBC_DRIVER - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_STATE_STORE_DB_TABLE - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_STATE_STORE_TYPE - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_TASK_EVENT_METADATA_GENERATOR_CLASS_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_TASK_EXECUTOR_THREADPOOL_SIZE - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_TASK_RETRY_INTERVAL_IN_SEC - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_TASK_RETRY_THREAD_POOL_CORE_SIZE - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_TASK_STATE_COLLECTOR_INTERVAL_SECONDS - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_TASK_STATE_TRACKER_THREAD_POOL_CORE_SIZE - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_TASK_STATUS_REPORT_INTERVAL_IN_MS - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_TASK_SYNCHRONOUS_EXECUTION_MODEL - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_TASK_TIMEOUT_SECONDS - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_THREADPOOL_SIZE_OF_LISTING_FS_DATASET_STATESTORE - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_TOPOLOGY_SPEC_FACTORY - Static variable in class org.apache.gobblin.service.ServiceConfigKeys

DEFAULT_TROUBLESHOOTER_IN_MEMORY_ISSUE_REPOSITORY_MAX_SIZE - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_WATERMARK_TYPE - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_WATERMARK_VALUE - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

Source default configurations.

DEFAULT_WRITER_BUILDER_CLASS - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_WRITER_EAGER_INITIALIZATION - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_WRITER_FILE_BASE_NAME - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DEFAULT_WRITER_FILE_PATH_TYPE - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

defaultScopeInstance() - Method in enum org.apache.gobblin.broker.gobblin_scopes.GobblinScopeTypes

defaultScopeInstance() - Method in interface org.apache.gobblin.broker.iface.ScopeType

defaultScopeInstance() - Method in enum org.apache.gobblin.broker.SimpleScopeType

delete(String) - Method in interface org.apache.gobblin.commit.CommitSequenceStore

delete a given job name from the store along with all CommitSequences associated with this job.

delete(String, String) - Method in interface org.apache.gobblin.commit.CommitSequenceStore

delete the CommitSequence for the given job name and dataset URN.

deleteSpec(URI) - Method in interface org.apache.gobblin.runtime.api.SpecProducer

deleteSpec(URI, Properties) - Method in interface org.apache.gobblin.runtime.api.SpecProducer

Delete a Spec being executed on SpecExecutor.

DeliverySemantics - Enum in org.apache.gobblin.commit

The semantics for data delivery.

Descriptor - Class in org.apache.gobblin.dataset

A descriptor is a simplified representation of a resource, which could be a dataset, dataset partition, file, etc.

Descriptor() - Constructor for class org.apache.gobblin.dataset.Descriptor

DescriptorResolver - Interface in org.apache.gobblin.dataset

A resolver transforms an existing Descriptor to a new one

DescriptorResolverFactory - Interface in org.apache.gobblin.dataset

Factory to create a DescriptorResolver instance

deserializeAddSpecResponse(String) - Method in interface org.apache.gobblin.runtime.api.SpecProducer

destination - Variable in class org.apache.gobblin.writer.DataWriterBuilder

Destination - Class in org.apache.gobblin.writer

A class representing a destination for a writer to write to.

Destination.DestinationType - Enum in org.apache.gobblin.writer

Enumeration of supported destination types.

DESTINATION_DATASET_HANDLER_CLASS - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DISABLED_JOB_CONFIG_FILE_MONITOR_POLLING_INTERVAL - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

DYNAMIC_CONFIG_GENERATOR_CLASS_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

Configuration for dynamic configuration generation

DynamicConfigGenerator - Interface in org.apache.gobblin.configuration

For generating dynamic configuration that gets added to the job configuration.

E

EDGE_SECURITY_KEY - Static variable in class org.apache.gobblin.service.ServiceConfigKeys

EMAIL_FROM_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

EMAIL_HOST_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

EMAIL_PASSWORD_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

EMAIL_SMTP_PORT_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

EMAIL_TOS_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

EMAIL_USER_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

ENABLE_DELIMITED_IDENTIFIER - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

encodeOutputStream(OutputStream) - Method in interface org.apache.gobblin.codec.StreamCodec

Wrap a bytestream and return a new stream that encodes the bytes written to it and writes the result into origStream

ENCRYPT_KEY_FS_URI - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

Password encryption and decryption properties.

ENCRYPT_KEY_LOC - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

ENCRYPT_USE_STRONG_ENCRYPTOR - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

EncryptedPasswordAuthenticator - Class in org.apache.gobblin.password

Authenticator that uses a username and password from the provided Properties to authenticate, and also decrypts the password using PasswordManager

EncryptedPasswordAuthenticator(Properties) - Constructor for class org.apache.gobblin.password.EncryptedPasswordAuthenticator

EncryptionProvider - Interface in org.apache.gobblin.crypto

Represents a builder that can construct encryption providers given a type and various configuration parameters.

encryptPassword(String) - Method in class org.apache.gobblin.password.PasswordManager

Encrypt a password.

endStep() - Method in class org.apache.gobblin.commit.CommitStepBase.Builder

EnumOptions - Annotation Type in org.apache.gobblin.typedconfig.compiletime

equals(Object) - Method in class org.apache.gobblin.broker.gobblin_scopes.JobScopeInstance

equals(Object) - Method in class org.apache.gobblin.broker.gobblin_scopes.TaskScopeInstance

equals(Object) - Method in class org.apache.gobblin.broker.StringNameSharedResourceKey

equals(Object) - Method in class org.apache.gobblin.configuration.SourceState

equals(Object) - Method in class org.apache.gobblin.configuration.WorkUnitState

equals(Object) - Method in class org.apache.gobblin.dataset.DatasetDescriptor

equals(Object) - Method in class org.apache.gobblin.dataset.PartitionDescriptor

equals(Object) - Method in class org.apache.gobblin.source.workunit.Extract

equals(Object) - Method in class org.apache.gobblin.source.workunit.MultiWorkUnit

equals(Object) - Method in class org.apache.gobblin.source.workunit.WorkUnit

EqualsCheck(Function<Void, T>, T, String) - Constructor for class org.apache.gobblin.testing.AssertWithBackoff.EqualsCheck

ERROR_MESSAGE_UNDECODABLE_COUNT - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

ERROR_PARTITION_COUNT - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

Configuration properties used internally by the KafkaExtractor.

ESCAPE_CHARS_IN_COLUMN_NAME - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

ESCAPE_CHARS_IN_TABLE_NAME - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

EVENT_METADATA_GENERATOR_CLASS_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

Configuration for emitting job events

execute() - Method in class org.apache.gobblin.commit.CommitSequence

Execute the CommitSteps in the order they are added to the commit sequence.

execute() - Method in interface org.apache.gobblin.commit.CommitStep

Execute the commit step.

executePolicy(Object) - Method in class org.apache.gobblin.qualitychecker.row.RowLevelPolicy

executePolicy() - Method in class org.apache.gobblin.qualitychecker.task.TaskLevelPolicy

exists(String) - Method in interface org.apache.gobblin.commit.CommitSequenceStore

Whether a CommitSequence with the given job name exists in the store.

exists(String, String) - Method in interface org.apache.gobblin.commit.CommitSequenceStore

Whether a CommitSequence with the given job name and dataset URN exists in a store.

EXPECTED_HIGH_WATERMARK_TO_JSON_KEY - Static variable in class org.apache.gobblin.source.extractor.WatermarkInterval

Extract - Class in gobblin.source.workunit

Shim layer for org.apache.gobblin.source.workunit.Extract

Extract(SourceState, Extract.TableType, String, String) - Constructor for class gobblin.source.workunit.Extract

Deprecated.

Extract(Extract.TableType, String, String) - Constructor for class gobblin.source.workunit.Extract

Extract(Extract) - Constructor for class gobblin.source.workunit.Extract

Extract - Class in org.apache.gobblin.source.workunit

A class representing all the base attributes required by all tables types.

Extract(SourceState, Extract.TableType, String, String) - Constructor for class org.apache.gobblin.source.workunit.Extract

Deprecated.

Extract does not use any property in SourceState. Use Extract.Extract(TableType, String, String)

Extract(Extract.TableType, String, String) - Constructor for class org.apache.gobblin.source.workunit.Extract

Constructor.

Extract(Extract) - Constructor for class org.apache.gobblin.source.workunit.Extract

Deep copy constructor.

Extract.TableType - Enum in gobblin.source.workunit

Extract.TableType - Enum in org.apache.gobblin.source.workunit

EXTRACT_DELTA_FIELDS_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

EXTRACT_EXTRACT_ID_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

EXTRACT_FULL_RUN_TIME_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

EXTRACT_ID_TIME_ZONE - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

EXTRACT_IS_FULL_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

EXTRACT_LIMIT_ENABLED_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

EXTRACT_NAMESPACE_NAME_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

EXTRACT_PRIMARY_KEY_FIELDS_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

EXTRACT_SCHEMA - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

EXTRACT_TABLE_NAME_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

EXTRACT_TABLE_TYPE_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

Extract related configuration properties.

ExtractFactory - Class in org.apache.gobblin.source.workunit

ExtractFactory(String) - Constructor for class org.apache.gobblin.source.workunit.ExtractFactory

Extractor<S,D> - Interface in org.apache.gobblin.source.extractor

An interface for classes that are responsible for extracting data from a data source.

EXTRACTOR_ROWS_EXPECTED - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

EXTRACTOR_ROWS_EXTRACTED - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

Configuration properties used by the row count policies.

F

FACTORY - Static variable in class org.apache.gobblin.dataset.NoopDatasetResolver

FAIL_TO_GET_OFFSET_COUNT - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

FAILURE_LOG_DIR_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

FAILURE_REPORTING_FILE_ENABLED_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

FieldDoesNotExistException - Exception in org.apache.gobblin.recordaccess

FieldDoesNotExistException(String) - Constructor for exception org.apache.gobblin.recordaccess.FieldDoesNotExistException

FieldDoesNotExistException(String, Throwable) - Constructor for exception org.apache.gobblin.recordaccess.FieldDoesNotExistException

FILEBASED_REPORT_STATUS_ON_COUNT - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

FileInfo() - Constructor for class org.apache.gobblin.writer.FsWriterMetrics.FileInfo

FileSystemDataset - Interface in org.apache.gobblin.dataset

Dataset in a file system, which can be characterized by a root Path.

filter(Predicate<WorkUnit>) - Method in class org.apache.gobblin.source.workunit.BasicWorkUnitStream

Apply a filtering function to this stream.

filter(Predicate<WorkUnit>) - Method in interface org.apache.gobblin.source.workunit.WorkUnitStream

Apply a filtering function to this stream.

FinalState - Interface in org.apache.gobblin.util

Interface for constructs that can report their final state.

findDatasets() - Method in interface org.apache.gobblin.dataset.DatasetsFinder

Find all Datasets in the file system.

findDatasets() - Method in class org.apache.gobblin.dataset.test.StaticDatasetsFinderForTesting

FLOW_ALLOW_CONCURRENT_EXECUTION - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

FLOW_APPLY_INPUT_RETENTION - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

FLOW_APPLY_RETENTION - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

FLOW_CONCURRENCY_ALLOWED - Static variable in class org.apache.gobblin.service.ServiceConfigKeys

FLOW_DESCRIPTION_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

FLOW_DESTINATION_IDENTIFIER_KEY - Static variable in class org.apache.gobblin.service.ServiceConfigKeys

FLOW_EXECUTION_ID_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

FLOW_EXPLAIN_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

FLOW_FAILURE_OPTION - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

FLOW_GROUP_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

FLOW_NAME_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

Common flow configuration properties.

FLOW_OWNING_GROUP_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

FLOW_RUN_IMMEDIATELY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

Configuration properties related to Flows

FLOW_SOURCE_IDENTIFIER_KEY - Static variable in class org.apache.gobblin.service.ServiceConfigKeys

FLOW_UNSCHEDULE_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

FlowEdge - Interface in org.apache.gobblin.runtime.api

A typical edge consists of two types of attributes: - Numerical value based: Return an numerical value for evaluation.

FluentDataWriterBuilder<S,D,B extends FluentDataWriterBuilder<S,D,B>> - Class in org.apache.gobblin.writer

A helper class to help create fluent DataWriterBuilders.

FluentDataWriterBuilder() - Constructor for class org.apache.gobblin.writer.FluentDataWriterBuilder

flush() - Method in interface org.apache.gobblin.writer.DataWriter

Flush data written by the writer.

flushable - Variable in class org.apache.gobblin.records.FlushControlMessageHandler

FlushControlMessage<D> - Class in org.apache.gobblin.stream

Control message for flushing writers

FlushControlMessage() - Constructor for class org.apache.gobblin.stream.FlushControlMessage

FlushControlMessage.FlushType - Enum in org.apache.gobblin.stream

FlushControlMessageHandler - Class in org.apache.gobblin.records

Flush control message handler that will flush a Flushable when handling a FlushControlMessage

FlushControlMessageHandler(Flushable) - Constructor for class org.apache.gobblin.records.FlushControlMessageHandler

Create a flush control message that will flush the given Flushable

FlushRecordEnvelope - Class in org.apache.gobblin.stream

FlushRecordEnvelope() - Constructor for class org.apache.gobblin.stream.FlushRecordEnvelope

forBranch(int) - Method in class org.apache.gobblin.writer.DataWriterBuilder

Tell the writer which branch it is associated with.

forBranch(int) - Method in class org.apache.gobblin.writer.FluentDataWriterBuilder

FORCE_LEADER - Static variable in class org.apache.gobblin.service.ServiceConfigKeys

FORK_BRANCH_ID_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

FORK_BRANCH_NAME_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

FORK_BRANCHES_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

Fork operator configuration properties.

FORK_CLOSE_WRITER_ON_COMPLETION - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

FORK_FINISHED_CHECK_INTERVAL - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

FORK_MAX_WAIT_MININUTES - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

FORK_OPERATOR_CLASS_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

FORK_RECORD_QUEUE_CAPACITY_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

FORK_RECORD_QUEUE_TIMEOUT_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

FORK_RECORD_QUEUE_TIMEOUT_UNIT_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

FORK_STATE_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

forkCloner() - Method in class org.apache.gobblin.stream.StreamEntity

forkDataRecord(WorkUnitState, D) - Method in interface org.apache.gobblin.fork.ForkOperator

Get a list of Booleans indicating if the record should go to each branch.

ForkedEntityBuilder() - Constructor for class org.apache.gobblin.stream.StreamEntity.ForkedEntityBuilder

ForkedStream() - Constructor for class org.apache.gobblin.fork.Forker.ForkedStream

Forker - Class in org.apache.gobblin.fork

Forks a RecordStreamWithMetadata into multiple branches specified by a ForkOperator.

Forker() - Constructor for class org.apache.gobblin.fork.Forker

Forker.ForkedStream<D,S> - Class in org.apache.gobblin.fork

An object containing the forked streams and a ConnectableFlowable used to connect the stream when all streams have been subscribed to.

ForkOperator<S,D> - Interface in org.apache.gobblin.fork

An interface for fork operators that convert one input data record into multiple records.

forkRecordBuilder() - Method in class org.apache.gobblin.stream.RecordEnvelope

Obtain a RecordEnvelope.ForkRecordBuilder to create derivative records to this record.

forkSchema(WorkUnitState, S) - Method in interface org.apache.gobblin.fork.ForkOperator

Get a list of Booleans indicating if the schema should go to each branch.

forkStream(RecordStreamWithMetadata<D, S>, ForkOperator<S, D>, WorkUnitState) - Method in class org.apache.gobblin.fork.Forker

Obtain the Forker.ForkedStream for the input RecordStreamWithMetadata and ForkOperator.

format - Variable in class org.apache.gobblin.writer.DataWriterBuilder

FRAMEWORK_JAR_FILES_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

fromDataMap(Map<String, String>) - Static method in class org.apache.gobblin.dataset.DatasetDescriptor

Deprecated.

use Descriptor#deserialize(String)

fromJson(String) - Static method in class org.apache.gobblin.dataset.Descriptor

Deserialize the json string to the a Descriptor object

fromJson(String, Class<T>) - Static method in class org.apache.gobblin.dataset.Descriptor

Deserialize the json string to the specified Descriptor object

fromJson(String) - Static method in class org.apache.gobblin.writer.FsWriterMetrics

Instantiate an object of this class from its JSON representation

fromJsonList(String) - Static method in class org.apache.gobblin.dataset.Descriptor

Deserialize the string, resulted from Descriptor.toJson(List), to a list of descriptors

fromPartitionJsonList(String) - Static method in class org.apache.gobblin.dataset.PartitionDescriptor

Deserialize the string, resulted from PartitionDescriptor.toPartitionJsonList(List), to a list of partition descriptors

FS_LOCATION - Static variable in class org.apache.gobblin.dataset.DatasetConstants

FS_PROXY_AS_USER_NAME - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

FS_PROXY_AS_USER_TOKEN_FILE - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

FS_PROXY_AUTH_METHOD - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

FS_SCHEME - Static variable in class org.apache.gobblin.dataset.DatasetConstants

FileSystem scheme and location

FS_URI - Static variable in class org.apache.gobblin.dataset.DatasetConstants

File system metadata

FS_URI_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

System configuration properties.

FsWriterMetrics - Class in org.apache.gobblin.writer

Metrics that can be stored in workUnitState by filesystem based writers.

FsWriterMetrics() - Constructor for class org.apache.gobblin.writer.FsWriterMetrics

FsWriterMetrics.FileInfo - Class in org.apache.gobblin.writer

G

generateDynamicConfig(Config) - Method in interface org.apache.gobblin.configuration.DynamicConfigGenerator

Generate dynamic configuration that should be added to the job configuration.

generateDynamicConfig(Config) - Method in class org.apache.gobblin.configuration.NoopDynamicConfigGenerator

get(String) - Method in interface org.apache.gobblin.commit.CommitSequenceStore

Get a Collection of dataset URNs with the given job name.

get(String, String) - Method in interface org.apache.gobblin.commit.CommitSequenceStore

Get the CommitSequence associated with the given job name and dataset URN.

get() - Method in class org.apache.gobblin.util.CompletedFuture

get(long, TimeUnit) - Method in class org.apache.gobblin.util.CompletedFuture

getActualHighWatermark() - Method in class org.apache.gobblin.configuration.WorkUnitState

Get the actual high Watermark as a JsonElement.

getActualHighWatermark(Class<T>, Gson) - Method in class org.apache.gobblin.configuration.WorkUnitState

Get the actual high Watermark.

getActualHighWatermark(Class<T>) - Method in class org.apache.gobblin.configuration.WorkUnitState

Get the actual high Watermark.

getAliasMap() - Method in class org.apache.gobblin.util.ClassAliasResolver

Get the map from found aliases to classes.

getAliasObjects() - Method in class org.apache.gobblin.util.ClassAliasResolver

getAllEncodedKeys() - Method in interface org.apache.gobblin.crypto.CredentialStore

List all binary encoded keys in the credential store

getAsInt(String) - Method in interface org.apache.gobblin.recordaccess.RecordAccessor

getAsLong(String) - Method in interface org.apache.gobblin.recordaccess.RecordAccessor

getAsString(String) - Method in interface org.apache.gobblin.recordaccess.RecordAccessor

getAttrs() - Method in interface org.apache.gobblin.runtime.api.SpecExecutor

SpecExecutor attributes include Location of SpecExecutor and the Type of it (Technology it used for data movement, like, gobblin-standalone/gobblin-cluster SpecExecutor attributes are supposed to be read-only once instantiated.

getAutoScope(SharedResourcesBroker<S>, ConfigView<S, K>) - Method in interface org.apache.gobblin.broker.iface.SharedResourceFactory

getBackoffFactor() - Method in class org.apache.gobblin.testing.AssertWithBackoff

The number to multiple the sleep after condition failure

getBranches(WorkUnitState) - Method in interface org.apache.gobblin.fork.ForkOperator

Get the number of branches after the fork.

getCapabilities() - Method in interface org.apache.gobblin.runtime.api.SpecExecutor

Source : Destination processing capabilities of SpecExecutor.

getChildCallback() - Method in class org.apache.gobblin.stream.StreamEntity.ForkedEntityBuilder

getClone() - Method in class org.apache.gobblin.stream.StreamEntity.ForkCloner

getCombinedWorkUnitAndDatasetState(String) - Method in interface org.apache.gobblin.configuration.CombinedWorkUnitAndDatasetStateFunctional

getCommittedWatermarks(Class<? extends CheckpointableWatermark>, Iterable<String>) - Method in interface org.apache.gobblin.writer.WatermarkStorage

Retrieve previously committed watermarks.

getConfig() - Method in interface org.apache.gobblin.broker.iface.ConfigView

getConfig() - Method in interface org.apache.gobblin.runtime.api.SpecExecutor

SpecExecutor config as a typesafe config object.

getDataDescriptor() - Method in interface org.apache.gobblin.writer.DataWriter

The method should return a Descriptor that represents what the writer is writing

getDataset() - Method in interface org.apache.gobblin.dataset.PartitionableDataset.DatasetPartition

getDatasetsIterator() - Method in interface org.apache.gobblin.dataset.IterableDatasetFinder

Deprecated.

use IterableDatasetFinder.getDatasetsStream(int, java.util.Comparator<T>) instead.

getDatasetsIterator() - Method in class org.apache.gobblin.dataset.IterableDatasetFinderImpl

getDatasetsIterator() - Method in class org.apache.gobblin.dataset.test.StaticDatasetsFinderForTesting

getDatasetsStream(int, Comparator<T>) - Method in interface org.apache.gobblin.dataset.IterableDatasetFinder

Get a stream of Datasets found.

getDatasetsStream(int, Comparator<Dataset>) - Method in class org.apache.gobblin.dataset.test.StaticDatasetsFinderForTesting

getDecoratedObject() - Method in interface org.apache.gobblin.util.Decorator

Get directly underlying object.

getDecoratorLineage(Object) - Static method in class org.apache.gobblin.util.DecoratorUtils

Finds the decorator lineage of the given object.

getDeltaFields() - Method in class org.apache.gobblin.source.workunit.Extract

Deprecated.

It is recommended to obtain delta fields from WorkUnit instead of Extract.

getDescription() - Method in interface org.apache.gobblin.runtime.api.Spec

getDescription() - Method in interface org.apache.gobblin.runtime.api.SpecExecutor

Human-readable description of the SpecExecutor .

getEdgeIdentity() - Method in interface org.apache.gobblin.runtime.api.FlowEdge

getEdgeProperties() - Method in interface org.apache.gobblin.runtime.api.FlowEdge

Return read-only Edge Properties .

getEncodedKey(String) - Method in interface org.apache.gobblin.crypto.CredentialStore

Get the binary encoded key with the given key

getErrFileLocation() - Method in class org.apache.gobblin.qualitychecker.row.RowLevelPolicy

getExecutionLink(Future<?>, String) - Method in interface org.apache.gobblin.runtime.api.SpecProducer

getExpectedHighWatermark() - Method in class org.apache.gobblin.source.extractor.WatermarkInterval

getExpectedHighWatermark() - Method in class org.apache.gobblin.source.workunit.WorkUnit

Get the expected high Watermark as a JsonElement.

getExpectedHighWatermark(Class<T>, Gson) - Method in class org.apache.gobblin.source.workunit.WorkUnit

Get the expected high Watermark.

getExpectedHighWatermark(Class<T>) - Method in class org.apache.gobblin.source.workunit.WorkUnit

Get the expected high Watermark.

getExpectedRecordCount() - Method in interface org.apache.gobblin.source.extractor.Extractor

Get the expected source record count.

getExtension() - Method in enum org.apache.gobblin.writer.WriterOutputFormat

Returns the file name extension for the enum type

getExtract() - Method in class org.apache.gobblin.configuration.WorkUnitState

Get the Extract associated with the WorkUnit.

getExtract() - Method in class org.apache.gobblin.source.workunit.WorkUnit

Get the Extract associated with this WorkUnit.

getExtractId() - Method in class org.apache.gobblin.source.workunit.Extract

Get a (non-globally) unique ID for this Extract.

getExtractor(WorkUnitState) - Method in interface org.apache.gobblin.source.Source

Get an Extractor based on a given WorkUnitState.

getFactoryName() - Method in interface org.apache.gobblin.broker.iface.ConfigView

getFinalConstructStates() - Method in class org.apache.gobblin.configuration.WorkUnitState

Builds a State containing all properties added with WorkUnitState.addFinalConstructState(java.lang.String, org.apache.gobblin.configuration.State) to this WorkUnitState.

getFinalState() - Method in class org.apache.gobblin.converter.Converter

Get final state for this object.

getFinalState() - Method in class org.apache.gobblin.qualitychecker.row.RowLevelPolicy

Get final state for this object.

getFinalState() - Method in interface org.apache.gobblin.util.FinalState

Called by tasks after all work for the construct has been executed.

getGeneric(String) - Method in interface org.apache.gobblin.recordaccess.RecordAccessor

getGson(Class<T>) - Static method in class org.apache.gobblin.util.io.GsonInterfaceAdapter

getHealth() - Method in interface org.apache.gobblin.runtime.api.SpecExecutor

Health of SpecExecutor.

getHighWaterMark() - Method in class org.apache.gobblin.configuration.WorkUnitState

Deprecated.

use WorkUnitState.getActualHighWatermark().

getHighWatermark() - Method in interface org.apache.gobblin.source.extractor.Extractor

Deprecated.

there is no longer support for reporting the high watermark via this method, please see Watermarks for more information.

getHighWaterMark() - Method in class org.apache.gobblin.source.workunit.WorkUnit

Deprecated.

use the WorkUnit.getExpectedHighWatermark() method.

getId() - Method in class org.apache.gobblin.configuration.State

Get the id of this instance.

getId() - Method in enum org.apache.gobblin.runtime.api.SpecExecutor.Verb

getInitializer(State, WorkUnitStream, int, int) - Method in class org.apache.gobblin.converter.Converter

getInitializer(State, WorkUnitStream, int, int) - Method in class org.apache.gobblin.writer.DataWriterBuilder

getInstance() - Static method in class org.apache.gobblin.password.PasswordManager

Get an instance with no master password, which cannot encrypt or decrypt passwords.

getInstance(State) - Static method in class org.apache.gobblin.password.PasswordManager

Get an instance.

getInstance(Properties) - Static method in class org.apache.gobblin.password.PasswordManager

Get an instance.

getInstance(Path) - Static method in class org.apache.gobblin.password.PasswordManager

Get an instance.

getInstance(Class<? extends DataPublisher>, State) - Static method in class org.apache.gobblin.publisher.DataPublisher

Get an instance of DataPublisher.

getInstance(Class<? extends DataPublisher>, State) - Static method in class org.apache.gobblin.publisher.SingleTaskDataPublisher

Get an instance of SingleTaskDataPublisher.

getIsFull() - Method in class org.apache.gobblin.source.workunit.Extract

Deprecated.

It is recommend to get this information from WorkUnit instead of Extract.

getKey() - Method in interface org.apache.gobblin.broker.iface.ConfigView

getLogger() - Method in class org.apache.gobblin.testing.AssertWithBackoff

The logger to use for logging waiting, results, etc.

getLowWatermark() - Method in class org.apache.gobblin.source.extractor.WatermarkInterval

getLowWatermark() - Method in class org.apache.gobblin.source.workunit.WorkUnit

Get the low Watermark as a JsonElement.

getLowWatermark(Class<T>, Gson) - Method in class org.apache.gobblin.source.workunit.WorkUnit

Get the low Watermark.

getLowWatermark(Class<T>) - Method in class org.apache.gobblin.source.workunit.WorkUnit

Get the low Watermark.

getLowWaterMark() - Method in class org.apache.gobblin.source.workunit.WorkUnit

Deprecated.

use the WorkUnit.getLowWatermark() method.

getMasterPassword(Path) - Static method in class org.apache.gobblin.password.PasswordManager

getMasterPassword(FileSystem, Path) - Static method in class org.apache.gobblin.password.PasswordManager

getMaterializedWorkUnitCollection() - Method in class org.apache.gobblin.source.workunit.BasicWorkUnitStream

Get a materialized collection of the WorkUnits in this stream.

getMaterializedWorkUnitCollection() - Method in interface org.apache.gobblin.source.workunit.WorkUnitStream

Get a materialized collection of the WorkUnits in this stream.

getMaxSleepMs() - Method in class org.apache.gobblin.testing.AssertWithBackoff

The max time to sleep between condition failures

getMergedMetadata() - Method in interface org.apache.gobblin.metadata.MetadataMerger

Get a metadata record that is a representation of all records passed into update().

getMessageHandler() - Method in class org.apache.gobblin.converter.Converter

getMessageHandler() - Method in class org.apache.gobblin.stream.ControlMessageInjector

getMessageHandler() - Method in interface org.apache.gobblin.writer.DataWriter

Default handler calls flush on this object when a FlushControlMessage is received

getMetadata() - Method in class org.apache.gobblin.dataset.DatasetDescriptor

getMetadata(State, String) - Method in interface org.apache.gobblin.runtime.api.TaskEventMetadataGenerator

Generate a map of additional metadata for the specified event name.

getMetadata(State, String) - Method in class org.apache.gobblin.runtime.NoopTaskEventMetadataGenerator

Generate a map of additional metadata for the specified event name.

getMultiAsInt(String) - Method in interface org.apache.gobblin.recordaccess.RecordAccessor

getMultiAsLong(String) - Method in interface org.apache.gobblin.recordaccess.RecordAccessor

getMultiAsString(String) - Method in interface org.apache.gobblin.recordaccess.RecordAccessor

getMultiGeneric(String) - Method in interface org.apache.gobblin.recordaccess.RecordAccessor

getName() - Method in interface org.apache.gobblin.broker.iface.SharedResourceFactory

getNamespace() - Method in class org.apache.gobblin.source.workunit.Extract

Get the dot-separated namespace of the table.

getNodeName() - Method in interface org.apache.gobblin.runtime.api.ServiceNode

getNodeProps() - Method in interface org.apache.gobblin.runtime.api.ServiceNode

getNotImplementedException(RecordCountProvider) - Method in class org.apache.gobblin.util.RecordCountProvider

getOutputFilePath() - Method in class org.apache.gobblin.configuration.WorkUnitState

getOutputFilePath() - Method in class org.apache.gobblin.source.workunit.Extract

Deprecated.

As this.getIsFull is deprecated.

getOutputFilePath() - Method in class org.apache.gobblin.source.workunit.WorkUnit

getPartitions(int, Comparator<T>) - Method in interface org.apache.gobblin.dataset.PartitionableDataset

Get a stream of partitions.

getPartitions(int, Comparator<SimpleDatasetPartitionForTesting>) - Method in class org.apache.gobblin.dataset.test.SimplePartitionableDatasetForTesting

getPasswordAuthentication() - Method in class org.apache.gobblin.password.EncryptedPasswordAuthenticator

getPreviousDatasetState(String) - Method in class org.apache.gobblin.configuration.SourceState

Get the state (in the form of a SourceState) of a dataset identified by a dataset URN of the previous job run.

getPreviousDatasetStatesByUrns() - Method in class org.apache.gobblin.configuration.SourceState

getPreviousSourceState() - Method in class org.apache.gobblin.configuration.SourceState

Get the SourceState of the previous job run.

getPreviousTableState() - Method in class org.apache.gobblin.configuration.WorkUnitState

Get properties set in the previous run for the same table as the WorkUnit.

getPreviousTableState() - Method in class org.apache.gobblin.qualitychecker.task.TaskLevelPolicy

getPreviousTableState() - Method in class org.apache.gobblin.source.workunit.Extract

Get the previous table State.

getPreviousWorkUnitStates() - Method in class org.apache.gobblin.configuration.SourceState

Get a List of previous WorkUnitStates.

getPreviousWorkUnitStates(String) - Method in class org.apache.gobblin.configuration.SourceState

Get a List of previous WorkUnitStates for a given datasetUrn.

getPreviousWorkUnitStatesByDatasetUrns() - Method in class org.apache.gobblin.configuration.SourceState

Get a Map from dataset URNs (as being specified by ConfigurationKeys.DATASET_URN_KEY to the WorkUnitState with the dataset URNs.

getPrimaryKeys() - Method in class org.apache.gobblin.source.workunit.Extract

Deprecated.

It is recommended to obtain primary keys from WorkUnit instead of Extract.

getProducer() - Method in interface org.apache.gobblin.runtime.api.SpecExecutor

A communication socket for generating spec to assigned physical executors, paired with a consumer on the physical executor side.

getProp(String) - Method in class org.apache.gobblin.configuration.State

Get the value of a property.

getProp(String, String) - Method in class org.apache.gobblin.configuration.State

Get the value of a property, using the given default value if the property is not set.

getProp(String) - Method in class org.apache.gobblin.configuration.WorkUnitState

getProp(String, String) - Method in class org.apache.gobblin.configuration.WorkUnitState

getProp(String) - Method in class org.apache.gobblin.source.workunit.WorkUnit

getProp(String, String) - Method in class org.apache.gobblin.source.workunit.WorkUnit

getPropAsBoolean(String) - Method in class org.apache.gobblin.configuration.State

Get the value of a property as a boolean.

getPropAsBoolean(String, boolean) - Method in class org.apache.gobblin.configuration.State

Get the value of a property as a boolean, using the given default value if the property is not set.

getPropAsCaseInsensitiveSet(String) - Method in class org.apache.gobblin.configuration.State

Get the value of a property as a case insensitive Set of strings.

getPropAsCaseInsensitiveSet(String, String) - Method in class org.apache.gobblin.configuration.State

Get the value of a property as a case insensitive Set of strings, using the given default value if the property is not set.

getPropAsDouble(String) - Method in class org.apache.gobblin.configuration.State

Get the value of a property as a double.

getPropAsDouble(String, double) - Method in class org.apache.gobblin.configuration.State

Get the value of a property as a double, using the given default value if the property is not set.

getPropAsInt(String) - Method in class org.apache.gobblin.configuration.State

Get the value of a property as an integer.

getPropAsInt(String, int) - Method in class org.apache.gobblin.configuration.State

Get the value of a property as an integer, using the given default value if the property is not set.

getPropAsJsonArray(String) - Method in class org.apache.gobblin.configuration.State

Get the value of a property as a JsonArray.

getPropAsList(String) - Method in class org.apache.gobblin.configuration.State

Get the value of a comma separated property as a List of strings.

getPropAsList(String, String) - Method in class org.apache.gobblin.configuration.State

Get the value of a property as a list of strings, using the given default value if the property is not set.

getPropAsLong(String) - Method in class org.apache.gobblin.configuration.State

Get the value of a property as a long integer.

getPropAsLong(String, long) - Method in class org.apache.gobblin.configuration.State

Get the value of a property as a long integer, using the given default value if the property is not set.

getPropAsSet(String) - Method in class org.apache.gobblin.configuration.State

Get the value of a comma separated property as a Set of strings.

getPropAsSet(String, String) - Method in class org.apache.gobblin.configuration.State

Get the value of a comma separated property as a Set of strings.

getPropAsShort(String) - Method in class org.apache.gobblin.configuration.State

Get the value of a property as a short.

getPropAsShort(String, short) - Method in class org.apache.gobblin.configuration.State

Get the value of a property as an short, using the given default value if the property is not set.

getPropAsShortWithRadix(String, int) - Method in class org.apache.gobblin.configuration.State

Get the value of a property as a short.

getPropAsShortWithRadix(String, short, int) - Method in class org.apache.gobblin.configuration.State

Get the value of a property as an short, using the given default value if the property is not set.

getProperties() - Method in class org.apache.gobblin.configuration.State

Return a copy of the underlying Properties object.

getProperties() - Method in class org.apache.gobblin.configuration.WorkUnitState

getProperties() - Method in class org.apache.gobblin.writer.Destination

Get configuration properties for the destination type.

getProperty(String) - Method in class org.apache.gobblin.configuration.State

Deprecated.

Use State.getProp(String)

getProperty(String, String) - Method in class org.apache.gobblin.configuration.State

Deprecated.

Use State.getProp(String, String)

getProperty(String) - Method in class org.apache.gobblin.configuration.WorkUnitState

Deprecated.

Use WorkUnitState.getProp(String)

getProperty(String, String) - Method in class org.apache.gobblin.configuration.WorkUnitState

Deprecated.

Use WorkUnitState.getProp(String, String)

getPropertyNames() - Method in class org.apache.gobblin.configuration.State

Get the names of all the properties set in a Set.

getPropertyNames() - Method in class org.apache.gobblin.configuration.WorkUnitState

getRecord() - Method in class org.apache.gobblin.stream.RecordEnvelope

getRecordCount(Path) - Method in class org.apache.gobblin.util.RecordCountProvider

Get record count from a given Path.

getRecordCount(Collection<Path>) - Method in class org.apache.gobblin.util.RecordCountProvider

Get record count for a list of paths.

getRecordMetadata(String) - Method in class org.apache.gobblin.stream.RecordEnvelope

getResource() - Method in interface org.apache.gobblin.broker.ResourceEntry

getResourceIfValid() - Method in interface org.apache.gobblin.broker.ResourceEntry

This method should guarantee that if all callers accessing the resource using this method then the object is returned atomically with respect to any validity state change.

getSchema() - Method in interface org.apache.gobblin.source.extractor.Extractor

Get the schema (metadata) of the extracted data records.

getScope() - Method in interface org.apache.gobblin.broker.iface.ScopedConfigView

getScope(S) - Method in interface org.apache.gobblin.broker.iface.SharedResourcesBroker

Get the ScopeInstance in this brokers topology at the provided ScopeType.

getScopedView(S) - Method in interface org.apache.gobblin.broker.iface.ConfigView

getScopeId() - Method in interface org.apache.gobblin.broker.iface.ScopeInstance

getSharedResource(SharedResourceFactory<T, K, S>, K) - Method in interface org.apache.gobblin.broker.iface.SharedResourcesBroker

Get a shared resource created by the input SharedResourceFactory.

getSharedResourceAtScope(SharedResourceFactory<T, K, S>, K, S) - Method in interface org.apache.gobblin.broker.iface.SharedResourcesBroker

Get a shared resource created by the input SharedResourceFactory at the ScopeInstance returned by #getScope) on the input ScopeType.

getSingleClone() - Method in class org.apache.gobblin.stream.StreamEntity

getSource() - Method in class org.apache.gobblin.runtime.CheckpointableWatermarkState

getSource() - Method in interface org.apache.gobblin.source.extractor.CheckpointableWatermark

getState() - Method in class org.apache.gobblin.publisher.DataPublisher

getTable() - Method in class org.apache.gobblin.source.workunit.Extract

Get the name of the table.

getTag() - Method in interface org.apache.gobblin.codec.StreamCodec

Get tag/file extension associated with encoder.

getTaskBroker() - Method in class org.apache.gobblin.configuration.WorkUnitState

Get a SharedResourcesBroker scoped for this task.

getTaskBrokerNullable() - Method in class org.apache.gobblin.configuration.WorkUnitState

Get a SharedResourcesBroker scoped for this task or null if it doesn't exist.

getTaskEventMetadataGenerator(State) - Static method in class org.apache.gobblin.util.TaskEventMetadataUtils

Construct a TaskEventMetadataGenerator from the task state.

getTaskState() - Method in class org.apache.gobblin.qualitychecker.row.RowLevelPolicy

getTaskState() - Method in class org.apache.gobblin.qualitychecker.task.TaskLevelPolicy

getTimeoutMs() - Method in class org.apache.gobblin.testing.AssertWithBackoff

the max time in milliseconds to wait for the condition to become true

getType() - Method in interface org.apache.gobblin.broker.iface.ScopeInstance

getType() - Method in class org.apache.gobblin.qualitychecker.row.RowLevelPolicy

getType() - Method in class org.apache.gobblin.qualitychecker.task.TaskLevelPolicy

getType() - Method in class org.apache.gobblin.source.workunit.Extract

Get the Extract.TableType of the table.

getType() - Method in class org.apache.gobblin.writer.Destination

Get the destination type.

getUniqueExtract(Extract.TableType, String, String) - Method in class org.apache.gobblin.source.workunit.ExtractFactory

Returns a unique Extract instance.

getUri() - Method in interface org.apache.gobblin.runtime.api.Spec

getUri() - Method in interface org.apache.gobblin.runtime.api.SpecExecutor

An URI identifying the SpecExecutor.

getUrn() - Method in interface org.apache.gobblin.dataset.Dataset

getUrn() - Method in interface org.apache.gobblin.dataset.PartitionableDataset.DatasetPartition

URN for this dataset.

getUrn() - Method in interface org.apache.gobblin.dataset.URNIdentified

URN for this object.

getVerb() - Method in enum org.apache.gobblin.runtime.api.SpecExecutor.Verb

getVersion() - Method in interface org.apache.gobblin.runtime.api.Spec

getWatermark() - Method in interface org.apache.gobblin.source.extractor.CheckpointableWatermark

getWatermark() - Method in class org.apache.gobblin.stream.RecordEnvelope

getWorkingState() - Method in class org.apache.gobblin.configuration.WorkUnitState

Get the current runtime state of the WorkUnit.

getWorkunit() - Method in class org.apache.gobblin.configuration.WorkUnitState

Get an ImmutableWorkUnit that wraps the internal WorkUnit.

getWorkunits(SourceState) - Method in interface org.apache.gobblin.source.Source

Get a list of WorkUnits, each of which is for extracting a portion of the data.

getWorkUnits() - Method in class org.apache.gobblin.source.workunit.BasicWorkUnitStream

getWorkUnits() - Method in class org.apache.gobblin.source.workunit.MultiWorkUnit

Get an immutable list of WorkUnits wrapped by this MultiWorkUnit.

getWorkUnits() - Method in interface org.apache.gobblin.source.workunit.WorkUnitStream

getWorkunitStream(SourceState) - Method in interface org.apache.gobblin.source.WorkUnitStreamSource

Get the WorkUnitStream to process.

GIT_MONITOR_BRANCH_NAME - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

GIT_MONITOR_CONFIG_BASE_DIR - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

GIT_MONITOR_JSCH_LOGGER_ENABLED - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

GIT_MONITOR_PASSWORD - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

GIT_MONITOR_POLLING_INTERVAL - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

GIT_MONITOR_REPO_DIR - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

GIT_MONITOR_REPO_URI - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

Configuration related to the Git based monitoring service

GIT_MONITOR_SSH_KNOWN_HOSTS - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

GIT_MONITOR_SSH_KNOWN_HOSTS_FILE - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

GIT_MONITOR_SSH_PASSPHRASE - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

GIT_MONITOR_SSH_PRIVATE_KEY_BASE64_ENCODED - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

GIT_MONITOR_SSH_PRIVATE_KEY_PATH - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

GIT_MONITOR_SSH_STRICT_HOST_KEY_CHECKING_ENABLED - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

GIT_MONITOR_SSH_WITH_PUBLIC_KEY_ENABLED - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

GIT_MONITOR_USERNAME - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

GLOBAL_WATERMARK_DATASET_URN - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

GlobalMetadata<S> - Class in org.apache.gobblin.metadata

Global metadata

GlobalMetadata() - Constructor for class org.apache.gobblin.metadata.GlobalMetadata

gobblin.configuration - package gobblin.configuration

gobblin.source.extractor - package gobblin.source.extractor

gobblin.source.workunit - package gobblin.source.workunit

GOBBLIN_BROKER_CONFIG_NAMESPACES - Static variable in class org.apache.gobblin.broker.BrokerConstants

GOBBLIN_BROKER_CONFIG_PREFIX - Static variable in class org.apache.gobblin.broker.BrokerConstants

GOBBLIN_FLOW_SLA_TIME - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

GOBBLIN_FLOW_SLA_TIME_UNIT - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

GOBBLIN_JOB_EVENT_REPORTING_FAILURE_FATAL - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

GOBBLIN_JOB_METRIC_REPORTING_FAILURE_FATAL - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

GOBBLIN_JOB_START_SLA_TIME - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

GOBBLIN_JOB_START_SLA_TIME_UNIT - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

GOBBLIN_METADATA_CHANGE_EVENT_ENABLED - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

Configuration and constant vale for GobblinMetadataChangeEvent

GOBBLIN_NON_TRANSIENT_ERRORS - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

GOBBLIN_RUNTIME_DELIVERY_SEMANTICS - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

Other configuration properties.

GOBBLIN_SERVICE_APP_LAUNCHER_PREFIX - Static variable in class org.apache.gobblin.service.ServiceConfigKeys

GOBBLIN_SERVICE_D2_ONLY_ANNOUNCE_LEADER - Static variable in class org.apache.gobblin.service.ServiceConfigKeys

GOBBLIN_SERVICE_DAG_MANAGER_ENABLED_KEY - Static variable in class org.apache.gobblin.service.ServiceConfigKeys

GOBBLIN_SERVICE_FLOW_CATALOG_ENABLED_KEY - Static variable in class org.apache.gobblin.service.ServiceConfigKeys

GOBBLIN_SERVICE_FLOW_CATALOG_LOCAL_COMMIT - Static variable in class org.apache.gobblin.service.ServiceConfigKeys

GOBBLIN_SERVICE_FLOWCOMPILER_CLASS_KEY - Static variable in class org.apache.gobblin.service.ServiceConfigKeys

GOBBLIN_SERVICE_FLOWSPEC - Static variable in class org.apache.gobblin.service.ServiceConfigKeys

GOBBLIN_SERVICE_GIT_CONFIG_MONITOR_ENABLED_KEY - Static variable in class org.apache.gobblin.service.ServiceConfigKeys

GOBBLIN_SERVICE_JOB_SCHEDULER_LISTENER_CLASS - Static variable in class org.apache.gobblin.service.ServiceConfigKeys

GOBBLIN_SERVICE_JOB_STATUS_MONITOR_ENABLED_KEY - Static variable in class org.apache.gobblin.service.ServiceConfigKeys

GOBBLIN_SERVICE_LOG4J_CONFIGURATION_FILE - Static variable in class org.apache.gobblin.service.ServiceConfigKeys

GOBBLIN_SERVICE_ORCHESTRATOR_ENABLED_KEY - Static variable in class org.apache.gobblin.service.ServiceConfigKeys

GOBBLIN_SERVICE_PREFIX - Static variable in class org.apache.gobblin.service.ServiceConfigKeys

GOBBLIN_SERVICE_RESTLI_SERVER_ENABLED_KEY - Static variable in class org.apache.gobblin.service.ServiceConfigKeys

GOBBLIN_SERVICE_SCHEDULER_ENABLED_KEY - Static variable in class org.apache.gobblin.service.ServiceConfigKeys

GOBBLIN_SERVICE_TOPOLOGY_CATALOG_ENABLED_KEY - Static variable in class org.apache.gobblin.service.ServiceConfigKeys

GOBBLIN_SERVICE_TOPOLOGY_SPEC_FACTORY_ENABLED_KEY - Static variable in class org.apache.gobblin.service.ServiceConfigKeys

GOBBLIN_TASK_EVENT_REPORTING_FAILURE_FATAL - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

GOBBLIN_TASK_METRIC_REPORTING_FAILURE_FATAL - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

GobblinScopeInstance - Class in org.apache.gobblin.broker.gobblin_scopes

ScopeInstance superclass for scopes used in Gobblin ingestion jobs.

GobblinScopeInstance(GobblinScopeTypes, String) - Constructor for class org.apache.gobblin.broker.gobblin_scopes.GobblinScopeInstance

GobblinScopeTypes - Enum in org.apache.gobblin.broker.gobblin_scopes

Scope topology for Gobblin ingestion applications.

GROUP_OWNERSHIP_SERVICE_CLASS - Static variable in class org.apache.gobblin.service.ServiceConfigKeys

GSON - Static variable in class org.apache.gobblin.dataset.Descriptor

Use gson for ser/de

GsonInterfaceAdapter - Class in org.apache.gobblin.util.io

A Gson interface adapter that makes it possible to serialize and deserialize polymorphic objects.

GsonInterfaceAdapter() - Constructor for class org.apache.gobblin.util.io.GsonInterfaceAdapter

H

handleMessage(ControlMessage) - Method in interface org.apache.gobblin.records.ControlMessageHandler

handle the input ControlMessage.

handleMessage(ControlMessage) - Method in class org.apache.gobblin.records.ControlMessageHandler.Noop

handleMessage(ControlMessage) - Method in class org.apache.gobblin.records.FlushControlMessageHandler

handleUnpublishedWorkUnits(Collection<? extends WorkUnitState>) - Method in interface org.apache.gobblin.publisher.UnpublishedHandling

This method will be called by Gobblin if it determines that a job should not be published.

hashCode() - Method in class org.apache.gobblin.broker.gobblin_scopes.JobScopeInstance

hashCode() - Method in class org.apache.gobblin.broker.gobblin_scopes.TaskScopeInstance

hashCode() - Method in class org.apache.gobblin.broker.StringNameSharedResourceKey

hashCode() - Method in class org.apache.gobblin.configuration.SourceState

hashCode() - Method in class org.apache.gobblin.configuration.WorkUnitState

hashCode() - Method in class org.apache.gobblin.dataset.DatasetDescriptor

hashCode() - Method in class org.apache.gobblin.dataset.PartitionDescriptor

hashCode() - Method in class org.apache.gobblin.source.workunit.Extract

hashCode() - Method in class org.apache.gobblin.source.workunit.MultiWorkUnit

hashCode() - Method in class org.apache.gobblin.source.workunit.WorkUnit

hasType() - Method in class org.apache.gobblin.source.workunit.Extract

If this Extract has extract table type defined.

HELIX_CLUSTER_NAME_KEY - Static variable in class org.apache.gobblin.service.ServiceConfigKeys

HELIX_FLOWSPEC_ADD - Static variable in class org.apache.gobblin.service.ServiceConfigKeys

HELIX_FLOWSPEC_REMOVE - Static variable in class org.apache.gobblin.service.ServiceConfigKeys

HELIX_FLOWSPEC_UPDATE - Static variable in class org.apache.gobblin.service.ServiceConfigKeys

HELIX_INSTANCE_NAME_KEY - Static variable in class org.apache.gobblin.service.ServiceConfigKeys

HELIX_INSTANCE_NAME_OPTION_NAME - Static variable in class org.apache.gobblin.service.ServiceConfigKeys

HierarchicalAckable - Class in org.apache.gobblin.ack

An Ackable used to ack other Ackables when a set of children ackables are all acknowledged.

HierarchicalAckable() - Constructor for class org.apache.gobblin.ack.HierarchicalAckable

HierarchicalAckable.ChildrenFailedException - Exception in org.apache.gobblin.ack

Indicates that at least one of the children Ackables was nacked.

HIVE_REG_PUBLISHER_CLASS - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

HIVE_REGISTRATION_POLICY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

Hive registration properties

HIVE_TABLE - Static variable in class org.apache.gobblin.dataset.HiveToHdfsDatasetResolver

HiveToHdfsDatasetResolver - Class in org.apache.gobblin.dataset

Singleton DatasetResolver to convert a Hive DatasetDescriptor to HDFS DatasetDescriptor

HiveToHdfsDatasetResolverFactory - Class in org.apache.gobblin.dataset

HiveToHdfsDatasetResolverFactory() - Constructor for class org.apache.gobblin.dataset.HiveToHdfsDatasetResolverFactory

I

ImmutableExtract - Class in gobblin.source.workunit

Shim layer for org.apache.gobblin.source.workunit.ImmutableExtract

ImmutableExtract(SourceState, Extract.TableType, String, String) - Constructor for class gobblin.source.workunit.ImmutableExtract

ImmutableExtract(Extract) - Constructor for class gobblin.source.workunit.ImmutableExtract

ImmutableExtract - Class in org.apache.gobblin.source.workunit

An immutable version of Extract.

ImmutableExtract(SourceState, Extract.TableType, String, String) - Constructor for class org.apache.gobblin.source.workunit.ImmutableExtract

ImmutableExtract(Extract) - Constructor for class org.apache.gobblin.source.workunit.ImmutableExtract

ImmutableWorkUnit - Class in gobblin.source.workunit

Shim layer for org.apache.gobblin.source.workunit.ImmutableWorkUnit

ImmutableWorkUnit(WorkUnit) - Constructor for class gobblin.source.workunit.ImmutableWorkUnit

ImmutableWorkUnit - Class in org.apache.gobblin.source.workunit

An immutable version of WorkUnit.

ImmutableWorkUnit(WorkUnit) - Constructor for class org.apache.gobblin.source.workunit.ImmutableWorkUnit

ImmutableWorkUnitState - Class in gobblin.configuration

Shim layer for org.apache.gobblin.configuration.ImmutableWorkUnitState

ImmutableWorkUnitState(WorkUnitState) - Constructor for class gobblin.configuration.ImmutableWorkUnitState

ImmutableWorkUnitState - Class in org.apache.gobblin.configuration

An immutable version of WorkUnitState.

ImmutableWorkUnitState(WorkUnitState) - Constructor for class org.apache.gobblin.configuration.ImmutableWorkUnitState

IncorrectTypeException - Exception in org.apache.gobblin.recordaccess

Thrown when a get() or set() call on an Accessor accesses a record with an incorrect underlying type.

IncorrectTypeException(String) - Constructor for exception org.apache.gobblin.recordaccess.IncorrectTypeException

IncorrectTypeException(String, Throwable) - Constructor for exception org.apache.gobblin.recordaccess.IncorrectTypeException

init(WorkUnitState) - Method in class org.apache.gobblin.converter.Converter

Initialize this Converter.

init(WorkUnitState) - Method in interface org.apache.gobblin.fork.ForkOperator

Initialize this ForkOperator.

init(WorkUnitState) - Method in class org.apache.gobblin.stream.ControlMessageInjector

Initialize this ControlMessageInjector.

initialize() - Method in class org.apache.gobblin.converter.initializer.NoopConverterInitializer

initialize() - Method in interface org.apache.gobblin.initializer.Initializer

Initialize for the writer.

initialize() - Method in class org.apache.gobblin.initializer.NoopInitializer

initialize() - Method in class org.apache.gobblin.publisher.DataPublisher

Deprecated.

DataPublisher initialization should be done in the constructor.

initialize() - Method in class org.apache.gobblin.writer.initializer.NoopWriterInitializer

Initializer - Interface in org.apache.gobblin.initializer

injectControlMessagesAfter(RecordEnvelope<DI>, WorkUnitState) - Method in class org.apache.gobblin.stream.ControlMessageInjector

Inject ControlMessages after the record

injectControlMessagesBefore(RecordEnvelope<DI>, WorkUnitState) - Method in class org.apache.gobblin.stream.ControlMessageInjector

Inject ControlMessages before the record

INSTANCE - Static variable in class org.apache.gobblin.converter.initializer.NoopConverterInitializer

INSTANCE - Static variable in class org.apache.gobblin.dataset.HiveToHdfsDatasetResolver

INSTANCE - Static variable in class org.apache.gobblin.dataset.NoopDatasetResolver

INSTANCE - Static variable in class org.apache.gobblin.initializer.NoopInitializer

INSTANCE - Static variable in class org.apache.gobblin.writer.initializer.NoopWriterInitializer

INTERMEDIATE_STATE_STORE_PREFIX - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

INTERMEDIATE_STATE_STORE_TYPE_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

IntRange - Annotation Type in org.apache.gobblin.typedconfig.compiletime

isCancelled() - Method in class org.apache.gobblin.util.CompletedFuture

isCompleted() - Method in interface org.apache.gobblin.commit.CommitStep

Determine whether the commit step has been completed.

isCopyable(Object) - Static method in class org.apache.gobblin.fork.CopyHelper

Check if an object is copyable using the CopyHelper.copy(Object) method.

isDone() - Method in class org.apache.gobblin.util.CompletedFuture

isEarlyStopped() - Method in interface org.apache.gobblin.source.Source

Instead of handling all WorkUnits in one run, some Source may choose to stop early in order to handle the proper workload, which can cause multiple runs after the initial run.

isEdgeEnabled() - Method in interface org.apache.gobblin.runtime.api.FlowEdge

isFiniteStream() - Method in interface org.apache.gobblin.source.workunit.WorkUnitStream

isLocal() - Method in enum org.apache.gobblin.broker.gobblin_scopes.GobblinScopeTypes

isLocal() - Method in interface org.apache.gobblin.broker.iface.ScopeType

isLocal() - Method in enum org.apache.gobblin.broker.SimpleScopeType

isNodeEnabled() - Method in interface org.apache.gobblin.runtime.api.ServiceNode

isSafeToMaterialize() - Method in interface org.apache.gobblin.source.workunit.WorkUnitStream

isThreadSafe() - Method in class org.apache.gobblin.publisher.DataPublisher

Returns true if the implementation of DataPublisher is thread-safe.

isValid() - Method in interface org.apache.gobblin.broker.ResourceEntry

isVirtual() - Method in interface org.apache.gobblin.dataset.FileSystemDataset

IterableDatasetFinder<T extends Dataset> - Interface in org.apache.gobblin.dataset

A DatasetsFinder that can return the Datasets as an Iterator.

IterableDatasetFinderImpl<T extends Dataset> - Class in org.apache.gobblin.dataset

Wraps a DatasetsFinder into an IterableDatasetFinder.

IterableDatasetFinderImpl() - Constructor for class org.apache.gobblin.dataset.IterableDatasetFinderImpl

J

JOB_COMMIT_POLICY_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

JOB_CONFIG_FILE_DIR_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

JOB_CONFIG_FILE_EXTENSIONS_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

JOB_CONFIG_FILE_GENERAL_PATH_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

JOB_CONFIG_FILE_MONITOR_POLLING_INTERVAL_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

JOB_CONFIG_FILE_PATH_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

JOB_DATA_PUBLISHER_TYPE - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

JOB_DEPENDENCIES - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

JOB_DESCRIPTION_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

JOB_DISABLED_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

JOB_EXECINFO_SERVER_ENABLED_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

Job execution info server and history store configuration properties.

JOB_EXECUTOR_THREAD_POOL_SIZE_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

JOB_FAILURE_EXCEPTION_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

JOB_FAILURES_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

JOB_FORK_ON_CONCAT - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

JOB_GROUP_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

JOB_HDFS_FILES_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

JOB_HISTORY_STORE_ENABLED_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

JOB_HISTORY_STORE_JDBC_DRIVER_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

JOB_HISTORY_STORE_PASSWORD_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

JOB_HISTORY_STORE_URL_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

JOB_HISTORY_STORE_USER_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

JOB_ID_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

Configuration properties used internally.

JOB_JAR_FILES_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

JOB_JAR_HDFS_FILES_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

JOB_KEY_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

JOB_LAUNCHER_TYPE_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

JOB_LISTENERS_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

JOB_LOCAL_FILES_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

JOB_LOCK_ENABLED_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

JOB_LOCK_TYPE - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

JOB_MAX_FAILURES_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

JOB_NAME_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

Common job configuration properties.

JOB_PROCEED_ON_TASK_STATE_COLLECOTR_SERVICE_FAILURE - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

Set to true so that job still proceed if TaskStateCollectorService failed.

JOB_RETRIGGERING_ENABLED - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

Job scheduler configuration properties.

JOB_RUN_ONCE_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

JOB_SCHEDULE_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

JOB_STATE_DISTRIBUTED_CACHE_NAME - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

JOB_STATE_FILE_PATH_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

JOB_STATE_IN_STATE_STORE - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

JOB_TAG_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

JOB_TEMPLATE_PATH - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

Optional, for user to specified which template to use, inside .job file

JOB_TRACKING_URL_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

JobScopeInstance - Class in org.apache.gobblin.broker.gobblin_scopes

A ScopeInstance for a Gobblin job.

JobScopeInstance(String, String) - Constructor for class org.apache.gobblin.broker.gobblin_scopes.JobScopeInstance

JobShutdownException - Exception in org.apache.gobblin.runtime

An exception thrown when a job cannot be graciously shutdown.

JobShutdownException(String) - Constructor for exception org.apache.gobblin.runtime.JobShutdownException

jsonObjectToState(JsonObject, String...) - Static method in class org.apache.gobblin.configuration.StateUtils

Converts a JsonObject to a State object.

K

KAFKA_BROKERS - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

Kafka job configurations.

KAFKA_SCHEMA_REGISTRY_HTTPCLIENT_CONN_TIMEOUT - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

KAFKA_SCHEMA_REGISTRY_HTTPCLIENT_METHOD_RETRY_COUNT - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

KAFKA_SCHEMA_REGISTRY_HTTPCLIENT_METHOD_RETRY_HANDLER_CLASS - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

KAFKA_SCHEMA_REGISTRY_HTTPCLIENT_REQUEST_RETRY_ENABLED - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

KAFKA_SCHEMA_REGISTRY_HTTPCLIENT_SO_TIMEOUT - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

Kafka schema registry HTTP client configuration

KAFKA_SCHEMA_REGISTRY_RETRY_INTERVAL_IN_MILLIS - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

KAFKA_SCHEMA_REGISTRY_RETRY_TIMES - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

Kafka schema registry retry configurations

KAFKA_SOURCE_AVG_FETCH_TIME_CAP - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

KAFKA_SOURCE_SHARE_CONSUMER_CLIENT - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

KAFKA_SOURCE_WORK_UNITS_CREATION_DEFAULT_THREAD_COUNT - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

KAFKA_SOURCE_WORK_UNITS_CREATION_THREADS - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

KERBEROS_AUTH - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

KERBEROS_REALM - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

Key - Annotation Type in org.apache.gobblin.typedconfig

L

LEADER_URL - Static variable in class org.apache.gobblin.service.ServiceConfigKeys

LIST_DELIMITER_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

listSpecs() - Method in interface org.apache.gobblin.runtime.api.SpecProducer

List all Spec being executed on SpecExecutor.

LOCAL_FS_URI - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

logger(Logger) - Method in class org.apache.gobblin.testing.AssertWithBackoff

Set the logger to use for logging waiting, results, etc.

LongRange - Annotation Type in org.apache.gobblin.typedconfig.compiletime

LOW_WATERMARK_TO_JSON_KEY - Static variable in class org.apache.gobblin.source.extractor.WatermarkInterval

M

mapRecords(Function<RecordEnvelope<D>, RecordEnvelope<DO>>) - Method in class org.apache.gobblin.records.RecordStreamWithMetadata

Apply the input mapping function to RecordEnvelopes, while letting other kinds of StreamEntity to pass through.

mapStream(Function<? super Flowable<StreamEntity<D>>, ? extends Flowable<StreamEntity<DO>>>) - Method in class org.apache.gobblin.records.RecordStreamWithMetadata

markCommit(Collection<? extends WorkUnitState>) - Method in class org.apache.gobblin.publisher.DataPublisher

MAX_FLOW_GROUP_LENGTH - Static variable in class org.apache.gobblin.service.ServiceConfigKeys

MAX_FLOW_NAME_LENGTH - Static variable in class org.apache.gobblin.service.ServiceConfigKeys

MAX_JOB_GROUP_LENGTH - Static variable in class org.apache.gobblin.service.ServiceConfigKeys

MAX_JOB_NAME_LENGTH - Static variable in class org.apache.gobblin.service.ServiceConfigKeys

MAX_TASK_RETRIES_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

MAXIMUM_JAR_COPY_RETRY_TIMES_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

maxSleepMs(long) - Method in class org.apache.gobblin.testing.AssertWithBackoff

Set the max time to sleep between condition failures

MetadataMerger<T> - Interface in org.apache.gobblin.metadata

Interface for an object that can merge metadata from several work units together.

MetadataUpdateControlMessage<S,D> - Class in org.apache.gobblin.stream

Control message for updating the GlobalMetadata used for processing records

MetadataUpdateControlMessage() - Constructor for class org.apache.gobblin.stream.MetadataUpdateControlMessage

METRIC_CONTEXT_NAME_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

METRIC_TIMER_WINDOW_SIZE_IN_MINUTES - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

METRICS_CONFIGURATIONS_PREFIX - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

Common metrics configuration properties.

METRICS_CUSTOM_BUILDERS - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

METRICS_ENABLED_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

METRICS_FILE_SUFFIX - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

METRICS_KAFKA_BROKERS - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

METRICS_KAFKA_TOPIC - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

METRICS_KAFKA_TOPIC_EVENTS - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

METRICS_KAFKA_TOPIC_METRICS - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

METRICS_LOG_DIR_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

METRICS_REPORT_INTERVAL_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

METRICS_REPORTING_CONFIGURATIONS_PREFIX - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

METRICS_REPORTING_EVENTS_CONFIGURATIONS_PREFIX - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

METRICS_REPORTING_EVENTS_KAFKA_AVRO_SCHEMA_ID - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

METRICS_REPORTING_EVENTS_KAFKA_FORMAT - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

METRICS_REPORTING_EVENTS_KAFKAPUSHERKEYS - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

METRICS_REPORTING_EVENTS_PUSHERKEYS - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

METRICS_REPORTING_FILE_ENABLED_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

METRICS_REPORTING_GRAPHITE_EVENTS_ENABLED_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

METRICS_REPORTING_GRAPHITE_EVENTS_PORT - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

METRICS_REPORTING_GRAPHITE_EVENTS_VALUE_AS_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

METRICS_REPORTING_GRAPHITE_HOSTNAME - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

METRICS_REPORTING_GRAPHITE_METRICS_ENABLED_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

METRICS_REPORTING_GRAPHITE_PORT - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

METRICS_REPORTING_GRAPHITE_PREFIX - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

METRICS_REPORTING_GRAPHITE_SENDING_TYPE - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

METRICS_REPORTING_INFLUXDB_DATABASE - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

METRICS_REPORTING_INFLUXDB_EVENTS_DATABASE - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

METRICS_REPORTING_INFLUXDB_EVENTS_ENABLED_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

METRICS_REPORTING_INFLUXDB_METRICS_ENABLED_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

METRICS_REPORTING_INFLUXDB_PASSWORD - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

METRICS_REPORTING_INFLUXDB_SENDING_TYPE - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

METRICS_REPORTING_INFLUXDB_URL - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

METRICS_REPORTING_INFLUXDB_USER - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

METRICS_REPORTING_JMX_ENABLED_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

METRICS_REPORTING_KAFKA_ENABLED_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

METRICS_REPORTING_KAFKA_FORMAT - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

METRICS_REPORTING_KAFKA_USE_SCHEMA_REGISTRY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

METRICS_REPORTING_KAFKAPUSHERKEYS - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

METRICS_REPORTING_METRICS_KAFKA_AVRO_SCHEMA_ID - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

METRICS_REPORTING_PUSHERKEYS - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

MissingExtractAttributeException - Exception in org.apache.gobblin.source.workunit

MissingExtractAttributeException(String) - Constructor for exception org.apache.gobblin.source.workunit.MissingExtractAttributeException

Thrown if a required attributes hasn't been set for an extract.

MR_JARS_DIR - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

Specifies a static location in HDFS to upload jars to.

MR_JOB_MAX_MAPPERS_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

MR_JOB_ROOT_DIR_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

Configuration properties used by the Hadoop MR job launcher.

MR_REPORT_METRICS_AS_COUNTERS_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

MR_TARGET_MAPPER_SIZE - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

MultiWorkUnit - Class in gobblin.source.workunit

Shim layer for org.apache.gobblin.source.workunit.MultiWorkUnit

MultiWorkUnit() - Constructor for class gobblin.source.workunit.MultiWorkUnit

Deprecated.

MultiWorkUnit - Class in org.apache.gobblin.source.workunit

A class that wraps multiple WorkUnits so they can executed within a single task.

MultiWorkUnit() - Constructor for class org.apache.gobblin.source.workunit.MultiWorkUnit

Deprecated.

Use MultiWorkUnit.createEmpty() instead.

N

nack(Throwable) - Method in interface org.apache.gobblin.ack.Ackable

Mark this entity as failed to process.

nack(Throwable) - Method in class org.apache.gobblin.ack.BasicAckableForTesting

nack(Throwable) - Method in class org.apache.gobblin.stream.StreamEntity

nacked - Variable in class org.apache.gobblin.ack.BasicAckableForTesting

name() - Method in interface org.apache.gobblin.broker.iface.ScopeType

The name of this ScopeType.

newBuilder() - Static method in class org.apache.gobblin.commit.CommitSequence

newChildAckable() - Method in class org.apache.gobblin.ack.HierarchicalAckable

newSubscopedBuilder(ScopeInstance<S>) - Method in interface org.apache.gobblin.broker.iface.SharedResourcesBroker

Get a builder to create a descendant SharedResourcesBroker.

NODE_SECURITY_KEY - Static variable in class org.apache.gobblin.service.ServiceConfigKeys

NOOP - Static variable in interface org.apache.gobblin.records.ControlMessageHandler

A ControlMessageHandler that does nothing.

Noop() - Constructor for class org.apache.gobblin.records.ControlMessageHandler.Noop

NoopAckable - Static variable in interface org.apache.gobblin.ack.Ackable

NoopConverterInitializer - Class in org.apache.gobblin.converter.initializer

NoopDatasetResolver - Class in org.apache.gobblin.dataset

The default DatasetResolver that directly uses Gobblin raw dataset as job dataset

NoopDynamicConfigGenerator - Class in org.apache.gobblin.configuration

NoOp dynamic config generator that returns an empty Config

NoopDynamicConfigGenerator() - Constructor for class org.apache.gobblin.configuration.NoopDynamicConfigGenerator

NoopInitializer - Class in org.apache.gobblin.initializer

NoopTaskEventMetadataGenerator - Class in org.apache.gobblin.runtime

NoopTaskEventMetadataGenerator() - Constructor for class org.apache.gobblin.runtime.NoopTaskEventMetadataGenerator

NoopWriterInitializer - Class in org.apache.gobblin.writer.initializer

NoSuchScopeException - Exception in org.apache.gobblin.broker.iface

Exception thrown when trying to access a ScopeInstance that is not defined in a SharedResourcesBroker.

NoSuchScopeException(S) - Constructor for exception org.apache.gobblin.broker.iface.NoSuchScopeException

NotConfiguredException - Exception in org.apache.gobblin.broker.iface

Exception thrown by a factory if there is not enough provided configuration to create the shared object.

NotConfiguredException() - Constructor for exception org.apache.gobblin.broker.iface.NotConfiguredException

NotConfiguredException(String) - Constructor for exception org.apache.gobblin.broker.iface.NotConfiguredException

NOTIFICATION_EMAIL_ENABLED_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

NUMBER_OF_ENCRYPT_KEYS - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

O

OBJECT_DATA - Static variable in class org.apache.gobblin.util.io.GsonInterfaceAdapter

OBJECT_TYPE - Static variable in class org.apache.gobblin.util.io.GsonInterfaceAdapter

of(Destination.DestinationType, State) - Static method in class org.apache.gobblin.writer.Destination

Create a new Destination instance.

OFFSET_TOO_EARLY_COUNT - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

Configuration properties used internally by the KafkaSource.

OFFSET_TOO_LATE_COUNT - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

onInvalidate() - Method in interface org.apache.gobblin.broker.ResourceEntry

This method will be called when the entry is invalidated.

org.apache.gobblin - package org.apache.gobblin

org.apache.gobblin.ack - package org.apache.gobblin.ack

org.apache.gobblin.annotation - package org.apache.gobblin.annotation

org.apache.gobblin.broker - package org.apache.gobblin.broker

org.apache.gobblin.broker.gobblin_scopes - package org.apache.gobblin.broker.gobblin_scopes

org.apache.gobblin.broker.iface - package org.apache.gobblin.broker.iface

org.apache.gobblin.capability - package org.apache.gobblin.capability

org.apache.gobblin.codec - package org.apache.gobblin.codec

org.apache.gobblin.commit - package org.apache.gobblin.commit

org.apache.gobblin.compat.hadoop - package org.apache.gobblin.compat.hadoop

org.apache.gobblin.configuration - package org.apache.gobblin.configuration

org.apache.gobblin.converter - package org.apache.gobblin.converter

org.apache.gobblin.converter.initializer - package org.apache.gobblin.converter.initializer

org.apache.gobblin.crypto - package org.apache.gobblin.crypto

org.apache.gobblin.dataset - package org.apache.gobblin.dataset

org.apache.gobblin.dataset.comparators - package org.apache.gobblin.dataset.comparators

Contains common dataset orders that DatasetsFinders can push down.

org.apache.gobblin.dataset.test - package org.apache.gobblin.dataset.test

org.apache.gobblin.fork - package org.apache.gobblin.fork

org.apache.gobblin.initializer - package org.apache.gobblin.initializer

org.apache.gobblin.metadata - package org.apache.gobblin.metadata

org.apache.gobblin.password - package org.apache.gobblin.password

org.apache.gobblin.publisher - package org.apache.gobblin.publisher

org.apache.gobblin.qualitychecker.row - package org.apache.gobblin.qualitychecker.row

org.apache.gobblin.qualitychecker.task - package org.apache.gobblin.qualitychecker.task

org.apache.gobblin.recordaccess - package org.apache.gobblin.recordaccess

org.apache.gobblin.records - package org.apache.gobblin.records

org.apache.gobblin.runtime - package org.apache.gobblin.runtime

org.apache.gobblin.runtime.api - package org.apache.gobblin.runtime.api

org.apache.gobblin.service - package org.apache.gobblin.service

org.apache.gobblin.source - package org.apache.gobblin.source

org.apache.gobblin.source.extractor - package org.apache.gobblin.source.extractor

org.apache.gobblin.source.workunit - package org.apache.gobblin.source.workunit

org.apache.gobblin.stream - package org.apache.gobblin.stream

org.apache.gobblin.testing - package org.apache.gobblin.testing

org.apache.gobblin.typedconfig - package org.apache.gobblin.typedconfig

org.apache.gobblin.typedconfig.compiletime - package org.apache.gobblin.typedconfig.compiletime

org.apache.gobblin.util - package org.apache.gobblin.util

org.apache.gobblin.util.io - package org.apache.gobblin.util.io

org.apache.gobblin.writer - package org.apache.gobblin.writer

org.apache.gobblin.writer.initializer - package org.apache.gobblin.writer.initializer

OUTPUT_DIR_DEFAULT_SUFFIX - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

overrideWith(Properties) - Method in class org.apache.gobblin.configuration.ImmutableWorkUnitState

overrideWith(State) - Method in class org.apache.gobblin.configuration.State

Add properties in a State instance that are in the current instance.

overrideWith(Properties) - Method in class org.apache.gobblin.configuration.State

Add properties in a Properties instance that are in the current instance.

overrideWith(Properties) - Method in class org.apache.gobblin.source.workunit.ImmutableExtract

overrideWith(State) - Method in class org.apache.gobblin.source.workunit.ImmutableExtract

overrideWith(Properties) - Method in class org.apache.gobblin.source.workunit.ImmutableWorkUnit

overrideWith(State) - Method in class org.apache.gobblin.source.workunit.ImmutableWorkUnit

OVERWRITE_CONFIGS_IN_STATESTORE - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

P

pack(List<WorkUnit>, WorkUnitWeighter) - Method in interface org.apache.gobblin.source.workunit.WorkUnitBinPacker

Packs the input WorkUnits into MultiWorkUnits.

PARALLELIZE_DATASET_COMMIT - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

parentScopes() - Method in enum org.apache.gobblin.broker.gobblin_scopes.GobblinScopeTypes

parentScopes() - Method in interface org.apache.gobblin.broker.iface.ScopeType

parentScopes() - Method in enum org.apache.gobblin.broker.SimpleScopeType

parse(State) - Static method in enum org.apache.gobblin.commit.DeliverySemantics

Get the devliery semantics type from ConfigurationKeys#DELIVERY_SEMANTICS.

PartitionableDataset<T extends PartitionableDataset.DatasetPartition> - Interface in org.apache.gobblin.dataset

A Dataset that can be partitioned into disjoint subsets of the dataset.

PartitionableDataset.DatasetPartition - Interface in org.apache.gobblin.dataset

A partition of a PartitionableDataset.

PartitionDescriptor - Class in org.apache.gobblin.dataset

A Descriptor to identifies a partition of a dataset

PartitionDescriptor(String, DatasetDescriptor) - Constructor for class org.apache.gobblin.dataset.PartitionDescriptor

PartitionIdentifier - Class in org.apache.gobblin.writer

PartitionIdentifier() - Constructor for class org.apache.gobblin.writer.PartitionIdentifier

PasswordManager - Class in org.apache.gobblin.password

A class for managing password encryption and decryption.

PERMISSION_PARSING_RADIX - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

PLATFORM_FILE - Static variable in class org.apache.gobblin.dataset.DatasetConstants

Platforms

PLATFORM_HDFS - Static variable in class org.apache.gobblin.dataset.DatasetConstants

PLATFORM_HIVE - Static variable in class org.apache.gobblin.dataset.DatasetConstants

PLATFORM_KAFKA - Static variable in class org.apache.gobblin.dataset.DatasetConstants

PLATFORM_MYSQL - Static variable in class org.apache.gobblin.dataset.DatasetConstants

PLATFORM_SALESFORCE - Static variable in class org.apache.gobblin.dataset.DatasetConstants

POLICY_BASED_BLOCKED_CONNECTION - Static variable in class org.apache.gobblin.service.ServiceConfigKeys

POLICY_BASED_BLOCKED_NODES - Static variable in class org.apache.gobblin.service.ServiceConfigKeys

POLICY_BASED_DATA_MOVEMENT_PATH - Static variable in class org.apache.gobblin.service.ServiceConfigKeys

processStream(RecordStreamWithMetadata<DI, SI>, WorkUnitState) - Method in class org.apache.gobblin.converter.Converter

Apply conversions to the input RecordStreamWithMetadata.

processStream(RecordStreamWithMetadata<DI, SI>, WorkUnitState) - Method in interface org.apache.gobblin.records.RecordStreamProcessor

Return a RecordStreamWithMetadata with the appropriate modifications.

processStream(RecordStreamWithMetadata<DI, SI>, WorkUnitState) - Method in class org.apache.gobblin.stream.ControlMessageInjector

Apply injections to the input RecordStreamWithMetadata.

props - Variable in class org.apache.gobblin.commit.CommitStepBase.Builder

props - Variable in class org.apache.gobblin.commit.CommitStepBase

PST_TIMEZONE_NAME - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

publish(Collection<? extends WorkUnitState>) - Method in class org.apache.gobblin.publisher.DataPublisher

First publish the metadata via DataPublisher.publishMetadata(Collection), and then publish the output data via the DataPublisher.publishData(Collection) method.

publish(WorkUnitState) - Method in class org.apache.gobblin.publisher.SingleTaskDataPublisher

First publish the metadata via DataPublisher#publishMetadata(WorkUnitState), and then publish the output data via the DataPublisher#publishData(WorkUnitState) method.

PUBLISH_DATA_AT_JOB_LEVEL - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

publishData(Collection<? extends WorkUnitState>) - Method in class org.apache.gobblin.publisher.DataPublisher

Publish the data for the given tasks.

publishData(WorkUnitState) - Method in class org.apache.gobblin.publisher.SingleTaskDataPublisher

Publish the data for the given task.

PUBLISHER_DIRS - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

PUBLISHER_LATEST_FILE_ARRIVAL_TIMESTAMP - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

publishMetadata(Collection<? extends WorkUnitState>) - Method in class org.apache.gobblin.publisher.DataPublisher

Publish the metadata (e.g., schema) for the given tasks.

publishMetadata(WorkUnitState) - Method in class org.apache.gobblin.publisher.SingleTaskDataPublisher

Publish the metadata (e.g., schema) for the given task.

put(String, String, CommitSequence) - Method in interface org.apache.gobblin.commit.CommitSequenceStore

Put a CommitSequence with the given job name and dataset URN.

Q

QUALITY_CHECKER_PREFIX - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

Configuration properties used by the quality checker.

QUALITY_CHECKER_TIMEZONE - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

QUEUED_TASK_TIME_MAX_AGE - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

QUEUED_TASK_TIME_MAX_SIZE - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

R

RANGE_DELIMITER_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

readFields(DataInput) - Method in interface org.apache.gobblin.compat.hadoop.WritableShim

readFields(DataInput) - Method in class org.apache.gobblin.configuration.ImmutableWorkUnitState

readFields(DataInput) - Method in class org.apache.gobblin.configuration.SourceState

readFields(DataInput) - Method in class org.apache.gobblin.configuration.State

readFields(DataInput) - Method in class org.apache.gobblin.configuration.WorkUnitState

readFields(DataInput) - Method in class org.apache.gobblin.source.workunit.ImmutableExtract

readFields(DataInput) - Method in class org.apache.gobblin.source.workunit.ImmutableWorkUnit

readFields(DataInput) - Method in class org.apache.gobblin.source.workunit.MultiWorkUnit

readFields(DataInput) - Method in class org.apache.gobblin.source.workunit.WorkUnit

readPassword(String) - Method in class org.apache.gobblin.password.PasswordManager

Decrypt a password if it is an encrypted password (in the form of ENC(.*)) and a master password file has been provided in the constructor.

readRecord(D) - Method in interface org.apache.gobblin.source.extractor.Extractor

Read the next data record from the data source.

readRecordEnvelope() - Method in interface org.apache.gobblin.source.extractor.Extractor

Read an RecordEnvelope.

readStreamEntity() - Method in interface org.apache.gobblin.source.extractor.Extractor

Read an StreamEntity.

readTextAsString(DataInput) - Static method in class org.apache.gobblin.compat.hadoop.TextSerializer

Deserialize a Hadoop Text object into a String

RECOMPACTION_WRITE_TO_NEW_FOLDER - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

Configuration properties related to Re-compaction

RECORD_STREAM_PROCESSOR_CLASSES_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

RecordAccessor - Interface in org.apache.gobblin.recordaccess

Interface for an object that can introspect and manipulate a record.

RecordAccessorException - Exception in org.apache.gobblin.recordaccess

Base class for exceptions thrown by RecordAccessors.

RecordAccessorException(String) - Constructor for exception org.apache.gobblin.recordaccess.RecordAccessorException

RecordAccessorException(String, Throwable) - Constructor for exception org.apache.gobblin.recordaccess.RecordAccessorException

recordAccessorForObject(Object) - Method in interface org.apache.gobblin.recordaccess.RecordAccessorProvider

Build and return a RecordAccessor that can manipulate the object passed in.

RecordAccessorProvider - Interface in org.apache.gobblin.recordaccess

A ServiceProvider that can construct a RecordAccessor based on an object's type.

RecordCountProvider - Class in org.apache.gobblin.util

A class for obtaining record counts from data.

RecordCountProvider() - Constructor for class org.apache.gobblin.util.RecordCountProvider

RecordEnvelope<D> - Class in org.apache.gobblin.stream

An envelope around a record containing metadata and allowing for ack'ing the record.

RecordEnvelope(D) - Constructor for class org.apache.gobblin.stream.RecordEnvelope

RecordEnvelope(D, CheckpointableWatermark) - Constructor for class org.apache.gobblin.stream.RecordEnvelope

RecordEnvelope.ForkRecordBuilder<DO> - Class in org.apache.gobblin.stream

Used to create derivative records with the same callbacks and watermarks.

recordStream(AtomicBoolean) - Method in interface org.apache.gobblin.source.extractor.Extractor

RecordStreamConsumer<S,D> - Interface in org.apache.gobblin.records

An object that a RecordStreamWithMetadata.

RecordStreamProcessor<SI,SO,DI,DO> - Interface in org.apache.gobblin.records

An object that applies some function to a RecordStreamWithMetadata.

RecordStreamProcessor.StreamProcessingException - Exception in org.apache.gobblin.records

Exception allowed by RecordStreamProcessor.processStream(RecordStreamWithMetadata, WorkUnitState).

RecordStreamWithMetadata<D,S> - Class in org.apache.gobblin.records

A stream of records along with metadata (e.g.

RecordStreamWithMetadata() - Constructor for class org.apache.gobblin.records.RecordStreamWithMetadata

recordsWritten() - Method in interface org.apache.gobblin.writer.DataWriter

Get the number of records written.

removeProp(String) - Method in class org.apache.gobblin.configuration.State

Remove a property if it exists.

removeProp(String) - Method in class org.apache.gobblin.configuration.WorkUnitState

removePropsWithPrefix(String) - Method in class org.apache.gobblin.configuration.State

Remove all properties with a certain keyPrefix

removePropsWithPrefix(String) - Method in class org.apache.gobblin.configuration.WorkUnitState

REPORT_JOB_PROGRESS - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

REQUIRED_ATRRIBUTES_LIST - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

Configuration property used only for job configuration file's template

resolve(DatasetDescriptor, State) - Method in interface org.apache.gobblin.dataset.DatasetResolver

Deprecated.

Given raw Gobblin dataset, resolve job specific dataset

resolve(Descriptor, State) - Method in interface org.apache.gobblin.dataset.DatasetResolver

Deprecated.

resolve(Descriptor, State) - Method in interface org.apache.gobblin.dataset.DescriptorResolver

Given raw Gobblin descriptor, resolve a job specific descriptor

resolve(DatasetDescriptor, State) - Method in class org.apache.gobblin.dataset.HiveToHdfsDatasetResolver

resolve(DatasetDescriptor, State) - Method in class org.apache.gobblin.dataset.NoopDatasetResolver

resolve(Descriptor, State) - Method in class org.apache.gobblin.dataset.NoopDatasetResolver

resolve(String) - Method in class org.apache.gobblin.util.ClassAliasResolver

Resolves the given alias to its name if a mapping exits.

resolveClass(String) - Method in class org.apache.gobblin.util.ClassAliasResolver

Attempts to resolve the given alias to a class.

resolveUnderlyingObject(Object) - Static method in class org.apache.gobblin.util.DecoratorUtils

Resolves the truly underlying object in a possible chain of Decorator.

ResourceCoordinate<T,K extends SharedResourceKey,S extends ScopeType<S>> - Class in org.apache.gobblin.broker

A SharedResourceFactoryResponse that indicates the broker should return the object at a different coordinate (ie.

ResourceCoordinate() - Constructor for class org.apache.gobblin.broker.ResourceCoordinate

ResourceEntry<T> - Interface in org.apache.gobblin.broker

A SharedResourceFactoryResponse containing a instance of a resource.

REST_SERVER_ADVERTISED_URI_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

REST_SERVER_HOST_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

Rest server configuration properties.

REST_SERVER_PORT_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

REUSABLE - Static variable in class org.apache.gobblin.publisher.DataPublisher

Reusable capability.

rootScope() - Method in enum org.apache.gobblin.broker.gobblin_scopes.GobblinScopeTypes

rootScope() - Method in interface org.apache.gobblin.broker.iface.ScopeType

rootScope() - Method in enum org.apache.gobblin.broker.SimpleScopeType

ROW_COUNT_RANGE - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

ROW_LEVEL_ERR_FILE - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

ROW_LEVEL_ERR_FILE_DEFAULT_SUFFIX - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

ROW_LEVEL_ERR_FILE_RECORDS_PER_TASK - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

Set the approximate max number of records to write in err_file for each task.

ROW_LEVEL_POLICY_LIST - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

ROW_LEVEL_POLICY_LIST_TYPE - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

RowLevelPolicy - Class in org.apache.gobblin.qualitychecker.row

A policy that operates on each row and executes a given check

RowLevelPolicy(State, RowLevelPolicy.Type) - Constructor for class org.apache.gobblin.qualitychecker.row.RowLevelPolicy

RowLevelPolicy.Result - Enum in org.apache.gobblin.qualitychecker.row

RowLevelPolicy.Type - Enum in org.apache.gobblin.qualitychecker.row

S

SCHEDULER_WAIT_FOR_JOB_COMPLETION_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

schema - Variable in class org.apache.gobblin.writer.DataWriterBuilder

SCHEMA_FILENAME - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

SCHEMA_IN_SOURCE_DIR - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

SchemaConversionException - Exception in org.apache.gobblin.converter

A type of Exception thrown when there's anything wrong with schema conversion.

SchemaConversionException(Throwable) - Constructor for exception org.apache.gobblin.converter.SchemaConversionException

SchemaConversionException(String, Throwable) - Constructor for exception org.apache.gobblin.converter.SchemaConversionException

SchemaConversionException(String) - Constructor for exception org.apache.gobblin.converter.SchemaConversionException

ScopedConfigView<S extends ScopeType<S>,K extends SharedResourceKey> - Interface in org.apache.gobblin.broker.iface

Provides a view of a Config specific to a factory, SharedResourceKey and ScopeInstance.

ScopeInstance<S extends ScopeType<S>> - Interface in org.apache.gobblin.broker.iface

A specific scope for a ScopeType.

ScopeType<S extends ScopeType<S>> - Interface in org.apache.gobblin.broker.iface

Represents a DAG of scope types.

selfScope() - Method in interface org.apache.gobblin.broker.iface.SharedResourcesBroker

serializeAddSpecResponse(Future<?>) - Method in interface org.apache.gobblin.runtime.api.SpecProducer

SERVICE_NAME - Static variable in class org.apache.gobblin.service.ServiceConfigKeys

SERVICE_POLICY_NAME - Static variable in class org.apache.gobblin.service.ServiceConfigKeys

SERVICE_PORT - Static variable in class org.apache.gobblin.service.ServiceConfigKeys

SERVICE_URL_PREFIX - Static variable in class org.apache.gobblin.service.ServiceConfigKeys

ServiceConfigKeys - Class in org.apache.gobblin.service

ServiceConfigKeys() - Constructor for class org.apache.gobblin.service.ServiceConfigKeys

ServiceNode - Interface in org.apache.gobblin.runtime.api

Abstraction of a Node in SpecExecutor 'Service' here refers to 'Service' in GaaS and it is not necessary related to a Service interface.

set(String, String) - Method in interface org.apache.gobblin.recordaccess.RecordAccessor

set(String, Integer) - Method in interface org.apache.gobblin.recordaccess.RecordAccessor

set(String, Long) - Method in interface org.apache.gobblin.recordaccess.RecordAccessor

setActualHighWatermark(Watermark) - Method in class org.apache.gobblin.configuration.ImmutableWorkUnitState

setActualHighWatermark(Watermark) - Method in class org.apache.gobblin.configuration.WorkUnitState

This method should set the actual, runtime high Watermark for this WorkUnitState.

setDeltaFields(String...) - Method in class org.apache.gobblin.source.workunit.Extract

Deprecated.

It is recommended to set delta fields in WorkUnit instead of Extract.

setDeltaFields(String...) - Method in class org.apache.gobblin.source.workunit.ImmutableExtract

setExtractId(String) - Method in class org.apache.gobblin.source.workunit.Extract

Set a (non-globally) unique ID for this Extract.

setExtractId(String) - Method in class org.apache.gobblin.source.workunit.ImmutableExtract

setFiniteStream(boolean) - Method in class org.apache.gobblin.source.workunit.BasicWorkUnitStream.Builder

setFullTrue(long) - Method in class org.apache.gobblin.source.workunit.Extract

Deprecated.

It is recommend to set this information in WorkUnit instead of Extract.

setFullTrue(long) - Method in class org.apache.gobblin.source.workunit.ImmutableExtract

setHighWaterMark(long) - Method in class org.apache.gobblin.configuration.ImmutableWorkUnitState

Deprecated.

setHighWaterMark(long) - Method in class org.apache.gobblin.configuration.WorkUnitState

Deprecated.

use WorkUnitState.setActualHighWatermark(Watermark).

setHighWaterMark(long) - Method in class org.apache.gobblin.source.workunit.ImmutableWorkUnit

Deprecated.

setHighWaterMark(long) - Method in class org.apache.gobblin.source.workunit.WorkUnit

Deprecated.

use WorkUnit.setWatermarkInterval(WatermarkInterval).

setId(String) - Method in class org.apache.gobblin.configuration.ImmutableWorkUnitState

setId(String) - Method in class org.apache.gobblin.configuration.State

Set the id used for state persistence and logging.

setId(String) - Method in class org.apache.gobblin.source.workunit.ImmutableExtract

setId(String) - Method in class org.apache.gobblin.source.workunit.ImmutableWorkUnit

setInputGlobalMetadata(GlobalMetadata<SI>, WorkUnitState) - Method in class org.apache.gobblin.stream.ControlMessageInjector

Set the global metadata of the input messages.

setJobState(State) - Method in class org.apache.gobblin.configuration.WorkUnitState

setLowWaterMark(long) - Method in class org.apache.gobblin.source.workunit.ImmutableWorkUnit

Deprecated.

setLowWaterMark(long) - Method in class org.apache.gobblin.source.workunit.WorkUnit

Deprecated.

use WorkUnit.setWatermarkInterval(WatermarkInterval).

setPrimaryKeys(String...) - Method in class org.apache.gobblin.source.workunit.Extract

Deprecated.

It is recommended to set primary keys in WorkUnit instead of Extract.

setPrimaryKeys(String...) - Method in class org.apache.gobblin.source.workunit.ImmutableExtract

setProp(String, Object) - Method in class org.apache.gobblin.configuration.ImmutableWorkUnitState

setProp(String, Object) - Method in class org.apache.gobblin.configuration.State

Set a property.

setProp(String, Object) - Method in class org.apache.gobblin.source.workunit.ImmutableExtract

setProp(String, Object) - Method in class org.apache.gobblin.source.workunit.ImmutableWorkUnit

setProp(String, Object) - Method in class org.apache.gobblin.source.workunit.MultiWorkUnit

Set the specified key, value pair in this MultiWorkUnit as well as in all the inner WorkUnits.

setPropExcludeInnerWorkUnits(String, Object) - Method in class org.apache.gobblin.source.workunit.MultiWorkUnit

Set the specified key, value pair in this MultiWorkUnit only, but do not propagate it to all the inner WorkUnits.

setProps(Properties, Properties) - Method in class org.apache.gobblin.configuration.State

Override existing State.commonProperties and State.specProperties.

setRecordMetadata(String, Object) - Method in class org.apache.gobblin.stream.RecordEnvelope

Set the record metadata

setSafeToMaterialize(boolean) - Method in class org.apache.gobblin.source.workunit.BasicWorkUnitStream.Builder

setStringArray(String, List<String>) - Method in interface org.apache.gobblin.recordaccess.RecordAccessor

setToNull(String) - Method in interface org.apache.gobblin.recordaccess.RecordAccessor

setWatermarkInterval(WatermarkInterval) - Method in class org.apache.gobblin.source.workunit.WorkUnit

Set WatermarkInterval for a WorkUnit.

setWorkingState(WorkUnitState.WorkingState) - Method in class org.apache.gobblin.configuration.ImmutableWorkUnitState

setWorkingState(WorkUnitState.WorkingState) - Method in class org.apache.gobblin.configuration.WorkUnitState

Set the current runtime state of the WorkUnit.

setWuProperties(Properties, Properties) - Method in class org.apache.gobblin.configuration.WorkUnitState

Override WorkUnitState.workUnit's properties with new commonProps and specProps.

SHARED_KAFKA_CONFIG_PREFIX - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

SharedResourceFactory<T,K extends SharedResourceKey,S extends ScopeType<S>> - Interface in org.apache.gobblin.broker.iface

A factory that creates ScopeInstance specific objects.

SharedResourceFactoryResponse<T> - Interface in org.apache.gobblin.broker.iface

An empty interface for responses from a SharedResourceFactory.

SharedResourceKey - Interface in org.apache.gobblin.broker.iface

A key to differentiate objects generated by the same factory for a SharedResourcesBroker.

SharedResourcesBroker<S extends ScopeType<S>> - Interface in org.apache.gobblin.broker.iface

A class that provides access to objects shared by multiple components within a process, as well as objects virtually shared among different processes (i.e.

SHOULD_FS_PROXY_AS_USER - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

Proxy Filesystem operation properties.

shouldPublishMetadataFirst() - Method in class org.apache.gobblin.publisher.DataPublisher

Generally metadata should be published before the data it represents, but this allows subclasses to override if they are dependent on data getting published first.

shutdown() - Method in interface org.apache.gobblin.source.extractor.Extractor

Called to notify the Extractor it should shut down as soon as possible.

shutdown(SourceState) - Method in interface org.apache.gobblin.source.Source

Shutdown this Source instance.

SIMPLE_WRITER_DELIMITER - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

SIMPLE_WRITER_PREPEND_SIZE - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

SimpleDatasetForTesting - Class in org.apache.gobblin.dataset.test

A dumb Dataset used for testing.

SimpleDatasetForTesting() - Constructor for class org.apache.gobblin.dataset.test.SimpleDatasetForTesting

SimpleDatasetPartitionForTesting - Class in org.apache.gobblin.dataset.test

A dumb PartitionableDataset.DatasetPartition used for testing.

SimpleDatasetPartitionForTesting() - Constructor for class org.apache.gobblin.dataset.test.SimpleDatasetPartitionForTesting

SimplePartitionableDatasetForTesting - Class in org.apache.gobblin.dataset.test

A PartitionableDataset that just returns a predefined set of SimpleDatasetPartitionForTesting used for testing.

SimplePartitionableDatasetForTesting(String, List<SimpleDatasetPartitionForTesting>) - Constructor for class org.apache.gobblin.dataset.test.SimplePartitionableDatasetForTesting

SimpleScope<S extends ScopeType<S>> - Class in org.apache.gobblin.broker

A simple ScopeInstance implementation containing just a ScopeType and an string id.

SimpleScope() - Constructor for class org.apache.gobblin.broker.SimpleScope

SimpleScopeType - Enum in org.apache.gobblin.broker

Simple ScopeType topology with only two levels.

SingleTaskDataPublisher - Class in org.apache.gobblin.publisher

A subclass of DataPublisher that can publish data of a single Task.

SingleTaskDataPublisher(State) - Constructor for class org.apache.gobblin.publisher.SingleTaskDataPublisher

skip() - Method in class org.apache.gobblin.source.workunit.WorkUnit

This method will allow a work unit to be skipped if needed.

Source<S,D> - Interface in org.apache.gobblin.source

An interface for classes that the end users implement to work with a data source from which schema and data records can be extracted.

SOURCE_CLASS_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

SOURCE_COLUMN_NAME_CASE - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

SOURCE_CONN_CLIENT_ID - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

SOURCE_CONN_CLIENT_SECRET - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

SOURCE_CONN_DECRYPT_CLIENT_SECRET - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

SOURCE_CONN_DEFAULT_PORT - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

SOURCE_CONN_DOMAIN - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

SOURCE_CONN_DRIVER - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

SOURCE_CONN_HOST_NAME - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

SOURCE_CONN_KNOWN_HOSTS - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

SOURCE_CONN_PASSWORD - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

SOURCE_CONN_PORT - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

SOURCE_CONN_PREFIX - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

Configuration properties for source connection.

SOURCE_CONN_PRIVATE_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

SOURCE_CONN_PROPERTIES - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

SOURCE_CONN_REFRESH_TOKEN - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

SOURCE_CONN_REST_URL - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

SOURCE_CONN_SECURITY_TOKEN - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

SOURCE_CONN_SID - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

SOURCE_CONN_TIMEOUT - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

SOURCE_CONN_USE_AUTHENTICATION - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

SOURCE_CONN_USE_PROXY_PORT - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

SOURCE_CONN_USE_PROXY_URL - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

SOURCE_CONN_USERNAME - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

SOURCE_CONN_VERSION - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

SOURCE_EARLY_STOP_ENABLED - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

SOURCE_ENTITIES - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

SOURCE_ENTITY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

Configuration properties used by the extractor.

SOURCE_FILEBASED_DATA_DIRECTORY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

Configuration properties used by the FileBasedExtractor

SOURCE_FILEBASED_ENCRYPTED_CONFIG_PATH - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

SOURCE_FILEBASED_FILES_TO_PULL - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

SOURCE_FILEBASED_FS_PRIOR_SNAPSHOT_REQUIRED - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

SOURCE_FILEBASED_FS_SNAPSHOT - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

SOURCE_FILEBASED_FS_URI - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

SOURCE_FILEBASED_MAX_FILES_PER_RUN - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

SOURCE_FILEBASED_OPTIONAL_DOWNLOADER_CLASS - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

SOURCE_FILEBASED_PLATFORM - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

SOURCE_FILEBASED_PRESERVE_FILE_NAME - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

SOURCE_MAX_NUMBER_OF_PARTITIONS - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

SOURCE_QUERYBASED_ALLOW_REMOVE_UPPER_BOUNDS - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

SOURCE_QUERYBASED_APPEND_MAX_WATERMARK_LIMIT - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

SOURCE_QUERYBASED_END_VALUE - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

SOURCE_QUERYBASED_EXCLUDED_COLUMNS - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

SOURCE_QUERYBASED_EXTRACT_TYPE - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

SOURCE_QUERYBASED_FETCH_SIZE - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

SOURCE_QUERYBASED_HOUR_COLUMN - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

SOURCE_QUERYBASED_IS_COMPRESSION_ENABLED - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

SOURCE_QUERYBASED_IS_HOURLY_EXTRACT - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

SOURCE_QUERYBASED_IS_METADATA_COLUMN_CHECK_ENABLED - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

SOURCE_QUERYBASED_IS_SPECIFIC_API_ACTIVE - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

SOURCE_QUERYBASED_IS_WATERMARK_OVERRIDE - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

SOURCE_QUERYBASED_JDBC_RESULTSET_FETCH_SIZE - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

SOURCE_QUERYBASED_LOW_WATERMARK_BACKUP_SECS - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

SOURCE_QUERYBASED_PARTITION_INTERVAL - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

SOURCE_QUERYBASED_PROMOTE_UNSIGNED_INT_TO_BIGINT - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

SOURCE_QUERYBASED_QUERY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

SOURCE_QUERYBASED_RESET_EMPTY_PARTITION_WATERMARK - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

SOURCE_QUERYBASED_SCHEMA - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

SOURCE_QUERYBASED_SKIP_COUNT_CALC - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

SOURCE_QUERYBASED_SKIP_HIGH_WATERMARK_CALC - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

SOURCE_QUERYBASED_START_VALUE - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

SOURCE_QUERYBASED_WATERMARK_TYPE - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

Configuration properties used by the QueryBasedExtractor.

SOURCE_SCHEMA - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

SOURCE_SKIP_FIRST_RECORD - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

SOURCE_TIMEZONE - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

SourceState - Class in gobblin.configuration

Shim layer for org.apache.gobblin.configuration.SourceState

SourceState() - Constructor for class gobblin.configuration.SourceState

Default constructor.

SourceState(State, Iterable<WorkUnitState>) - Constructor for class gobblin.configuration.SourceState

SourceState(State, Map<String, ? extends SourceState>, Iterable<WorkUnitState>) - Constructor for class gobblin.configuration.SourceState

SourceState - Class in org.apache.gobblin.configuration

A container for all meta data related to a particular source.

SourceState() - Constructor for class org.apache.gobblin.configuration.SourceState

Default constructor.

SourceState(State) - Constructor for class org.apache.gobblin.configuration.SourceState

Constructor.

SourceState(State, Iterable<WorkUnitState>) - Constructor for class org.apache.gobblin.configuration.SourceState

Constructor.

SourceState(State, Map<String, ? extends SourceState>, Iterable<WorkUnitState>) - Constructor for class org.apache.gobblin.configuration.SourceState

Constructor.

Spec - Interface in org.apache.gobblin.runtime.api

A basic interface for an object with a URI, version, and description.

SPEC_EXECUTOR_KEY - Static variable in class org.apache.gobblin.service.ServiceConfigKeys

SPEC_PRODUCER_SERIALIZED_FUTURE - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

Configuration properties related to Spec Producer

SpecConsumer<V> - Interface in org.apache.gobblin.runtime.api

A communication socket (receiving side for this class) for each SpecExecutor to receive spec to execute from Orchestrator.

SpecExecutor - Interface in org.apache.gobblin.runtime.api

Defines a representation of JobSpec-Executor in GaaS.

SpecExecutor.Verb - Enum in org.apache.gobblin.runtime.api

SPECEXECUTOR_CONFIGS_PREFIX_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

SPECEXECUTOR_INSTANCE_CAPABILITIES_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

SPECEXECUTOR_INSTANCE_URI_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

Configuration properties related to Spec Executor Instance

SpecProducer<V> - Interface in org.apache.gobblin.runtime.api

Defines a SpecProducer to produce jobs to SpecExecutor that can execute a Spec.

SQL_SERVER_CONNECTION_PARAMETERS - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

Stable - Annotation Type in org.apache.gobblin.annotation

Denotes an API that is still stable and should not change.

STAGING_DIR_DEFAULT_SUFFIX - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

start(WatermarkStorage) - Method in interface org.apache.gobblin.source.extractor.StreamingExtractor

Initialize the extractor to be ready to read records

State - Class in gobblin.configuration

Shim layer for org.apache.gobblin.configuration.State

State() - Constructor for class gobblin.configuration.State

State(Properties) - Constructor for class gobblin.configuration.State

State(State) - Constructor for class gobblin.configuration.State

State - Class in org.apache.gobblin.configuration

A serializable wrapper class that can be persisted for Properties.

State() - Constructor for class org.apache.gobblin.configuration.State

State(Properties) - Constructor for class org.apache.gobblin.configuration.State

State(State) - Constructor for class org.apache.gobblin.configuration.State

state - Variable in class org.apache.gobblin.publisher.DataPublisher

state - Variable in class org.apache.gobblin.qualitychecker.row.RowLevelPolicy

STATE_STORE_COMPRESSED_VALUES_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

STATE_STORE_DB_CONN_MIN_EVICTABLE_IDLE_TIME_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

STATE_STORE_DB_JDBC_DRIVER_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

STATE_STORE_DB_PASSWORD_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

STATE_STORE_DB_TABLE_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

STATE_STORE_DB_URL_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

STATE_STORE_DB_USER_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

STATE_STORE_ENABLED - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

STATE_STORE_FACTORY_CLASS_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

STATE_STORE_FS_URI_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

STATE_STORE_KEY_SEPARATION_CHARACTER - Static variable in class org.apache.gobblin.service.ServiceConfigKeys

STATE_STORE_ROOT_DIR_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

STATE_STORE_TABLE_SUFFIX - Static variable in class org.apache.gobblin.service.ServiceConfigKeys

STATE_STORE_TYPE_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

State store configuration properties.

STATE_STORE_TYPE_NOOP - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

StateUtils - Class in org.apache.gobblin.configuration

Utility class for dealing with State objects.

StateUtils() - Constructor for class org.apache.gobblin.configuration.StateUtils

StaticDatasetsFinderForTesting - Class in org.apache.gobblin.dataset.test

A DatasetsFinder that returns a predefined set of Datasets for testing.

StaticDatasetsFinderForTesting() - Constructor for class org.apache.gobblin.dataset.test.StaticDatasetsFinderForTesting

StreamCodec - Interface in org.apache.gobblin.codec

Interface for an object that can encrypt a bytestream.

StreamEntity<D> - Class in org.apache.gobblin.stream

An entity in the Gobblin ingestion stream.

StreamEntity() - Constructor for class org.apache.gobblin.stream.StreamEntity

StreamEntity(StreamEntity<?>, boolean) - Constructor for class org.apache.gobblin.stream.StreamEntity

StreamEntity(StreamEntity<D>.ForkedEntityBuilder, boolean) - Constructor for class org.apache.gobblin.stream.StreamEntity

StreamEntity.ForkCloner - Class in org.apache.gobblin.stream

Used to clone a StreamEntity retaining the correct callbacks.

StreamEntity.ForkedEntityBuilder - Class in org.apache.gobblin.stream

Used to generate derived StreamEntitys using a HierarchicalAckable to make sure parent ackable is automatically acked when all children are acked.

StreamingExtractor<S,D> - Interface in org.apache.gobblin.source.extractor

An interface for implementing streaming / continuous extractors

StreamProcessingException(String) - Constructor for exception org.apache.gobblin.records.RecordStreamProcessor.StreamProcessingException

StreamProcessingException(String, Throwable) - Constructor for exception org.apache.gobblin.records.RecordStreamProcessor.StreamProcessingException

StreamProcessingException(Throwable) - Constructor for exception org.apache.gobblin.records.RecordStreamProcessor.StreamProcessingException

StringNameSharedResourceKey - Class in org.apache.gobblin.broker

A SharedResourceKey with only a string name

StringNameSharedResourceKey(String) - Constructor for class org.apache.gobblin.broker.StringNameSharedResourceKey

StringRegex - Annotation Type in org.apache.gobblin.typedconfig.compiletime

SubscopedBrokerBuilder<S extends ScopeType<S>,B extends SharedResourcesBroker<S>> - Interface in org.apache.gobblin.broker.iface

A builder used to create new SharedResourcesBroker compatible with an existing SharedResourcesBroker (i.e.

SUPER_USER_KEY_TAB_LOCATION - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

SUPER_USER_NAME_TO_PROXY_AS_OTHERS - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

supportsCapability(Capability, Map<String, Object>) - Method in interface org.apache.gobblin.capability.CapabilityAware

Checks if this object supports the given Capability with the given properties.

supportsCapability(Capability, Map<String, Object>) - Method in class org.apache.gobblin.publisher.DataPublisher

T

TASK_ATTEMPT_ID_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

TASK_DATA_PUBLISHER_TYPE - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

TASK_DATA_ROOT_DIR_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

TASK_EVENT_METADATA_GENERATOR_CLASS_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

Configuration for emitting task events

TASK_EXECUTOR_THREADPOOL_SIZE_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

Task executor and state tracker configuration properties.

TASK_FAILURE_EXCEPTION_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

TASK_ID_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

TASK_IGNORE_CLOSE_FAILURES - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

TASK_INTERRUPT_ON_CANCEL - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

TASK_ISSUES_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

TASK_KEY_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

TASK_LEVEL_POLICY_LIST - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

TASK_LEVEL_POLICY_LIST_TYPE - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

TASK_RETRIES_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

TASK_RETRY_INTERVAL_IN_SEC_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

TASK_RETRY_THREAD_POOL_CORE_SIZE_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

TASK_START_TIME_MILLIS_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

TASK_STATE_COLLECTOR_HANDLER_CLASS - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

TASK_STATE_COLLECTOR_INTERVAL_SECONDS - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

Configuration properties used by the distributed job launcher.

TASK_STATE_TRACKER_THREAD_POOL_CORE_SIZE_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

TASK_STATUS_REPORT_INTERVAL_IN_MS_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

Configuration properties for the task status.

TASK_SYNCHRONOUS_EXECUTION_MODEL_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

Task execution properties.

TASK_TIMEOUT_SECONDS - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

TaskEventMetadataGenerator - Interface in org.apache.gobblin.runtime.api

For generating additional event metadata to associate with Task Events.

TaskEventMetadataUtils - Class in org.apache.gobblin.util

TaskEventMetadataUtils() - Constructor for class org.apache.gobblin.util.TaskEventMetadataUtils

TaskLevelPolicy - Class in org.apache.gobblin.qualitychecker.task

TaskLevelPolicy(State, TaskLevelPolicy.Type) - Constructor for class org.apache.gobblin.qualitychecker.task.TaskLevelPolicy

TaskLevelPolicy.Result - Enum in org.apache.gobblin.qualitychecker.task

TaskLevelPolicy.Type - Enum in org.apache.gobblin.qualitychecker.task

TaskScopeInstance - Class in org.apache.gobblin.broker.gobblin_scopes

A ScopeInstance for a Gobblin task.

TaskScopeInstance(String) - Constructor for class org.apache.gobblin.broker.gobblin_scopes.TaskScopeInstance

TEMPLATE_CATALOGS_FULLY_QUALIFIED_PATH_KEY - Static variable in class org.apache.gobblin.service.ServiceConfigKeys

TEST_HARNESS_LAUNCHER_IMPL - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

TextSerializer - Class in org.apache.gobblin.compat.hadoop

Serializes Java Strings similar to a Hadoop Text without depending on the underlying Hadoop libraries.

TextSerializer() - Constructor for class org.apache.gobblin.compat.hadoop.TextSerializer

THREADPOOL_SIZE_OF_LISTING_FS_DATASET_STATESTORE - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

THREADSAFE - Static variable in class org.apache.gobblin.capability.Capability

Threadsafe capability.

throwable - Variable in class org.apache.gobblin.ack.BasicAckableForTesting

timeoutMs(long) - Method in class org.apache.gobblin.testing.AssertWithBackoff

Set the max time in milliseconds to wait for the condition to become true

TMP_DIR - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

toConfigurationKey() - Method in interface org.apache.gobblin.broker.iface.SharedResourceKey

toConfigurationKey() - Method in class org.apache.gobblin.broker.StringNameSharedResourceKey

toDataMap() - Method in class org.apache.gobblin.dataset.DatasetDescriptor

Deprecated.

use Descriptor#serialize(Descriptor)

toJson(Descriptor) - Static method in class org.apache.gobblin.dataset.Descriptor

Serialize any Descriptor object as json string

toJson(List<Descriptor>) - Static method in class org.apache.gobblin.dataset.Descriptor

Serialize a list of descriptors as json string

toJson() - Method in interface org.apache.gobblin.source.extractor.Watermark

Convert this Watermark into a JsonElement.

toJson() - Method in class org.apache.gobblin.source.extractor.WatermarkInterval

toJson() - Method in class org.apache.gobblin.writer.FsWriterMetrics

Serialize this class to Json

TOKEN_AUTH - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

toPartitionJsonList(List<PartitionDescriptor>) - Static method in class org.apache.gobblin.dataset.PartitionDescriptor

Serialize a list of partition descriptors as json string

TOPOLOGY_DESCRIPTION_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

TOPOLOGY_FACTORY_PREFIX - Static variable in class org.apache.gobblin.service.ServiceConfigKeys

TOPOLOGY_FACTORY_TOPOLOGY_NAMES_KEY - Static variable in class org.apache.gobblin.service.ServiceConfigKeys

TOPOLOGY_GROUP_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

TOPOLOGY_NAME_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

Common topology configuration properties.

TOPOLOGYSPEC_DESCRIPTION_KEY - Static variable in class org.apache.gobblin.service.ServiceConfigKeys

TOPOLOGYSPEC_FACTORY_KEY - Static variable in class org.apache.gobblin.service.ServiceConfigKeys

TOPOLOGYSPEC_SERDE_CLASS_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

TOPOLOGYSPEC_STORE_CLASS_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

Configuration properties related to TopologySpec Store

TOPOLOGYSPEC_STORE_DIR_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

TOPOLOGYSPEC_URI_KEY - Static variable in class org.apache.gobblin.service.ServiceConfigKeys

TOPOLOGYSPEC_VERSION_KEY - Static variable in class org.apache.gobblin.service.ServiceConfigKeys

toProp() - Method in class org.apache.gobblin.typedconfig.TypedConfig

convert data to property

toString() - Method in class org.apache.gobblin.configuration.State

toString() - Method in class org.apache.gobblin.configuration.WorkUnitState

toString() - Method in enum org.apache.gobblin.Constructs

toString() - Method in class org.apache.gobblin.dataset.Descriptor

toString() - Method in class org.apache.gobblin.qualitychecker.row.RowLevelPolicy

toString() - Method in class org.apache.gobblin.qualitychecker.task.TaskLevelPolicy

TOTAL_WORK_UNIT_COUNT - Static variable in class org.apache.gobblin.service.ServiceConfigKeys

TOTAL_WORK_UNIT_SIZE - Static variable in class org.apache.gobblin.service.ServiceConfigKeys

transform(Function<WorkUnit, WorkUnit>) - Method in class org.apache.gobblin.source.workunit.BasicWorkUnitStream

Apply a transformation function to this stream.

transform(Function<WorkUnit, WorkUnit>) - Method in interface org.apache.gobblin.source.workunit.WorkUnitStream

Apply a transformation function to this stream.

TROUBLESHOOTER_DISABLE_EVENT_REPORTING - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

Disables reporting troubleshooter issues as GobblinTrackingEvents

TROUBLESHOOTER_DISABLED - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

Disables all troubleshooter functions

TROUBLESHOOTER_IN_MEMORY_ISSUE_REPOSITORY_MAX_SIZE - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

The maximum number of issues that In-memory troubleshooter repository will keep.

TypedConfig - Class in org.apache.gobblin.typedconfig

TypedConfig(Properties) - Constructor for class org.apache.gobblin.typedconfig.TypedConfig

typedSelf() - Method in class org.apache.gobblin.writer.FluentDataWriterBuilder

U

UnpublishedHandling - Interface in org.apache.gobblin.publisher

DataPublishers implementing this interface will cause Gobblin to call UnpublishedHandling.handleUnpublishedWorkUnits(java.util.Collection<? extends org.apache.gobblin.configuration.WorkUnitState>) if it determines the job will not be published.

update(T) - Method in interface org.apache.gobblin.metadata.MetadataMerger

Process a metadata record, merging it with all previously processed records.

update(FsWriterMetrics) - Method in interface org.apache.gobblin.metadata.MetadataMerger

Process a metrics record, merging it with all previously processed records.

updateSpec(V) - Method in interface org.apache.gobblin.runtime.api.SpecProducer

Update a Spec being executed on SpecExecutor.

URNIdentified - Interface in org.apache.gobblin.dataset

An object that can be identified by URN.

URNLexicographicalComparator - Class in org.apache.gobblin.dataset.comparators

Dataset comparator that compares by dataset urn.

URNLexicographicalComparator() - Constructor for class org.apache.gobblin.dataset.comparators.URNLexicographicalComparator

USE_DATASET_LOCAL_WORK_DIR - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

Configuration for sharded directory files

USER_DEFINED_STAGING_DIR_FLAG - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

USER_DEFINED_STATIC_STAGING_DIR - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

V

valueOf(String) - Static method in enum gobblin.source.workunit.Extract.TableType

Returns the enum constant of this type with the specified name.

valueOf(String) - Static method in enum org.apache.gobblin.broker.gobblin_scopes.GobblinScopeTypes

Returns the enum constant of this type with the specified name.

valueOf(String) - Static method in enum org.apache.gobblin.broker.SimpleScopeType

Returns the enum constant of this type with the specified name.

valueOf(String) - Static method in enum org.apache.gobblin.commit.DeliverySemantics

Returns the enum constant of this type with the specified name.

valueOf(String) - Static method in enum org.apache.gobblin.configuration.WorkUnitState.WorkingState

Returns the enum constant of this type with the specified name.

valueOf(String) - Static method in enum org.apache.gobblin.Constructs

Returns the enum constant of this type with the specified name.

valueOf(String) - Static method in enum org.apache.gobblin.qualitychecker.row.RowLevelPolicy.Result

Returns the enum constant of this type with the specified name.

valueOf(String) - Static method in enum org.apache.gobblin.qualitychecker.row.RowLevelPolicy.Type

Returns the enum constant of this type with the specified name.

valueOf(String) - Static method in enum org.apache.gobblin.qualitychecker.task.TaskLevelPolicy.Result

Returns the enum constant of this type with the specified name.

valueOf(String) - Static method in enum org.apache.gobblin.qualitychecker.task.TaskLevelPolicy.Type

Returns the enum constant of this type with the specified name.

valueOf(String) - Static method in enum org.apache.gobblin.runtime.api.SpecExecutor.Verb

Returns the enum constant of this type with the specified name.

valueOf(String) - Static method in enum org.apache.gobblin.source.workunit.Extract.TableType

Returns the enum constant of this type with the specified name.

valueOf(String) - Static method in enum org.apache.gobblin.stream.FlushControlMessage.FlushType

Returns the enum constant of this type with the specified name.

valueOf(String) - Static method in enum org.apache.gobblin.writer.Destination.DestinationType

Returns the enum constant of this type with the specified name.

valueOf(String) - Static method in enum org.apache.gobblin.writer.WriterOutputFormat

Returns the enum constant of this type with the specified name.

values() - Static method in enum gobblin.source.workunit.Extract.TableType

Returns an array containing the constants of this enum type, in the order they are declared.

values() - Static method in enum org.apache.gobblin.broker.gobblin_scopes.GobblinScopeTypes

Returns an array containing the constants of this enum type, in the order they are declared.

values() - Static method in enum org.apache.gobblin.broker.SimpleScopeType

Returns an array containing the constants of this enum type, in the order they are declared.

values() - Static method in enum org.apache.gobblin.commit.DeliverySemantics

Returns an array containing the constants of this enum type, in the order they are declared.

values() - Static method in enum org.apache.gobblin.configuration.WorkUnitState.WorkingState

Returns an array containing the constants of this enum type, in the order they are declared.

values() - Static method in enum org.apache.gobblin.Constructs

Returns an array containing the constants of this enum type, in the order they are declared.

values() - Static method in enum org.apache.gobblin.qualitychecker.row.RowLevelPolicy.Result

Returns an array containing the constants of this enum type, in the order they are declared.

values() - Static method in enum org.apache.gobblin.qualitychecker.row.RowLevelPolicy.Type

Returns an array containing the constants of this enum type, in the order they are declared.

values() - Static method in enum org.apache.gobblin.qualitychecker.task.TaskLevelPolicy.Result

Returns an array containing the constants of this enum type, in the order they are declared.

values() - Static method in enum org.apache.gobblin.qualitychecker.task.TaskLevelPolicy.Type

Returns an array containing the constants of this enum type, in the order they are declared.

values() - Static method in enum org.apache.gobblin.runtime.api.SpecExecutor.Verb

Returns an array containing the constants of this enum type, in the order they are declared.

values() - Static method in enum org.apache.gobblin.source.workunit.Extract.TableType

Returns an array containing the constants of this enum type, in the order they are declared.

values() - Static method in enum org.apache.gobblin.stream.FlushControlMessage.FlushType

Returns an array containing the constants of this enum type, in the order they are declared.

values() - Static method in enum org.apache.gobblin.writer.Destination.DestinationType

Returns an array containing the constants of this enum type, in the order they are declared.

values() - Static method in enum org.apache.gobblin.writer.WriterOutputFormat

Returns an array containing the constants of this enum type, in the order they are declared.

VERB_KEY - Static variable in interface org.apache.gobblin.runtime.api.SpecExecutor

W

Watermark - Interface in gobblin.source.extractor

Shim layer for org.apache.gobblin.source.extractor.Watermark

Watermark - Interface in org.apache.gobblin.source.extractor

A Watermark represents a checkpoint in data processing, and indicates that all data up to specific point has been pulled from some source.

WATERMARK_INTERVAL_VALUE_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

Watermark interval related configuration properties.

WatermarkInterval - Class in gobblin.source.extractor

Shim layer for org.apache.gobblin.source.extractor.WatermarkInterval

WatermarkInterval(Watermark, Watermark) - Constructor for class gobblin.source.extractor.WatermarkInterval

WatermarkInterval - Class in org.apache.gobblin.source.extractor

Each WorkUnit has a corresponding WatermarkInterval.

WatermarkInterval(Watermark, Watermark) - Constructor for class org.apache.gobblin.source.extractor.WatermarkInterval

WatermarkSerializerHelper - Class in org.apache.gobblin.source.extractor

Provides default implementation for converting a Watermark to a JsonElement, and vice versa.

WatermarkSerializerHelper() - Constructor for class org.apache.gobblin.source.extractor.WatermarkSerializerHelper

WatermarkStorage - Interface in org.apache.gobblin.writer

Storage for Watermarks.

weight(WorkUnit) - Method in interface org.apache.gobblin.source.workunit.WorkUnitWeighter

The weight of this work unit.

WHITELISTED_EDGE_IDS - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

withAdditionalParentBroker(SharedResourcesBroker<S>) - Method in interface org.apache.gobblin.broker.iface.SubscopedBrokerBuilder

Specify additional ancestor SharedResourcesBroker.

withAttemptId(String) - Method in class org.apache.gobblin.writer.DataWriterBuilder

Attempt Id for this writer.

withBranches(int) - Method in class org.apache.gobblin.writer.DataWriterBuilder

Tell the writer how many branches are being used.

withBranches(int) - Method in class org.apache.gobblin.writer.FluentDataWriterBuilder

withDatasetUrn(String) - Method in class org.apache.gobblin.commit.CommitSequence.Builder

Set the dataset URN for the commit sequence.

withJobName(String) - Method in class org.apache.gobblin.commit.CommitSequence.Builder

Set the job name for the commit sequence.

withOverridingConfig(Config) - Method in interface org.apache.gobblin.broker.iface.SubscopedBrokerBuilder

Specify Config overrides.

withProps(State) - Method in class org.apache.gobblin.commit.CommitStepBase.Builder

withRecord(DO) - Method in class org.apache.gobblin.stream.RecordEnvelope

withRecordStream(Flowable<StreamEntity<DO>>) - Method in class org.apache.gobblin.records.RecordStreamWithMetadata

withRecordStream(Flowable<StreamEntity<DO>>, SO) - Method in class org.apache.gobblin.records.RecordStreamWithMetadata

Deprecated.

withRecordStream(Flowable<StreamEntity<DO>>, GlobalMetadata<SO>) - Method in class org.apache.gobblin.records.RecordStreamWithMetadata

withSchema(S) - Method in class org.apache.gobblin.writer.DataWriterBuilder

Tell the writer the data schema.

withSchema(S) - Method in class org.apache.gobblin.writer.FluentDataWriterBuilder

withWriterId(String) - Method in class org.apache.gobblin.writer.DataWriterBuilder

Give the writer a unique ID.

withWriterId(String) - Method in class org.apache.gobblin.writer.FluentDataWriterBuilder

WORK_UNIT_CREATION_AND_RUN_INTERVAL - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

WORK_UNIT_CREATION_TIME_IN_MILLIS - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

WORK_UNIT_DATE_PARTITION_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

WORK_UNIT_DATE_PARTITION_NAME - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

WORK_UNIT_ENABLE_TRACKING_LOGS - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

WORK_UNIT_HIGH_WATER_MARK_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

WORK_UNIT_LOW_WATER_MARK_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

Work unit related configuration properties.

WORK_UNIT_RETRY_ENABLED_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

WORK_UNIT_RETRY_POLICY_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

WORK_UNIT_SIZE - Static variable in class org.apache.gobblin.service.ServiceConfigKeys

WORK_UNIT_SKIP_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

WORK_UNIT_STATE_ACTUAL_HIGH_WATER_MARK_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

WORK_UNIT_STATE_RUNTIME_HIGH_WATER_MARK - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

WORK_UNIT_WORKING_STATE_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

Work unit runtime state related configuration properties.

WorkUnit - Class in gobblin.source.workunit

Shim layer for org.apache.gobblin.source.workunit.WorkUnit

WorkUnit() - Constructor for class gobblin.source.workunit.WorkUnit

Deprecated.

WorkUnit(SourceState, Extract) - Constructor for class gobblin.source.workunit.WorkUnit

Deprecated.

WorkUnit(SourceState, Extract, WatermarkInterval) - Constructor for class gobblin.source.workunit.WorkUnit

Deprecated.

WorkUnit(Extract) - Constructor for class gobblin.source.workunit.WorkUnit

WorkUnit(WorkUnit) - Constructor for class gobblin.source.workunit.WorkUnit

Deprecated.

WorkUnit - Class in org.apache.gobblin.source.workunit

A logic concept that defines a unit of work or task for extracting a portion of the data to be pulled in a job run.

WorkUnit() - Constructor for class org.apache.gobblin.source.workunit.WorkUnit

Deprecated.

Use WorkUnit.createEmpty()

WorkUnit(SourceState, Extract) - Constructor for class org.apache.gobblin.source.workunit.WorkUnit

Deprecated.

Properties in SourceState should not be added to a WorkUnit. Having each WorkUnit contain a copy of SourceState is a waste of memory. Use WorkUnit.create(Extract).

WorkUnit(SourceState, Extract, WatermarkInterval) - Constructor for class org.apache.gobblin.source.workunit.WorkUnit

Deprecated.

Properties in SourceState should not be added to a WorkUnit. Having each WorkUnit contain a copy of SourceState is a waste of memory. Use WorkUnit.create(Extract, WatermarkInterval).

WorkUnit(Extract) - Constructor for class org.apache.gobblin.source.workunit.WorkUnit

Constructor.

WorkUnit(WorkUnit) - Constructor for class org.apache.gobblin.source.workunit.WorkUnit

Deprecated.

Use WorkUnit.copyOf(WorkUnit)

WorkUnitBinPacker - Interface in org.apache.gobblin.source.workunit

A bin packing algorithm for packing WorkUnits into MultiWorkUnits.

WorkUnitState - Class in gobblin.configuration

Shim layer for org.apache.gobblin.configuration.WorkUnitState

WorkUnitState() - Constructor for class gobblin.configuration.WorkUnitState

Default constructor used for deserialization.

WorkUnitState(WorkUnit) - Constructor for class gobblin.configuration.WorkUnitState

Deprecated.

WorkUnitState(WorkUnit, State) - Constructor for class gobblin.configuration.WorkUnitState

WorkUnitState - Class in org.apache.gobblin.configuration

This class encapsulates a WorkUnit instance and additionally holds all the task runtime state of that WorkUnit.

WorkUnitState() - Constructor for class org.apache.gobblin.configuration.WorkUnitState

Default constructor used for deserialization.

WorkUnitState(WorkUnit) - Constructor for class org.apache.gobblin.configuration.WorkUnitState

Deprecated.

It is recommended to use WorkUnitState.WorkUnitState(WorkUnit, State) rather than combining properties in the job state into the workunit.

WorkUnitState(WorkUnit, State) - Constructor for class org.apache.gobblin.configuration.WorkUnitState

If creating a WorkUnitState for use by a task, use WorkUnitState.WorkUnitState(WorkUnit, State, SharedResourcesBroker) instead.

WorkUnitState(WorkUnit, State, SubscopedBrokerBuilder<GobblinScopeTypes, ?>) - Constructor for class org.apache.gobblin.configuration.WorkUnitState

WorkUnitState(WorkUnit, State, SharedResourcesBroker<GobblinScopeTypes>) - Constructor for class org.apache.gobblin.configuration.WorkUnitState

WorkUnitState.WorkingState - Enum in org.apache.gobblin.configuration

Runtime state of the WorkUnit.

WorkUnitStream - Interface in org.apache.gobblin.source.workunit

A stream of WorkUnits, allows for working with large numbers of work units in a memory-efficient way, as well as processing infinite streams of work units.

WorkUnitStreamSource<S,D> - Interface in org.apache.gobblin.source

A Source that produces a WorkUnitStream.

WorkUnitWeighter - Interface in org.apache.gobblin.source.workunit

Provides weights for WorkUnits to use by a WorkUnitBinPacker.

WritableShim - Interface in org.apache.gobblin.compat.hadoop

An interface that mirrors Hadoop's Writable interface; this allows objects in gobblin-api to implement similar serializers without explicitly depending on Hadoop itself for the interface definition.

write(DataOutput) - Method in interface org.apache.gobblin.compat.hadoop.WritableShim

write(DataOutput) - Method in class org.apache.gobblin.configuration.SourceState

write(DataOutput, boolean) - Method in class org.apache.gobblin.configuration.SourceState

write(DataOutput) - Method in class org.apache.gobblin.configuration.State

write(DataOutput) - Method in class org.apache.gobblin.configuration.WorkUnitState

write(DataOutput) - Method in class org.apache.gobblin.source.workunit.MultiWorkUnit

write(DataOutput) - Method in class org.apache.gobblin.source.workunit.WorkUnit

write(D) - Method in interface org.apache.gobblin.writer.DataWriter

Write a data record.

write(D) - Method in class org.apache.gobblin.writer.WriterWrapper

writeEnvelope(RecordEnvelope<D>) - Method in interface org.apache.gobblin.writer.DataWriter

Write the input RecordEnvelope.

writeInFormat(WriterOutputFormat) - Method in class org.apache.gobblin.writer.DataWriterBuilder

Tell the writer the output format of type WriterOutputFormat.

writeInFormat(WriterOutputFormat) - Method in class org.apache.gobblin.writer.FluentDataWriterBuilder

WRITER_ADD_TASK_TIMESTAMP - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

WRITER_AVERAGE_TIMESTAMP - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

WRITER_BUFFER_SIZE - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

WRITER_BUILDER_CLASS - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

WRITER_BYTES_WRITTEN - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

WRITER_CODEC_TYPE - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

WRITER_DEFLATE_LEVEL - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

WRITER_DESTINATION_TYPE_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

WRITER_DIR_PERMISSIONS - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

WRITER_EAGER_INITIALIZATION_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

WRITER_EARLIEST_TIMESTAMP - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

WRITER_FILE_BLOCK_SIZE - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

WRITER_FILE_GROUP - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

WRITER_FILE_NAME - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

WRITER_FILE_OWNER - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

WRITER_FILE_PATH - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

WRITER_FILE_PATH_TYPE - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

WRITER_FILE_PERMISSIONS - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

WRITER_FILE_REPLICATION_FACTOR - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

WRITER_FILE_SYSTEM_URI - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

WRITER_FINAL_OUTPUT_FILE_PATHS - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

Writer configuration properties used internally.

WRITER_GROUP_NAME - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

WRITER_METADATA_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

WRITER_OUTPUT_DIR - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

WRITER_OUTPUT_FORMAT_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

WRITER_PARTITION_PATH_KEY - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

WRITER_PARTITIONER_CLASS - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

WRITER_PREFIX - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

Writer configuration properties.

WRITER_PRESERVE_FILE_NAME - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

WRITER_RECORDS_WRITTEN - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

WRITER_ROWS_WRITTEN - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

WRITER_SKIP_NULL_RECORD - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

WRITER_STAGING_DIR - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

WRITER_STAGING_TABLE - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

WRITER_TRUNCATE_STAGING_TABLE - Static variable in class org.apache.gobblin.configuration.ConfigurationKeys

writerAttemptId - Variable in class org.apache.gobblin.writer.DataWriterBuilder

writerId - Variable in class org.apache.gobblin.writer.DataWriterBuilder

WriterInitializer - Interface in org.apache.gobblin.writer.initializer

WriterOutputFormat - Enum in org.apache.gobblin.writer

An enumeration of writer output formats.

WriterWrapper<D> - Class in org.apache.gobblin.writer

An interface that should be used by all DataWriters that don't actually write records but instead wrap a different DataWriter.

WriterWrapper() - Constructor for class org.apache.gobblin.writer.WriterWrapper

writeStringAsText(DataOutput, String) - Static method in class org.apache.gobblin.compat.hadoop.TextSerializer

Serialize a String using the same logic as a Hadoop Text object

writeTo(Destination) - Method in class org.apache.gobblin.writer.DataWriterBuilder

Tell the writer the destination to write to.

writeTo(Destination) - Method in class org.apache.gobblin.writer.FluentDataWriterBuilder

Z

ZK_CONNECTION_STRING_KEY - Static variable in class org.apache.gobblin.service.ServiceConfigKeys

A B C D E F G H I J K L M N O P Q R S T U V W Z