Package org.apache.pinot.spi.ingestion.batch.spec

Class SegmentGenerationJobSpec

  • All Implemented Interfaces:
    Serializable
    public class SegmentGenerationJobSpec
extends Object
implements Serializable
    SegmentGenerationJobSpec defines all the required information in order to kick off a Pinot data ingestion job.
    See Also:
    Serialized Form

    • Constructor Detail

      • SegmentGenerationJobSpec

        public SegmentGenerationJobSpec()

    • Method Detail

      • setExecutionFrameworkSpec

        public void setExecutionFrameworkSpec​(ExecutionFrameworkSpec executionFrameworkSpec)

      • getJobType

        public String getJobType()

      • setJobType

        public void setJobType​(String jobType)
        Set the job type for the ingestion job.
        Parameters:
        jobType - Job type for the ingestion job.

      • getInputDirURI

        public String getInputDirURI()

      • setInputDirURI

        public void setInputDirURI​(String inputDirURI)

      • isSearchRecursively

        public boolean isSearchRecursively()

      • setSearchRecursively

        public void setSearchRecursively​(boolean searchRecursively)

      • getIncludeFileNamePattern

        public String getIncludeFileNamePattern()

      • setIncludeFileNamePattern

        public void setIncludeFileNamePattern​(String includeFileNamePattern)
        include file name pattern, supported glob pattern. Sample usage: 'glob:*.avro' will include all avro files just under the inputDirURI, not sub directories; 'glob:**\/*.avro' will include all the avro files under inputDirURI recursively.
        Parameters:
        includeFileNamePattern -

      • getExcludeFileNamePattern

        public String getExcludeFileNamePattern()

      • setExcludeFileNamePattern

        public void setExcludeFileNamePattern​(String excludeFileNamePattern)
        exclude file name pattern, supported glob pattern. Sample usage: 'glob:*.avro' will exclude all avro files just under the inputDirURI, not sub directories; 'glob:**\/*.avro' will exclude all the avro files under inputDirURI recursively.
        Parameters:
        excludeFileNamePattern -

      • getOutputDirURI

        public String getOutputDirURI()

      • setOutputDirURI

        public void setOutputDirURI​(String outputDirURI)

      • isOverwriteOutput

        public boolean isOverwriteOutput()

      • setOverwriteOutput

        public void setOverwriteOutput​(boolean overwriteOutput)

      • setPinotFSSpecs

        public void setPinotFSSpecs​(List<PinotFSSpec> pinotFSSpecs)

      • getTableSpec

        public TableSpec getTableSpec()

      • setTableSpec

        public void setTableSpec​(TableSpec tableSpec)

      • setRecordReaderSpec

        public void setRecordReaderSpec​(RecordReaderSpec recordReaderSpec)

      • setPinotClusterSpecs

        public void setPinotClusterSpecs​(PinotClusterSpec[] pinotClusterSpecs)

      • setSegmentNameGeneratorSpec

        public void setSegmentNameGeneratorSpec​(SegmentNameGeneratorSpec segmentNameGeneratorSpec)

      • setPushJobSpec

        public void setPushJobSpec​(PushJobSpec pushJobSpec)

      • getSegmentCreationJobParallelism

        public int getSegmentCreationJobParallelism()

      • setSegmentCreationJobParallelism

        public void setSegmentCreationJobParallelism​(int segmentCreationJobParallelism)

      • setCleanUpOutputDir

        public void setCleanUpOutputDir​(boolean cleanUpOutputDir)

      • isCleanUpOutputDir

        public boolean isCleanUpOutputDir()

      • getTlsSpec

        public TlsSpec getTlsSpec()

      • setTlsSpec

        public void setTlsSpec​(TlsSpec tlsSpec)

      • isFailOnEmptySegment

        public boolean isFailOnEmptySegment()

      • setFailOnEmptySegment

        public void setFailOnEmptySegment​(boolean failOnEmptySegment)

      • getAuthToken

        public String getAuthToken()

      • setAuthToken

        public void setAuthToken​(String authToken)

      • toJSONString

        public String toJSONString​(boolean removeSensitiveKeys)