Package org.apache.pinot.spi.utils

Class IngestionConfigUtils

  • public final class IngestionConfigUtils
extends Object
    Helper methods for extracting fields from IngestionConfig in a backward compatible manner

    • Method Detail

      • getStreamConfigMap

        public static Map<String,​String> getStreamConfigMap​(TableConfig tableConfig)
        Fetches the streamConfig from the given realtime table. First, the ingestionConfigs->stream->streamConfigs will be checked. If not found, the indexingConfig->streamConfigs will be checked (which is deprecated).
        Parameters:
        tableConfig - realtime table config
        Returns:
        streamConfigs map

      • getBatchSegmentIngestionConsistentDataPushEnabled

        public static boolean getBatchSegmentIngestionConsistentDataPushEnabled​(TableConfig tableConfig)
        Fetches the configured consistentDataPush boolean from the table config

      • getBatchSegmentIngestionType

        public static String getBatchSegmentIngestionType​(TableConfig tableConfig)
        Fetches the configured segmentIngestionType (APPEND/REFRESH) from the table config First checks in the ingestionConfig. If not found, checks in the segmentsConfig (has been deprecated from here in favor of ingestion config)

      • getBatchSegmentIngestionFrequency

        public static String getBatchSegmentIngestionFrequency​(TableConfig tableConfig)
        Fetches the configured segmentIngestionFrequency from the table config First checks in the ingestionConfig. If not found, checks in the segmentsConfig (has been deprecated from here in favor of ingestion config)

      • getRecordReaderProps

        public static Map<String,​String> getRecordReaderProps​(Map<String,​String> batchConfigMap)
        Fetch the properties which belong to record reader, by removing the identifier prefix

      • getSegmentNameGeneratorProps

        public static Map<String,​String> getSegmentNameGeneratorProps​(Map<String,​String> batchConfigMap)
        Fetch the properties which belong to segment name generator, by removing the identifier prefix

      • extractPropsMatchingPrefix

        public static Map<String,​String> extractPropsMatchingPrefix​(Map<String,​String> batchConfigMap,
                                                                  String prefix)
        Extracts entries where keys start with given prefix

      • getSegmentNameGeneratorType

        public static String getSegmentNameGeneratorType​(Map<String,​String> batchConfigMap)
        Extracts the segment name generator type from the batchConfigMap, or returns default value if not found

      • getPushMode

        public static String getPushMode​(Map<String,​String> batchConfigMap)
        Extracts the push mode from the batchConfigMap, or returns default value if not found

      • getPushAttempts

        public static int getPushAttempts​(Map<String,​String> batchConfigMap)
        Extracts the push attempts from the batchConfigMap, or returns default value if not found

      • getPushParallelism

        public static int getPushParallelism​(Map<String,​String> batchConfigMap)
        Extracts the push parallelism from the batchConfigMap, or returns default value if not found

      • getPushRetryIntervalMillis

        public static long getPushRetryIntervalMillis​(Map<String,​String> batchConfigMap)
        Extracts the push return interval millis from the batchConfigMap, or returns default value if not found