Package org.apache.druid.data.input.impl

Class JsonInputFormat

java.lang.Object

org.apache.druid.data.input.impl.NestedInputFormat

org.apache.druid.data.input.impl.JsonInputFormat

All Implemented Interfaces:

InputFormat

public class JsonInputFormat
extends NestedInputFormat

Field Summary

Fields

Modifier and Type	Field	Description
static String	TYPE_KEY

Fields inherited from interface org.apache.druid.data.input.InputFormat

TYPE_PROPERTY

Constructor Summary

Constructors

Constructor	Description
JsonInputFormat(JSONPathSpec flattenSpec, Map<String,Boolean> featureSpec, Boolean keepNullColumns, boolean lineSplittable, Boolean assumeNewlineDelimited, Boolean useJsonNodeReader)
JsonInputFormat(JSONPathSpec flattenSpec, Map<String,Boolean> featureSpec, Boolean keepNullColumns, Boolean assumeNewlineDelimited, Boolean useJsonNodeReader)

Method Summary

Modifier and Type	Method	Description
InputEntityReader	createReader(InputRowSchema inputRowSchema, InputEntity source, File temporaryDirectory)
boolean	equals(Object o)
Map<String,Boolean>	getFeatureSpec()
long	getWeightedSize(String path, long size)	Computes the weighted size of a given input object of the underyling input format type, weighted for its cost during ingestion.
int	hashCode()
boolean	isAssumeNewlineDelimited()
boolean	isKeepNullColumns()
boolean	isSplittable()	Trait to indicate that a file can be split into multiple InputSplits.
boolean	isUseJsonNodeReader()
String	toString()
JsonInputFormat	withLineSplittable(boolean lineSplittable)	Create a new JsonInputFormat object based on the given parameter sub-classes may need to override this method to create an object with correct sub-class type
static InputFormat	withLineSplittable(InputFormat format, boolean lineSplittable)	If the provided format is JsonInputFormat, return a version with withLineSplittable(boolean) called.

Methods inherited from class org.apache.druid.data.input.impl.NestedInputFormat

getFlattenSpec

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.druid.data.input.InputFormat

createRowAdapter

Field Detail

TYPE_KEY

public static final String TYPE_KEY

See Also:

Constant Field Values

Constructor Detail

JsonInputFormat

public JsonInputFormat(@Nullable
                       JSONPathSpec flattenSpec,
                       @Nullable
                       Map<String,Boolean> featureSpec,
                       @Nullable
                       Boolean keepNullColumns,
                       @Nullable
                       Boolean assumeNewlineDelimited,
                       @Nullable
                       Boolean useJsonNodeReader)

JsonInputFormat

public JsonInputFormat(@Nullable
                       JSONPathSpec flattenSpec,
                       Map<String,Boolean> featureSpec,
                       Boolean keepNullColumns,
                       boolean lineSplittable,
                       Boolean assumeNewlineDelimited,
                       Boolean useJsonNodeReader)

Method Detail

getFeatureSpec

public Map<String,Boolean> getFeatureSpec()

isKeepNullColumns

public boolean isKeepNullColumns()

isAssumeNewlineDelimited

public boolean isAssumeNewlineDelimited()

isUseJsonNodeReader

public boolean isUseJsonNodeReader()

isSplittable

public boolean isSplittable()

Description copied from interface: InputFormat

Trait to indicate that a file can be split into multiple InputSplits.

This method is not being used anywhere for now, but should be considered in SplittableInputSource.createSplits(org.apache.druid.data.input.InputFormat, org.apache.druid.data.input.SplitHintSpec) in the future.

createReader

public InputEntityReader createReader(InputRowSchema inputRowSchema,
                                      InputEntity source,
                                      File temporaryDirectory)

getWeightedSize

public long getWeightedSize(String path,
                            long size)

Description copied from interface: InputFormat

Computes the weighted size of a given input object of the underyling input format type, weighted for its cost during ingestion. The weight calculated is dependent on the format type and compression type (CompressionUtils.Format) used if any. Uncompressed newline delimited json is used as baseline with scale factor 1. This means that when computing the byte weight that an uncompressed newline delimited json input object has towards ingestion, we take the file size as is, 1:1.

Parameters:

path - The path of the input object. Used to tell whether any compression is used.

size - The size of the input object in bytes.

Returns:

The weighted size of the input object.

withLineSplittable

public JsonInputFormat withLineSplittable(boolean lineSplittable)

Create a new JsonInputFormat object based on the given parameter sub-classes may need to override this method to create an object with correct sub-class type

equals

public boolean equals(Object o)

Overrides:

equals in class NestedInputFormat

hashCode

public int hashCode()

Overrides:

hashCode in class NestedInputFormat

toString

public String toString()

Overrides:

toString in class Object

withLineSplittable

public static InputFormat withLineSplittable(InputFormat format,
                                             boolean lineSplittable)

If the provided format is JsonInputFormat, return a version with withLineSplittable(boolean) called. Otherwise return the provided format itself. This is a hack in order to get the same "json" input format to use JsonReader by default for streaming ingestion, and JsonLineReader by default for batch file-based ingestion.