Package org.apache.druid.catalog.model.table

Interface InputSourceDefn

  • All Known Implementing Classes:
    BaseInputSourceDefn, FormattedInputSourceDefn, HttpInputSourceDefn, InlineInputSourceDefn, LocalInputSourceDefn
    public interface InputSourceDefn
    Metadata definition for one Druid input source.

    This class models the merger of a catalog definition (where the input source is stored as serialized JSON), and table function parameters to produce a Druid input source. We use the following names:

    Property
    a top-level entry in the properties map of an (external) table specification. The entire JSON-serialized input source spec is a property. There may be others. Property names are defined by constants that end with _PROPERTY.
    Field
    The code often has the work with individual entries within the input source. To do that, we convert the input source to a Java map. Each entry in the map is a "field". Field names are implicit in the input source code. For convenience here, we declare constants for each field name, each ends with _FIELD.
    Parameter
    A SQL table function provides one or more named parameters. Each tends to map, directly or indirectly, to an input source field. Table parameter name constants end with _PARAMETER.
    Argument
    The actual value passed for a table function parameter for a specific query. Arguments correspond to a parameter. In Calcite, arguments are ordered. For convenience in the merging code, this layer converts the ordered parameter/argument lists into a map to allow simply by (parameter) name access.
    You will see constants that have the same value, but have each of the three suffixes, depending on how we've chosen to do the mapping for each input source. Since the input sources were not designed for this use case, we can't do a simple mapping: some creativity is required in each case.

    • Method Detail

      • bind

        void bind​(TableDefnRegistry registry)
        Gather information about the set of format definitions.

      • typeValue

        String typeValue()
        Type value for this format: same as the type string used in the serialized JSON for this input source. Used as the key for this definition within the table registry, and associates the serialized JSON with the corresponding input source definition.

      • validate

        void validate​(ResolvedExternalTable table)
        Given a external table catalog spec, with the JSON input source and format properties parsed to generic Java maps, validate that the properties are valid prior to saving the spec into the catalog.
        Parameters:
        table - a catalog table spec with the input source and input format properties parsed into generic Java maps
        Throws:
        org.apache.druid.java.util.common.IAE - if the spec properties are invalid

      • adHocTableFn

        TableFunction adHocTableFn()
        Provide a definition for a SQL table function that defines an ad-hoc external table "from scratch" for this input source. Typically defines parameters for the input source and all allowed input formats.
        Returns:
        a fully-defined external table to be handed off to the Calcite planner.
        Throws:
        org.apache.druid.java.util.common.IAE - if the function arguments are invalid

      • partialTableFn

        TableFunction partialTableFn​(ResolvedExternalTable table)
        Provide a definition for a SQL table function that completes a partial table spec from the catalog. Used when the spec represents a "partial table" or a "connection". The function provides parameters needed to complete the table (typically the set of input files, objects, etc.) If the catalog table spec does not provide a format, and this input source requires a format, then the parameters also include parameters for all supported input formats, same as for adHocTableFn().
        Parameters:
        table - a catalog table spec with the input source and input format properties parsed into generic Java maps
        Returns:
        a fully-defined external table to be handed off to the Calcite planner.
        Throws:
        org.apache.druid.java.util.common.IAE - if the function arguments are invalid

      • convertTable

        ExternalTableSpec convertTable​(ResolvedExternalTable table)
        Convert a complete (that is, fully-defined) table spec to an external table spec. Used when SQL references the catalog table directly by name in the FROM clause without using a table function.
        Returns:
        a fully-defined external table to be handed off to the Calcite planner.
        Throws:
        org.apache.druid.java.util.common.IAE - if the spec properties are invalid, or if the spec is partial and requires the use of a table function to complete