public class HttpInputSourceDefn extends FormattedInputSourceDefn
Provides a parameterized form where the user defines a value for the
uriTemplate table property in the table spec, then provides the partial URLs
in a table function to use for that one query. The final URIs are created by combining
the template and the arguments. Example:
uriTemplate property): "http://example.com/data/kttm-{}.json"uris function argument: "22-Nov-21, 22-Nov-22"When the template is used, the format is optional: it can be provided either with the table spec or at runtime, depending on what the user wants to accomplish. In the above, where the ".json" is encoded in the template, it makes sense to include the format with the spec. If the template was "http://example.com/data/{}", and the data comes in multiple formats, it might make sense to specify the format in the query. In this case, the table spec acts more like a connection.
If the template is not used, then the uris property must be provided in the
table spec, along with the corresponding format.
The above semantics make a bit more sense when we realize that the spec can also provide a user name and password. When those are provided, then the input source must name a single site: the one for which the credentials are valid. Given this, the only table spec that makes sense is one where the URI is defined: either as a template or explicitly.
When used as an ad-hoc function, the user specifies the uris and optional user name and password: the template is not available (or useful) in the ad-hoc case.
Table function parameters are cleaned up relative to the input source field names to make them a bit easier to use.
BaseInputSourceDefn.AdHocTableFunction, BaseInputSourceDefn.PartialTableFunction| Modifier and Type | Field and Description |
|---|---|
static String |
PASSWORD_ENV_VAR_PARAMETER |
protected static String |
PASSWORD_FIELD |
static String |
PASSWORD_PARAMETER |
static String |
TYPE_KEY |
static String |
URI_TEMPLATE_PROPERTY |
protected static String |
URIS_FIELD |
static String |
URIS_PARAMETER |
static String |
USER_PARAMETER |
protected static String |
USERNAME_FIELD |
FORMAT_PARAMETER| Constructor and Description |
|---|
HttpInputSourceDefn() |
| Modifier and Type | Method and Description |
|---|---|
protected List<TableFunction.ParameterDefn> |
adHocTableFnParameters()
Overridden by subclasses to provide the list of table function parameters for
this specific input format.
|
protected void |
auditInputSource(Map<String,Object> jsonMap)
Optional step to audit or adjust the input source properties prior to
conversion via Jackson.
|
protected void |
convertArgsToSourceMap(Map<String,Object> jsonMap,
Map<String,Object> args)
Convert SQL arguments to the corresponding "generic JSON" form in the given map.
|
protected ExternalTableSpec |
convertCompletedTable(ResolvedExternalTable table,
Map<String,Object> args,
List<ColumnSpec> columns)
Complete a partial table using the table function arguments and columns provided.
|
protected Class<? extends InputSource> |
inputSourceClass()
Overridden by each subclass to return the input source class to be
used for JSON conversions.
|
TableFunction |
partialTableFn(ResolvedExternalTable table)
Provide a definition for a SQL table function that completes a partial table
spec from the catalog.
|
String |
typeValue()
Type value for this format: same as the type string used in the serialized
JSON for this input source.
|
void |
validate(ResolvedExternalTable table)
Given a external table catalog spec, with the JSON input source and format
properties parsed to generic Java maps, validate that the properties are
valid prior to saving the spec into the catalog.
|
addFormatParameters, bind, convertArgsToFormat, convertPartialFormattedTable, convertTableToFormat, defineAdHocTableFunctionadHocTableFn, convertArgsToSource, convertArgsToTable, convertSource, convertTable, convertTableToSource, selectPartialTableColumnspublic static final String TYPE_KEY
public static final String URI_TEMPLATE_PROPERTY
public static final String URIS_PARAMETER
public static final String USER_PARAMETER
public static final String PASSWORD_PARAMETER
public static final String PASSWORD_ENV_VAR_PARAMETER
protected static final String URIS_FIELD
protected static final String PASSWORD_FIELD
protected static final String USERNAME_FIELD
public String typeValue()
InputSourceDefnprotected Class<? extends InputSource> inputSourceClass()
BaseInputSourceDefninputSourceClass in class BaseInputSourceDefnpublic void validate(ResolvedExternalTable table)
InputSourceDefnvalidate in interface InputSourceDefnvalidate in class FormattedInputSourceDefntable - a catalog table spec with the input source and input format
properties parsed into generic Java mapsprotected void auditInputSource(Map<String,Object> jsonMap)
BaseInputSourceDefnjsonMap.auditInputSource in class BaseInputSourceDefnprotected List<TableFunction.ParameterDefn> adHocTableFnParameters()
FormattedInputSourceDefnadHocTableFnParameters in class FormattedInputSourceDefnprotected void convertArgsToSourceMap(Map<String,Object> jsonMap, Map<String,Object> args)
BaseInputSourceDefnconvertArgsToSourceMap in class BaseInputSourceDefnpublic TableFunction partialTableFn(ResolvedExternalTable table)
InputSourceDefnInputSourceDefn.adHocTableFn().table - a catalog table spec with the input source and input format
properties parsed into generic Java mapsprotected ExternalTableSpec convertCompletedTable(ResolvedExternalTable table, Map<String,Object> args, List<ColumnSpec> columns)
BaseInputSourceDefnEXTENDS clause: the implementation should decide
if columns are required (or allowed) depending on whether the partial spec already
defines columns.convertCompletedTable in class BaseInputSourceDefntable - the partial table spec, with input source and format parsed into a
generic Java mapargs - the argument values provided in the SQL table function call. The arguments
use the Java types defined in the parameter definitions.columns - the set of columns (if any) from the SQL EXTEND clauseCopyright © 2011–2023 The Apache Software Foundation. All rights reserved.