org.apache.beam.sdk.extensions.python.transforms

Class RunInference<OutputT>

java.lang.Object

org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PCollection<?>,org.apache.beam.sdk.values.PCollection<OutputT>>

org.apache.beam.sdk.extensions.python.transforms.RunInference<OutputT>

All Implemented Interfaces:

java.io.Serializable, org.apache.beam.sdk.transforms.display.HasDisplayData

public class RunInference<OutputT>
extends org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PCollection<?>,org.apache.beam.sdk.values.PCollection<OutputT>>

Wrapper for invoking external Python RunInference.

See Also:

Serialized Form

Field Summary

Fields inherited from class org.apache.beam.sdk.transforms.PTransform

annotations, displayData, name, resourceHints

Method Summary

Modifier and Type	Method and Description
org.apache.beam.sdk.values.PCollection<OutputT>	expand(org.apache.beam.sdk.values.PCollection<?> input)
static RunInference<org.apache.beam.sdk.values.Row>	of(java.lang.String modelLoader, org.apache.beam.sdk.schemas.Schema.FieldType exampleType, org.apache.beam.sdk.schemas.Schema.FieldType inferenceType) Instantiates a multi-language wrapper for a Python RunInference with a given model loader.
static RunInference<org.apache.beam.sdk.values.Row>	of(java.lang.String modelLoader, org.apache.beam.sdk.schemas.Schema schema) Instantiates a multi-language wrapper for a Python RunInference with a given model loader.
static <KeyT> RunInference<org.apache.beam.sdk.values.KV<KeyT,org.apache.beam.sdk.values.Row>>	ofKVs(java.lang.String modelLoader, org.apache.beam.sdk.schemas.Schema.FieldType exampleType, org.apache.beam.sdk.schemas.Schema.FieldType inferenceType, org.apache.beam.sdk.coders.Coder<KeyT> keyCoder) Similar to RunInference#of(String, FieldType, FieldType) but the input is a PCollection of KVs.
static <KeyT> RunInference<org.apache.beam.sdk.values.KV<KeyT,org.apache.beam.sdk.values.Row>>	ofKVs(java.lang.String modelLoader, org.apache.beam.sdk.schemas.Schema schema, org.apache.beam.sdk.coders.Coder<KeyT> keyCoder) Similar to of(String, Schema) but the input is a PCollection of KVs.
RunInference<OutputT>	withExpansionService(java.lang.String expansionService) Sets an expansion service endpoint for RunInference.
RunInference<OutputT>	withExtraPackages(java.util.List<java.lang.String> extraPackages) Specifies any extra packages required by the RunInference model handler.
RunInference<OutputT>	withKwarg(java.lang.String key, java.lang.Object arg) Sets keyword arguments for the model loader.

Methods inherited from class org.apache.beam.sdk.transforms.PTransform

addAnnotation, compose, compose, getAdditionalInputs, getAnnotations, getDefaultOutputCoder, getDefaultOutputCoder, getDefaultOutputCoder, getKindString, getName, getResourceHints, populateDisplayData, setDisplayData, setResourceHints, toString, validate, validate

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Method Detail

of

public static RunInference<org.apache.beam.sdk.values.Row> of(java.lang.String modelLoader,
                                                              org.apache.beam.sdk.schemas.Schema.FieldType exampleType,
                                                              org.apache.beam.sdk.schemas.Schema.FieldType inferenceType)

Instantiates a multi-language wrapper for a Python RunInference with a given model loader.

Parameters:

modelLoader - A Python callable for a model loader class object.

exampleType - A schema field type for the example column in output rows.

inferenceType - A schema field type for the inference column in output rows.

Returns:

A RunInference for the given model loader.

ofKVs

public static <KeyT> RunInference<org.apache.beam.sdk.values.KV<KeyT,org.apache.beam.sdk.values.Row>> ofKVs(java.lang.String modelLoader,
                                                                                                            org.apache.beam.sdk.schemas.Schema.FieldType exampleType,
                                                                                                            org.apache.beam.sdk.schemas.Schema.FieldType inferenceType,
                                                                                                            org.apache.beam.sdk.coders.Coder<KeyT> keyCoder)

Similar to RunInference#of(String, FieldType, FieldType) but the input is a PCollection of KVs.

Also outputs a PCollection of KVs of the same key type.

For example, use this if you are using Python KeyedModelHandler as the model handler.

Type Parameters:

KeyT - input and output Key type. Inferred by the provided coder.

Parameters:

modelLoader - A Python callable for a model loader class object.

exampleType - A schema field type for the example column in output rows.

inferenceType - A schema field type for the inference column in output rows.

keyCoder - a Coder for the input and output Key type.

Returns:

A RunInference for the given model loader.

of

public static RunInference<org.apache.beam.sdk.values.Row> of(java.lang.String modelLoader,
                                                              org.apache.beam.sdk.schemas.Schema schema)

Instantiates a multi-language wrapper for a Python RunInference with a given model loader.

Parameters:

modelLoader - A Python callable for a model loader class object.

schema - A schema for output rows.

Returns:

A RunInference for the given model loader.

ofKVs

public static <KeyT> RunInference<org.apache.beam.sdk.values.KV<KeyT,org.apache.beam.sdk.values.Row>> ofKVs(java.lang.String modelLoader,
                                                                                                            org.apache.beam.sdk.schemas.Schema schema,
                                                                                                            org.apache.beam.sdk.coders.Coder<KeyT> keyCoder)

Similar to of(String, Schema) but the input is a PCollection of KVs.

Type Parameters:

KeyT - input and output Key type. Inferred by the provided coder.

Parameters:

modelLoader - A Python callable for a model loader class object.

schema - A schema for output rows.

keyCoder - a Coder for the input and output Key type.

Returns:

A RunInference for the given model loader.

withKwarg

public RunInference<OutputT> withKwarg(java.lang.String key,
                                       java.lang.Object arg)

Sets keyword arguments for the model loader.

Returns:

A RunInference with keyword arguments.

withExtraPackages

public RunInference<OutputT> withExtraPackages(java.util.List<java.lang.String> extraPackages)

Specifies any extra packages required by the RunInference model handler.

This should only be specified when using the default expansion service, i.e. when not using withExpansionService(String) to provide an expansion service.

The package can either be a PyPi package or the path to a locally available Python package.

For model handlers provided by Beam Python SDK, the implementation will automatically try to infer correct packages needed, so this may be omitted.

Parameters:

extraPackages - a list of PyPi packages. May include the version.

Returns:

A RunInference with extra packages.

withExpansionService

public RunInference<OutputT> withExpansionService(java.lang.String expansionService)

Sets an expansion service endpoint for RunInference.

Parameters:

expansionService - A URL for a Python expansion service.

Returns:

A RunInference for the given expansion service endpoint.

expand

public org.apache.beam.sdk.values.PCollection<OutputT> expand(org.apache.beam.sdk.values.PCollection<?> input)

Specified by:

expand in class org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PCollection<?>,org.apache.beam.sdk.values.PCollection<OutputT>>