org.apache.beam.fn.harness

Class ExternalWorkerService

java.lang.Object

org.apache.beam.model.fnexecution.v1.BeamFnExternalWorkerPoolGrpc.BeamFnExternalWorkerPoolImplBase

org.apache.beam.fn.harness.ExternalWorkerService

All Implemented Interfaces:

java.lang.AutoCloseable, org.apache.beam.model.fnexecution.v1.BeamFnExternalWorkerPoolGrpc.AsyncService, org.apache.beam.sdk.fn.server.FnService, org.apache.beam.vendor.grpc.v1p60p1.io.grpc.BindableService

public class ExternalWorkerService
extends org.apache.beam.model.fnexecution.v1.BeamFnExternalWorkerPoolGrpc.BeamFnExternalWorkerPoolImplBase
implements org.apache.beam.sdk.fn.server.FnService

Implements the BeamFnExternalWorkerPool service by starting a fresh SDK harness for each request.

Constructor Summary

Constructors

Constructor and Description
ExternalWorkerService(org.apache.beam.sdk.options.PipelineOptions options)

Method Summary

Modifier and Type	Method and Description
void	close()
static void	main(java.lang.String[] args) Worker pool entry point.
org.apache.beam.sdk.fn.server.GrpcFnServer<ExternalWorkerService>	start()
void	startWorker(org.apache.beam.model.fnexecution.v1.BeamFnApi.StartWorkerRequest request, org.apache.beam.vendor.grpc.v1p60p1.io.grpc.stub.StreamObserver<org.apache.beam.model.fnexecution.v1.BeamFnApi.StartWorkerResponse> responseObserver)
void	stopWorker(org.apache.beam.model.fnexecution.v1.BeamFnApi.StopWorkerRequest request, org.apache.beam.vendor.grpc.v1p60p1.io.grpc.stub.StreamObserver<org.apache.beam.model.fnexecution.v1.BeamFnApi.StopWorkerResponse> responseObserver)

Methods inherited from class org.apache.beam.model.fnexecution.v1.BeamFnExternalWorkerPoolGrpc.BeamFnExternalWorkerPoolImplBase

bindService

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.beam.vendor.grpc.v1p60p1.io.grpc.BindableService

bindService

Constructor Detail

ExternalWorkerService

public ExternalWorkerService(org.apache.beam.sdk.options.PipelineOptions options)

Method Detail

startWorker

public void startWorker(org.apache.beam.model.fnexecution.v1.BeamFnApi.StartWorkerRequest request,
                        org.apache.beam.vendor.grpc.v1p60p1.io.grpc.stub.StreamObserver<org.apache.beam.model.fnexecution.v1.BeamFnApi.StartWorkerResponse> responseObserver)

Specified by:

startWorker in interface org.apache.beam.model.fnexecution.v1.BeamFnExternalWorkerPoolGrpc.AsyncService

stopWorker

public void stopWorker(org.apache.beam.model.fnexecution.v1.BeamFnApi.StopWorkerRequest request,
                       org.apache.beam.vendor.grpc.v1p60p1.io.grpc.stub.StreamObserver<org.apache.beam.model.fnexecution.v1.BeamFnApi.StopWorkerResponse> responseObserver)

Specified by:

stopWorker in interface org.apache.beam.model.fnexecution.v1.BeamFnExternalWorkerPoolGrpc.AsyncService

close

public void close()

Specified by:

close in interface java.lang.AutoCloseable

Specified by:

close in interface org.apache.beam.sdk.fn.server.FnService

start

public org.apache.beam.sdk.fn.server.GrpcFnServer<ExternalWorkerService> start()
                                                                        throws java.lang.Exception

Throws:

java.lang.Exception

main

public static void main(java.lang.String[] args)

Worker pool entry point.

The worker pool exposes an RPC service that is used with EXTERNAL environment to start and stop the SDK workers.

The worker pool uses threads for parallelism;

This entry point is used by the Java SDK container in worker pool mode and expects the following environment variables:

• PIPELINE_OPTIONS: A serialized form of PipelineOptions. It needs to be known up-front and matches the running job. See PipelineOptions for further details.