com.netflix.genie.web.configs

Class GenieServicesAutoConfiguration

java.lang.Object

com.netflix.genie.web.configs.GenieServicesAutoConfiguration

@Configuration
 @EnableConfigurationProperties(value={DataServiceRetryProperties.class,FileCacheProperties.class,HealthProperties.class,JobsCleanupProperties.class,JobsForwardingProperties.class,JobsLocationsProperties.class,JobsMaxProperties.class,JobsMemoryProperties.class,JobsUsersProperties.class,ExponentialBackOffTriggerProperties.class,JobsActiveLimitProperties.class})
public class GenieServicesAutoConfiguration
extends java.lang.Object

Configuration for all the services.

Since:

3.0.0

Constructor Summary

Constructors

Constructor and Description
GenieServicesAutoConfiguration()

Method Summary

Modifier and Type	Method and Description
AgentFilterService	agentFilterService() Get a NOOP/fallback AgentFilterService instance if there isn't already one.
AgentJobService	agentJobService(JobPersistenceService jobPersistenceService, JobSpecificationService jobSpecificationService, AgentFilterService agentFilterService, io.micrometer.core.instrument.MeterRegistry meterRegistry) Get a AgentJobService instance if there isn't already one.
AgentMetricsServiceImpl	agentMetricsService(com.netflix.genie.common.internal.util.GenieHostInfo genieHostInfo, AgentConnectionPersistenceService agentConnectionPersistenceService, io.micrometer.core.instrument.MeterRegistry registry) Provide an implementation of AgentMetricsService if one hasn't been provided.
AgentRoutingService	agentRoutingService(AgentConnectionPersistenceService agentConnectionPersistenceService, com.netflix.genie.common.internal.util.GenieHostInfo genieHostInfo) Get an implementation of AgentRoutingService if one hasn't already been defined.
AttachmentService	attachmentService(JobsProperties jobsProperties) The attachment service to use.
GenieFileTransferService	cacheGenieFileTransferService(FileTransferFactory fileTransferFactory, FileCacheProperties fileCacheProperties, LocalFileTransferImpl localFileTransfer, io.micrometer.core.instrument.MeterRegistry registry) Get an instance of the Cache Genie File Transfer service.
AgentFileStreamService	fallbackAgentFileStreamService() Get a fallback implementation of AgentFileStreamService in case gRPC is disabled.
JobKillServiceV4	fallbackJobKillServiceV4() Get a fallback implementation of JobKillServiceV4 in case gRPC is disabled.
org.springframework.beans.factory.config.ServiceLocatorFactoryBean	fileTransferFactory() FileTransfer factory.
GenieFileTransferService	genieFileTransferService(FileTransferFactory fileTransferFactory) Get an instance of the Genie File Transfer service.
JobCompletionService	jobCompletionService(JobPersistenceService jobPersistenceService, JobSearchService jobSearchService, com.netflix.genie.common.internal.services.JobArchiveService jobArchiveService, org.springframework.core.io.Resource genieWorkingDir, MailService mailService, io.micrometer.core.instrument.MeterRegistry registry, JobsProperties jobsProperties, org.springframework.retry.support.RetryTemplate retryTemplate) Get an implementation of JobCompletionService if one hasn't already been defined.
JobCoordinatorService	jobCoordinatorService(JobPersistenceService jobPersistenceService, JobKillService jobKillService, JobStateService jobStateService, JobSearchService jobSearchService, JobsProperties jobsProperties, ApplicationPersistenceService applicationPersistenceService, ClusterPersistenceService clusterPersistenceService, CommandPersistenceService commandPersistenceService, JobSpecificationService specificationService, io.micrometer.core.instrument.MeterRegistry registry, com.netflix.genie.common.internal.util.GenieHostInfo genieHostInfo) Get an instance of the JobCoordinatorService.
JobDirectoryServerService	jobDirectoryServerService(org.springframework.core.io.ResourceLoader resourceLoader, JobPersistenceService jobPersistenceService, JobFileService jobFileService, AgentFileStreamService agentFileStreamService, io.micrometer.core.instrument.MeterRegistry meterRegistry) Provide the default implementation of JobDirectoryServerService for serving job directory resources.
JobFileService	jobFileService(org.springframework.core.io.Resource jobsDir) Get a JobFileService implementation if one is required.
JobKillService	jobKillService(JobKillServiceV3 jobKillServiceV3, JobKillServiceV4 jobKillServiceV4, JobPersistenceService jobPersistenceService) Get an local implementation of the JobKillService.
JobKillServiceV3	jobKillServiceV3(com.netflix.genie.common.internal.util.GenieHostInfo genieHostInfo, JobSearchService jobSearchService, org.apache.commons.exec.Executor executor, JobsProperties jobsProperties, GenieEventBus genieEventBus, org.springframework.core.io.Resource genieWorkingDir, com.fasterxml.jackson.databind.ObjectMapper objectMapper, ProcessChecker.Factory processCheckerFactory) Get an local implementation of the JobKillService.
JobSpecificationService	jobSpecificationService(ApplicationPersistenceService applicationPersistenceService, ClusterPersistenceService clusterPersistenceService, CommandPersistenceService commandPersistenceService, @NotEmpty java.util.List<ClusterLoadBalancer> clusterLoadBalancers, io.micrometer.core.instrument.MeterRegistry registry, JobsProperties jobsProperties) Get an implementation of JobSpecificationService if one hasn't already been defined.
JobsProperties	jobsProperties(JobsCleanupProperties cleanup, JobsForwardingProperties forwarding, JobsLocationsProperties locations, JobsMaxProperties max, JobsMemoryProperties memory, JobsUsersProperties users, ExponentialBackOffTriggerProperties completionCheckBackOff, JobsActiveLimitProperties activeLimit) Collection of properties related to job execution.
JobSubmitterService	jobSubmitterService(JobPersistenceService jobPersistenceService, GenieEventBus genieEventBus, java.util.List<WorkflowTask> workflowTasks, org.springframework.core.io.Resource genieWorkingDir, io.micrometer.core.instrument.MeterRegistry registry) Get a implementation of the JobSubmitterService that runs jobs locally.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

GenieServicesAutoConfiguration

public GenieServicesAutoConfiguration()

Method Detail

jobsProperties

@Bean
public JobsProperties jobsProperties(JobsCleanupProperties cleanup,
                                           JobsForwardingProperties forwarding,
                                           JobsLocationsProperties locations,
                                           JobsMaxProperties max,
                                           JobsMemoryProperties memory,
                                           JobsUsersProperties users,
                                           ExponentialBackOffTriggerProperties completionCheckBackOff,
                                           JobsActiveLimitProperties activeLimit)

Collection of properties related to job execution.

Parameters:

cleanup - cleanup properties

forwarding - forwarding properties

locations - locations properties

max - max properties

memory - memory properties

users - users properties

completionCheckBackOff - completion back-off properties

activeLimit - active limit properties

Returns:

a JobsProperties instance

jobKillServiceV3

@Bean
 @ConditionalOnMissingBean(value=JobKillServiceV3.class)
public JobKillServiceV3 jobKillServiceV3(com.netflix.genie.common.internal.util.GenieHostInfo genieHostInfo,
                                                                                                        JobSearchService jobSearchService,
                                                                                                        org.apache.commons.exec.Executor executor,
                                                                                                        JobsProperties jobsProperties,
                                                                                                        GenieEventBus genieEventBus,
                                                                                                        @Qualifier(value="jobsDir")
                                                                                                        org.springframework.core.io.Resource genieWorkingDir,
                                                                                                        com.fasterxml.jackson.databind.ObjectMapper objectMapper,
                                                                                                        ProcessChecker.Factory processCheckerFactory)

Get an local implementation of the JobKillService.

Parameters:

genieHostInfo - Information about the host the Genie process is running on

jobSearchService - The job search service to use to locate job information.

executor - The executor to use to run system processes.

jobsProperties - The jobs properties to use

genieEventBus - The application event bus to use to publish system wide events

genieWorkingDir - Working directory for genie where it creates jobs directories.

objectMapper - The Jackson ObjectMapper used to serialize from/to JSON

processCheckerFactory - The process checker factory

Returns:

A job kill service instance.

jobKillService

@Bean
 @ConditionalOnMissingBean(value=JobKillService.class)
public JobKillService jobKillService(JobKillServiceV3 jobKillServiceV3,
                                                                                                  JobKillServiceV4 jobKillServiceV4,
                                                                                                  JobPersistenceService jobPersistenceService)

Get an local implementation of the JobKillService.

Parameters:

jobKillServiceV3 - Service to kill V3 jobs.

jobKillServiceV4 - Service to kill V4 jobs.

jobPersistenceService - Job persistence service

Returns:

A job kill service instance.

fallbackJobKillServiceV4

@Bean
 @ConditionalOnMissingBean(value=JobKillServiceV4.class)
public JobKillServiceV4 fallbackJobKillServiceV4()

Get a fallback implementation of JobKillServiceV4 in case gRPC is disabled.

Returns:

a placeholder V4 job kill service for tests.

genieFileTransferService

@Bean
 @ConditionalOnMissingBean(name="genieFileTransferService")
public GenieFileTransferService genieFileTransferService(FileTransferFactory fileTransferFactory)
                                                                                                                    throws com.netflix.genie.common.exceptions.GenieException

Get an instance of the Genie File Transfer service.

Parameters:

fileTransferFactory - file transfer implementation factory

Returns:

A singleton for GenieFileTransferService

Throws:

com.netflix.genie.common.exceptions.GenieException - If there is any problem

cacheGenieFileTransferService

@Bean
 @ConditionalOnMissingBean(name="cacheGenieFileTransferService")
public GenieFileTransferService cacheGenieFileTransferService(FileTransferFactory fileTransferFactory,
                                                                                                                                     FileCacheProperties fileCacheProperties,
                                                                                                                                     LocalFileTransferImpl localFileTransfer,
                                                                                                                                     io.micrometer.core.instrument.MeterRegistry registry)
                                                                                                                              throws com.netflix.genie.common.exceptions.GenieException

Get an instance of the Cache Genie File Transfer service.

Parameters:

fileTransferFactory - file transfer implementation factory

fileCacheProperties - Properties related to the file cache that can be set by the admin

localFileTransfer - local file transfer service

registry - Registry

Returns:

A singleton for GenieFileTransferService

Throws:

com.netflix.genie.common.exceptions.GenieException - If there is any problem

jobSubmitterService

@Bean
 @ConditionalOnMissingBean(value=JobSubmitterService.class)
public JobSubmitterService jobSubmitterService(JobPersistenceService jobPersistenceService,
                                                                                                                 GenieEventBus genieEventBus,
                                                                                                                 java.util.List<WorkflowTask> workflowTasks,
                                                                                                                 @Qualifier(value="jobsDir")
                                                                                                                 org.springframework.core.io.Resource genieWorkingDir,
                                                                                                                 io.micrometer.core.instrument.MeterRegistry registry)

Get a implementation of the JobSubmitterService that runs jobs locally.

Parameters:

jobPersistenceService - Implementation of the job persistence service.

genieEventBus - The genie event bus implementation to use

workflowTasks - List of all the workflow tasks to be executed.

genieWorkingDir - Working directory for genie where it creates jobs directories.

registry - The metrics registry to use

Returns:

An instance of the JobSubmitterService.

jobCoordinatorService

@Bean
 @ConditionalOnMissingBean(value=JobCoordinatorService.class)
public JobCoordinatorService jobCoordinatorService(JobPersistenceService jobPersistenceService,
                                                                                                                       JobKillService jobKillService,
                                                                                                                       @Qualifier(value="jobMonitoringCoordinator")
                                                                                                                       JobStateService jobStateService,
                                                                                                                       JobSearchService jobSearchService,
                                                                                                                       JobsProperties jobsProperties,
                                                                                                                       ApplicationPersistenceService applicationPersistenceService,
                                                                                                                       ClusterPersistenceService clusterPersistenceService,
                                                                                                                       CommandPersistenceService commandPersistenceService,
                                                                                                                       JobSpecificationService specificationService,
                                                                                                                       io.micrometer.core.instrument.MeterRegistry registry,
                                                                                                                       com.netflix.genie.common.internal.util.GenieHostInfo genieHostInfo)

Get an instance of the JobCoordinatorService.

Parameters:

jobPersistenceService - implementation of job persistence service interface

jobKillService - The job kill service to use

jobStateService - The running job metrics service to use

jobSearchService - Implementation of job search service interface

jobsProperties - The jobs properties to use

applicationPersistenceService - Implementation of application service interface

clusterPersistenceService - Implementation of cluster service interface

commandPersistenceService - Implementation of command service interface

specificationService - The job specification service to use

registry - The metrics registry to use

genieHostInfo - Information about the host the Genie process is running on

Returns:

An instance of the JobCoordinatorService.

attachmentService

@Bean
 @ConditionalOnMissingBean(value=AttachmentService.class)
public AttachmentService attachmentService(JobsProperties jobsProperties)

The attachment service to use.

Parameters:

jobsProperties - All properties related to jobs

Returns:

The attachment service to use

fileTransferFactory

@Bean
 @ConditionalOnMissingBean(name="fileTransferFactory",
                          value=org.springframework.beans.factory.config.ServiceLocatorFactoryBean.class)
public org.springframework.beans.factory.config.ServiceLocatorFactoryBean fileTransferFactory()

FileTransfer factory.

Returns:

FileTransfer factory

agentJobService

@Bean
 @ConditionalOnMissingBean(value=AgentJobService.class)
public AgentJobService agentJobService(JobPersistenceService jobPersistenceService,
                                                                                                     JobSpecificationService jobSpecificationService,
                                                                                                     AgentFilterService agentFilterService,
                                                                                                     io.micrometer.core.instrument.MeterRegistry meterRegistry)

Get a AgentJobService instance if there isn't already one.

Parameters:

jobPersistenceService - The persistence service to use

jobSpecificationService - The specification service to use

agentFilterService - The agent filter service to use

meterRegistry - The metrics registry to use

Returns:

An AgentJobServiceImpl instance.

jobFileService

@Bean
 @ConditionalOnMissingBean(value=JobFileService.class)
public JobFileService jobFileService(@Qualifier(value="jobsDir")
                                                                                                  org.springframework.core.io.Resource jobsDir)
                                                                                           throws java.io.IOException

Get a JobFileService implementation if one is required.

Parameters:

jobsDir - The job directory resource

Returns:

A DiskJobFileServiceImpl instance

Throws:

java.io.IOException - When the job directory can't be created or isn't a directory

jobSpecificationService

@Bean
 @ConditionalOnMissingBean(value=JobSpecificationService.class)
public JobSpecificationService jobSpecificationService(ApplicationPersistenceService applicationPersistenceService,
                                                                                                                             ClusterPersistenceService clusterPersistenceService,
                                                                                                                             CommandPersistenceService commandPersistenceService,
                                                                                                                             @NotEmpty
                                                                                                                             @NotEmpty java.util.List<ClusterLoadBalancer> clusterLoadBalancers,
                                                                                                                             io.micrometer.core.instrument.MeterRegistry registry,
                                                                                                                             JobsProperties jobsProperties)

Get an implementation of JobSpecificationService if one hasn't already been defined.

Parameters:

applicationPersistenceService - The service to use to manipulate applications

clusterPersistenceService - The service to use to manipulate clusters

commandPersistenceService - The service to use to manipulate commands

clusterLoadBalancers - The load balancer implementations to use

registry - The metrics repository to use

jobsProperties - The properties for running a job set by the user

Returns:

A JobSpecificationServiceImpl instance

agentRoutingService

@Bean
 @ConditionalOnMissingBean(value=AgentRoutingService.class)
public AgentRoutingService agentRoutingService(AgentConnectionPersistenceService agentConnectionPersistenceService,
                                                                                                                 com.netflix.genie.common.internal.util.GenieHostInfo genieHostInfo)

Get an implementation of AgentRoutingService if one hasn't already been defined.

Parameters:

agentConnectionPersistenceService - The persistence service to use for agent connections

genieHostInfo - The local genie host information

Returns:

A AgentRoutingServiceImpl instance

jobCompletionService

@Bean
 @ConditionalOnMissingBean(value=JobCompletionService.class)
public JobCompletionService jobCompletionService(JobPersistenceService jobPersistenceService,
                                                                                                                    JobSearchService jobSearchService,
                                                                                                                    com.netflix.genie.common.internal.services.JobArchiveService jobArchiveService,
                                                                                                                    @Qualifier(value="jobsDir")
                                                                                                                    org.springframework.core.io.Resource genieWorkingDir,
                                                                                                                    MailService mailService,
                                                                                                                    io.micrometer.core.instrument.MeterRegistry registry,
                                                                                                                    JobsProperties jobsProperties,
                                                                                                                    @Qualifier(value="genieRetryTemplate")
                                                                                                                    org.springframework.retry.support.RetryTemplate retryTemplate)
                                                                                                             throws com.netflix.genie.common.exceptions.GenieException

Get an implementation of JobCompletionService if one hasn't already been defined.

Parameters:

jobPersistenceService - The job persistence service to use

jobSearchService - The job search service to use

jobArchiveService - The JobArchiveService implementation to use

genieWorkingDir - Working directory for genie where it creates jobs directories.

mailService - The mail service

registry - Registry

jobsProperties - The jobs properties to use

retryTemplate - The retry template

Returns:

an instance of JobCompletionService

Throws:

com.netflix.genie.common.exceptions.GenieException - if the bean fails during construction

agentFilterService

@Bean
 @ConditionalOnMissingBean(value=AgentFilterService.class)
public AgentFilterService agentFilterService()

Get a NOOP/fallback AgentFilterService instance if there isn't already one.

Returns:

An AgentFilterService instance.

See Also:

GenieAgentFilterAutoConfiguration

jobDirectoryServerService

@Bean
 @ConditionalOnMissingBean(value=JobDirectoryServerService.class)
public JobDirectoryServerService jobDirectoryServerService(org.springframework.core.io.ResourceLoader resourceLoader,
                                                                                                                                   JobPersistenceService jobPersistenceService,
                                                                                                                                   JobFileService jobFileService,
                                                                                                                                   AgentFileStreamService agentFileStreamService,
                                                                                                                                   io.micrometer.core.instrument.MeterRegistry meterRegistry)

Provide the default implementation of JobDirectoryServerService for serving job directory resources.

Parameters:

resourceLoader - The application resource loader used to get references to resources

jobPersistenceService - The job persistence service used to get information about a job

jobFileService - The service responsible for managing the job working directory on disk for V3 Jobs

agentFileStreamService - The service to request a file from an agent running a job

meterRegistry - The meter registry used to keep track of metrics

Returns:

An instance of JobDirectoryServerServiceImpl

fallbackAgentFileStreamService

@Bean
 @ConditionalOnMissingBean(value=AgentFileStreamService.class)
public AgentFileStreamService fallbackAgentFileStreamService()

Get a fallback implementation of AgentFileStreamService in case gRPC is disabled.

Returns:

a placeholder agent file stream service for tests.

agentMetricsService

@Bean
 @ConditionalOnMissingBean(value=AgentMetricsService.class)
public AgentMetricsServiceImpl agentMetricsService(com.netflix.genie.common.internal.util.GenieHostInfo genieHostInfo,
                                                                                                                     AgentConnectionPersistenceService agentConnectionPersistenceService,
                                                                                                                     io.micrometer.core.instrument.MeterRegistry registry)

Provide an implementation of AgentMetricsService if one hasn't been provided.

Parameters:

genieHostInfo - The Genie host information

agentConnectionPersistenceService - Implementation of AgentConnectionPersistenceService to get information about running agents in the ecosystem

registry - The metrics repository

Returns:

An instance of AgentMetricsServiceImpl