com.netflix.genie.web.jpa.services

Class JpaJobPersistenceServiceImpl

java.lang.Object

com.netflix.genie.web.jpa.services.JpaJobPersistenceServiceImpl

All Implemented Interfaces:

JobPersistenceService

@Transactional(rollbackFor={com.netflix.genie.common.exceptions.GenieException.class,com.netflix.genie.common.internal.exceptions.unchecked.GenieRuntimeException.class,javax.validation.ConstraintViolationException.class})
public class JpaJobPersistenceServiceImpl
extends java.lang.Object
implements JobPersistenceService

JPA implementation of the job persistence service.

Constructor Summary

Constructors

Constructor and Description
JpaJobPersistenceServiceImpl(JpaTagPersistenceService tagPersistenceService, JpaFilePersistenceService filePersistenceService, JpaApplicationRepository applicationRepository, JpaClusterRepository clusterRepository, JpaCommandRepository commandRepository, JpaJobRepository jobRepository) Constructor.

Method Summary

Modifier and Type	Method and Description
void	claimJob(@NotBlank(message="Job id is missing and is required") java.lang.String id, @Valid com.netflix.genie.common.internal.dto.v4.AgentClientMetadata agentClientMetadata) Set a job identified by id to be owned by the agent identified by agentClientMetadata.
void	createJob(com.netflix.genie.common.dto.JobRequest jobRequest, com.netflix.genie.common.dto.JobMetadata jobMetadata, @NotNull com.netflix.genie.common.dto.Job job, @NotNull com.netflix.genie.common.dto.JobExecution jobExecution) Save all the initial job fields in the data store.
long	deleteBatchOfJobsCreatedBeforeDate(@NotNull java.time.Instant date, @Min(value=1L) int maxDeleted, @Min(value=1L) int pageSize) This method will delete a chunk of jobs whose creation time is earlier than the given date.
java.util.Optional<java.lang.String>	getJobArchiveLocation(@NotBlank(message="Job id is missing and is required") java.lang.String id) Get the location a job directory was archived to if at all.
java.util.Optional<com.netflix.genie.common.internal.dto.v4.JobRequest>	getJobRequest(@NotBlank(message="Id is missing and is required") java.lang.String id) Get the original request for a job.
java.util.Optional<com.netflix.genie.common.internal.dto.v4.JobSpecification>	getJobSpecification(@NotBlank(message="Id is missing and is required") java.lang.String id) Get the saved job specification for the given job.
com.netflix.genie.common.dto.JobStatus	getJobStatus(@NotBlank(message="Job id is missing and is required") java.lang.String id) Get the status for a job with the given id.
boolean	isV4(@NotBlank(message="Id is missing and is required") java.lang.String id) Get whether the job is a v4 job.
java.lang.String	saveJobRequest(@Valid com.netflix.genie.common.internal.dto.v4.JobRequest jobRequest, @Valid com.netflix.genie.common.internal.dto.v4.JobRequestMetadata jobRequestMetadata) Save the job request information.
void	saveJobSpecification(@NotBlank(message="Id is missing and is required") java.lang.String id, @Valid com.netflix.genie.common.internal.dto.v4.JobSpecification specification) Save the given job specification details for a job.
void	setJobCompletionInformation(@NotBlank(message="No job id entered. Unable to update.") java.lang.String id, int exitCode, @NotNull(message="No job status entered. Unable to update") com.netflix.genie.common.dto.JobStatus status, @NotBlank(message="Status message can\'t be blank. Unable to update") java.lang.String statusMessage, java.lang.Long stdOutSize, java.lang.Long stdErrSize) Method to set all job completion information for a job execution.
void	setJobRunningInformation(@NotBlank java.lang.String id, @Min(value=0L,message="Must be no lower than zero") int processId, @Min(value=1L,message="Must be at least 1 millisecond, preferably much more") long checkDelay, @NotNull java.time.Instant timeout) Update the job with information for the running job process.
void	updateJobStatus(@NotBlank(message="Id is missing and is required") java.lang.String id, @NotNull com.netflix.genie.common.dto.JobStatus currentStatus, @NotNull com.netflix.genie.common.dto.JobStatus newStatus, java.lang.String newStatusMessage) Update the status of the job identified with id to be newStatus provided that the current status of the job matches newStatus.
void	updateJobStatus(@NotBlank(message="No job id entered. Unable to update.") java.lang.String id, @NotNull(message="Status cannot be null.") com.netflix.genie.common.dto.JobStatus jobStatus, @NotBlank(message="Status message cannot be empty.") java.lang.String statusMsg) Update the status and status message of the job.
void	updateJobWithRuntimeEnvironment(@NotBlank java.lang.String jobId, @NotBlank java.lang.String clusterId, @NotBlank java.lang.String commandId, @NotNull java.util.List<java.lang.String> applicationIds, @Min(value=1L) int memory) Update the job with the various resources used to run the job including the cluster, command and applications.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

JpaJobPersistenceServiceImpl

public JpaJobPersistenceServiceImpl(JpaTagPersistenceService tagPersistenceService,
                                    JpaFilePersistenceService filePersistenceService,
                                    JpaApplicationRepository applicationRepository,
                                    JpaClusterRepository clusterRepository,
                                    JpaCommandRepository commandRepository,
                                    JpaJobRepository jobRepository)

Constructor.

Parameters:

tagPersistenceService - The JpaTagPersistenceService to use

filePersistenceService - The JpaFilePersistenceService to use

applicationRepository - The JpaApplicationRepository to use

clusterRepository - The JpaClusterRepository to use

commandRepository - The JpaCommandRepository to use

jobRepository - The JpaJobRepository to use

Method Detail

createJob

public void createJob(@NotNull
                      com.netflix.genie.common.dto.JobRequest jobRequest,
                      @NotNull
                      com.netflix.genie.common.dto.JobMetadata jobMetadata,
                      @NotNull
                      @NotNull com.netflix.genie.common.dto.Job job,
                      @NotNull
                      @NotNull com.netflix.genie.common.dto.JobExecution jobExecution)
               throws com.netflix.genie.common.exceptions.GenieException

Save all the initial job fields in the data store.

Specified by:

createJob in interface JobPersistenceService

Parameters:

jobRequest - the Job request object to save. Not null

jobMetadata - metadata about the job request. Not null

job - The Job object to create

jobExecution - The job execution object to create

Throws:

com.netflix.genie.common.exceptions.GenieException - if there is an error

updateJobWithRuntimeEnvironment

public void updateJobWithRuntimeEnvironment(@NotBlank
                                            @NotBlank java.lang.String jobId,
                                            @NotBlank
                                            @NotBlank java.lang.String clusterId,
                                            @NotBlank
                                            @NotBlank java.lang.String commandId,
                                            @NotNull
                                            @NotNull java.util.List<java.lang.String> applicationIds,
                                            @Min(value=1L)
                                            @Min(value=1L) int memory)
                                     throws com.netflix.genie.common.exceptions.GenieException

Update the job with the various resources used to run the job including the cluster, command and applications.

Specified by:

updateJobWithRuntimeEnvironment in interface JobPersistenceService

Parameters:

jobId - The id of the job to update

clusterId - The id of the cluster the job runs on

commandId - The id of the command the job runs with

applicationIds - The ids of the applications used to run the job

memory - The amount of memory (in MB) to run the job with

Throws:

com.netflix.genie.common.exceptions.GenieException - For any problems while updating

updateJobStatus

public void updateJobStatus(@NotBlank(message="No job id entered. Unable to update.")
                            @NotBlank(message="No job id entered. Unable to update.") java.lang.String id,
                            @NotNull(message="Status cannot be null.")
                            @NotNull(message="Status cannot be null.") com.netflix.genie.common.dto.JobStatus jobStatus,
                            @NotBlank(message="Status message cannot be empty.")
                            @NotBlank(message="Status message cannot be empty.") java.lang.String statusMsg)
                     throws com.netflix.genie.common.exceptions.GenieException

Update the status and status message of the job.

Specified by:

updateJobStatus in interface JobPersistenceService

Parameters:

id - The id of the job to update the status for.

jobStatus - The updated status of the job.

statusMsg - The updated status message of the job.

Throws:

com.netflix.genie.common.exceptions.GenieException - if there is an error

setJobRunningInformation

public void setJobRunningInformation(@NotBlank
                                     @NotBlank java.lang.String id,
                                     @Min(value=0L,message="Must be no lower than zero")
                                     @Min(value=0L,message="Must be no lower than zero") int processId,
                                     @Min(value=1L,message="Must be at least 1 millisecond, preferably much more")
                                     @Min(value=1L,message="Must be at least 1 millisecond, preferably much more") long checkDelay,
                                     @NotNull
                                     @NotNull java.time.Instant timeout)
                              throws com.netflix.genie.common.exceptions.GenieException

Update the job with information for the running job process.

Specified by:

setJobRunningInformation in interface JobPersistenceService

Parameters:

id - the id of the job to update the process id for

processId - The id of the process on the box for this job

checkDelay - The delay to check the process with

timeout - The date at which this job should timeout

Throws:

com.netflix.genie.common.exceptions.GenieException - if there is an error

setJobCompletionInformation

public void setJobCompletionInformation(@NotBlank(message="No job id entered. Unable to update.")
                                        @NotBlank(message="No job id entered. Unable to update.") java.lang.String id,
                                        int exitCode,
                                        @NotNull(message="No job status entered. Unable to update")
                                        @NotNull(message="No job status entered. Unable to update") com.netflix.genie.common.dto.JobStatus status,
                                        @NotBlank(message="Status message can\'t be blank. Unable to update")
                                        @NotBlank(message="Status message can\'t be blank. Unable to update") java.lang.String statusMessage,
                                        @Nullable
                                        java.lang.Long stdOutSize,
                                        @Nullable
                                        java.lang.Long stdErrSize)
                                 throws com.netflix.genie.common.exceptions.GenieException

Method to set all job completion information for a job execution.

Specified by:

setJobCompletionInformation in interface JobPersistenceService

Parameters:

id - the id of the job to update the exit code

exitCode - The exit code of the process

status - The job status for the job

statusMessage - The job status message

stdOutSize - The size (in bytes) of the standard out file or null if there isn't one

stdErrSize - The size (in bytes) of the standard error file or null if there isn't one

Throws:

com.netflix.genie.common.exceptions.GenieException - if there is an error

deleteBatchOfJobsCreatedBeforeDate

public long deleteBatchOfJobsCreatedBeforeDate(@NotNull
                                               @NotNull java.time.Instant date,
                                               @Min(value=1L)
                                               @Min(value=1L) int maxDeleted,
                                               @Min(value=1L)
                                               @Min(value=1L) int pageSize)

This method will delete a chunk of jobs whose creation time is earlier than the given date.

Specified by:

deleteBatchOfJobsCreatedBeforeDate in interface JobPersistenceService

Parameters:

date - The date before which all jobs should be deleted

maxDeleted - The maximum number of jobs that should be deleted (soft limit, can be rounded up to multiple of page size)

pageSize - Page size used to iterate through jobs

Returns:

the number of deleted jobs

saveJobRequest

public java.lang.String saveJobRequest(@Valid
                                       @Valid com.netflix.genie.common.internal.dto.v4.JobRequest jobRequest,
                                       @Valid
                                       @Valid com.netflix.genie.common.internal.dto.v4.JobRequestMetadata jobRequestMetadata)

Save the job request information.

Specified by:

saveJobRequest in interface JobPersistenceService

Parameters:

jobRequest - All the metadata provided by the user about the job

jobRequestMetadata - Metadata about the request gathered by the system not provided by the user

Returns:

The id that was reserved in the system for this job

getJobRequest

@Transactional(readOnly=true)
public java.util.Optional<com.netflix.genie.common.internal.dto.v4.JobRequest> getJobRequest(@NotBlank(message="Id is missing and is required")
                                                                                                                           @NotBlank(message="Id is missing and is required") java.lang.String id)

Get the original request for a job.

Specified by:

getJobRequest in interface JobPersistenceService

Parameters:

id - The unique id of the job to get

Returns:

The job request if one was found. Wrapped in Optional so empty optional returned if no job found

saveJobSpecification

public void saveJobSpecification(@NotBlank(message="Id is missing and is required")
                                 @NotBlank(message="Id is missing and is required") java.lang.String id,
                                 @Valid
                                 @Valid com.netflix.genie.common.internal.dto.v4.JobSpecification specification)

Save the given job specification details for a job. Sets the job status to JobStatus.RESOLVED.

Specified by:

saveJobSpecification in interface JobPersistenceService

Parameters:

id - The id of the job

specification - The job specification

getJobSpecification

@Transactional(readOnly=true)
public java.util.Optional<com.netflix.genie.common.internal.dto.v4.JobSpecification> getJobSpecification(@NotBlank(message="Id is missing and is required")
                                                                                                                                       @NotBlank(message="Id is missing and is required") java.lang.String id)

Get the saved job specification for the given job. If the job hasn't had a job specification resolved an empty Optional will be returned.

Specified by:

getJobSpecification in interface JobPersistenceService

Parameters:

id - The id of the job

Returns:

The job specification if one is present else an empty Optional

claimJob

public void claimJob(@NotBlank(message="Job id is missing and is required")
                     @NotBlank(message="Job id is missing and is required") java.lang.String id,
                     @Valid
                     @Valid com.netflix.genie.common.internal.dto.v4.AgentClientMetadata agentClientMetadata)

Set a job identified by id to be owned by the agent identified by agentClientMetadata. The job status in the system will be set to JobStatus.CLAIMED

Specified by:

claimJob in interface JobPersistenceService

Parameters:

id - The id of the job to claim. Must exist in the system.

agentClientMetadata - The metadata about the client claiming the job

updateJobStatus

public void updateJobStatus(@NotBlank(message="Id is missing and is required")
                            @NotBlank(message="Id is missing and is required") java.lang.String id,
                            @NotNull
                            @NotNull com.netflix.genie.common.dto.JobStatus currentStatus,
                            @NotNull
                            @NotNull com.netflix.genie.common.dto.JobStatus newStatus,
                            @Nullable
                            java.lang.String newStatusMessage)

Update the status of the job identified with id to be newStatus provided that the current status of the job matches newStatus. Optionally a status message can be provided to provide more details to users. If the newStatus is JobStatus.RUNNING the start time will be set. If the newStatus is a member of JobStatus.getFinishedStatuses() and the job had a started time set the finished time of the job will be set.

Specified by:

updateJobStatus in interface JobPersistenceService

Parameters:

id - The id of the job to update status for. Must exist in the system.

currentStatus - The status the caller to this API thinks the job currently has

newStatus - The new status the caller would like to update the status to

newStatusMessage - An optional status message to associate with this change

isV4

@Transactional(readOnly=true)
public boolean isV4(@NotBlank(message="Id is missing and is required")
                                                  @NotBlank(message="Id is missing and is required") java.lang.String id)

Get whether the job is a v4 job.

Specified by:

isV4 in interface JobPersistenceService

Parameters:

id - The id of the job

Returns:

true if its a v4 job

getJobStatus

@Transactional(readOnly=true)
public com.netflix.genie.common.dto.JobStatus getJobStatus(@NotBlank(message="Job id is missing and is required")
                                                                                         @NotBlank(message="Job id is missing and is required") java.lang.String id)
                                                                                  throws com.netflix.genie.common.exceptions.GenieNotFoundException

Get the status for a job with the given id.

Specified by:

getJobStatus in interface JobPersistenceService

Parameters:

id - The id of the job to get status for

Returns:

The job status

Throws:

com.netflix.genie.common.exceptions.GenieNotFoundException - If no job with the given id exists

getJobArchiveLocation

@Transactional(readOnly=true)
public java.util.Optional<java.lang.String> getJobArchiveLocation(@NotBlank(message="Job id is missing and is required")
                                                                                                @NotBlank(message="Job id is missing and is required") java.lang.String id)
                                                                                         throws com.netflix.genie.common.exceptions.GenieNotFoundException

Get the location a job directory was archived to if at all.

Specified by:

getJobArchiveLocation in interface JobPersistenceService

Parameters:

id - The id of the job to get the location for

Returns:

The archive location or Optional.empty()

Throws:

com.netflix.genie.common.exceptions.GenieNotFoundException - When there is no job with id id