com.netflix.genie.web.data.services.impl.jpa.repositories

Interface JpaJobRepository

All Superinterfaces:

JpaBaseRepository<JobEntity>

public interface JpaJobRepository
extends JpaBaseRepository<JobEntity>

Job repository.

Field Summary

Fields

Modifier and Type	Field and Description
static java.lang.String	FIND_OLD_JOBS_QUERY The query used to find batches of jobs before a certain time.

Method Summary

Modifier and Type	Method and Description
java.lang.Long	countJobsByUserAndStatusIn(@NotBlank java.lang.String user, @NotEmpty java.util.Set<java.lang.String> statuses) Count all jobs that belong to a given user and are in any of the given states.
java.util.Set<JobProjection>	findByAgentHostnameAndStatusIn(java.lang.String agentHostname, java.util.Set<java.lang.String> statuses) Find jobs by host name and status.
java.util.Set<java.lang.Long>	findJobsCreatedBefore(java.time.Instant createdThreshold, java.util.Set<java.lang.String> excludeStatuses, int limit) Find a batch of jobs that were created before the given time.
java.util.Set<java.lang.String>	getAgentJobIdsWithStatusIn(@NotEmpty java.util.Set<java.lang.String> statuses) Find agent jobs in the given set of states.
java.util.Optional<java.lang.String>	getArchiveStatus(java.lang.String id) Get only the archive status of a job.
JobInfoAggregate	getHostJobInfo(java.lang.String agentHostname, java.util.Set<java.lang.String> activeStatuses, java.util.Set<java.lang.String> usedStatuses) In a single query get aggregate information for the amount of memory used and count of active jobs on a given host.
java.util.Optional<JobApplicationsProjection>	getJobApplications(java.lang.String id) Get the applications for a job.
java.util.Optional<JobClusterProjection>	getJobCluster(java.lang.String id) Get the cluster for a job.
java.util.Optional<JobCommandProjection>	getJobCommand(java.lang.String id) Get the command for a job.
java.util.Optional<java.lang.String>	getJobHostname(java.lang.String id) Get only the hostname of a job.
java.util.Optional<java.lang.String>	getJobStatus(java.lang.String id) Get only the status of a job.
java.util.Set<java.lang.String>	getJobsWithStatusAndArchiveStatusUpdatedBefore(@NotEmpty java.util.Set<java.lang.String> statuses, @NotEmpty java.util.Set<java.lang.String> archiveStatuses, java.time.Instant updateThreshold) Find agent jobs in the given set of job and archive states that were marked finished before a given threshold.
java.util.Optional<com.fasterxml.jackson.databind.JsonNode>	getLauncherExt(java.lang.String id) Get only the launcher ext of a job.
java.util.Optional<com.fasterxml.jackson.databind.JsonNode>	getRequestedLauncherExt(java.lang.String id) Get only the requested launcher ext of a job.
long	getTotalMemoryUsedOnHost(java.lang.String agentHostname, java.util.Set<java.lang.String> statuses) Given the hostname that agents are running on return the total memory their jobs are currently using.
java.util.Set<UserJobResourcesAggregate>	getUserJobResourcesAggregates(java.util.Set<java.lang.String> statuses, boolean api) Returns resources usage for each user that has a running job.
java.util.Optional<JobProjection>	getV3Job(java.lang.String id) Get the data needed to create a V3 Job DTO.
java.util.Optional<JobRequestProjection>	getV3JobRequest(java.lang.String id) Get the data needed to create a V3 Job Request DTO.
java.util.Optional<V4JobRequestProjection>	getV4JobRequest(java.lang.String id) Get the data needed to create a V4 Job Request DTO.
java.util.Optional<JobSpecificationProjection>	getV4JobSpecification(java.lang.String id) Get the data needed to create a V4 Job Specification DTO.
java.util.Optional<java.lang.Boolean>	isAPI(java.lang.String id) Return whether the job was submitted via the API or the Agent CLI.

Methods inherited from interface com.netflix.genie.web.data.services.impl.jpa.repositories.JpaBaseRepository

deleteByIdIn, existsByUniqueId, findByUniqueId, findByUniqueId

Methods inherited from interface org.springframework.data.jpa.repository.JpaRepository

deleteAllInBatch, deleteInBatch, findAll, findAll, findAll, findAll, findAllById, flush, getOne, saveAll, saveAndFlush

Methods inherited from interface org.springframework.data.repository.PagingAndSortingRepository

findAll

Methods inherited from interface org.springframework.data.repository.CrudRepository

count, delete, deleteAll, deleteAll, deleteById, existsById, findById, save

Methods inherited from interface org.springframework.data.repository.query.QueryByExampleExecutor

count, exists, findAll, findOne

Methods inherited from interface org.springframework.data.jpa.repository.JpaSpecificationExecutor

count, findAll, findAll, findAll, findOne

Field Detail

FIND_OLD_JOBS_QUERY

static final java.lang.String FIND_OLD_JOBS_QUERY

The query used to find batches of jobs before a certain time.

See Also:

Constant Field Values

Method Detail

findByAgentHostnameAndStatusIn

@EntityGraph(value="Job.v3.dto.job",
             type=LOAD)
java.util.Set<JobProjection> findByAgentHostnameAndStatusIn(java.lang.String agentHostname,
                                                                                                                         java.util.Set<java.lang.String> statuses)

Find jobs by host name and status.

Parameters:

agentHostname - The host name to search for

statuses - The job statuses to filter by

Returns:

The jobs

getTotalMemoryUsedOnHost

@Query(value="SELECT COALESCE(SUM(j.memoryUsed), 0) FROM JobEntity j WHERE j.agentHostname = :agentHostname AND j.status IN (:statuses)")
long getTotalMemoryUsedOnHost(@Param(value="agentHostname")
                                                                                                                                                                        java.lang.String agentHostname,
                                                                                                                                                                        @Param(value="statuses")
                                                                                                                                                                        java.util.Set<java.lang.String> statuses)

Given the hostname that agents are running on return the total memory their jobs are currently using.

Parameters:

agentHostname - The agent hostname

statuses - The job statuses to filter by e.g. JobStatus.getActiveStatuses()

Returns:

The total memory used in MB

getHostJobInfo

@Query(value="SELECT (SELECT COALESCE(SUM(j.memory_used), 0) FROM jobs j WHERE j.agent_hostname = :agentHostname and j.status IN (:activeStatuses)) as totalMemoryAllocated, (SELECT COALESCE(SUM(j.memory_used), 0) FROM jobs j WHERE j.agent_hostname = :agentHostname and j.status IN (:usedStatuses)) as totalMemoryUsed, (SELECT COUNT(*) FROM jobs j WHERE j.agent_hostname = :agentHostname and j.status IN (:activeStatuses)) as numberOfActiveJobs",
       nativeQuery=true)
JobInfoAggregate getHostJobInfo(@Param(value="agentHostname")
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       java.lang.String agentHostname,
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       @Param(value="activeStatuses")
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       java.util.Set<java.lang.String> activeStatuses,
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       @Param(value="usedStatuses")
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       java.util.Set<java.lang.String> usedStatuses)

In a single query get aggregate information for the amount of memory used and count of active jobs on a given host.

Parameters:

agentHostname - The hostname where the agent is running the job

activeStatuses - The set of statuses to use in order to consider a job "active"

usedStatuses - The set of statuses to use in order to consider a job to actively be using memory

Returns:

A JobInfoAggregate instance with the requested information

countJobsByUserAndStatusIn

java.lang.Long countJobsByUserAndStatusIn(@NotBlank
                                          @NotBlank java.lang.String user,
                                          @NotEmpty
                                          @NotEmpty java.util.Set<java.lang.String> statuses)

Count all jobs that belong to a given user and are in any of the given states.

Parameters:

user - the user name

statuses - the set of statuses

Returns:

the count of jobs matching the search criteria

findJobsCreatedBefore

@Query(value="SELECT id FROM jobs WHERE created < :createdThreshold AND status NOT IN (:excludedStatuses) LIMIT :batchSize",
       nativeQuery=true)
java.util.Set<java.lang.Long> findJobsCreatedBefore(@Param(value="createdThreshold")
                                                                                                                                                                                                          java.time.Instant createdThreshold,
                                                                                                                                                                                                          @Param(value="excludedStatuses")
                                                                                                                                                                                                          java.util.Set<java.lang.String> excludeStatuses,
                                                                                                                                                                                                          @Param(value="batchSize")
                                                                                                                                                                                                          int limit)

Find a batch of jobs that were created before the given time.

Parameters:

createdThreshold - The time before which the jobs were submitted. Exclusive

excludeStatuses - The set of statuses which should be excluded from the results

limit - The maximum number of jobs to to find

Returns:

The number of deleted jobs

getUserJobResourcesAggregates

@Query(value="SELECT j.user AS user, COUNT(j) as runningJobsCount, COALESCE(SUM(j.memoryUsed), 0) as usedMemory FROM JobEntity j WHERE j.status IN (:statuses) AND j.api = :isApi GROUP BY j.user")
java.util.Set<UserJobResourcesAggregate> getUserJobResourcesAggregates(@Param(value="statuses")
                                                                                                                                                                                                                                                                           java.util.Set<java.lang.String> statuses,
                                                                                                                                                                                                                                                                           @Param(value="isApi")
                                                                                                                                                                                                                                                                           boolean api)

Returns resources usage for each user that has a running job. Only jobs running on Genie servers are considered (i.e. no Agent jobs)

Parameters:

statuses - The set of statuses a job has to be in to be considered

api - Whether the job was submitted through the api (true) or agent cli (false)

Returns:

The user resource aggregates

getAgentJobIdsWithStatusIn

@Query(value="SELECT j.uniqueId FROM JobEntity j WHERE j.status IN (:statuses) AND j.v4 = TRUE")
java.util.Set<java.lang.String> getAgentJobIdsWithStatusIn(@Param(value="statuses") @NotEmpty
                                                                                                                                                            @NotEmpty java.util.Set<java.lang.String> statuses)

Find agent jobs in the given set of states.

Parameters:

statuses - the job statuses filter

Returns:

a set of job projections

getJobsWithStatusAndArchiveStatusUpdatedBefore

@Query(value="SELECT j.uniqueId FROM JobEntity j WHERE j.status IN (:statuses) AND j.archiveStatus IN (:archiveStatuses) AND j.updated < :updatedThreshold AND j.v4 = TRUE")
java.util.Set<java.lang.String> getJobsWithStatusAndArchiveStatusUpdatedBefore(@Param(value="statuses") @NotEmpty
                                                                                                                                                                                                                                                            @NotEmpty java.util.Set<java.lang.String> statuses,
                                                                                                                                                                                                                                                            @Param(value="archiveStatuses") @NotEmpty
                                                                                                                                                                                                                                                            @NotEmpty java.util.Set<java.lang.String> archiveStatuses,
                                                                                                                                                                                                                                                            @Param(value="updatedThreshold")
                                                                                                                                                                                                                                                            java.time.Instant updateThreshold)

Find agent jobs in the given set of job and archive states that were marked finished before a given threshold.

Parameters:

statuses - the job statuses filter

archiveStatuses - the job archive statuses filter

updateThreshold - select jobs last updated before this threshold

Returns:

a set of job ids

getJobStatus

@Query(value="SELECT j.status FROM JobEntity j WHERE j.uniqueId = :id")
java.util.Optional<java.lang.String> getJobStatus(@Param(value="id")
                                                                                                                          java.lang.String id)

Get only the status of a job.

Parameters:

id - The id of the job to get the status for

Returns:

The job status string or Optional.empty() if no job with the given id exists

isAPI

@Query(value="SELECT j.api FROM JobEntity j WHERE j.uniqueId = :id")
java.util.Optional<java.lang.Boolean> isAPI(@Param(value="id")
                                                                                                                 java.lang.String id)

Return whether the job was submitted via the API or the Agent CLI.

Parameters:

id - The unique id of the job

Returns:

true if the job was submitted via the API. Optional.empty() if the job doesn't exist

getJobHostname

@Query(value="SELECT j.agentHostname FROM JobEntity j WHERE j.uniqueId = :id")
java.util.Optional<java.lang.String> getJobHostname(@Param(value="id")
                                                                                                                                   java.lang.String id)

Get only the hostname of a job.

Parameters:

id - The id of the job to get the hostname for

Returns:

The job hostname or Optional.empty() if no job with the given id exists

getArchiveStatus

@Query(value="SELECT COALESCE(j.archiveStatus, \'UNKNOWN\') FROM JobEntity j WHERE j.uniqueId = :id")
java.util.Optional<java.lang.String> getArchiveStatus(@Param(value="id")
                                                                                                                                                            java.lang.String id)

Get only the archive status of a job.

Parameters:

id - The id of the job to get the archive status for

Returns:

The job archive status or Optional.empty() if no job with the given id exists

getRequestedLauncherExt

@Query(value="SELECT j.requestedLauncherExt FROM JobEntity j WHERE j.uniqueId = :id")
java.util.Optional<com.fasterxml.jackson.databind.JsonNode> getRequestedLauncherExt(@Param(value="id")
                                                                                                                                                                          java.lang.String id)

Get only the requested launcher ext of a job.

Parameters:

id - The id of the job

Returns:

The requested launcher ext JSON node Optional.empty() if no job with the given id exists

getLauncherExt

@Query(value="SELECT j.launcherExt FROM JobEntity j WHERE j.uniqueId = :id")
java.util.Optional<com.fasterxml.jackson.databind.JsonNode> getLauncherExt(@Param(value="id")
                                                                                                                                                        java.lang.String id)

Get only the launcher ext of a job.

Parameters:

id - The id of the job

Returns:

The launcher ext JSON node Optional.empty() if no job with the given id exists

getV3Job

@Query(value="SELECT j FROM JobEntity j WHERE j.uniqueId = :id")
 @EntityGraph(value="Job.v3.dto.job",
             type=LOAD)
java.util.Optional<JobProjection> getV3Job(@Param(value="id")
                                                                                                                                                                          java.lang.String id)

Get the data needed to create a V3 Job DTO.

Parameters:

id - The unique id of the job

Returns:

The JobProjection data or Optional.empty() if the job doesn't exist

getV3JobRequest

@Query(value="SELECT j FROM JobEntity j WHERE j.uniqueId = :id")
 @EntityGraph(value="Job.v3.dto.request",
             type=LOAD)
java.util.Optional<JobRequestProjection> getV3JobRequest(@Param(value="id")
                                                                                                                                                                                            java.lang.String id)

Get the data needed to create a V3 Job Request DTO.

Parameters:

id - The unique id of the job

Returns:

The JobRequestProjection data or Optional.empty() if the job doesn't exist

getV4JobRequest

@Query(value="SELECT j FROM JobEntity j WHERE j.uniqueId = :id")
 @EntityGraph(value="Job.v4.dto.request",
             type=LOAD)
java.util.Optional<V4JobRequestProjection> getV4JobRequest(@Param(value="id")
                                                                                                                                                                                              java.lang.String id)

Get the data needed to create a V4 Job Request DTO.

Parameters:

id - The unique id of the job

Returns:

The V4JobRequestProjection data or Optional.empty() if the job doesn't exist

getV4JobSpecification

@Query(value="SELECT j FROM JobEntity j WHERE j.uniqueId = :id")
 @EntityGraph(value="Job.v4.dto.specification",
             type=LOAD)
java.util.Optional<JobSpecificationProjection> getV4JobSpecification(@Param(value="id")
                                                                                                                                                                                                              java.lang.String id)

Get the data needed to create a V4 Job Specification DTO.

Parameters:

id - The unique id of the job

Returns:

The JobSpecificationProjection data or Optional.empty() if the job doesn't exist

getJobApplications

@Query(value="SELECT j FROM JobEntity j WHERE j.uniqueId = :id")
 @EntityGraph(value="Job.applications",
             type=LOAD)
java.util.Optional<JobApplicationsProjection> getJobApplications(@Param(value="id")
                                                                                                                                                                                                  java.lang.String id)

Get the applications for a job.

Parameters:

id - The unique id of the job

Returns:

The JobApplicationsProjection data or Optional.empty() if the job doesn't exist

getJobCluster

@Query(value="SELECT j FROM JobEntity j WHERE j.uniqueId = :id")
 @EntityGraph(value="Job.cluster",
             type=LOAD)
java.util.Optional<JobClusterProjection> getJobCluster(@Param(value="id")
                                                                                                                                                                                   java.lang.String id)

Get the cluster for a job.

Parameters:

id - The unique id of the job

Returns:

The JobClusterProjection data or Optional.empty() if the job doesn't exist

getJobCommand

@Query(value="SELECT j FROM JobEntity j WHERE j.uniqueId = :id")
 @EntityGraph(value="Job.command",
             type=LOAD)
java.util.Optional<JobCommandProjection> getJobCommand(@Param(value="id")
                                                                                                                                                                                   java.lang.String id)

Get the command for a job.

Parameters:

id - The unique id of the job

Returns:

The JobCommandProjection data or Optional.empty() if the job doesn't exist