com.linkedin.pegasus.gradle

Class PegasusPlugin

java.lang.Object

com.linkedin.pegasus.gradle.PegasusPlugin

All Implemented Interfaces:

org.gradle.api.Plugin<org.gradle.api.Project>

public class PegasusPlugin
extends java.lang.Object
implements org.gradle.api.Plugin<org.gradle.api.Project>

Pegasus code generation plugin. The supported project layout for this plugin is as follows:

   --- api/
   |   --- build.gradle
   |   --- src/
   |       --- <sourceSet>/
   |       |   --- idl/
   |       |   |   --- <published idl (.restspec.json) files>
   |       |   --- java/
   |       |   |   --- <packageName>/
   |       |   |       --- <common java files>
   |       |   --- pegasus/
   |       |       --- <packageName>/
   |       |           --- <data schema (.pdsc) files>
   |       --- <sourceSet>GeneratedDataTemplate/
   |       |   --- java/
   |       |       --- <packageName>/
   |       |           --- <data template source files generated from data schema (.pdsc) files>
   |       --- <sourceSet>GeneratedAvroSchema/
   |       |   --- avro/
   |       |       --- <packageName>/
   |       |           --- <avsc avro schema files (.avsc) generated from pegasus schema files>
   |       --- <sourceSet>GeneratedRest/
   |           --- java/
   |               --- <packageName>/
   |                   --- <rest client source (.java) files generated from published idl>
   --- impl/
   |   --- build.gradle
   |   --- src/
   |       --- <sourceSet>/
   |       |   --- java/
   |       |       --- <packageName>/
   |       |           --- <resource class source (.java) files>
   |       --- <sourceSet>GeneratedRest/
   |           --- idl/
   |               --- <generated idl (.restspec.json) files>
   --- <other projects>/
 

• api: contains all the files which are commonly depended by the server and client implementation. The common files include the data schema (.pdsc) files, the idl (.restspec.json) files and potentially Java interface files used by both sides.
• impl: contains the resource class for server implementation.

Performs the following functions:

Generate data model and data template jars for each source set.

Overview:

In the api project, the plugin generates the data template source (.java) files from the data schema (.pdsc) files, and furthermore compiles the source files and packages them to jar files. Details of jar contents will be explained in following paragraphs. In general, data schema files should exist only in api projects.

Configure the server and client implementation projects to depend on the api project's dataTemplate configuration to get access to the generated data templates from within these projects. This allows api classes to be built first so that implementation projects can consume them. We recommend this structure to avoid circular dependencies (directly or indirectly) among implementation projects.

Detail:

Generates data template source (.java) files from data schema (.pdsc) files, compiles the data template source (.java) files into class (.class) files, creates a data model jar file and a data template jar file. The data model jar file contains the source data schema (.pdsc) files. The data template jar file contains both the source data schema (.pdsc) files and the generated data template class (.class) files.

In the data template generation phase, the plugin creates a new target source set for the generated files. The new target source set's name is the input source set name's suffixed with "GeneratedDataTemplate", e.g. "mainGeneratedDataTemplate". The plugin invokes PegasusDataTemplateGenerator to generate data template source (.java) files for all data schema (.pdsc) files present in the input source set's pegasus directory, e.g. "src/main/pegasus". The generated data template source (.java) files will be in the new target source set's java source directory, e.g. "src/mainGeneratedDataTemplate/java". In addition to the data schema (.pdsc) files in the pegasus directory, the dataModel configuration specifies resolver path for the PegasusDataTemplateGenerator. The resolver path provides the data schemas and previously generated data template classes that may be referenced by the input source set's data schemas. In most cases, the dataModel configuration should contain data template jars.

The next phase is the data template compilation phase, the plugin compiles the generated data template source (.java) files into class files. The dataTemplateCompile configuration specifies the pegasus jars needed to compile these classes. The compileClasspath of the target source set is a composite of the dataModel configuration which includes the data template classes that were previously generated and included in the dependent data template jars, and the dataTemplateCompile configuration. This configuration should specify a dependency on the Pegasus data jar.

The following phase is creating the the data model jar and the data template jar. This plugin creates the data model jar that includes the contents of the input source set's pegasus directory, and sets the jar file's classification to "data-model". Hence, the resulting jar file's name should end with "-data-model.jar". It adds the data model jar as an artifact to the dataModel configuration. This jar file should only contain data schema (.pdsc) files.

This plugin also create the data template jar that includes the contents of the input source set's pegasus directory and the java class output directory of the target source set. It sets the jar file's classification to "data-template". Hence, the resulting jar file's name should end with "-data-template.jar". It adds the data template jar file as an artifact to the dataTemplate configuration. This jar file contains both data schema (.pdsc) files and generated data template class (.class) files.

This plugin will ensure that data template source files are generated before compiling the input source set and before the idea and eclipse tasks. It also adds the generated classes to the compileClasspath of the input source set.

The configurations that apply to generating the data model and data template jars are as follow:

• The dataTemplateCompile configuration specifies the classpath for compiling the generated data template source (.java) files. In most cases, it should be the Pegasus data jar. (The default compile configuration is not used for compiling data templates because it is not desirable to include non data template dependencies in the data template jar.) The configuration should not directly include data template jars. Data template jars should be included in the dataModel configuration.
• The dataModel configuration provides the value of the "generator.resolver.path" system property that is passed to PegasusDataTemplateGenerator. In most cases, this configuration should contain only data template jars. The data template jars contain both data schema (.pdsc) files and generated data template (.class) files. PegasusDataTemplateGenerator will not generate data template (.java) files for classes that can be found in the resolver path. This avoids redundant generation of the same classes, and inclusion of these classes in multiple jars. The dataModel configuration is also used to publish the data model jar which contains only data schema (.pdsc) files.
• The testDataModel configuration is similar to the dataModel configuration except it is used when generating data templates from test source sets. It extends from the dataModel configuration. It is also used to publish the data model jar from test source sets.
• The dataTemplate configuration is used to publish the data template jar which contains both data schema (.pdsc) files and the data template class (.class) files generated from these data schema (.pdsc) files.
• The testDataTemplate configuration is similar to the dataTemplate configuration except it is used when publishing the data template jar files generated from test source sets.

Performs the following functions:

Generate avro schema jars for each source set.

Overview:

In the api project, the task 'generateAvroSchema' generates the avro schema (.avsc) files from pegasus schema (.pdsc) files. In general, data schema files should exist only in api projects.

Configure the server and client implementation projects to depend on the api project's avroSchema configuration to get access to the generated avro schemas from within these projects.

This plugin also create the avro schema jar that includes the contents of the input source set's avro directory and the avsc schema files. The resulting jar file's name should end with "-avro-schema.jar".

Generate rest model and rest client jars for each source set.

Overview:

In the api project, generates rest client source (.java) files from the idl, compiles the rest client source (.java) files to rest client class (.class) files and puts them in jar files. In general, the api project should be only place that contains the publishable idl files. If the published idl changes an existing idl in the api project, the plugin will emit message indicating this has occurred and suggest that the entire project be rebuilt if it is desirable for clients of the idl to pick up the newly published changes.

In the impl project, generates the idl (.restspec.json) files from the input source set's resource class files, then compares them against the existing idl files in the api project for compatibility checking. If incompatible changes are found, the build fails (unless certain flag is specified, see below). If the generated idl passes compatibility checks (see compatibility check levels below), publishes the generated idl (.restspec.json) to the api project.

Detail:

rest client generation phase: in api project

In this phase, the rest client source (.java) files are generated from the api project idl (.restspec.json) files using RestRequestBuilderGenerator. The generated rest client source files will be in the new target source set's java source directory, e.g. "src/mainGeneratedRest/java".

RestRequestBuilderGenerator requires access to the data schemas referenced by the idl. The dataModel configuration specifies the resolver path needed by RestRequestBuilderGenerator to access the data schemas referenced by the idl that is not in the source set's pegasus directory. This plugin automatically includes the data schema (.pdsc) files in the source set's pegasus directory in the resolver path. In most cases, the dataModel configuration should contain data template jars. The data template jars contains both data schema (.pdsc) files and generated data template class (.class) files. By specifying data template jars instead of data model jars, redundant generation of data template classes is avoided as classes that can be found in the resolver path are not generated.

rest client compilation phase: in api project

In this phase, the plugin compiles the generated rest client source (.java) files into class files. The restClientCompile configuration specifies the pegasus jars needed to compile these classes. The compile classpath is a composite of the dataModel configuration which includes the data template classes that were previously generated and included in the dependent data template jars, and the restClientCompile configuration. This configuration should specify a dependency on the Pegasus restli-client jar.

The following stage is creating the the rest model jar and the rest client jar. This plugin creates the rest model jar that includes the generated idl (.restspec.json) files, and sets the jar file's classification to "rest-model". Hence, the resulting jar file's name should end with "-rest-model.jar". It adds the rest model jar as an artifact to the restModel configuration. This jar file should only contain idl (.restspec.json) files.

This plugin also create the rest client jar that includes the generated idl (.restspec.json) files and the java class output directory of the target source set. It sets the jar file's classification to "rest-client". Hence, the resulting jar file's name should end with "-rest-client.jar". It adds the rest client jar file as an artifact to the restClient configuration. This jar file contains both idl (.restspec.json) files and generated rest client class (.class) files.

idl generation phase: in server implementation project

Before entering this phase, the plugin will ensure that generating idl will occur after compiling the input source set. It will also ensure that IDEA and Eclipse tasks runs after rest client source (.java) files are generated.

In this phase, the plugin creates a new target source set for the generated files. The new target source set's name is the input source set name's* suffixed with "GeneratedRest", e.g. "mainGeneratedRest". The plugin invokes RestLiResourceModelExporter to generate idl (.restspec.json) files for each IdlItem in the input source set's pegasus IdlOptions. The generated idl files will be in target source set's idl directory, e.g. "src/mainGeneratedRest/idl". For example, the following adds an IdlItem to the source set's pegasus IdlOptions. This line should appear in the impl project's build.gradle. If no IdlItem is added, this source set will be excluded from generating idl and checking idl compatibility, even there are existing idl files.

   pegasus.main.idlOptions.addIdlItem(["com.linkedin.restli.examples.groups.server"])
 

After the idl generation phase, each included idl file is checked for compatibility against those in the api project. In case the current interface breaks compatibility, by default the build fails and reports all compatibility errors and warnings. Otherwise, the build tasks in the api project later will package the resource classes into jar files. User can change the compatibility requirement between the current and published idl by setting the "rest.model.compatibility" project property, i.e. "gradle -Prest.model.compatibility= ..." The following levels are supported:

• ignore: idl compatibility check will occur but its result will be ignored. The result will be aggregated and printed at the end of the build.
• backwards: build fails if there are backwards incompatible changes in idl. Build continues if there are only compatible changes.
• equivalent (default): build fails if there is any functional changes (compatible or incompatible) in the current idl. Only docs and comments are allowed to be different.

The plugin needs to know where the api project is. It searches the api project in the following steps. If all searches fail, the build fails.

1. Use the specified project from the impl project build.gradle file. The ext.apiProject property explicitly assigns the api project. E.g.

          ext.apiProject = project(':groups:groups-server-api')
        

   If multiple such statements exist, the last will be used. Wrong project path causes Gradle evaluation error.
2. If no ext.apiProject property is defined, the plugin will try to guess the api project name with the following conventions. The search stops at the first successful match.
   1. If the impl project name ends with the following suffixes, substitute the suffix with "-api".
      1. -impl
      2. -service
      3. -server
      4. -server-impl
      This list can be overridden by inserting the following line to the project build.gradle:

                 ext.apiProjectSubstitutionSuffixes = ['-new-suffix-1', '-new-suffix-2']
               

      Alternatively, this setting could be applied globally to all projects by putting it in the subprojects section of the root build.gradle.
   2. Append "-api" to the impl project name.

The plugin invokes RestLiResourceModelCompatibilityChecker to check compatibility.

The idl files in the api project are not generated by the plugin, but rather "published" from the impl project. The publishRestModel task is used to copy the idl files to the api project. This task is invoked automatically if the idls are verified to be "safe". "Safe" is determined by the "rest.model.compatibility" property. Because this task is skipped if the idls are functionally equivalent (not necessarily identical, e.g. differ in doc fields), if the default "equivalent" compatibility level is used, no file will be copied. If such automatic publishing is intended to be skip, set the "rest.model.skipPublish" property to true. Note that all the properties are per-project and can be overridden in each project's build.gradle file.

Please always keep in mind that if idl publishing is happened, a subsequent whole-project rebuild is necessary to pick up the changes. Otherwise, the Hudson job will fail and the source code commit will fail.

The configurations that apply to generating the rest model and rest client jars are as follow:

• The restClientCompile configuration specifies the classpath for compiling the generated rest client source (.java) files. In most cases, it should be the Pegasus restli-client jar. (The default compile configuration is not used for compiling rest client because it is not desirable to include non rest client dependencies, such as the rest server implementation classes, in the data template jar.) The configuration should not directly include data template jars. Data template jars should be included in the dataModel configuration.
• The dataModel configuration provides the value of the "generator.resolver.path" system property that is passed to RestRequestBuilderGenerator. This configuration should contain only data template jars. The data template jars contain both data schema (.pdsc) files and generated data template (.class) files. The RestRequestBuilderGenerator will only generate rest client classes. The dataModel configuration is also included in the compile classpath for the generated rest client source files. The dataModel configuration does not include generated data template classes, then the Java compiler may not able to find the data template classes referenced by the generated rest client.
• The testDataModel configuration is similar to the dataModel configuration except it is used when generating rest client source files from test source sets.
• The restModel configuration is used to publish the rest model jar which contains generated idl (.restspec.json) files.
• The testRestModel configuration is similar to the restModel configuration except it is used to publish rest model jar files generated from test source sets.
• The restClient configuration is used to publish the rest client jar which contains both generated idl (.restspec.json) files and the rest client class (.class) files generated from from these idl (.restspec.json) files.
• The testRestClient configuration is similar to the restClient configuration except it is used to publish rest client jar files generated from test source sets.

This plugin considers test source sets whose names begin with 'test' or 'integTest' to be test source sets.

Field Summary

Fields

Modifier and Type	Field and Description
static java.lang.String	DATA_TEMPLATE_FILE_SUFFIX
static java.util.Collection<java.lang.String>	DATA_TEMPLATE_FILE_SUFFIXES
static boolean	debug
static java.lang.String	IDL_COMPAT_REQUIREMENT
static java.lang.String	IDL_FILE_SUFFIX
static java.lang.String	PDL_FILE_SUFFIX
static java.lang.String	SNAPSHOT_COMPAT_REQUIREMENT
static java.lang.String	SNAPSHOT_FILE_SUFFIX

Constructor Summary

Constructors

Constructor and Description
PegasusPlugin()

Method Summary

Modifier and Type	Method and Description
void	apply(org.gradle.api.Project project)
protected void	configureAvroSchemaGeneration(org.gradle.api.Project project, org.gradle.api.tasks.SourceSet sourceSet)
protected void	configureConversionUtilities(org.gradle.api.Project project, org.gradle.api.tasks.SourceSet sourceSet)
protected void	configureDataTemplateGeneration(org.gradle.api.Project project, org.gradle.api.tasks.SourceSet sourceSet)
protected void	configureGeneratedSourcesAndJavadoc(org.gradle.api.Project project)
protected void	configureRestClientGeneration(org.gradle.api.Project project, org.gradle.api.tasks.SourceSet sourceSet)
protected void	configureRestModelGeneration(org.gradle.api.Project project, org.gradle.api.tasks.SourceSet sourceSet)
static java.lang.String	getGeneratedDirPath(org.gradle.api.Project project, org.gradle.api.tasks.SourceSet sourceSet, java.lang.String genType)
static java.lang.String	getNonEmptyProperty(org.gradle.api.Project project, java.lang.String propertyName) return the property value if the property exists and is not empty (-Pname=value) return null if property does not exist or the property is empty (-Pname)
static boolean	isPropertyTrue(org.gradle.api.Project project, java.lang.String propertyName) Return true if the given property exists and its value is true
void	setJavadocJarTask(org.gradle.api.Task javadocJarTask)
void	setPluginType(java.lang.Class<? extends org.gradle.api.Plugin<org.gradle.api.Project>> pluginType)
void	setSourcesJarTask(org.gradle.api.Task sourcesJarTask)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

debug

public static boolean debug

DATA_TEMPLATE_FILE_SUFFIX

public static final java.lang.String DATA_TEMPLATE_FILE_SUFFIX

See Also:

Constant Field Values

PDL_FILE_SUFFIX

public static final java.lang.String PDL_FILE_SUFFIX

See Also:

Constant Field Values

DATA_TEMPLATE_FILE_SUFFIXES

public static final java.util.Collection<java.lang.String> DATA_TEMPLATE_FILE_SUFFIXES

IDL_FILE_SUFFIX

public static final java.lang.String IDL_FILE_SUFFIX

See Also:

Constant Field Values

SNAPSHOT_FILE_SUFFIX

public static final java.lang.String SNAPSHOT_FILE_SUFFIX

See Also:

Constant Field Values

SNAPSHOT_COMPAT_REQUIREMENT

public static final java.lang.String SNAPSHOT_COMPAT_REQUIREMENT

See Also:

Constant Field Values

IDL_COMPAT_REQUIREMENT

public static final java.lang.String IDL_COMPAT_REQUIREMENT

See Also:

Constant Field Values

Constructor Detail

PegasusPlugin

public PegasusPlugin()

Method Detail

setPluginType

public void setPluginType(java.lang.Class<? extends org.gradle.api.Plugin<org.gradle.api.Project>> pluginType)

setSourcesJarTask

public void setSourcesJarTask(org.gradle.api.Task sourcesJarTask)

setJavadocJarTask

public void setJavadocJarTask(org.gradle.api.Task javadocJarTask)

apply

public void apply(org.gradle.api.Project project)

Specified by:

apply in interface org.gradle.api.Plugin<org.gradle.api.Project>

configureGeneratedSourcesAndJavadoc

protected void configureGeneratedSourcesAndJavadoc(org.gradle.api.Project project)

getGeneratedDirPath

public static java.lang.String getGeneratedDirPath(org.gradle.api.Project project,
                                                   org.gradle.api.tasks.SourceSet sourceSet,
                                                   java.lang.String genType)

configureRestModelGeneration

protected void configureRestModelGeneration(org.gradle.api.Project project,
                                            org.gradle.api.tasks.SourceSet sourceSet)

configureAvroSchemaGeneration

protected void configureAvroSchemaGeneration(org.gradle.api.Project project,
                                             org.gradle.api.tasks.SourceSet sourceSet)

configureConversionUtilities

protected void configureConversionUtilities(org.gradle.api.Project project,
                                            org.gradle.api.tasks.SourceSet sourceSet)

configureDataTemplateGeneration

protected void configureDataTemplateGeneration(org.gradle.api.Project project,
                                               org.gradle.api.tasks.SourceSet sourceSet)

configureRestClientGeneration

protected void configureRestClientGeneration(org.gradle.api.Project project,
                                             org.gradle.api.tasks.SourceSet sourceSet)

getNonEmptyProperty

public static java.lang.String getNonEmptyProperty(org.gradle.api.Project project,
                                                   java.lang.String propertyName)

return the property value if the property exists and is not empty (-Pname=value) return null if property does not exist or the property is empty (-Pname)

Parameters:

project - the project where to look for the property

propertyName - the name of the property

isPropertyTrue

public static boolean isPropertyTrue(org.gradle.api.Project project,
                                     java.lang.String propertyName)

Return true if the given property exists and its value is true

Parameters:

project - the project where to look for the property

propertyName - the name of the property