Package io.openliberty.tools.common.plugins.util

Class DevUtil

    • Constructor Detail

      • DevUtil

        public DevUtil​(File buildDirectory,
               File serverDirectory,
               File sourceDirectory,
               File testSourceDirectory,
               File configDirectory,
               File projectDirectory,
               File multiModuleProjectDirectory,
               List<File> resourceDirs,
               boolean hotTests,
               boolean skipTests,
               boolean skipUTs,
               boolean skipITs,
               String applicationId,
               long serverStartTimeout,
               int appStartupTimeout,
               int appUpdateTimeout,
               long compileWaitMillis,
               boolean libertyDebug,
               boolean useBuildRecompile,
               boolean gradle,
               boolean pollingTest,
               boolean container,
               File dockerfile,
               File dockerBuildContext,
               String dockerRunOpts,
               int dockerBuildTimeout,
               boolean skipDefaultPorts,
               JavaCompilerOptions compilerOptions,
               boolean keepTempDockerfile,
               String mavenCacheLocation,
               List<ProjectModule> upstreamProjects,
               boolean recompileDependencies,
               String packagingType,
               File buildFile,
               Map<String,​List<String>> parentBuildFiles,
               boolean generateFeatures,
               Set<String> compileArtifactPaths,
               Set<String> testArtifactPaths,
               List<Path> webResourceDirs)

    • Method Detail

      • debug

        public abstract void debug​(String msg,
                           Throwable e)
        Log debug
        Parameters:
        msg -
        e -

      • debug

        public abstract void debug​(Throwable e)
        Log debug
        Parameters:
        e -

      • warn

        public abstract void warn​(String msg)
        Log warning
        Parameters:
        msg -

      • info

        public abstract void info​(String msg)
        Log info
        Parameters:
        msg -

      • error

        public abstract void error​(String msg)
        Log error
        Parameters:
        msg -

      • isDebugEnabled

        public abstract boolean isDebugEnabled()
        Returns whether debug is enabled by the current logger
        Returns:
        whether debug is enabled

      • recompileBuildFile

        public abstract boolean recompileBuildFile​(File buildFile,
                                           Set<String> compileArtifactPaths,
                                           Set<String> testArtifactPaths,
                                           boolean generateFeatures,
                                           ThreadPoolExecutor executor)
                                    throws PluginExecutionException
        Recompile the build file
        Parameters:
        buildFile -
        compileArtifactPaths -
        testArtifactPaths -
        generateFeatures - boolean, true if auto generation of features is on
        executor - The thread pool executor
        Returns:
        true if the build file was recompiled with changes
        Throws:
        PluginExecutionException - if there was an error when restarting the server

      • updateArtifactPaths

        public abstract boolean updateArtifactPaths​(ProjectModule projectModule,
                                            boolean redeployCheck,
                                            boolean generateFeatures,
                                            ThreadPoolExecutor executor)
                                     throws PluginExecutionException
        Updates the compile artifact paths of the given project module. Only used in multi module scenario.
        Parameters:
        projectModule - The corresponding project module to update artifact paths for
        redeployCheck - Whether to redeploy the application if changes in the dependencies are detected
        generateFeatures - boolean, true if auto generation of features is on
        executor - The thread pool executor
        Returns:
        true if the compile artifact paths are updated
        Throws:
        PluginExecutionException - if there was an error when restarting the server

      • updateArtifactPaths

        public abstract boolean updateArtifactPaths​(File parentBuildFile)
        Update the compile artifact paths of any child modules of the given build file.
        Parameters:
        parentBuildFile - The parent build file
        Returns:
        true if the compile artifact paths are updated

      • installFeatures

        public abstract void installFeatures​(File configFile,
                                     File serverDir,
                                     boolean generateFeatures)
        Check the configuration file for new features and install features if needed
        Parameters:
        configFile -
        serverDir -
        generateFeatures -

      • getServerFeatureUtilObj

        public abstract ServerFeatureUtil getServerFeatureUtilObj()
        Get the ServerFeatureUtil object
        Returns:
        ServerFeatureUtil object

      • getExistingFeatures

        public abstract Set<String> getExistingFeatures()
        Get the set of existing features
        Returns:
        existing features set

      • updateExistingFeatures

        public abstract void updateExistingFeatures()
        Update the existing features list using the files in the server directory. Called on configuration file change to ensure we have the most current feature list.

      • compile

        public abstract boolean compile​(File dir)
        Compile the specified directory
        Parameters:
        dir -
        Returns:

      • compile

        public abstract boolean compile​(File dir,
                                ProjectModule project)
        Compile the specified directory for the project module
        Parameters:
        dir -
        project - project module (used in multi-module scenario)
        Returns:

      • stopServer

        public abstract void stopServer()
        Stop the server

      • getServerTask

        public abstract io.openliberty.tools.ant.ServerTask getServerTask()
                                                           throws Exception
        Get the ServerTask to start the server, which can be in either "run" or "debug" mode
        Returns:
        ServerTask the task to start the server
        Throws:
        Exception - if there was an error copying/creating config files

      • getServerStartTimeoutExample

        public abstract String getServerStartTimeoutExample()
        Get an example command using the server start timeout parameter. The example command is unique to each plugin.
        Returns:
        String containing the example command

      • getProjectName

        public abstract String getProjectName()
        Get the name of the current project running dev mode.
        Returns:
        String of the project name

      • isLooseApplication

        public abstract boolean isLooseApplication()
        Is the application deployed as a loose application.

      • getLooseApplicationFile

        public abstract File getLooseApplicationFile()
        Get the loose application configuration file.
        Returns:
        File loose application configuration file

      • runTests

        public void runTests​(boolean waitForApplicationUpdate,
                     int messageOccurrences,
                     ThreadPoolExecutor executor,
                     boolean forceSkipTests,
                     boolean forceSkipUTs,
                     boolean forceSkipITs,
                     File currentBuildFile,
                     String projectName)
        Run unit and/or integration tests
        Parameters:
        waitForApplicationUpdate - Whether to wait for the application to update before running integration tests
        messageOccurrences - The previous number of times the application updated message has appeared.
        executor - The thread pool executor
        forceSkipTests - Whether to force skip all tests
        forceSkipUTs - Whether to force skip the unit tests
        forceSkipITs - Whether to force skip the integration tests
        currentBuildFile - The build file to run tests against
        projectName - The name of the current project, null if only one project exists

      • countApplicationUpdatedMessages

        public int countApplicationUpdatedMessages()
        Get the number of times the application updated message has appeared in the application log
        Returns:
        the number of times the application has updated

      • startServer

        public void startServer​(boolean buildContainer,
                        boolean pullParentImage)
                 throws PluginExecutionException
        Start the server and keep it running in a background thread.
        Parameters:
        buildContainer - Force a Docker build when in container mode. Ignored otherwise.
        pullParentImage - If buildContainer is true, this determines whether the Docker build should also pull the latest parent image. Ignored otherwise.
        Throws:
        PluginExecutionException - If the server startup could not be verified within the timeout, or server startup failed.

      • getEscapeCharacter

        protected static char getEscapeCharacter​(List<String> dockerfileLines)
                                  throws PluginExecutionException
        Get escape character from the escape directive at the top of the Dockerfile. Docker documents a couple of directives, but it seems escape must always be the first line to work.
        Throws:
        PluginExecutionException

      • disableOpenJ9SCC

        protected void disableOpenJ9SCC​(List<String> dockerfileLines)
        Disables OpenJ9 SCC to speed up build times by injecting environment variable before "RUN configure.sh" is called. Reference: https://github.com/OpenLiberty/ci.docker#openj9-shared-class-cache-scc
        Note: lines must have been trimmed and cleaned of comments using getCleanedLines() before calling this.

      • detectFeaturesSh

        protected void detectFeaturesSh​(List<String> dockerfileLines)

      • getLooseAppProjectRoot

        public static File getLooseAppProjectRoot​(File projectDirectory,
                                          File multiModuleProjectDirectory)
        Get the root directory for mounting loose app in container. This is the longest common directory between the projectDirectory and multiModuleProjectDirectory.
        Parameters:
        projectDirectory - The current project directory. Must not be null.
        multiModuleProjectDirectory - The multi module project directory. Can be null.
        Returns:
        The longest common directory, or projectDirectory if multiModuleProjectDirectory is null

      • getLongestCommonDir

        protected static File getLongestCommonDir​(File dir1,
                                          File dir2)

      • parseNetworks

        protected static String[] parseNetworks​(String dockerResult)
        Parses Docker network names from a "docker inspect" command result on a container.
        Parameters:
        dockerResult - the result from the command "docker inspect -f '{{.NetworkSettings.Networks}}' containerName". The dockerResult must not contain surrounding quotes or leading/trailing whitespace.
        Returns:
        a String array containing the names of the networks contained in the dockerResult parameter

      • removeSurroundingQuotes

        protected static String removeSurroundingQuotes​(String str)

      • libertyGenerateFeatures

        public abstract boolean libertyGenerateFeatures​(Collection<String> classes,
                                                boolean optimize)
        Generate features for the application
        Parameters:
        classes - class file paths features should be generated for (can be null if no modified classes)
        optimize - if true, generate optimized feature list
        Returns:
        true if feature generation was successful

      • restartServer

        public void restartServer​(boolean buildContainer)
                   throws PluginExecutionException
        Stop the server, set up Liberty and restart it.
        Parameters:
        buildContainer - Force a Docker build when in container mode. Ignored otherwise.
        Throws:
        PluginExecutionException

      • parseHttpPort

        protected void parseHttpPort​(String webAppMessage,
                             int portPrefixIndex)

      • cleanUpServerEnv

        public void cleanUpServerEnv()

      • cleanUpTempConfig

        public void cleanUpTempConfig()

      • cleanUpTempDockerfile

        public void cleanUpTempDockerfile()

      • setDevStop

        public void setDevStop​(boolean devStop)
        Whether dev mode intentionally caused the server to stop.
        Parameters:
        devStop - If true, stopping the server will not cause dev mode to print an error message.

      • getDebugEnvironmentVariables

        public Map<String,​String> getDebugEnvironmentVariables()
                                                      throws IOException
        Gets a map of the environment variables to set for debug mode.
        Returns:
        a Map of debug environment variables with name as key
        Throws:
        IOException

      • enableServerDebug

        public void enableServerDebug()
                       throws IOException
        Enable server debug variables in server.env, using the user specified debug port if it's available, otherwise uses a random available port.
        Throws:
        IOException - if there was an IO exception when reading or writing the server.env

      • findAvailablePort

        public int findAvailablePort​(int preferredPort,
                             boolean isDebugPort)
                      throws IOException
        Finds an available port to use. There are two semantics. If looking for a port for the server debug connection and the port is in use then return an ephemeral port. If looking for a port for the server http connection then try sequential port numbers. In the case of the server debug connection, if the preferred port is not available, return a random available port and cache the result which will override the preferredPort if this method is called again.
        Parameters:
        preferredPort - The number of the port to start the search for an available port.
        isDebugPort - Whether to choose an ephemeral port. True to choose an ephemeral port, false to search sequentially.
        Returns:
        An available port.
        Throws:
        IOException - if it could not find any available port, or there was an error when opening a server socket regardless of port.

      • runHotkeyReaderThread

        public void runHotkeyReaderThread​(ThreadPoolExecutor executor)
        Run a hotkey reader thread. If the thread is already running, re-prints the message about pressing enter to run tests.
        Parameters:
        executor - the test thread executor

      • setFeatureGeneration

        protected void setFeatureGeneration​(boolean generateFeatures)

      • watchFiles

        public void watchFiles​(File outputDirectory,
                       File testOutputDirectory,
                       ThreadPoolExecutor executor,
                       File serverXmlFile,
                       File bootstrapPropertiesFile,
                       File jvmOptionsFile)
                throws Exception
        Watch files for changes.
        Parameters:
        outputDirectory -
        testOutputDirectory -
        executor -
        serverXmlFile - Can be null when using the server.xml from the configDirectory, which has a default value.
        bootstrapPropertiesFile -
        jvmOptionsFile -
        Throws:
        Exception

      • getOmitFilesList

        protected Collection<File> getOmitFilesList​(File looseAppFile,
                                            String srcDirectoryPath)
        Given the loose app file and the source directory path, return a list of files that are specified in the loose app file and are in the source directory and should be omitted from watching.
        Parameters:
        looseAppFile - Loose Application configuration file
        srcDirectoryPath - the source directory path
        Returns:
        a list of files that should be omitted from watching as they are on the source directory path and exist in the loose app config file

      • resourceDirectoryCreated

        protected abstract void resourceDirectoryCreated()
                                          throws IOException
        Throws:
        IOException

      • resourceModifiedOrCreated

        protected abstract void resourceModifiedOrCreated​(File fileChanged,
                                                  File resourceParent,
                                                  File outputDirectory)
                                           throws IOException
        Throws:
        IOException

      • isGeneratedConfigFile

        protected boolean isGeneratedConfigFile​(File fileChanged,
                                        File srcDir,
                                        File targetDir)
                                 throws IOException
        Determines if the corresponding target config file was generated by a Liberty plugin
        Parameters:
        fileChanged - the file that was changed
        srcDir - the directory of the file changed
        targetDir - the target directory
        Throws:
        IOException - unable to resolve canonical path

      • readFile

        public String readFile​(File file)
                throws IOException
        Reads the file to a String
        Parameters:
        file -
        Returns:
        String representation of the file
        Throws:
        IOException - unable to read file to string

      • installFeaturesToTempDir

        public void installFeaturesToTempDir​(File fileChanged,
                                     File srcDir,
                                     String targetFileName,
                                     boolean generateFeaturesSuccess)
                              throws IOException
        Creates a temporary copy of the configuration file and checks the configFile in the temporary directory to avoid install-feature timing issues. Installs features if new features are detected.
        Parameters:
        fileChanged - the file that was changed
        srcDir - the directory of the file changed
        targetFileName - if not null renames the fileChanged to targetFileName in the targetDir
        generateFeaturesSuccess - if features were successfully generated, skip install features if false. Defaults to true if generateFeatures is off
        Throws:
        IOException - creating and copying to tempConfig directory

      • copyFile

        public void copyFile​(File fileChanged,
                     File srcDir,
                     File targetDir,
                     String targetFileName)
              throws IOException
        Copies the fileChanged from the srcDir to the targetDir.
        Parameters:
        fileChanged - the file that was changed
        srcDir - the directory of the file changed
        targetDir - the target directory
        targetFileName - if not null renames the fileChanged to targetFileName in the targetDir
        Throws:
        IOException - unable to resolve canonical path

      • deleteFile

        protected void deleteFile​(File deletedFile,
                          File dir,
                          File targetDir,
                          String targetFileName)
                   throws IOException
        Deletes the corresponding file in the targetDir.
        Parameters:
        deletedFile - the file that was deleted
        dir - the directory of the deletedFile
        targetDir - the corresponding targetDir of the deletedFile
        targetFileName - if not null deletes the targetFile with this name
        Throws:
        IOException - unable to resolve canonical path

      • cleanTargetDir

        protected void cleanTargetDir​(File outputDirectory)
        Delete all the Java class files within the specified directory. If the directory is empty, deletes the directory as well.
        Parameters:
        outputDirectory - the directory for compiled classes

      • registerAll

        protected void registerAll​(Path start,
                           ThreadPoolExecutor executor)
                    throws IOException
        Register the parent directory and all sub-directories with the WatchService
        Parameters:
        start - parent directory
        executor - the test thread executor
        Throws:
        IOException - unable to walk through file tree

      • registerAll

        protected void registerAll​(Path start,
                           ThreadPoolExecutor executor,
                           boolean removeOnContainerRebuild)
                    throws IOException
        Register the parent directory and all sub-directories with the WatchService
        Parameters:
        start - parent directory
        executor - the test thread executor
        removeOnContainerRebuild - whether the files should be unwatched if the container is rebuilt
        Throws:
        IOException - unable to walk through file tree

      • getFileFromConfigDirectory

        protected File getFileFromConfigDirectory​(String file)
        Get the file from the configDirectory if it exists
        Parameters:
        file -
        Returns:
        file or null if it does not exist

      • deleteJavaFile

        protected void deleteJavaFile​(File fileChanged,
                              File classesDir,
                              File compileSourceRoot)
                       throws IOException
        Given the fileChanged delete the corresponding Java class or directory
        Parameters:
        fileChanged - Java file changed
        classesDir - the directory for compiled classes
        compileSourceRoot - the source directory for the Java classes
        Throws:
        IOException - unable to resolve canonical path

      • recompileJavaSource

        protected boolean recompileJavaSource​(Collection<File> javaFilesChanged,
                                      Set<String> artifactPaths,
                                      ThreadPoolExecutor executor,
                                      File outputDirectory,
                                      File testOutputDirectory,
                                      String projectName,
                                      File projectBuildFile,
                                      JavaCompilerOptions projectCompilerOptions,
                                      boolean forceSkipUTs,
                                      boolean skipRunningTests)
                               throws PluginExecutionException
        Recompile Java source files and run tests after application update
        Parameters:
        javaFilesChanged - collection of Java files changed
        artifactPaths - list of project artifact paths for building the classpath
        executor - the test thread executor
        outputDirectory - the directory for compiled classes
        testOutputDirectory - the directory for compiled test classes
        projectName - the name of the current project (artifactId), null if only one project exists
        projectBuildFile - the build file of the current project
        projectCompilerOptions - the Java compiler options of the current project
        forceSkipUTs - whether to force skipping the unit tests
        skipRunningTests - whether to skip running tests, takes precedence over the forceSkipUTs param
        Throws:
        PluginExecutionException - if the classes output directory doesn't exist and can't be created

      • recompileJavaTest

        protected boolean recompileJavaTest​(Collection<File> javaFilesChanged,
                                    Set<String> artifactPaths,
                                    ThreadPoolExecutor executor,
                                    File outputDirectory,
                                    File testOutputDirectory,
                                    String projectName,
                                    File projectBuildFile,
                                    JavaCompilerOptions projectCompilerOptions,
                                    boolean forceSkipUTs,
                                    boolean skipRunningTests)
                             throws PluginExecutionException
        Recompile test source files and run tests immediately
        Parameters:
        javaFilesChanged - collection of Java files changed
        artifactPaths - list of project artifact paths for building the classpath
        executor - the test thread executor
        outputDirectory - the directory for compiled classes
        testOutputDirectory - the directory for compiled test classes
        projectName - the name of the current project (artifactId), null if only one project exists
        projectBuildFile - the build file of the current project
        projectCompilerOptions - the Java compiler options of the current project
        forceSkipUTs - whether to force skipping the unit tests
        skipRunningTests - whether to skip running tests, takes precedence over the forceSkipUTs param
        Throws:
        PluginExecutionException - if the classes output directory doesn't exist and can't be created

      • recompileJava

        protected boolean recompileJava​(Collection<File> javaFilesChanged,
                                Set<String> artifactPaths,
                                ThreadPoolExecutor executor,
                                boolean tests,
                                File outputDirectory,
                                File testOutputDirectory,
                                String projectName,
                                File projectBuildFile,
                                JavaCompilerOptions projectCompilerOptions,
                                boolean forceSkipUTs,
                                boolean skipRunningTests)
                         throws PluginExecutionException
        Recompile source files
        Parameters:
        javaFilesChanged - collection of Java files changed
        artifactPaths - list of project artifact paths for building the classpath
        executor - the test thread executor
        tests - indicates whether the files changed were test files
        outputDirectory - the directory for compiled classes
        testOutputDirectory - the directory for compiled test classes
        projectName - the name of the current project (artifactId), null if only one project exists
        projectBuildFile - the build file of the current project
        projectCompilerOptions - the Java compiler options of the current project
        forceSkipUTs - whether to force skipping the unit tests
        skipRunningTests - whether to skip running tests, takes precedence over the forceSkipUTs param
        Throws:
        PluginExecutionException - if the classes output directory doesn't exist and can't be created

      • getClassPath

        protected Set<File> getClassPath​(Set<String> artifactPaths,
                                 List<File> outputDirs)
                          throws IOException
        Gets the class path for the specified artifactPaths and outputDirs.
        Parameters:
        artifactPaths - list of artifacts for the current project
        outputDirs - list of output directories for the current project
        Returns:
        set of classpath files
        Throws:
        IOException - unable to resolve canonical path

      • runTestThread

        public void runTestThread​(boolean waitForApplicationUpdate,
                          ThreadPoolExecutor executor,
                          int messageOccurrences,
                          boolean manualInvocation,
                          File... currentBuildFiles)
        Run tests in a new thread.
        Parameters:
        waitForApplicationUpdate - whether it should wait for the application to update before running integration tests
        executor - the thread pool executor
        messageOccurrences - how many times the application updated message has occurred in the log
        manualInvocation - whether the tests were manually invoked
        currentBuildFiles - the build file(s) to run tests against

      • runTestThread

        public void runTestThread​(boolean waitForApplicationUpdate,
                          ThreadPoolExecutor executor,
                          int messageOccurrences,
                          boolean skipUnitTests,
                          boolean manualInvocation,
                          File... currentBuildFiles)
        Run tests in a new thread. Only call this method directly if you want to override the skip unit tests value.
        Parameters:
        waitForApplicationUpdate - whether it should wait for the application to update before running integration tests
        executor - the thread pool executor
        messageOccurrences - how many times the application updated message has occurred in the log
        skipUnitTests - whether to skip the unit tests
        manualInvocation - whether the tests were manually invoked
        currentBuildFiles - the build file(s) to run tests against

      • getHostName

        public String getHostName()
        Gets the Liberty server's host name.
        Returns:
        hostName the host name, or null if the server is not started

      • getHttpPort

        public String getHttpPort()
        Gets the Liberty server's http port.
        Returns:
        httpPort the http port, or null if the server is not started or there is no http port bound

      • getHttpsPort

        public String getHttpsPort()
        Gets the Liberty server's https port.
        Returns:
        httpsPort the https port, or null if the server is not started or there is no https port bound

      • setLibertyDebugPort

        public void setLibertyDebugPort​(int libertyDebugPort)
        Sets the preferred debug port.
        Parameters:
        libertyDebugPort - the preferred debug port

      • setPropertyFiles

        public void setPropertyFiles​(List<File> propertyFiles)
        This is needed for Gradle only. Sets additional property files that may be used by the build. Loads the properties for later comparison of changes.
        Parameters:
        propertyFiles - list of property files

      • getContainerName

        public String getContainerName()

      • getCompileArtifacts

        public Set<String> getCompileArtifacts()

      • getTestArtifacts

        public Set<String> getTestArtifacts()

      • updateJavaCompilerOptions

        public void updateJavaCompilerOptions​(JavaCompilerOptions updatedCompilerOptions)
        Updates the Java compiler options
        Parameters:
        updatedCompilerOptions -

      • getProjectModule

        public ProjectModule getProjectModule​(File buildFile)
                               throws IOException
        Given a build file returns the corresponding ProjectModule otherwise returns null
        Parameters:
        buildFile -
        Returns:
        Project module
        Throws:
        IOException

      • isMultiModuleProject

        public boolean isMultiModuleProject()
        Checks if upstreamProjects exist, indicating that this is a multi module project
        Returns:
        true if this is a multi module project, false if not

      • triggerMainModuleCompile

        protected void triggerMainModuleCompile​(boolean testsOnly)
                                 throws IOException
        Trigger a compile of the entire main module. The main module is the module with the Liberty configuration.Adds all Java files to the to be compiled list so that they will be compiled on next watch loop.
        Parameters:
        testsOnly - true if ONLY tests should be compiled
        Throws:
        IOException

      • triggerUpstreamModuleCompile

        protected void triggerUpstreamModuleCompile​(ProjectModule project,
                                            boolean testsOnly)
                                     throws IOException
        Trigger a compile of the entire specified module. This is only used in a multi-module scenario. Adds all Java files to the to be compiled list so that they will be compiled on next watch loop.
        Parameters:
        project - ProjectModule, the module to be compiled
        testsOnly - true if ONLY tests should be compiled
        Throws:
        IOException

      • writeDevcMetadata

        public void writeDevcMetadata​(boolean alive)
        Create metadata when running devc mode and containers Language server then uses metadata file to connect