Skip to content

Generalized Maven Build Scripts

David Diederich requested to merge maven-build-script into master

Background - Cached Maven Build Scripts

A while back (!186 (merged)) we started caching the output of maven to a file, then uploading the file as an artifact. For still unknown reasons, this dramatically improved the performance of the compile-and-unit-test job. Users could no longer watch builds as they ran, but since the overall runs were up to 10x faster, it was a good trade. The last few lines of the run are echoed to the screen, while the full log is available to download as an artifact.

Generalize to the .maven job

The main point of this MR is to generalize that behavior beyond compile-and-unit-test to anything that relies on the .maven job. By moving the script creation up to this "base class" job, we can use it as part of integration testing, fossa scans, and anything else that needs to use maven as part of its operation.

To use this, anywhere you would run $MAVEN args..., instead use $MAVEN_BUILD . maven-OPERATION-output.txt args.... The first argument to $MAVEN_BUILD is the directory to run in. Typically this is ., but you could use it to build in subdirectories without having to cd to it. The second argument is the filename to capture the output to (relative to the path specified in the first argument). Anything can be used, but if it follows the pattern maven-*-output.txt, it will automatically be picked up as an artifact.

Extra Features

  • Output is colored: Commands in bright green, results in cyan or red. Makes it easier to find the start / end of each maven command.
  • Collapsible sections: The truncated output of each command is independently collapsible, if you only want to look at one command
  • Timed sections: The time for each maven command is included in the job log output. Note: This doesn't show if you are watching the build while it operates. Must refresh after the job completes to see this.

Simpler Loop Execution for Maven Projects

In addition to the $MAVEN_BUILD script, there is now a $MAVEN_FOREACH script. This script runs the $MAVEN_BUILD script for each of the parent pom files. The directories of the parent pom files are (as before) printed at the beginning of each job log, and stored in the null-separated variable $ALL_MAVEN_BUILD_DIRS_Z. To use this script, run $MAVEN_FOREACH maven-OPERATION-output.txt args... -- basically the same as $MAVEN_BUILD, but without the directory. Remember, the output file will be relative to the execution directory, so this creates many different output files, each sorted into the directories of the main pom.

Upgrade Other Parts of the CI-CD Pipeline

In addition to changing the build/maven.yml file, I also updated the fossa-maven.yml and several cloud-provider/ files that seemed to be using maven. I believe I preserved the behavior, but I would like experts for each of these files to take a look.

I ran a multi-pipeline on several different services / libraries, restricting the pipeline stages to only the files that I changed by this MR.

Edited by David Diederich

Merge request reports