ADR: Dependabot Integration for Auto Dependency Update to Improve Vulnerability Fix Process

Status

  • Proposed
  • Trialing
  • Under review
  • Approved
  • Retired

Context & Scope

Dependency update for vulnerability fixes has been a manual process in the community. Sometimes, a vulnerability from one outdated dependency can impact many projects and require many files to be changed to fix it. The manual process is time consuming and error prone. It is also a reactive instead of a proactive process.

Dependabot integrated in a GitLab pipeline can automatically identify and create merge requests for updating outdated dependencies in the project, essentially minimizing the manual effort required to keep software secure by ensuring dependencies are regularly updated with the latest security patches and bug fixes, which is crucial for maintaining a healthy software ecosystem.

Dependabot-gitlab is available for free for public GitLab repositories, while premium plans are available for private repositories.

This ADR proposes to integrate the free Dependabot-gitlab tool (https://dependabot-gitlab.gitlab.io/dependabot/guide/) into the community GitLab CICD pipelines to enable auto dependency scan and dependency update merge request creation thus to improve the manual vulnerability fix process.

Decision

Adoption of the open-source Dependabot-gitlab tool still in alpha phase

Dependabot-gitlab tool uses GitHub’s open source dependabot-core for dependency update logic and adds additional functionality to integrate these updates with GitLab. It is under MIT license and still considered in alpha status.

The container image we use will be from Dockerhub “docker.io/andrcuns/dependabot-gitlab:latest”. It has been around for over four years with over one million downloads. GitHub has Dependabot as a build-in tool which GitLab lacks. This tool will only be used as part of development process in CICD pipelines and will not be included as part of Data Platform distribution itself and we will not enable auto-approval for the merge requests generated by the tool, we think it is an acceptable option considering the benefits.

Gradual integration

Rationale

Dependabot-gitlab can support a wide range of programming languages and package managers, including JavaScript (npm, yarn), Python (pip), Ruby (Bundler), Java (Maven, Gradle), and more.

  • We will gradually add integration for all projects starting from Java projects first and then Python projects.
  • The dependency scan frequency by the Dependabot tool can be configured in the pipeline for each project separately. We plan to start with weekly scan and adjust the frequency as needed.
  • In each project, we can configure to scan different folders so that each folder can generate its own merge requests. Each merge request will include only one discovered dependency upgrade. This will allow the flexibility for each CSP and CI folders to be scanned separately and optionally and each merge request will be clearly targeting the configured folder. We plan to add scan for azure folders first to make sure things work as expected and make configuration adjustments after some experimenting period. Scanning for other CSPs and CI folders can be added later gradually.
  • The generated merge request can be customized through the Dependendabot pipeline parameters in multiple ways including prefix, assignee, approvers and auto-approval option. We plan not to enable auto-approval.

Vulnerability alerts

When dependabot-gitlab detects security vulnerability in a dependency but is unable to update it with a merge request, it will create security vulnerability issue instead. It is possible to configure assignee for vulnerability issue via vulnerability-alert configuration option.

Consequences

With the tool adoption and integration with CICD pipelines, we will have an automated, faster and proactive way to detect and update outdated dependencies. It will not completely replace all the manual update needed though because the tool might miss or fail to update some. Also the generated merge requests still should be manually examined by an approval. More complex dependency update that will require code changes to support a new version of the dependency will still need to be done carefully manually.

PoC work

PoC has been done to figure out the detailed pipeline changes needed and related configurations. Dataset service, Indexer service and Seismic ddms were used as the pilot services.

When to revisit

If GitLab starts to offer build-in Dependabot support similar to GitHub, we will not need to use the additional Dependabot-gitlab open-source tool and the mechanism for integration with pipelines probably will change and become easier.

Tradeoff Analysis - Input to decision

Alternatives and implications

Decision criteria and tradeoffs

Decision timeline

We will start the work in M25 once ADR is approved and continue with completing all services repos in Venus.

Edited by Lucy Liu