Draft: hybrid trusted-pipeline routing (IBM disconnect + AWS/Azure/GCP advisory)

Purpose

This MR implements Phase 1 from ADR #58.

  • CIMPL becomes the voting merge gate.
  • AWS / Azure / GCP remain in the MR-connected trusted child pipeline as advisory reference signals.
  • IBM moves to a disconnected trusted-push pipeline.

Changes

standard-setup.yml

  • Workflow rule (push/web on trusted-*): allow push and web triggers on trusted-* branches so the disconnected trusted-push pipeline can exist for IBM. Previously only pipeline source was permitted.
  • Workflow rule scope (IBM-only): restrict the new push/web trusted-* pipeline type to IBM services only. This prevents legacy non-IBM templates from being triggered there.
  • trusted-merge-branch-verification: keep this restricted to source=pipeline, because disconnected trusted-push has no MR source branch to verify.

CSP files: AWS/Azure/GCP (advisory in MR-connected child, skipped on disconnected push)

File Jobs touched
cloud-providers/aws-global.yml aws-containerize, aws-update-helm, aws-update-tf, aws-update-eks
cloud-providers/aws-maven.yml aws-test-java, aws-acceptance-test
cloud-providers/aws-python.yml aws-test-python, aws-acceptance-test-python
cloud-providers/aws-one-pipeline.yml aws-compile-and-unit-test (csp-build)
cloud-providers/azure.yml azure_containerize, azure_containerize_helm, azure_deploy, azure_test, azure_test_py, azure_swagger_test, azure-acceptance-test, azure_code_coverage
cloud-providers/azure-one-pipeline.yml azure-compile-and-unit-test (csp-build)
cloud-providers/gc-com-gke.yml gc-deploy, gc-test, gc-acceptance-test, gc-test-python, gc-acceptance-test-python
cloud-providers/gc-containerize.yml gc-containerize-gitlab
cloud-providers/gc-helm.yml gc-helm-charts-gitlab
cloud-providers/gc-one-pipeline.yml gc-compile-and-unit-test (csp-build)

IBM files: disconnect to trusted-push pipeline

File Jobs touched
cloud-providers/ibm.yml ibm-deploy, ibm-deploy-devpri, ibm-test, ibm-acceptance-test, ibm-test-py
cloud-providers/ibm-one-pipeline.yml ibm-compile-and-unit-test (csp-build)

CIMPL files: guard against running in disconnected push pipeline

File Jobs touched
cloud-providers/cimpl-containerize.yml cimpl-containerize-gitlab, cimpl-containerize-bootstrap-gitlab
cloud-providers/cimpl-helm.yml cimpl-helm-charts-gitlab
cloud-providers/cimpl-rosa.yml cimpl-deploy, cimpl-test, cimpl-test-python, cimpl-acceptance-test, cimpl-acceptance-test-python
cloud-providers/cimpl-dag.yml cimpl-render-dag, cimpl-publish-dag, cimpl-deploy-dag, cimpl-postman-tests

Downstream needs:-dependency scope fixes

These downstream jobs have hard needs: edges to provider jobs whose routing changes in this MR. Without matching scope fixes, GitLab would reject pipeline instantiation because the upstream job would be absent.

File Job Upstream Treatment
cloud-providers/azure.yml container_scanning azure_containerize skip guard only; posture in MR-connected trusted child unchanged (stays voting)
cloud-providers/cimpl-openapi.yml cimpl-check-openapi-spec cimpl-deploy skip guard only; job already carries allow_failure: true
cloud-providers/gc-bootstrap.yml gc-containerize-bootstrap-gitlab gc-compile-and-unit-test (via gc-one-pipeline.yml override) skip guard plus advisory clause (matches sibling gc-containerize-gitlab treatment)

Legacy template scope fixes

Opening push / web on trusted-* affects workflow globally, but not all legacy provider templates initially had matching source guards.

As a result, legacy non-IBM templates using only: variables: [$CSP == '1'] could start provider jobs from the new disconnected pipeline in multi-cloud services that set both $IBM == '1' and another CSP flag. Converted to the skip-plus-advisory pattern:

File Jobs converted
cloud-providers/aws-dag.yml aws-bootstrap-dag, aws-publish-dag, aws-tests-end-to-end
cloud-providers/azure-function.yml azure_containerize, azure_deploy
cloud-providers/azure_dag.yml azure_standalone_tests, azure_build_dag, azure_copy_dag, azure_register_dag, azure_dag_postman_tests (pre-existing except: clauses preserved as when: never rules)
cloud-providers/azure-seismic-store-service.yml azure_deploy, azure_test
cloud-providers/gc-dag.yaml gc-render-dag, gc-publish-dag, gc-deploy-dag:community, gc-postman-tests:community (the :dev2 and :preship variants are already release/tag-scoped and safe)
cloud-providers/gc-mappers.yml gc-test (already rules:, added skip plus advisory clauses)

The IBM-scoped workflow rule above is the first line of defense. These template fixes handle multi-cloud services where that rule alone is not enough.

Legacy only:/except: to rules: conversion

AWS and most Azure jobs still used only:/except: on master. The conversion was required to express the new rule-based routing. GCP and CIMPL jobs already used rules: and needed only new clauses inserted.

Known scope notes

  • container_scanning and azure_code_coverage (Azure) carry the skip guard so they do not break the disconnected pipeline's dependency graph. Their posture in the MR-connected trusted child is unchanged; they do not gain advisory behavior from this MR.
  • Acceptance-test jobs already carrying allow_failure: true in master (gc-acceptance-test, gc-acceptance-test-python, aws-acceptance-test, aws-acceptance-test-python, azure-acceptance-test) keep that master behavior unchanged. This branch only adds the source guard.
  • IBM entrypoint variants (ibm-deploy-only.yml, ibm-java-git.yml, ibm-python-test.yml, ibm-policy.yml, ibm-policy-int-test.yml, ibm-seismic-store-service.yml, ibm-wellbore.yml, ibm-wellbore-git.yml, ibm-wellbore-bulk-worker.yml, ibm_search.yml, ibm_csv.yml) are not updated with the Phase 1a source guard. Services using these variants would run IBM jobs in both pipelines, creating duplicate runs until follow-up guard work lands. Services using the primary ibm.yml path are isolated.

Test plan

Acceptance criteria for Phase 1 pilot validation:

  • MR-connected trusted child on a service MR routes CIMPL as voting, AWS/Azure/GCP as advisory, and omits IBM.
  • Disconnected trusted-push on an IBM-enabled service runs IBM jobs only; CIMPL and AWS/Azure/GCP jobs absent.
  • Disconnected trusted-push is not created for non-IBM services.
  • trusted-merge-branch-verification runs only on the MR-connected trusted child and is skipped on disconnected trusted-push.
Edited by Daniel Scholl (MS]

Merge request reports

Loading