This changes the tags on most of the jobs to be osdu-* (where the * is 'small', 'medium', or 'large').
These tags cause the jobs to be picked up by different runners, which are specifically available for OSDU's use (and not generally available on community). The new runners are configured to launch new machine instances for each job, rather than sharing machine resources among many jobs. As a result, each job should indicate how many resources they need available to execute (the small/medium/large bit).
As of this MR, the runner configuration enables a greater concurrence limit for the small jobs, and also maintains several small machines in hot reserve -- decreasing the startup time of the job. These configuration will undoubtedly change based on experience, but in general, jobs should stay at the smallest level that gives adequate performance -- leaving the larger machine instances available for jobs that can actually utilize the extra vCPUS / Memory.
Important Note About Forks
These runners only work for projects under the OSDU group. Forks in private namespaces will not be able to use them. I plan to address this later by enabling a configuration to switch to the community runners in private namespaces. If this workflow is important to you, let me know so I can provide you workarounds and prioritize a general fix.
Example Pipeline
https://community.opengroup.org/osdu/platform/security-and-compliance/legal/-/pipelines/2992
Example Contributor MR