This page serves as a documentation of the different environments that are used to develop, test, qualify and release the OSDU data platform on the various supported cloud backends.
This page serves as a documentation of the different environments that are used to develop, test, qualify and release the OSDU data platform on the various supported cloud backends.
## Environments Proposal:
These are the proposed environments for handling development, Automated QA and manual validation of the final product in preshipping.
## Challenges with Mercury baseline setup
## Challenges with Mercury baseline setup
* With only one dev environment in community Gitlab, trusted builds cant be run in parallel without impacting the CD/integration tests in the target environment. This forces a code freeze at milestones to allow tagging and completion of
* With only one dev environment in community Gitlab, trusted builds cant be run in parallel without impacting the CD/integration tests in the target environment. This forces a code freeze at milestones to allow tagging and completion of
* _Note _- OSDU data platform is composed of multiple projects and each project's CICD only tests their own integration tests to identify a green build. However the platform wide sanity would require a consistent green build across projects. Example a change to indexer service that happened after search service may deem indexer as passed but you need to retest the search to catch the disconnect.
* _Note _- OSDU data platform is composed of multiple projects and each project's CICD only tests their own integration tests to identify a green build. However the platform wide sanity would require a consistent green build across projects. Example a change to indexer service that happened after search service may deem indexer as passed but you need to retest the search to catch the disconnect.
* Manual testing of end-to-end and exploratory tests require some assurance for the testers that the system has passed the automation tests for integration and functionality. Without a clear separation of the manual test pre-ship/emulated production environment and the automated test execution QA environment, this assurance doesn't exist in Mercury
* Manual testing of end-to-end and exploratory tests require some assurance for the testers that the system has passed the automation tests for integration and functionality. Without a clear separation of the manual test pre-ship/emulated production environment and the automated test execution QA environment, this assurance doesn't exist in Mercury
* QA automation test developers also require a stable baseline with green build to develop/validate their test suites. Today this is done either in pre-ship or in QA/execution environment. Doing in pre-ship implies test suites are developed where production users are doing e2e tests already (see assurance comment above). Doing in QA environment implies having a durable QA environment (not available in all CSP) and qualified milestone deliveries into the QA.
* QA automation test developers also require a stable baseline with green build to develop/validate their test suites. Today this is done either in pre-ship or in QA/execution environment. Doing in pre-ship implies test suites are developed where production users are doing e2e tests already (see assurance comment above). Doing in QA environment implies having a durable QA environment (not available in all CSP) and qualified milestone deliveries into the QA.
* QA and pre-ship production test environments require test data sets (currently Volve and TNO) to be loaded so the system is tested with real-world data rather than fabricated ones in the unit tests within the CICD space in Gitlab projects.
## Environments Proposal:
These are the proposed environments for handling development, Automated QA and manual validation of the final product in preshipping.
## Environment Definition
## Environment Definition
Recommendation is for some durable environments to be created
Cognizant of costs and maintenance overhead, we categorize the environments to a smaller set of durable environments and an on-demand transient environment that can be instantiated for the duration of a given test or engagement.
### Durable Environments
***Dev Primary Environment**
* This exists today and linked to the community Gitlab CICD. The master branch regular builds rely on this environment for validating new MRs submitted to Gitlab. The environment currently runs project level unit and integration tests in an automated fashion. No platform level validation/integration tests are performed.
***Dev Secondary Environment**
* This is a proposed environment also linked to the community Gitlab CICD. This allows us to avoid a code freeze when the community reaches a milestone and to allow for parallel trusted builds to run for both the tagging/milestone builds in Dev2 and the mater branch MR builds in Dev primary. No platform level validation/integration tests are performed.
***QA Automation Environment**
* It is important for the community that the automated QA tests are done with a) real-world data sets (TNO, Volve, etc.) and b) with the representative deliverable they will receive from the provider/CSP own CICD. Note that the deployed environment does not pick up binaries from the milestone tagged version in community Gitlab, so the need to integrate the outputs from the provider/CSP CICD into the automated QA environment. Milestone releases should be pushed in here and all the integration (functional) tests will be run in an automated fashion here.
***Preship Production Environment**
* This is an environment that provides an opportunity for operators and potential OSDU consumers to volunteer and test as a representative end-user/customer. The feedback will improve both the community code quality and also the operational/security/deployment aspects brought to fruition by the provider/CSP.
### Transient Environments
***QA Development Secondary Environment**
***Preship Secondary Environment for Operational Procedure Testing**
***Preship Secondary Environment**
## Recommendation:
The following picture is a high-level recommendation for both durable and transient environments to be created
