@@ -4,7 +4,7 @@ This page serves as a documentation of the different environments that are used
## Challenges with Mercury baseline setup
* With only one dev environment in community Gitlab, trusted builds cant be run in parallel without impacting the CD/integration tests in the target environment. This forces a code freeze at milestones to allow tagging and completion of
* _Note_- OSDU data platform is composed of multiple projects and each project's CICD only tests their own integration tests to identify a green build. However the platform wide sanity would require a consistent green build across projects. Example a change to indexer service that happened after search service may deem indexer as passed but you need to retest the search to catch the disconnect.
* _Note_- OSDU data platform is composed of multiple projects and each project's CICD only tests their own integration tests to identify a green build. However the platform wide sanity would require a consistent green build across projects. Example a change to indexer service that happened after search service may deem indexer as passed but you need to retest the search to catch the disconnect.
* Manual testing of end-to-end and exploratory tests require some assurance for the testers that the system has passed the automation tests for integration and functionality. Without a clear separation of the manual test pre-ship/emulated production environment and the automated test execution QA environment, this assurance doesn't exist in Mercury
* QA automation test developers also require a stable baseline with green build to develop/validate their test suites. Today this is done either in pre-ship or in QA/execution environment. Doing in pre-ship implies test suites are developed where production users are doing e2e tests already (see assurance comment above). Doing in QA environment implies having a durable QA environment (not available in all CSP) and qualified milestone deliveries into the QA.
* QA and pre-ship production test environments require test data sets (currently Volve and TNO) to be loaded so the system is tested with real-world data rather than fabricated ones in the unit tests within the CICD space in Gitlab projects.
...
...
@@ -33,10 +33,13 @@ Cognizant of costs and maintenance overhead, we categorize the environments to a
### Transient Environments
***QA Development Secondary Environment**
This a recommended environment to provide a stable environment for the QA team to build functional tests for new capabilities in the platform. This will start initially manual during test development before they can automate this to create postman/newman scripts. The goal is to explore, document and automate the test cases and append the suite to be ready for push into durable QA automation environment. At the moment the QA engineers will need to compete with automation tests and milestone updates in QA automation environment for their own development. Because this requires test data and the assurance before pre-ship it should also be fed with build artifacts from the CSP/provider's CICD rather than just the community Gitlab - so tests run in simulated prod environment.
***Preship Secondary Environment for Operational Procedure Testing**
Because Production environment is mainly for operator and representative users to exercise the system before shipping out, the tests should assure the environment will not be impacted negatively. This means any tests around load/performance tests or operational tests such as disaster recovery, backup/restore, service rollback/roll-forward strategies need to be done in an isolated space. This environment is a transient setup that can allow for such potentially destructive tests - as a mock production environment, this should be populated from the CSP/provider's CICD rather than just the community Gitlab and the test data sets (Volve, TNO) should be pre-populated here.
***Preship Secondary Environment**
***Preship Secondary Environment for Security Testing**
This is similar to the previous transient preship secondary environment - any tests around security such as pen tests, port-scans, DOS attacks etc. will be done in this temporary but isolated space. As a mock production environment, this should be populated from the CSP/provider's CICD rather than just the community Gitlab and the test data sets (Volve, TNO) should be pre-populated here.
