Draft: MultipleVendor_SingleTestSuite
We have been noticing below issues in maintaining vendor-specific test suites-
- Most of the tests are common to each vendor but still they are run in every vendor-specific pipelines.
- For every feature being added, we may again be writing new test cases/modifying existing ones in every vendor module. Mostly duplication of effort.
- It is difficult to have insight on coverage of individual test suites.
- There are 2 different implementations of the test - cucumber and JUnit.
- Overall journey for the MR is getting increased.
We intend to convert all JUnit test cases into cucumber-based test and merge all vendor specific tests under file-test-core module.This will help us resolve all the issues listed above.
In this MR, We have converted Azure JUnit tests into cucumber and kept them under file-test-core module along with GCP tests. The test suite has few parameterized values which need to be passed from pipeline. We can collect and share the list later.
Request you all to check the feature files in this MR to see if the current tests already cover all your individual JUnit test cases and if not we can add them. If you feel few test cases need to be run only for a given vendor, we can device a way to handle this situation.
NOTE: We have currently disabled few test cases in the test-core module temporarily as there were some issues in them but are working to fix them soon. But we still can start making necessary changes in our pipelines to adopt this testing strategy.
Questions/suggestions are welcome!