Skip to content
GitLab
Projects Groups Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
  • Sign in / Register
  • I Ingestion Workflow
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
    • Locked Files
  • Issues 41
    • Issues 41
    • List
    • Boards
    • Service Desk
    • Milestones
    • Iterations
    • Requirements
  • Merge requests 8
    • Merge requests 8
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
    • Test Cases
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Packages and registries
    • Packages and registries
    • Package Registry
    • Container Registry
    • Infrastructure Registry
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Code review
    • Insights
    • Issue
    • Repository
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • Open Subsurface Data Universe SoftwareOpen Subsurface Data Universe Software
  • Platform
  • Data Flow
  • Data IngestionData Ingestion
  • Ingestion Workflow
  • Issues
  • #124
Closed
Open
Issue created Jul 30, 2021 by Aalekh Jain@aalekhjMaintainer

Contributing Integration tests to core [Failing ITs for AWS]

Current behaviour

There are too few integration tests in core which leads to non-robust testing of ingestion workflow across the cloud providers. This had also lead to non-consistent behaviour for some of the APIs across the various cloud provider's implementation which sometimes become a huge bottleneck in the development process by becoming a hurdle in this setting of cross-cloud collaboration.

Expected behaviour

The target is to make sure that the ingestion workflow APIs exhibit same behaviour across all the cloud providers and to also have a robust mechanism to improve the integration test coverage thereby resulting into a reliable service development going forward.


MAJOR CONTRIBUTIONS

The integration tests have been contributed by Azure which will benefit all the other cloud providers as well. This has been done as part of the following MR: !131 (merged)

A brief comparison of the contributions (number of ITs) are as follows -

AREA API ENDPOINT TYPE BEFORE AFTER
WORKFLOW /v1/workflow POST 3 8
WORKFLOW /v1/workflow GET 2 7
WORKFLOW /v1/workflow/{workflow_name} DELETE 1 5
WORKFLOW /v1/workflow/{workflow_name} GET 2 7
WORKFLOW RUN /v1/workflow/{workflow_name}/workflowRun POST 1 6
WORKFLOW RUN /v1/workflow/{workflow_name}/workflowRun GET 1 5
WORKFLOW RUN /v1/workflow/{workflow_name}/workflowRun/{runId} GET 1 6
WORKFLOW RUN /v1/workflow/{workflow_name}/workflowRun/{runId} PUT 1 10
COMBINED - TOTAL 12 54

Integration tests helps in identifying the functionality gap. Going forward, the expectation is to add/update the integration tests as per the common api spec in the core module which will aid in robust service development for all the cloud providers.


Current State of the MR

Link: !131 (merged)

Majority of the tests are passing for majority of the cloud providers, a brief summary of the failing tests are as follows -

Cloud Provider Number of failing tests LINK TO TEST RUN
AWS 0 https://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/jobs/483006
AZURE 0 https://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/jobs/483007
GCP 0 https://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/jobs/483005
IBM 0 https://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/jobs/483004

Would request the respective owners to have a look into this.

cc: @kibattul @shrikgar @wsmatth @Kateryna_Kurach

Edited Aug 09, 2021 by Aalekh Jain
Assignee
Assign to
Time tracking