Skip to content
GitLab
  • Menu
Projects Groups Snippets
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
  • Sign in / Register
  • P Python SDK
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
    • Locked Files
  • Issues 2
    • Issues 2
    • List
    • Boards
    • Service Desk
    • Milestones
    • Iterations
    • Requirements
  • Merge requests 1
    • Merge requests 1
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
    • Test Cases
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Packages & Registries
    • Packages & Registries
    • Package Registry
    • Container Registry
    • Infrastructure Registry
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Code review
    • Insights
    • Issue
    • Repository
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • Open Subsurface Data Universe Software
  • Platform
  • System
  • SDKs
  • Python SDK
  • Merge requests
  • !23

GONRG-2696: Manifest integrity batch search

  • Review changes

  • Download
  • Email patches
  • Plain diff
Merged Yan Sushchynski (EPAM) requested to merge GONRG-2696_Manifest_integrity_batch_search into master Aug 10, 2021
  • Overview 0
  • Commits 1
  • Pipelines 2
  • Changes 25

Type of change

  • Bug Fix
  • Feature

Does this introduce a change in the core logic?

  • [Yes]

Does this introduce a change in the cloud provider implementation, if so which cloud?

  • AWS
  • Azure
  • GCP
  • IBM

Updates description?

Reimplement logic of Manifest Integrity Check.

Before this MR, each entity of the Manifest was traversed during Manifest Integrity check. During this traversing, references to other entities (both in the Manifest or on OSDU) were collected and if they weren't present in Manifest, they were searched in Search service. And if there were missing entities on OSDU, then the entity containing these missing references was excluded from the manifest. This sequence of actions were repeating for each entity in the Manifest.

Now, the process of Integrity Check has following steps:

  1. Traverse the Manifest and create the dependency graph with parent-child relations between entities. Referrer is a child, referent is a parent.
  2. If there are references to entities not in the Manifest, mark such entities as external.
  3. Try to find all external entities in Search Service splitting the list of external entities into chunks.
  4. If there are missing entities (records) on OSDU, mark them and their children (referrers) as invalid and exclude them from the manifest.
Edited Aug 20, 2021 by Yan Sushchynski (EPAM)
Assignee
Assign to
Reviewer
Request review from
Time tracking
Source branch: GONRG-2696_Manifest_integrity_batch_search