Skip to content
GitLab
Projects Groups Topics Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
  • Register
  • Sign in
  • S Storage
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributor statistics
    • Graph
    • Compare revisions
    • Locked files
  • Issues 53
    • Issues 53
    • List
    • Boards
    • Service Desk
    • Milestones
    • Iterations
    • Requirements
  • Merge requests 16
    • Merge requests 16
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
    • Test cases
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Packages and registries
    • Packages and registries
    • Package Registry
    • Container Registry
    • Infrastructure Registry
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Code review
    • Insights
    • Issue
    • Repository
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • Open Subsurface Data Universe SoftwareOpen Subsurface Data Universe Software
  • Platform
  • System
  • Storage
  • Merge requests
  • !605

Conversion: validate records at the start using json schemas

  • Review changes

  • Download
  • Email patches
  • Plain diff
Closed Mykyta Savchuk requested to merge batch-spatial-location-array-500 into master Jan 26, 2023
  • Overview 8
  • Commits 7
  • Pipelines 20
  • Changes 10

There is a bug, when SpatialLocation in record is array, storage returns 500 on batch. Example record data:

"SpatialLocation": [
            {
                "SpatialGeometryTypeID": "Point",
....
]

And there were a lot of similar bugs related to record geo data validation lately (like misspelling field names and so on, that resulted in 500), but the existing validation logic is not intuitive and is spread across the code with random checks, so to fix them we had to add different checks (if's) in different parts of the code, and to make this logic more robust and prevent future bugs it was decided to do a refactoring and place the validation in one place using json schemas, since we already have them, and remove any further validation checks. Also, as we already have a well defined schema for the SpatialLocation geo attribute, I added a validation for such case in addition to the existing generic geo attribute validation, which I also converted to json schema (GenericGeoAttribute.json).

  • Changed conversion logic to add validation against the schema at the start of the flow;
  • Removed unnecessary validation checks;
  • In case of validation error, the message from validation library is returned, instead of constant predefined messages;
  • I changed tests' data (unit and integration), because the schema for AnyCrsFeatureCollection says that if bbox is present, it should not be null, so I added a sample value.
Edited Feb 13, 2023 by Mykyta Savchuk
Assignee
Assign to
Reviewers
Request review from
Time tracking
Source branch: batch-spatial-location-array-500