[Validation] Dataset file or file collection has been already ingested into OSDU before ingesting it’s metadata
Validation whether Dataset file or file collection has been already ingested into OSDU before ingesting it’s metadata.
Scope: Dataset
The logic for this check will be a little bit different depending on the type of the Dataset (File or File Collection):
For File type (schema “AbstractFileSourceInfo” is used: https://gitlab.opengroup.org/osdu/subcommittees/data-def/work-products/schema/-/blob/master/Authoring/abstract/AbstractFileSourceInfo.1.0.0.json ) validation should be performed to check that “FileSource” parameter exists
For File Collection type (schema “AbstractFileCollection” schema https://gitlab.opengroup.org/osdu/subcommittees/data-def/work-products/schema/-/blob/master/Authoring/abstract/AbstractFileCollection.1.0.0.json ) the following validation steps should be performed: Step 1: Does “IndexFilePath” exist?
If yes -> validation pass
If no -> proceed to Step 2
Step 2: For each file in the collection check whether “FileSource” parameter exists.
If yes -> validation pass
If no -> validation fail
If validation fails, reject the whole WPC this Dataset belongs to.