Manifest ingestion fail on non-file-based datasets
The validation script validate_file_source.py, is rejecting datasets that are not matching the hardcoded types:
- FILE = ":dataset--File."
- FILE_COLLECTION = ":dataset--FileCollection."
- EDS_FILE = ":dataset--ConnectedSource."
Other dataset types that are not associated with files, should not be checked for any of the file attributes such as FileSource. For example, the following manifest that contains an ETP dataset should be bypassing the file associated tests.
{
"kind": "osdu:wks:Manifest:1.0.0",
"Data": {
"Datasets": [
{
"acl": {
"viewers": [
"data.default.viewers@opendes.contoso.com"
],
"owners": [
"data.default.owners@opendes.contoso.com"
]
},
"kind": "osdu:wks:dataset--ETPDataspace:1.0.0",
"legal": {
"legaltags": [
"opendes-ReservoirDDMS-Legal-Tag"
],
"otherRelevantDataCountries": [
"US",
"UK"
]
},
"createTime": "2023-03-21T16:33:19.651Z",
"modifyTime": "2023-03-21T16:33:19.651Z",
"id": "opendes:dataset--ETPDataspace:M16_Demo-Volve_Reservoir",
"version": 1,
"data": {
"ExistenceKind": "opendes:reference-data--ExistenceKind:Actual:",
"DatasetProperties": {
"URI": "eml:///dataspace('M16_Demo/Volve_Reservoir')"
},
"Name": "M16_Demo/Volve_Reservoir"
}
}
]
}
}
Edited by Laurent Deny