Manifest ingestion fail on non-file-based datasets
The validation script validate_file_source.py, is rejecting datasets that are not matching the hardcoded types:
- FILE = ":dataset--File."
- FILE_COLLECTION = ":dataset--FileCollection."
- EDS_FILE = ":dataset--ConnectedSource."
Other dataset types that are not associated with files, should not be checked for any of the file attributes such as FileSource
. For example, the following manifest that contains an ETP dataset should be bypassing the file associated tests.
{
"kind": "osdu:wks:Manifest:1.0.0",
"Data": {
"Datasets": [
{
"acl": {
"viewers": [
"data.default.viewers@opendes.contoso.com"
],
"owners": [
"data.default.owners@opendes.contoso.com"
]
},
"kind": "osdu:wks:dataset--ETPDataspace:1.0.0",
"legal": {
"legaltags": [
"opendes-ReservoirDDMS-Legal-Tag"
],
"otherRelevantDataCountries": [
"US",
"UK"
]
},
"createTime": "2023-03-21T16:33:19.651Z",
"modifyTime": "2023-03-21T16:33:19.651Z",
"id": "opendes:dataset--ETPDataspace:M16_Demo-Volve_Reservoir",
"version": 1,
"data": {
"ExistenceKind": "opendes:reference-data--ExistenceKind:Actual:",
"DatasetProperties": {
"URI": "eml:///dataspace('M16_Demo/Volve_Reservoir')"
},
"Name": "M16_Demo/Volve_Reservoir"
}
}
]
}
}
Edited by Laurent Deny