While performing the same workflow using the same data to ingest reference data type, I see that the new record ids are not getting created each time, Instead the version is getting incremented
When, try to ingest data where entity type is master, I see the new record id getting created each time I run the workflow, even though the collection and data file being used are same. To me this is the expected behavior.
When I try to do the same with entity type reference, I am seeing that the ids getting generated are same (not new) and the only new version is getting generated. So e.g. if I get count of the records for the entity type before and after the ingestion, the count remains same except for the first time. (in my case the count goes up by 4 for the first time and then it stays the same as my data has 4 records).
`So I modified the file to have one more record (total 5). When I ran the workflow again with additional one record, I saw the count going up by 1 and not by 5.
Before (inserting 5 records) { "results": [ { "id": "opendes:reference-data--ContractorType:LineClearing" } ], "aggregations": [ { "key": "osdu:wks:reference-data--ContractorType:1.0.0", "count": 9 } ], "totalCount": 9 }
After (inserting 5 records) { "results": [ { "id": "opendes:reference-data--ContractorType:LineClearing" } ], "aggregations": [ { "key": "osdu:wks:reference-data--ContractorType:1.0.0", "count": 10 } ], "totalCount": 10 }
Before (again inserting 5 records) { "results": [ { "id": "opendes:reference-data--ContractorType:LineClearing" } ], "aggregations": [ { "key": "osdu:wks:reference-data--ContractorType:1.0.0", "count": 10 } ], "totalCount": 10 }
After (again inserting 5 records) { "results": [ { "id": "opendes:reference-data--ContractorType:LineClearing" } ], "aggregations": [ { "key": "osdu:wks:reference-data--ContractorType:1.0.0", "count": 10 } ], "totalCount": 10 } ` CSVWorkflow__CI-CD_v2.0-ReferenceData.postman_collection.json
ReferenceData.csv (with f5 records)
The environment file can be gotten from https://community.opengroup.org/osdu/platform/pre-shipping/-/tree/main/R3-M16/QA_Artifacts_M16/envFilesAndCollections/envFiles OR https://community.opengroup.org/osdu/platform/testing/-/tree/master/Postman%20Collection/00_CICD_Setup_Environment