LQCIndicator.1.0.0 Custom Reference Schema OSDU Search API Issue results in WellLogTypeID field not appearing in search result where data is available at storage
Hello OSDU Forum Team,
I from Shell LQC OSDU Migration Project reaching out to you regarding an OSDU Search Issue facing with an LQC Custom Reference Schema/Data type. Jake from DP team (Jake.J.Pearce@shell.com) is working on this issue, he suggested us to raise this as an issue with OSDU Forum to get some assistance on this. I have explained the whole issue in step-by-step as below, hope it helps you in understanding it.
In case if need any more information this issue, I am happy to explain the issue in detail over a call. Please share me the required associate's names and your convenient time to setup a call. My Email-id: vedantha.gowda@shell.com
PFA LQCIndicator Schema JSON file for your reference. LQCIndicator_Custom_Reference_Schema.json
Quick Summary of the issue Field "WellLogTypeID" displaying "null" for LQCIndicator:1.0.0 custom reference schema when results are returned for the kind via the Search API. The custom schema is referencing OSDU public data.
Please find the below details regarding the issue,
- LQCIndicator.1.0.0 is an LQC Custom Reference Schema/Data Type designed and approved from Shell Data Architect team.
- DP Team has ingested/registered the schema in all OSDU Environments – US Instance.
- LQC Team prepared the reference data as per the business requirement and handed over it to RDM Team in an Excel Sheet to ingest into LQCIndicator Schema.
- RDM Team has built an Informatica Pipeline, prepared an manifest payload and ingested the reference data into LQCIndicator Schema in OSDU Acceptance Environment US Instance.
- RDM team were using OSDU Manifest Ingestion API Service for ingesting the LQCIndicator Reference data into OSDU through Informatica pipeline.
- LQCIndicator has total 42 records and all of them successfully ingested into OSDU Acc Env, but for only 8 records the WellLogTypeID field is appearing as NULL, even though expected data is available in storage.
- Out of 42 LQCIndicator records 8 records are referencing to LOGTYPE:Interpreted and rest 34 records referencing to ConveyanceMethod:LoggingWhileDrilling and ConveyanceMethod:ElectricWirelineConveyed. All of the records referencing LogType:Interpreted are displaying "null" in the WellLogTypeID field. The other referenced fields (for ConveyanceMethod:LoggingWhileDrilling and ConveyanceMethod:ElectricWirelineConveyed) are displaying values as they should.
- When we queried the LQCIndicator schema kind through OSDU Search API service the WellLogTypeID field is showing NULL
- When we queried the LQCIndicator schema kind through OSDU Storage API Service the WellLogTypeID field is showing the correct data value - ("WellLogTypeID": "osdu:reference-data--LogType:Interpreted:").
- The 8 records (id) where we are facing WellLogTypeID field NULL issue are listed at the bottom.
- The WellLogTypeID & ConveyanceMethodID both are similar fields as per schema design. Both are referencing to OSDU Forum Schema’s. ConveyanceMethod referencing to schema – “osdu:wks:reference-data--ConveyanceMethod:1.0.0” and WellLogTypeID referencing to schema “osdu:wks:reference-data--LogType:1.0.0”.
- ConveyanceMethodID attribute works off the same referencing mechanism as WellLogTypeID, however this field is displaying values properly when obtained through the Search API - "ConveyanceMethodID": "osdu:reference-data--ConveyanceMethod:LoggingWhileDrilling:", as per the business requirement (The rest 34 LQCIndicator records).
- A new OSDU environment was created at Shell and we have successfully re-created the LQCIndicator Schema WellLogTypeID field NULL Issue in the new OSDU Environment.
- As suggested by DP team we are raising this issue to OSDU Forum and requesting you to help us on resolving this issue.
- I have attached the LQC Custom Reference Schema for your reference.
- I am happy to supply any further information as required
Screenshot's of WellLogTypeID field NULL issue in LQCIndicator.1.0.0 Schema/Data type
Queried through - OSDU Search API Service:
Queried through - OSDU Storage API Service:
Below is Search Query to find it in AWS@shell OSDU Acceptance Env - US Instance { "kind": "shell:wks:reference-data--LQCIndicator:1.0.0", "returnedFields": ["id", "data.WellLogTypeID"] }
Below are the 8 - LQCIndicator id’s out of 42 the WellLogTypeID data NULL issue (for the rest 34 LQCIndicator – id’s the WellLogTypeID should be NULL, where ConveyanceMethodID attribute will be populated)
osdu:reference-data--LQCIndicator:24 osdu:reference-data--LQCIndicator:25 osdu:reference-data--LQCIndicator:26 osdu:reference-data--LQCIndicator:27 osdu:reference-data--LQCIndicator:28 osdu:reference-data--LQCIndicator:29 osdu:reference-data--LQCIndicator:30 osdu:reference-data--LQCIndicator:31
Looking forward for your guidance and help in resolving this issue. Thank you