As mentioned, I shared this issue with a couple of members. Eric Griffith from PDS Group (very active in our WITSML SIG and a leader of our Architecture Team, ETP development and WITSML-OSDU mapping and gap analysis) replied:
I can’t see internal details as we’re not OSDU members, but Etienne is reporting that the Energistics provided parsers are generating OSDU wellbore documents with incorrect references to VerticalMeasurementType and VerticalMeasurementPath reference data.
I’m unable to find official documentation on how a reference is supposed to be constructed, but what I’m inferring from some schemas and examples is that each reference data value defines an ID that should be used when referencing it (either using a code like MD or TVD or a more complex identifier like BoundCRS::OSDU::23031018).
Etienne is saying that the parsers are not using the expected ID for some of the references it is generating. He implies that the parsers are using an older version of the ID, which is no longer in use. This highlights a potential versioning issue in OSDU: there does not appear to be a way to indicate which version of the reference data you’re referring to. Again, since I don’t know the formal rules for constructing a reference, this may not actually be an issue.
Handling known and potentially unknown variations of reference data IDs seems to be something Thomas and Debasis are considering for future work.
I think the expected action at present is for the parsers to be updated to provide the currently expected reference data IDs in the references to reference data. While Etienne reported a specific issue with the wellbore parser, similar issues may exist with the other parsers.
Jana: As you know, we no longer have a developer on staff and Laurent is fully immersed in the Reservoir DDMS project at this time. It would be helpful if the OSDU community can identify someone who can support the parsers going forward. We will assist with background as needed.
I am seeking input from members who have worked on the parsers and who have been working on a mapping and gap analysis for the next version of WITSML to help ensure WITSML v2.1 includes elements currently in the JSON schemas but not in WITSML v2.0. Our goal is to put v2.1 out for public review before year-end so we have been very focused on this as well as updates to both PRODML and RESQML before year end.
Jay Hollingsworth should be included on this issue as well. He is out of office through next week, back October 18th. I will update this issue if I get feedback from our SIG team members.
Jana Schey Chief Operating Officer [cid:image001.jpg@01D7BB57.BA80EFC0]
From: Debasis Chatterjee (@debasisc) notifications@community.opengroup.org Sent: Wednesday, October 6, 2021 8:17 PM To: Jana Schey jana.schey@energistics.org Subject: Energistics - OSDU Integration | IBM R3M8 - Failure to ingest Wellbore data from WITSML source (#43 (closed))
Debasis Chatterjeehttps://urldefense.proofpoint.com/v2/url?u=https-3A__community.opengroup.org_debasisc&d=DwMCaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=7aHxb10vdT9B-HnJRxkATcLVAWPjzM-wUneEZGYXiR0&m=iaAWnN-MKX1XPSVhP4fgPxmKMoa3aWr8e5ZrjA9tTGY&s=cFVH4fqiUqvux3ApRvEl5N_iQ7UHbtrjlgIqCc41ATI&e= created an issue: #43 (closed)https://urldefense.proofpoint.com/v2/url?u=https-3A__community.opengroup.org_osdu_platform_data-2Dflow_ingestion_energistics-2Dosdu-2Dintegration_-2D_issues_43&d=DwMCaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=7aHxb10vdT9B-HnJRxkATcLVAWPjzM-wUneEZGYXiR0&m=iaAWnN-MKX1XPSVhP4fgPxmKMoa3aWr8e5ZrjA9tTGY&s=PzbqaiHxIe1MZurNoUtUfKcm9fFgt8fDwtTvWn-5ZU8&e=
Reported by @epeyssonhttps://urldefense.proofpoint.com/v2/url?u=https-3A__community.opengroup.org_epeysson&d=DwMCaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=7aHxb10vdT9B-HnJRxkATcLVAWPjzM-wUneEZGYXiR0&m=iaAWnN-MKX1XPSVhP4fgPxmKMoa3aWr8e5ZrjA9tTGY&s=NbdfkkCKq3VYStMhHMPx14m73Klg89EoGOMTs0GQW08&e= .
From Airflow log, we see the following as reason of failure.
"Missing referential id: {
'opendes:reference-data--VerticalMeasurementType:TotalDepth:',
'opendes:reference-data--VerticalMeasurementPath:MeasuredDepth:',
'opendes:reference-data--VerticalMeasurementPath:TrueVerticalDepth:',
After some investigation, it was determined that OSDU Reference value convention has changed to use code instead.
reference-data--VerticalMeasurementType:TD instead of TotalDepth.
reference-data--VerticalMeasurementPath:TVD instead of TrueVerticalDepth
reference-data--VerticalMeasurementPath:MD instead of MeasuredDepth
One possible solution is to change code for such changes.
Another alternative can be to hold field mapping in some configuration file so that it is easy to handle this kind of change in the future.
For other data types (Well, Marker, Log, Trajectory, Tubular), there will be potentially other mismatches too.
Copying to @janas712https://urldefense.proofpoint.com/v2/url?u=https-3A__community.opengroup.org_janas712&d=DwMCaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=7aHxb10vdT9B-HnJRxkATcLVAWPjzM-wUneEZGYXiR0&m=iaAWnN-MKX1XPSVhP4fgPxmKMoa3aWr8e5ZrjA9tTGY&s=i-usDqbhk3qUaMRwqiFmRs0JJD3OnBRTNfft5s9qMes&e= for her input on the subject
Also adding @gehrmannhttps://urldefense.proofpoint.com/v2/url?u=https-3A__community.opengroup.org_gehrmann&d=DwMCaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=7aHxb10vdT9B-HnJRxkATcLVAWPjzM-wUneEZGYXiR0&m=iaAWnN-MKX1XPSVhP4fgPxmKMoa3aWr8e5ZrjA9tTGY&s=PZF5b4GoHb3GxcUqj7cB_DnSJ4Nf2uVhmdwdZnkEddc&e= for his awareness
cc - @ChrisZhanghttps://urldefense.proofpoint.com/v2/url?u=https-3A__community.opengroup.org_ChrisZhang&d=DwMCaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=7aHxb10vdT9B-HnJRxkATcLVAWPjzM-wUneEZGYXiR0&m=iaAWnN-MKX1XPSVhP4fgPxmKMoa3aWr8e5ZrjA9tTGY&s=0J79lQpw_dMSpmbUqE38AtC8toKSItvImYiOt5r-2kw&e= , @chadhttps://urldefense.proofpoint.com/v2/url?u=https-3A__community.opengroup.org_chad&d=DwMCaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=7aHxb10vdT9B-HnJRxkATcLVAWPjzM-wUneEZGYXiR0&m=iaAWnN-MKX1XPSVhP4fgPxmKMoa3aWr8e5ZrjA9tTGY&s=pqZ-bek9PUn8jdHqtkYqZCksVHptelD77lX2IO-x3M8&e= , @Keith_Wallhttps://urldefense.proofpoint.com/v2/url?u=https-3A__community.opengroup.org_Keith-5FWall&d=DwMCaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=7aHxb10vdT9B-HnJRxkATcLVAWPjzM-wUneEZGYXiR0&m=iaAWnN-MKX1XPSVhP4fgPxmKMoa3aWr8e5ZrjA9tTGY&s=Pe2wfFkYnWN2B3KK0zT0U1GyiOFBIZo6Yko59GnEvWU&e= for information
— Reply to this email directly or view it on GitLabhttps://urldefense.proofpoint.com/v2/url?u=https-3A__community.opengroup.org_osdu_platform_data-2Dflow_ingestion_energistics-2Dosdu-2Dintegration_-2D_issues_43&d=DwMCaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=7aHxb10vdT9B-HnJRxkATcLVAWPjzM-wUneEZGYXiR0&m=iaAWnN-MKX1XPSVhP4fgPxmKMoa3aWr8e5ZrjA9tTGY&s=PzbqaiHxIe1MZurNoUtUfKcm9fFgt8fDwtTvWn-5ZU8&e=. You're receiving this email because you have been mentioned on community.opengroup.org. If you'd like to receive fewer emails, you can unsubscribehttps://urldefense.proofpoint.com/v2/url?u=https-3A__community.opengroup.org_-2D_sent-5Fnotifications_16cbbccec73c6637e9d71763d465c21d_unsubscribe&d=DwMCaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=7aHxb10vdT9B-HnJRxkATcLVAWPjzM-wUneEZGYXiR0&m=iaAWnN-MKX1XPSVhP4fgPxmKMoa3aWr8e5ZrjA9tTGY&s=6NqeLHSLeQfUz6ULjRf9l6caGDTmjCm4co21o8Ozpdw&e= from this thread or adjust your notification settings.
No issues other than Microsoft us not yet scheduled implementation.
Get Outlook for iOShttps://aka.ms/o0ukef