CRS Conversion not working if persistableReferenceCrs left off --> Move Storage to crs-conversion v3
According to the documentation we are supposed to be able to leave off the persistableReferenceCrs
now (although, because of it being required in the schema the best we can do is make it an empty string) and that the CRS conversions will use the CoordinateReferenceSystemID
to look up the value of the persistable reference. However, this does not appear to work.
Take the following simple Well for example:
{
"acl": {
"owners": [
"data.default.owners@dp.myosdu.com"
],
"viewers": [
"data.default.viewers@dp.myosdu.com"
]
},
"data": {
"FacilityName": "Dummy 1 - Do Not Use Me",
"SpatialLocation": {
"AsIngestedCoordinates": {
"CoordinateReferenceSystemID": "dp:reference-data--CoordinateReferenceSystem:Geographic2D:EPSG::4326:",
"features": [
{
"geometry": {
"coordinates": [
-45.944904,
18.12565
],
"type": "AnyCrsPoint"
},
"properties": {},
"type": "AnyCrsFeature"
}
],
"persistableReferenceCrs": "",
"type": "AnyCrsFeatureCollection"
},
"SpatialGeometryTypeID": "dp:reference-data--SpatialGeometryType:Point:"
}
},
"id": "dp:master-data--Well:TEST_CRS_METHODS",
"kind": "osdu:wks:master-data--Well:1.3.0",
"legal": {
"legaltags": [
"dp-default-legal"
],
"otherRelevantDataCountries": [
"US"
],
"status": "compliant"
},
"meta": [ ]
}
Which should populate the Wgs84Coordinates when sent to the indexer, but you do not see it in the indexer record:
{
"acl": {
"owners": [
"data.default.owners@dp.myosdu.com"
],
"viewers": [
"data.default.viewers@dp.myosdu.com"
]
},
"authority": "osdu",
"createTime": "2024-01-23T19:16:46.001Z",
"createUser": "bryan.j.dawson@exxonmobil.com",
"data": {
"FacilityName": "Dummy 1 - Do Not Use Me",
"SpatialLocation.SpatialGeometryTypeID": "dp:reference-data--SpatialGeometryType:Point:",
"VirtualProperties.DefaultLocation.SpatialGeometryTypeID": "dp:reference-data--SpatialGeometryType:Point:",
"VirtualProperties.DefaultName": "Dummy 1 - Do Not Use Me"
},
"id": "dp:master-data--Well:TEST_CRS_METHODS",
"kind": "osdu:wks:master-data--Well:1.3.0",
"legal": {
"legaltags": [
"dp-default-legal"
],
"otherRelevantDataCountries": [
"US"
],
"status": "compliant"
},
"namespace": "osdu:wks",
"source": "wks",
"tags": {
"normalizedKind": "osdu:wks:master-data--Well:1"
},
"type": "master-data--Well",
"version": 1706037405365334
}
However, if we fill in the persistableReferenceCrs
and meta block it works as expected.
{
"acl": {
"owners": [
"data.default.owners@dp.myosdu.com"
],
"viewers": [
"data.default.viewers@dp.myosdu.com"
]
},
"data": {
"FacilityName": "Dummy 1 - Do Not Use Me",
"SpatialLocation": {
"AsIngestedCoordinates": {
"CoordinateReferenceSystemID": "dp:reference-data--CoordinateReferenceSystem:Geographic2D:EPSG::4326:",
"features": [
{
"geometry": {
"coordinates": [
-45.944904,
18.12565
],
"type": "AnyCrsPoint"
},
"properties": {},
"type": "AnyCrsFeature"
}
],
"persistableReferenceCrs": "{\"authCode\":{\"auth\":\"EPSG\",\"code\":\"4326\"},\"name\":\"GCS_WGS_1984\",\"type\":\"LBC\",\"ver\":\"PE_10_9_1\",\"wkt\":\"GEOGCS[\\\"GCS_WGS_1984\\\",DATUM[\\\"D_WGS_1984\\\",SPHEROID[\\\"WGS_1984\\\",6378137.0,298.257223563]],PRIMEM[\\\"Greenwich\\\",0.0],UNIT[\\\"Degree\\\",0.0174532925199433],AUTHORITY[\\\"EPSG\\\",4326]]\"}",
"type": "AnyCrsFeatureCollection"
},
"SpatialGeometryTypeID": "dp:reference-data--SpatialGeometryType:Point:"
}
},
"id": "dp:master-data--Well:TEST_CRS_METHODS",
"kind": "osdu:wks:master-data--Well:1.3.0",
"legal": {
"legaltags": [
"dp-default-legal"
],
"otherRelevantDataCountries": [
"US"
],
"status": "compliant"
},
"meta": [
{
"coordinateReferenceSystemID": "dp:reference-data--CoordinateReferenceSystem:Geographic2D:EPSG::4326:",
"kind": "CRS",
"name": "WGS 84",
"persistableReference": "{\"authCode\":{\"auth\":\"EPSG\",\"code\":\"4326\"},\"name\":\"GCS_WGS_1984\",\"type\":\"LBC\",\"ver\":\"PE_10_9_1\",\"wkt\":\"GEOGCS[\\\"GCS_WGS_1984\\\",DATUM[\\\"D_WGS_1984\\\",SPHEROID[\\\"WGS_1984\\\",6378137.0,298.257223563]],PRIMEM[\\\"Greenwich\\\",0.0],UNIT[\\\"Degree\\\",0.0174532925199433],AUTHORITY[\\\"EPSG\\\",4326]]\"}",
"propertyNames": [
"SpatialLocation.AsIngestedCoordinates"
]
}
]
}
and indexer record looks like what we expected
{
"acl": {
"owners": [
"data.default.owners@dp.myosdu.com"
],
"viewers": [
"data.default.viewers@dp.myosdu.com"
]
},
"authority": "osdu",
"createTime": "2024-01-23T19:16:46.001Z",
"createUser": "bryan.j.dawson@exxonmobil.com",
"data": {
"FacilityName": "Dummy 1 - Do Not Use Me",
"SpatialLocation.SpatialGeometryTypeID": "dp:reference-data--SpatialGeometryType:Point:",
"SpatialLocation.Wgs84Coordinates": {
"geometries": [
{
"coordinates": [
-45.944904,
18.12565
],
"type": "point"
}
],
"type": "geometrycollection"
},
"VirtualProperties.DefaultLocation.IsDecimated": false,
"VirtualProperties.DefaultLocation.SpatialGeometryTypeID": "dp:reference-data--SpatialGeometryType:Point:",
"VirtualProperties.DefaultLocation.Wgs84Coordinates": {
"geometries": [
{
"coordinates": [
-45.944904,
18.12565
],
"type": "point"
}
],
"type": "geometrycollection"
},
"VirtualProperties.DefaultName": "Dummy 1 - Do Not Use Me"
},
"id": "dp:master-data--Well:TEST_CRS_METHODS",
"kind": "osdu:wks:master-data--Well:1.3.0",
"legal": {
"legaltags": [
"dp-default-legal"
],
"otherRelevantDataCountries": [
"US"
],
"status": "compliant"
},
"modifyTime": "2024-01-23T19:33:06.905Z",
"modifyUser": "bryan.j.dawson@exxonmobil.com",
"namespace": "osdu:wks",
"source": "wks",
"tags": {
"normalizedKind": "osdu:wks:master-data--Well:1"
},
"type": "master-data--Well",
"version": 1706038386690196
}
CoordinateReferenceSystemID
dynamic lookup feature was implemented on crs-conversionv3
endpoint.
Storage service conversion endpoint (used by indexer-service) calls crs-conversion service to get converted records. As of current release (M22), Storage service is using
v2
endpoints from crs-conversion. This is the reason we see this behavior.
So to solve this one will require moving to the v3 endpoints for crs-conversion service.