CRS Catalog points-in-aou enable terminating colon for CRS record id

The CRS conversion APIs all require a terminating colon when referencing a CRS record id in the request. The CRS Catalog APIs for some endpoints accept a terminating colon, and in some cases do not. This issue is intended to make the Catalog APIs optionally accept a colon if they currently do not. This way it is backward compatible, and the various CRS APIs in Convert and Catalog are all consistent with each other and with the OSDU platform standards.

(Side note: the Core Services Search and Storage APIs may not all be complying, but those will be ignored and the understanding is those will not be changed anyway. This issue is only about the CRS Catalog APIs).

It is low priority, but presumed to be easy change of input parsing. Wherever currently an id string is passed in the request like osdu:reference-data\--CoordinateReferenceSystem:Geographic2D:EPSG::4158 (without a terminating colon), then check that passing osdu:reference-data\--CoordinateReferenceSystem:Geographic2D:EPSG::4158: (with a terminating colon) does not work.
If it fails with a terminating colon, then make it work (both ways).

The suggested code fix is as follows. For API that work without a terminating colon, but not with. Then when handling the input id string, remove the terminating colon if the string has it. Then pass it on as before with colon removed. This should fix the issue, i.e., this will allow input with and without terminating colon, and the response will be unchanged (i.e., it will execute the logic as if the request was as before, without a colon).

check swagger and update as needed.
update tutorial with text and examples.
- NOTE: the updated tutorial should have footnotes or remarks, i.e., the examples can consistently show usage of the colon, but then a footnote or note that prior to M21 the syntax did not accept the colon at the end of the id.
- This should be done in issue !297 (merged) if that is still open and not in this branch. Actually !297 (merged) branch could be used to do the code changes rather than creating a branch here (or at least merge 297 before creating a branch here to avoid conflicts in tutorial updates).
find for all APIs any record id (optionally) used in request body (or url parameter). This started by a report that points-in-aou was inconsistent with other APIs, but lets check swagger and tutorial for all APIs. An initial check shows the following:

{{osduonaws_base_url}}/api/crs/catalog/v3/coordinate-reference-system?recordId=osdu:reference-data\--CoordinateReferenceSystem:Geographic2D:EPSG::4158

This GET currently works without a terminating colon for recordId, and I believe it fails if a colon is added (TBC). We can leave it that way, but ideally we fix it and change it to accept either with or without terminating colon. if this is easy to do then fix it (I mean to say, the priority is points-in-aou and we can live with this if need be).
NOTE: the dataId= should never have a terminating colon.

{{osduonaws_base_url}}/api/crs/catalog/v3/coordinate-transformation?recordId=osdu:reference-data\--CoordinateTransformation:EPSG::15851

This GET currently works without a terminating colon for recordId, and I believe it fails if a colon is added (TBC). (same as above).
NOTE: the dataId= should never have a terminating colon.

{{osduonaws_base_url}}/api/crs/catalog POST /v3/coordinate-reference-system`

This example in tutorial shows BaseCRS without a colon. This should be made to work with and without colon.
NOTE: the above example in tutorial uses "opendes". This has been changed to "osdu" of course.
NOTE: tutorial alternatives using Search or Storage Services shows the query does not have a terminating colon and that may be why AWS did not implement the CRS Service with a colon (because it wraps to search). This does not have to (should not be) addressed! I.e., This issue is only about CRS Catalog, very simple, and not about the platform or other Core Services.

{{osduonaws_base_url}}/api/crs/catalog POST /v3/coordinate-transformation`

This example shows SourceCRS and TargetCrs already work with terminating colon. Please confirm in a test, and if it works with terminating colon then there is no action.

{{osduonaws_base_url}}/api/crs/catalog POST /v3/points-in-aou`

This example shows that "recordId": "osdu:reference-data--CoordinateTransformation:EPSG::15851" does not have a terminating colon. Also we have tested a terminating colon is not accepted. To fix this, it is suggested to not change the code, except for the input parsing of the recordId string, i.e., if the request has a terminating colon then remove that and pass on the request further as in the current version. That way it should allow both a terminating colon and not.

Example - Click to expand

CRS Catalog Service references CRS and CT records by id without a closing colon (error if terminating colon is used). CRS Conversion Service requires a terminating colon (error if colon is omitted). What is way forward for this (is it provider dependent? What is the OSDU standard? Does it need to be fixed? I just added it here as placeholder and have a comment in the system. If the way forward is to fix this then a new issue should be opened for it. Also see attached email. RE__CRS_Catalog_Service_should_use_a_terminating_colon_in_requests_using_a_record_id_.msg

The standard is: record id come without trailing colon/without version as the version number is explicitly represented in the system property version.
Relationships, e.g. as in data.BaseCRSID can refer to a specific version ({id}:{version}) or to implicitly the latest version by omitting the version number ({id}:), this is the trailing colon variant as used in the majority of the cases.

No change required for 2.

Ad 1). The documentation of EpsgManifestGenerator mentions the Multipolygon.

The current generated manifest does not. I did not check (yet) if perhaps somehow it is related to an option running Generator, or for special case with Operator that has multiple usages. The first thing would be to check the Generator with the standard -crs 3851 option to epsg.org to see what that returns.

Test that fails:

curl --location 'https://osdu.com/api/crs/catalog/v3/points-in-aou' \
--header 'data-partition-id: osdu' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer eyJhbGciOiJm8lCA' \
--header 'Cookie: PF=lOekY5BExFoeE8n9DvELw2' \
--data '{
"recordId": "osdu:reference-data--CoordinateReferenceSystem:Projected:EPSG::3851",
    "points": [
        {
            "latitude": -40,
            "longitude": 175
        },
        {
            "latitude": -40,
            "longitude": -175
        },
        {
            "latitude": 25,
            "longitude": -90
        },
        {
            "latitude": 45,
            "longitude": -92
        }
    ]
}
'

Gives a (not so helpful) error. Expected result (Response):

Edited Oct 26, 2023 by Bert Kampes