CRS Catalog issueshttps://community.opengroup.org/osdu/platform/system/reference/crs-catalog-service/-/issues2020-12-16T23:42:57Zhttps://community.opengroup.org/osdu/platform/system/reference/crs-catalog-service/-/issues/3Add Devops for Azure2020-12-16T23:42:57ZNicholas KarskyAdd Devops for Azure- [x] Enable Containerize and Deploy
- [x] Debug Deploy
- [x] Enable Int Tests
- [x] Debug Int Tests- [x] Enable Containerize and Deploy
- [x] Debug Deploy
- [x] Enable Int Tests
- [x] Debug Int TestsNicholas KarskyNicholas Karskyhttps://community.opengroup.org/osdu/platform/system/reference/crs-catalog-service/-/issues/7Upgrade Core IBM Dependency2021-07-21T06:37:25ZDavid Diederichd.diederich@opengroup.orgUpgrade Core IBM DependencyDavid Diederichd.diederich@opengroup.orgDavid Diederichd.diederich@opengroup.orghttps://community.opengroup.org/osdu/platform/system/reference/crs-catalog-service/-/issues/11Catalog service fails at deployment stage after update of core libraries2021-09-07T20:25:32ZRostislav Vatolinvatolinrp@gmail.comCatalog service fails at deployment stage after update of core librariesCatalog service fails after updating the core libraries because it doesn't have the configuration to call partition service. This started happening after this MR: https://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-co...Catalog service fails after updating the core libraries because it doesn't have the configuration to call partition service. This started happening after this MR: https://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/134
To resolve the issue please set the property azure.serviceBus.manager.enabled to falsehttps://community.opengroup.org/osdu/platform/system/reference/crs-catalog-service/-/issues/15Use HPA for kubernetes service2021-10-12T13:57:53ZRostislav Vatolinvatolinrp@gmail.comUse HPA for kubernetes serviceImplement practices described here: https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/Implement practices described here: https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/https://community.opengroup.org/osdu/platform/system/reference/crs-catalog-service/-/issues/14[BUG] CORS configuration missing allowed header2021-10-20T14:33:44ZYifan Ye[BUG] CORS configuration missing allowed headerThe allowed header is missing: Content-Type causing such error:
Access to fetch has been blocked by CORS policy: Request header field content-type is not allowed by Access-Control-Allow-Headers in preflight response.The allowed header is missing: Content-Type causing such error:
Access to fetch has been blocked by CORS policy: Request header field content-type is not allowed by Access-Control-Allow-Headers in preflight response.Yifan YeYifan Yehttps://community.opengroup.org/osdu/platform/system/reference/crs-catalog-service/-/issues/13Add memory limits2021-10-20T14:33:52ZYifan YeAdd memory limitsAdding memory limits:
AKS node autoscaler uses memory limits to add nodes to cluster when HPA(Horizontal Pod Autoscaler) needs more capacity. Values were experimentally determined. Given implementation allows managing the list of envs w...Adding memory limits:
AKS node autoscaler uses memory limits to add nodes to cluster when HPA(Horizontal Pod Autoscaler) needs more capacity. Values were experimentally determined. Given implementation allows managing the list of envs where limits are enabled.Yifan YeYifan Yehttps://community.opengroup.org/osdu/platform/system/reference/crs-catalog-service/-/issues/5Fix security vulnerabilities2021-10-25T14:48:30ZRostislav Vatolinvatolinrp@gmail.comFix security vulnerabilitiesFix security vulnerabilities by replacing old dependencies with new.
About the dependencies more info may be found in descriptions here:
https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/76
...Fix security vulnerabilities by replacing old dependencies with new.
About the dependencies more info may be found in descriptions here:
https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/merge_requests/76
and here:
https://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/issues/8https://community.opengroup.org/osdu/platform/system/reference/crs-catalog-service/-/issues/16Log4J Expedient Updates and Patches2021-12-16T02:39:57ZDavid Diederichd.diederich@opengroup.orgLog4J Expedient Updates and PatchesThis issue associates MRs that were applied to this project quickly to get a patched version ready as soon as possible. The intent is to provide a reference point for later, more thoughtful, analysis.This issue associates MRs that were applied to this project quickly to get a patched version ready as soon as possible. The intent is to provide a reference point for later, more thoughtful, analysis.David Diederichd.diederich@opengroup.orgDavid Diederichd.diederich@opengroup.orghttps://community.opengroup.org/osdu/platform/system/reference/crs-catalog-service/-/issues/17Upgrade to Log4J 2.172021-12-21T01:50:29ZDavid Diederichd.diederich@opengroup.orgUpgrade to Log4J 2.17The Apache Foundation released another Log4j2 update, version 2.17, which address a denial of service vulnerability.
This issue tracks progress to upgrade this dependency for this project.The Apache Foundation released another Log4j2 update, version 2.17, which address a denial of service vulnerability.
This issue tracks progress to upgrade this dependency for this project.https://community.opengroup.org/osdu/platform/system/reference/crs-catalog-service/-/issues/8EpsgManifestGenerator for custom entities (WKT and names)2022-02-05T23:37:10ZDebasis ChatterjeeEpsgManifestGenerator for custom entities (WKT and names)---
discussion 2022-01-17
Operators have a requirement to add custom CRS, CT and BoundCRSs to OSDU geodetic reference data.
[populating-OSDUgeodetic-reference-data.docx](/uploads/5f046d69bf1c152e6c6041d228a40d16/populating-OSDUgeodetic...---
discussion 2022-01-17
Operators have a requirement to add custom CRS, CT and BoundCRSs to OSDU geodetic reference data.
[populating-OSDUgeodetic-reference-data.docx](/uploads/5f046d69bf1c152e6c6041d228a40d16/populating-OSDUgeodetic-reference-data.docx)Operators with an in-house GeoRepository would point EpsgManifestGenerator to it in order to overwrite the OSDU data. Such database would contain Operator defined CRS, CT, and BoundCRSs.
Several changes described in the attached (outdated) have been implemented (in particular the "id" and ability to generate reference data from a list). As of 2022-01-17 there are 2 unresolved items bundled in this issue:
1. The ability for an operator to put in a naming convention for BoundCRSs. (More generically, for any CRS or CT - the main issue is that Operators will have naming conventions for BoundCRSs that should be used in OSDU.
2. Creation of WKT. Ideally the WKT is created dynamically from parameters stored in the database and not from hardcoded information to avoid mistakes - but mainly to enable WKT for custom entities.
This issue was discussed in the geomatics workstream with Phil (XOM), Sigrid (Equinor), Bert (Shell), Josh (BP) and Jin (Chevron).
For the first issue, a NameAlias could perhaps be used, but that solution is not understood and still requires a mechanism to generate reference data such that end users in OSDU applications see the desired CRS names. The easier solution appears to be that the an output name is added to the script/input file so that this name is written.
For the second issue, several approaches have lightly been discussed but nobody in the Geomatics workstream knows of a clear and easy method to generate this WKT in ESRI style. (a) EPSG GeoRepository does not generate this "ESRI flavor" - it can output OGC/ISO WKT1 and WKT2. Some attempts were made to output ESRI methods and parameters but that is not fully compliant even with a simple LCC1 method because it uses a naming system only. (b) gdal or other utilities (e.g., epsg.io) appear to be able to generate ESRI WKT. However it is not clear to us what utility can do this, nor what license is required for this and if OSDU would accept that. (c) An ESRI web service might be able to somehow export ESRI WKT - this would need to be checked but seemed unlikely; (d) a geospatial open source developer may know the answer and be able to code this, i.e., by extending the EpsgManifestGenerator.py script to retrieve the method and parameters, then use that to define a spatial reference object, and then export that as ESRI WKT which essentially can be converted to what is required for OSDU PersistenReference.
---
Discussion on 27-Aug-2021
Need some kind of translator from common format such as WKThttps://community.opengroup.org/osdu/platform/system/reference/crs-catalog-service/-/issues/19Incorrect Info endpoint [GONRG-4529]2022-03-18T17:55:20ZDenis Karpenok (EPAM)Incorrect Info endpoint [GONRG-4529]`GET 'https://preship.gcp.gnrg-osdu.projects.epam.com/api/crs/catalog/v2/info' - 401, "Unauthorized"`
Expected:
Works well`GET 'https://preship.gcp.gnrg-osdu.projects.epam.com/api/crs/catalog/v2/info' - 401, "Unauthorized"`
Expected:
Works wellM11 - Release 0.14Chris ZhangDzmitry Malkevich (EPAM)Chris Zhanghttps://community.opengroup.org/osdu/platform/system/reference/crs-catalog-service/-/issues/20Use actuator endpoint to check readiness of the service2022-04-05T16:52:47ZRostislav Vatolinvatolinrp@gmail.comUse actuator endpoint to check readiness of the serviceUsing actuator endpoint to check health of the service brings benefits:
1. no need to use custom or swagger API
2. no need to flood the system with useless logs
This practice is already used in such services as entitlements, legal, par...Using actuator endpoint to check health of the service brings benefits:
1. no need to use custom or swagger API
2. no need to flood the system with useless logs
This practice is already used in such services as entitlements, legal, partitionM11 - Release 0.14Rostislav Vatolinvatolinrp@gmail.comRostislav Vatolinvatolinrp@gmail.comhttps://community.opengroup.org/osdu/platform/system/reference/crs-catalog-service/-/issues/6CRS Catalog APIs need to retrieve values from Reference data CoordinateRefere...2022-04-18T11:52:04ZDebasis ChatterjeeCRS Catalog APIs need to retrieve values from Reference data CoordinateReferenceSystem and CoordinateTransformationThe openAPI CRS Catalog APIs currently retrieve information from a catalog file, for example to obtain a list of Coordinate Reference Systems.
This is a legacy solution and now incorrect, i.e., the geodetic information in OSDU is stored ...The openAPI CRS Catalog APIs currently retrieve information from a catalog file, for example to obtain a list of Coordinate Reference Systems.
This is a legacy solution and now incorrect, i.e., the geodetic information in OSDU is stored as reference data CoordinateReferenceSystem and CoordinateTransformation.
Developers could use the Search API to retrieve any information but because the dedicated CRS Catalog API is there the EA and PMC desire is to keep this platform service (CRS Catalog APIs). Hence the required change is that these APIs either directly or indirectly retrieve the information from the reference data such that the same source is used.
1. Search for CRS entries in Reference data -
POST {{search_api_url}}/query
Body
```
{
"kind": "{{data-partition-id}}:wks:reference-data--CoordinateReferenceSystem:1.0.0",
"query": "id:\"odesprod:reference-data--CoordinateReferenceSystem:Geographic2D:EPSG::4267\"",
"limit": 20
}
```
2. Leverage CRS Catalog service
GET {{CRSCatalog_HOST}}/crs?limit=2300M11 - Release 0.14https://community.opengroup.org/osdu/platform/system/reference/crs-catalog-service/-/issues/21Fix failing pipeline in release version of catalog2022-05-25T13:42:37ZRostislav Vatolinvatolinrp@gmail.comFix failing pipeline in release version of catalogFix failing pipeline:
https://community.opengroup.org/osdu/platform/system/reference/crs-catalog-service/-/pipelines/93466Fix failing pipeline:
https://community.opengroup.org/osdu/platform/system/reference/crs-catalog-service/-/pipelines/93466M10 - Release 0.13Rostislav Vatolinvatolinrp@gmail.comRostislav Vatolinvatolinrp@gmail.comhttps://community.opengroup.org/osdu/platform/system/reference/crs-catalog-service/-/issues/12Missing high-level properties for transformation (and likely coordinaterefere...2022-05-26T23:51:13ZBert KampesMissing high-level properties for transformation (and likely coordinatereferencesystem)**CoordinateTransformation.v.1.0.0 seems to miss properties that would be useful**
1. Remarks (optional in EPSG). For example CT EPSG::1612 has https://epsg.org/transformation_1612/ED50-to-WGS-84-23.html
| Parameter values are taken f...**CoordinateTransformation.v.1.0.0 seems to miss properties that would be useful**
1. Remarks (optional in EPSG). For example CT EPSG::1612 has https://epsg.org/transformation_1612/ED50-to-WGS-84-23.html
| Parameter values are taken from ED50 to ETRS89 (1), code 1588. Adopted for ED50 to WGS84 transformations offshore Norway north of 62N from April 2001 when it replaced code 1590. Included in Statens Kartverk programme wsktrans from v4.0.
but this metadata is not there?
2. The Usages are there, with scope and extent. For extent the Name is there. But not the description
For example for CT EPSG::1149there is `"Name": "Europe - ETRF by country"`
https://epsg.org/transformation_1149/ETRS89-to-WGS-84-1.html
but the extent description is not there : Europe - onshore and offshore: Albania; Andorra; Austria; Belgium; Bosnia and Herzegovina; Bulgaria; Croatia; Cyprus; Czechia; Denmark; Estonia; Faroe Islands; Finland; France; Germany; Gibraltar; Greece; Hungary; Ireland; Italy; Kosovo; Latvia; Liechtenstein; Lithuania; Luxembourg; Malta; Moldova; Monaco; Montenegro; Netherlands; North Macedonia; Norway including Svalbard and Jan Mayen; Poland; Portugal; Romania; San Marino; Serbia; Slovakia; Slovenia; Spain; Sweden; Switzerland; United Kingdom (UK) including Channel Islands and Isle of Man; Vatican City State.
When this is in reference data these countries can be very handy.
3. no validity status (deprecation). Maybe OSDU does not directly require this but I'd still think it should be included for cases that CRS/CT definition is transferred and then deprecated. This is bit tricky with BoundCRSs.
* This likely is similar for CRS.
* I understand the OSDU model is not 100% complete and mainly focuses on the persistent reference string.
But both top-level entity remarks as well as Usage extent description are very handy metadata.
**For CRS properties**
* It seems there is a CoordinateReferenceSystemType as enum per EPSG data model. But it seems awkward/not clear how to know the subtype of geodetic (i.e., geocentric or geog2D or geog3D). There is a "Kind" but that is a string. I'd argue somehow there needs to be a very clear and simple way of knowing if a CRS or coordinates are geocentric or geog2D or geog3Dhttps://community.opengroup.org/osdu/platform/system/reference/crs-catalog-service/-/issues/10Need mechanism to batch load CRS values into Reference data from Operator's r...2022-05-26T23:54:01ZDebasis ChatterjeeNeed mechanism to batch load CRS values into Reference data from Operator's reliable sourceSource - Phil, Bert
Similar to IOGP loading script provided by Thomas.Source - Phil, Bert
Similar to IOGP loading script provided by Thomas.https://community.opengroup.org/osdu/platform/system/reference/crs-catalog-service/-/issues/4Terminology alignment2022-05-31T20:49:04ZBert KampesTerminology alignmentI suggest various changes to the text and API to align with ISO 19111 terminology where possible, even though it is understood that the text has a vintage and is intended for non-specialist. After a quick read the following changes are s...I suggest various changes to the text and API to align with ISO 19111 terminology where possible, even though it is understood that the text has a vintage and is intended for non-specialist. After a quick read the following changes are suggested; which may need to be made also in other parts (e.g., in engine API i/o or help text)
1. It says "Latitude and longitude are measured in plane angles (spherical coordinate system - wiki). Since th…
Suggest we avoid links to wikipedia, but more importantly these are not spherical. Geodetic lat and lon are ellipsoidal. Geocentric lat and lon are something too as well as astronomical lat and lon. This would be way too much for first paragraph to get into.
Suggest to replace the link with (unfortunately perhaps still wikipedia:)) https://en.wikipedia.org/wiki/Latitude
2. It says "With the introduction of satellites, one world-wide approximation has been defined, which is the World Geodetic System of 1984 (WGS 84). This is the reference used in the Global Positioning System. WGS 84 is still a mathematical approximation. Over time, plate tectonics moves the continents under the ellipsoid. At the moment, the relative shifts are in the order of normal GPS resolution, but will accumulate further over time. Eventually the geospatial references will become time dependent. The current catalog does not contain any time-dependent elements yet."
Suggest this should be reviewed. The point is that coordinates are never in "WGS 84" but in a (national) static CRS or potentially offshore in surveyed in a specific dynamic CRS realization such as ITRF2014. This is not solved by adding time-dependent transformations but by storing data in the original CRS (possible a specific dynamic CRS realization and with coordinate epoch) and when transformed to "WGS 84" for analytics or global mapping to keep track of the transformation used or accept 1-2m "fuzz". Since I also do not know good words right now I suggest to add a link to IOGP Guidance Note for further reading:
https://www.iogp.org/bookstore/product/geomatics-guidance-note-25-dynamic-versus-static-crss-and-use-of-the-itrf/
3. In various places for historic reason CT is used as Cartographic Transformation. In the ISO 19111 model there are coordinate operations, and one of those when a datum changes is Coordinate Transformation.
Suggest to replace all occurrences somehow.
4. There is some text that states that the choice of transformation is the task of geodesist and ordinary people cannot make such decisions. I see a link with early-binding here; where I still advocate a late-binding data model. The difference is not in who makes that choice necessarily, but more in the pre-definition and book keeping. For example the CT to use can be stored as property and instead of the data loader selecting an EB CRS, he or she would select the (LB) CRS and a CT to WGS 84 for ingestion. I don't see the difference for responsibility, except that with the EB CRS we need to maintain an additional layer of geodetic definitions. And when data is transformed not to WGS 84 but to some other CRS, e.g., GDA94 to GDA2020 if not having the notion it must go through WGS 84 then a better transformation can be selected.
However, this is just a comment to perhaps rephrase somewhat, again, I do not have the words ready.
In my view (in ISO 19111) a CRS is LB, so there is no need to say that. I can understand that it is specified for full clarity.
but in the API and documentation I would expect to get (LB) CRSs back if I get CRSs and BoundCRSs if I ask for BoundCRSs.
The current api description seems to have getCRS for all objects and getLBCRS for LB and getEBCRS for EB. I'd argue there would be only 2 functions, one for getCRS (LB by definition) and one for getEBCRS. There could be an issue with that. If WGS 84 based systems are not seen as EB then they would be missed out in a dropdown if EB would be used. But there could be a function to get all projCRS, all geog2DCRSs, and all EB CRS. There is some discussion here needed perhaps but my feeling is not to mix and match EB with LB. Also the EPSG v10 model may have boundCRS as an empty defined type of CRS, which might argue that they should be returned as CRS.
5. There are definitions of singleCT and compoundCT which I do not quite understand in detail.
However, in ISO 19111 there is a term "concatenated operation". This can be a multi-step transformation path defined with a single code.
Hence I first thought it was a mistake like compoundCRS, compoundCT, not realizing it is concatenatedCT.
But then I read a little but further and see a policy of 'fallback' and 'concatenated'.
I suggest that the usage of compoundCT should be reviewed if appropriate, and since the policy is not clear to me that some example should be given (e.g., it reads as if a list of transformations are given between two datums and they are tried for a point or list of points until an appropriate one is found. In itself that is a method, i.e., if the polygon area of use would be used.)
Suggest a reference to following could be included in the documentation
http://docs.opengeospatial.org/as/18-005r4/18-005r4.html
6. lon_min vs. longitudeLeft vs. westboundlongitude, etc
a. Suggest the example should use fractional with 2 decimal places (there is an example using “100” for example). EPSG policy has been to use 2 decimal places for BBOX and 5 for polygon nodes though the new system may have some issues still in this regard that are being resolved.
b. api shows "lon_min", etc. This has an issue with a bbox spanning the antimeridian in the sense the smaller value longitude in decimal degrees in interval <-180,+180]. On the page is a section “BoundBox” which show “lon_min” which is not aligned.
Suggest remove all min/max and replace with "longitudeRight", etc. as used in other parts of the page.
c. And although longitudeRight is clear to people I presume, in EPSG this is eastboundlongitude etc. Hence we may want to rather align with "east/west/north/south" rather than "right/left/up/down". Since the referenced CRS is WGS 84 for the BBOX, I do think the bound most on the right is unambiguously for the most east.
7. usage of "Authority"
I know in ESRI flavor of WKT v1 this word is used. In ISO WKT I think this is called ID which is the data source and the code, or what we called codespace and code (together making the identifier) in the concept architecture document.
I tried to check in the ISO whether this is called issuer or codespace or database (i.e., EPSG is not the authority of a national CRS definition; that would be a national agency, but EPSG database would be the data source, where the national agency is the information source). It seems ISO 19111 used “Identifier” and I always say that as “EPSG::4326” where the empty space between colons is the database version. It refers in section 8 to ISO 19115-1:2014 which I don’t know.
Suggest a consistent way of doing this.
https://epsg.org/crs/wkt/id/32065 has it as "ID["EPSG",32065]]
8. Essence.
I did not understand from the description what essence is or what it is for. I think it is kind of a subset of practical information, but not as "light" as just a name, a code and unit.
Is it planned at some point to call the engine with ID only implicitly, or would still always an explicit definition be send to the engine?https://community.opengroup.org/osdu/platform/system/reference/crs-catalog-service/-/issues/9Need simple mechanism to perform regional search, with further filtering2022-06-27T17:48:19ZDebasis ChatterjeeNeed simple mechanism to perform regional search, with further filteringEx: What are the available CRS definitions for country Nigeria?
Ex: Search by polygon (currently polygon is not included in Reference data definition)
To be elaborated by Bert K, showing summary of Shell implementation
Need more than si...Ex: What are the available CRS definitions for country Nigeria?
Ex: Search by polygon (currently polygon is not included in Reference data definition)
To be elaborated by Bert K, showing summary of Shell implementation
Need more than simple bounding box.
And further restrict to ProjectedCRS only.
Check EPSG or IOGP site for similar search features.https://community.opengroup.org/osdu/platform/system/reference/crs-catalog-service/-/issues/25CRS Catalog API doc is missing healthcheck endpoint2022-07-11T21:34:00ZBert KampesCRS Catalog API doc is missing healthcheck endpoint@fhoueto.amz
cc - @debasisc
In AWS collection /CRS v3/CRS Catalog (and Convert services) there is an endpoint {{osduonaws_base_url}}/api/crs/catalog/actuator/health.
this endpoint seems undocumented in the API swagger page. Request ...@fhoueto.amz
cc - @debasisc
In AWS collection /CRS v3/CRS Catalog (and Convert services) there is an endpoint {{osduonaws_base_url}}/api/crs/catalog/actuator/health.
this endpoint seems undocumented in the API swagger page. Request is
* [ ] add endpoint to doc
* [ ] it would be opportune to ensure all endpoints are covered. I did not do a 1 by 1.
thanks - bertOkoun-Ola Fabien HouetoOkoun-Ola Fabien Houetohttps://community.opengroup.org/osdu/platform/system/reference/crs-catalog-service/-/issues/27CRS Catalog release/0.15 build Failure2022-08-23T15:16:56ZShrikant GargCRS Catalog release/0.15 build FailureIndexer release/0.15 build Failure start failing as it is referring to core-common 0.15.0-SNAPSHOT which is cleaned up. Ideally SNAPSHOT versions should not be referenced from release branch/tag. So upgrading it to latest version is reco...Indexer release/0.15 build Failure start failing as it is referring to core-common 0.15.0-SNAPSHOT which is cleaned up. Ideally SNAPSHOT versions should not be referenced from release branch/tag. So upgrading it to latest version is recommended.M12 - Release 0.15Shrikant GargShrikant Garg