CRS Catalog query_with_cursor needed
@puneet.bhardwaj @kiranallamsety @nanigopal.roy - this issue is to fix the identified bug that query returns duplicates when using limit and offset. CRS Catalog services return all records (more than 1000 if there are more than 1000). Relates to #31 (closed).
-
change query to query_with_cursor and check that APIs coordinate-reference-system
andcoordinate-transformation
do not return duplicates.
@gehrmann @chad - We found a confusing thing working on EpsgManifestPublisher. We get duplicates in the responses when Search POST query is called with limit 1000 and offset 0, then offset 1000, then offset 2000, etc. until we get TotalCount. That seems to defeat the purpose of the limit/offset parameter. We resolved it in EpsgPublisher by switching code to use query_with_cursor which does not return duplicates (using the returned cursor in the subsequent calls). I asked for an example and then I might log an issue in the Search repo, assuming this is unexpected, or is this expected behavior? To reproduce, basically call
{{osduonaws_base_url}}/api/search/v2/query
{
"kind": "osdu:wks:reference-data--CoordinateReferenceSystem:1.0.0",
"returnedFields": ["id"],
"limit": 1000,
"offset": 0
}
{
"kind": "osdu:wks:reference-data--CoordinateReferenceSystem:1.0.0",
"returnedFields": ["id"],
"limit": 1000,
"offset": 1000
}
{
"kind": "osdu:wks:reference-data--CoordinateReferenceSystem:1.0.0",
"returnedFields": ["id"],
"limit": 1000,
"offset": 2000
}
Once I have run this I will add the responses.