[System/Storage] Frame of Reference API for Normalized Records
The Storage service needs an API method to fetch records with numerical and date-valued attributes in a common frame of reference. Since ingested records can currently be in a variety of CRS and Unit systems as well as time zones, there needs to be a way for consumption zones to fetch records with a common frame of reference for meaningful comparisons as well as map displays, data quality metrics, and essentially any workflow requiring the same measurement system for an attribute.
The Storage API does not need to persist converted records, but should be an API enabling on-the-fly, in-memory representations of records to be retrieved. This is a common concern of all Storage consumers.
The API needs to handle:
- NaN in data: NaN values are allowed in Storage, so normalization needs to handle it correctly by treating it as an invalid value when trying to convert.
- If prerequisites such as lat/lon are not provided, then system should not try to do any conversion.
- Consistent error message needs to be provided for empty/missing/null values.
Edited by An Ngo