Drive search-indexing of last updated/created dates with systems team
As a Solution Architect I need a way to only retrieve the records from an external source that have been updated or created since the last time the data job ran so that I can eliminate unnecessary processing on the fetch and ingest.
- This is accomplished by comparing the Last run date (or equivalent property name) in the ConnectedSourceDataJob record. This value is passed to the Search API to return records that have been created or updated since that date. However the Search API does not currently have the record last updated/created dates search indexed.
- OSDU Core Search API has record last updated/created dates search-indexed
- EDS Developers can using Search API to query by last updated/crated dates
- Interface with Core Services team and drive this change
- Train EDS developers on how to use the new query functionality