Date-Time validation causing ingestion failure
Date values are failing schema validation on ingestion if the dates are not in UTC, or don't contain a time-zone offset. This is a new validation, that requires date-times to conform to RFC3339. The intent is good, but it does not conform to schemas, or to our data.
There is a large volume of date in which a date is known, but for which no time or time zone is provided. Recognizing this, the OSDU schema only require that dates be a string.
As a large volume of data is managed that does not have time zone information, our options are either to reject all these dates, or to ingest and maintain them in original format.
If we force a time zone change on data by putting it into a UTC format when we really do not know the time zone, we are corrupting the data.
I have consulted the Enterprise Architecture Geomatics team, and asked if we should (1) Do not load dates with unknown time zones or (2) maintain the dates in as-provided form. There was complete agreement that the industry has a large volume of data with dates without time zones, and we can still make use of those dates, but must not modify them by adding a default time zone.
Please remove the date validation from ingestion.