Ingestion - move createRecord into File Service instead of OSDU DAG
The intent is to expose this more transparently to ISVs and get them thinking in terms of file record.
- AWS sees an issue with requiring with ISVs having to provide FileIDs, though this is optional
Status
- Proposed
- Trialing
- Under review
- Approved
- Retired
Context & Scope
In future, when have Event-based model, may be able to turn Ingestion steps around since assured that file already exists when creating record.
Decision
Rationale
Consequences
When to revisit
Tradeoff Analysis - Input to decision
Alternatives and implications
Moving create record out of the DAG:
- System still should create records for WP/WPC. If this logic is moved into the service, we don't need DAG anymore
Leaving create record in the DAG:
- Future-proof for extending the logic (extra indexers, etc.)
Decision criteria and tradeoffs
Decision timeline
To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information