Refactor Transformer for batch `ingest->process` workflow
- Leverage schema service for authoritative source on property types
- Combine with inferred types (assumed string) for maximum coverage and compatibility
- Introduce
loadInBatch
capability which enables a substantially more performant and memory-conscious workflow for ingestion and processing (stages 1 & 2 of Transformer)-
batchSize
is configurable for fine-tuning the performance to an operator's environment
-
- Docs updated to reflect new capabilities
- Minor bug fixes to Provider