Ingestion updates

Type of change

• Bug Fix
• Feature

Does this introduce a change in the core logic?

• [Yes]

Does this introduce a change in the cloud provider implementation, if so which cloud?

• AWS
• Azure
• GCP
• IBM

Updates description?

This MR comes with a batch of updates:

Features:

• Validate entire manifest entity (GONRG-1783)
• Surrogate keys replacement (GONRG-1652)
• Auth logic uses Python SDK implementation (GONRG-1689)
• New operator for manifest integrity (GONRG-1700)
• New operator for schema validation (GONRG-1770)
• Logic for ensuring Datasets, WPCs and WP referencial integrity (GONRG-1653)
• Implementation of the batch uploading (GONRG-1650)
• Added FileSource validation for Datasets (GONRG-1651)

Structure updates:

• Removed obsolete dags (GONRG-1567)
• README.md has been updated (GONRG-1591)
• Fix id composing (GONRG-1700)
• Cleaned-up and renamed airflow variables (GONRG-1719)

Bugfixes:

• Handle file variable fix
• Fix download_as_bytes not supported in storage==1.13.2
• Removed : (colon) symbol from the end of reference ids (GONRG-1911)