Commits · 6ce118284bbdee4fb1ef59479edc4e12f657f0a4 · OSDU / OSDU Data Platform / Data Flow / Data Ingestion / Manifest Ingestion DAG · GitLab

Snippets Groups Projects

Jun 24, 2021
- GONRG-2327: Added possibility to specify custom SA-file for credentials logic · 6ce11828
  Siarhei Khaletski (EPAM) authored 3 years ago
  
  6ce11828
May 28, 2021
- GONRG-2320: Switch file handler API to v2 · ed7d2d91
  Yan Sushchynski (EPAM) authored 3 years ago
  
  ed7d2d91
May 07, 2021
- GONRG-2366: Due to the issues with Search with cursor it replaced to query with offsets · c2b41cce
  Siarhei Khaletski (EPAM) authored 3 years ago
  
  c2b41cce
May 05, 2021
- GONRG-2017: Add manifest_ingestion test file to repo · 30d037b0
  Yan Sushchynski (EPAM) authored 3 years ago
  
  30d037b0
- GONRG-2213: Fix different validation results · 1bd7632c
  Yan Sushchynski (EPAM) authored 3 years ago
  
  1bd7632c
May 03, 2021
- GONRG-2291: Skipped reason truncated · 212bc01e
  Yan Sushchynski (EPAM) authored 3 years ago
  
  212bc01e
- GONRG-2103: Fix entity passes validation with wrong formats · e992b3ef
  Yan Sushchynski (EPAM) authored 3 years ago
  
  e992b3ef
Apr 28, 2021
- GONRG-2170: Fixed request body · 4cf32ff6
  Siarhei Khaletski (EPAM) authored 3 years ago
  
  4cf32ff6
Apr 27, 2021
- GONRG-2293: Fix Missing colon in parent srn with surrogate-key · 63528eeb
  Yan Sushchynski (EPAM) authored 3 years ago
  
  63528eeb
- GONRG-2292: Find references with no version in Search · b49b28c3
  Yan Sushchynski (EPAM) authored 3 years ago
  
  b49b28c3
- GONRG-2170: Added cursor for search service requests · f2a87b36
  Siarhei Khaletski (EPAM) authored 3 years ago
  
  f2a87b36
Apr 26, 2021
- GONRG-2142: Add lru cache · e59469e5
  Yan Sushchynski (EPAM) authored 3 years ago and Siarhei Khaletski (EPAM) committed 3 years ago
  
  e59469e5
Apr 21, 2021
- Ingestion updates · b9c1a071
  Yan Sushchynski (EPAM) authored 3 years ago and Rostislav Dublin (EPAM) committed 3 years ago
  
  b9c1a071
Apr 19, 2021
- Aws impl, id version bug fix · 545b62be
  Spencer Sutton authored 3 years ago
  
  545b62be
Apr 09, 2021
- GONRG-2185: Single manifest validation hiden under the flag · 7bda9686
  Siarhei Khaletski (EPAM) authored 3 years ago
  
  7bda9686
Mar 23, 2021
- Add BaseTokenRefresher · 48e9789a
  Yan Sushchynski (EPAM) authored 4 years ago and Siarhei Khaletski (EPAM) committed 4 years ago
  
  48e9789a
Mar 18, 2021
- GONRG-1961: Fix pipline fails · 64df438a
  Yan Sushchynski (EPAM) authored 4 years ago and Siarhei Khaletski (EPAM) committed 4 years ago
  
  64df438a
- GONRG-1941: Fix space char in FileSource · d38d6f2e
  Yan Sushchynski (EPAM) authored 4 years ago and Siarhei Khaletski (EPAM) committed 4 years ago
  
  d38d6f2e
Mar 11, 2021
- GONRG-1932: Refactor referential integrity · 4f02c1db
  Yan Sushchynski (EPAM) authored 4 years ago and Siarhei Khaletski (EPAM) committed 4 years ago
  
  4f02c1db
Mar 05, 2021
- Fix adding to valid wpc · b75370d5
  Yan Sushchynski (EPAM) authored 4 years ago and Siarhei Khaletski (EPAM) committed 4 years ago
  
  b75370d5
- Add constant to single manifest processor · f7c5e607
  Yan Sushchynski (EPAM) authored 4 years ago and Siarhei Khaletski (EPAM) committed 4 years ago
  
  f7c5e607
- GONRG-1912: Add surrogate keys to patterns · 92bbd40c
  Yan Sushchynski (EPAM) authored 4 years ago and Siarhei Khaletski (EPAM) committed 4 years ago
  
  92bbd40c
- GONRG-1891: Artifacts processing in WPC · c448c54e
  Yan Sushchynski (EPAM) authored 4 years ago and Siarhei Khaletski (EPAM) committed 4 years ago
  
  c448c54e
Mar 04, 2021
- GONRG-1866: add condition statement to import vendors dependencies · 8938fc43
  Dmitrii Valuiskii (EPAM) authored 4 years ago and Siarhei Khaletski (EPAM) committed 4 years ago
  
  8938fc43
- GONRG-1910: fix bug with ingesting master data with pre-defined id · b1778896
  Dmitrii Valuiskii (EPAM) authored 4 years ago and Siarhei Khaletski (EPAM) committed 4 years ago
  
  b1778896
- GONRG-1936: fix error with colon symbol in referential ids · b223c6c8
  Dmitrii Valuiskii (EPAM) authored 4 years ago and Siarhei Khaletski (EPAM) committed 4 years ago
  
  b223c6c8
Mar 03, 2021
- GONRG-1910: add validation reference ids with versions · fb11caae
  Dmitrii Valuiskii (EPAM) authored 4 years ago and Siarhei Khaletski (EPAM) committed 4 years ago
  
  fb11caae
Mar 02, 2021
- GONRG-1881: add check for WP manifest completeness · c1cf95ec
  Dmitrii Valuiskii (EPAM) authored 4 years ago and Siarhei Khaletski (EPAM) committed 4 years ago
  
  c1cf95ec
Mar 01, 2021
- GONRG-1719: updated variables to support ENV approach · 3d0e0f7b
  Siarhei Khaletski (EPAM) authored 4 years ago
  
  3d0e0f7b
Feb 25, 2021
- Add trigger rule to skip batch tasks · 7f1c24f6
  Yan Sushchynski (EPAM) authored 4 years ago and Siarhei Khaletski (EPAM) committed 4 years ago
  
  (cherry picked from commit 8af8301d)
  7f1c24f6
- GONRG-1911: remove : (colon) symbol from the end of reference ids · a4643d8a
  Dmitrii Valuiskii (EPAM) authored 4 years ago and Siarhei Khaletski (EPAM) committed 4 years ago
  
  a4643d8a
- GONRG-1651: Add FileSource validation. · 6fbb5750
  Ernesto Gutierrez authored 4 years ago and Siarhei Khaletski (EPAM) committed 4 years ago
  
  6fbb5750
- GONRG-1770: hotfix variable name · 0b1dfba1
  Dmitrii Valuiskii (EPAM) authored 4 years ago and Siarhei Khaletski (EPAM) committed 4 years ago
  
  0b1dfba1
- GONRG-1770: add composable operator for schema validation · dbdd530b
  Dmitrii Valuiskii (EPAM) authored 4 years ago and Siarhei Khaletski (EPAM) committed 4 years ago
  
  dbdd530b
- GONRG-1719: clean-up and rename airflow variables · 4f2f58a2
  Dmitrii Valuiskii (EPAM) authored 4 years ago and Siarhei Khaletski (EPAM) committed 4 years ago
  
  4f2f58a2
- Handle file variable fix · d2cebc9c
  Yan Sushchynski (EPAM) authored 4 years ago and Siarhei Khaletski (EPAM) committed 4 years ago
  
  d2cebc9c
- GONRG-1567: Removed obsolete dags: · 008c172b
  Siarhei Khaletski (EPAM) authored 4 years ago
  
  - default-ingest - other-log-ingest - well-log-ingest
  008c172b
- GONRG-1653: Validate Datasets, WPCs and WP referencial integrity. · e2fc2d0a
  Ernesto Gutierrez authored 4 years ago and Siarhei Khaletski (EPAM) committed 4 years ago
  
  e2fc2d0a
- Fix dag. Remove subdags · b97d68d2
  Yan Sushchynski (EPAM) authored 4 years ago and Siarhei Khaletski (EPAM) committed 4 years ago
  
  b97d68d2
- GONRG-1700: fix id composing · 5a58168e
  Dmitrii Valuiskii (EPAM) authored 4 years ago and Siarhei Khaletski (EPAM) committed 4 years ago
  
  5a58168e