OpenETP Client cannot ingest wells with big Stratigraphic Column
With the new dataset that @dani.alsaab provided to replace the old Volve dataset MR, we are facing of this bug in AWS: https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/reservoir/open-etp-server/-/jobs/3224741
After some investigation:
- With this data I could confirm this issue see the log attached it looks related to the chunk
- With same wells but shorter Stratigraphic Column, it looks working fine
`dalsaab@lp-ncy-fitou:~$ docker run -it --rm -v /mnt/d/OES/Data:/Data open-etp:ssl-client openETPServer space -S wss://${RDDMS_URL} -s Dani/WellTest --import-epc /Data/Volve_Demo_Wells_Depth_light.epc --data-partition-id ${PARTITION} --auth bearer --jwt-token ${TOKEN} [2024-11-15 13:30:23.027] Info: CLT connect($1 on ~1, wss://api.awsinteropm22.edi.awsenergy.47lining.com/api/reservoir-ddms-etp/v2) OK (283.5ms)
[2024-11-15 13:30:23.306] Info: Import Options: timeout = 10 [s]; max_msg_size (negotiated) = 1000000; transaction_retries = 0; reject_duplicated = false; [2024-11-15 13:30:23.493] Info: Prolog progress: 3% [2024-11-15 13:30:33.999] Info: PUT 1,886 objects (13,562,092 bytes + 2.5%, 26 messages) in 10.505s [2024-11-15 13:30:33.999] Info: (1.2 MiB/s, 40 MB peak, 0.045s encoding, 10.245s waiting) [2024-11-15 13:30:33.999] Info: Send objects progress: 4% [2024-11-15 13:30:44.110] Info: Send small-enough arrays progress: 24% [2024-11-15 13:30:44.313] Info: PUT 636 arrays (38,952,412 bytes + 0.4%, 57 messages) in 10.313s [2024-11-15 13:30:44.313] Info: (3.6 MiB/s, 61 MB peak, 0.160s encoding, 10.025s waiting) [2024-11-15 13:30:44.313] Info: Send small-enough arrays progress: 27% [2024-11-15 13:30:44.416] Info: INI 1 arrays (214 bytes = 100%, 1 messages) in 0.103s [2024-11-15 13:30:44.416] Info: (0.0 MiB/s, 61 MB peak, 0.000s encoding, 0.103s waiting) [2024-11-15 13:30:44.416] Info: Declare large arrays progress: 30% [2024-11-15 13:30:44.651] Info: Send large arrays progress: 97% [2024-11-15 13:30:44.890] Info: PUT 1 arrays (903,400 bytes + 0.0%, 2 messages) in 0.474s [2024-11-15 13:30:44.890] Info: (1.8 MiB/s, 61 MB peak, 0.003s encoding, 0.418s waiting) [2024-11-15 13:30:44.890] Info: Send large arrays progress: 98% [2024-11-15 13:30:45.041] Info: PUT 1,886 objects, 637 arrays (53,417,904 bytes + 0.9%, 86 messages) in 21.735s [2024-11-15 13:30:45.041] Info: (2.3 MiB/s, 61 MB peak, 0.208s encoding, 20.790s waiting) [2024-11-15 13:30:45.041] Info: Commit progress: 99% [2024-11-15 13:30:45.044] Info: Final progress: 100%
dalsaab@lp-ncy-fitou:~$ docker run -it --rm open-etp:ssl-client openETPServer space -S wss://${RDDMS_URL} -s Dani/WellTest --stats --data-partition-id ${PARTITION} --auth bearer --jwt-token ${TOKEN} [2024-11-15 13:30:56.711] Info: CLT connect($1 on ~1, wss://api.awsinteropm22.edi.awsenergy.47lining.com/api/reservoir-ddms-etp/v2) OK (395.5ms)
[2024-11-15 13:30:57.243] Warning: 1 eml20.obj_EpcExternalPartReference [2024-11-15 13:30:57.243] Warning: 17 resqml20.obj_CategoricalProperty [2024-11-15 13:30:57.243] Warning: 490 resqml20.obj_ContinuousProperty [2024-11-15 13:30:57.243] Warning: 384 resqml20.obj_GeneticBoundaryFeature [2024-11-15 13:30:57.243] Warning: 384 resqml20.obj_HorizonInterpretation [2024-11-15 13:30:57.243] Warning: 1 resqml20.obj_LocalDepth3dCrs [2024-11-15 13:30:57.243] Warning: 13 resqml20.obj_MdDatum [2024-11-15 13:30:57.243] Warning: 1 resqml20.obj_OrganizationFeature [2024-11-15 13:30:57.243] Warning: 70 resqml20.obj_PropertyKind [2024-11-15 13:30:57.243] Warning: 127 resqml20.obj_PropertySet [2024-11-15 13:30:57.243] Warning: 1 resqml20.obj_StratigraphicColumn [2024-11-15 13:30:57.243] Warning: 1 resqml20.obj_StratigraphicColumnRankInterpretation [2024-11-15 13:30:57.243] Warning: 10 resqml20.obj_StratigraphicOccurrenceInterpretation [2024-11-15 13:30:57.243] Warning: 5 resqml20.obj_StratigraphicUnitFeature [2024-11-15 13:30:57.243] Warning: 5 resqml20.obj_StratigraphicUnitInterpretation [2024-11-15 13:30:57.243] Warning: 4 resqml20.obj_StringTableLookup [2024-11-15 13:30:57.243] Warning: 13 resqml20.obj_WellboreFeature [2024-11-15 13:30:57.243] Warning: 322 resqml20.obj_WellboreFrameRepresentation [2024-11-15 13:30:57.243] Warning: 13 resqml20.obj_WellboreInterpretation [2024-11-15 13:30:57.244] Warning: 11 resqml20.obj_WellboreMarkerFrameRepresentation [2024-11-15 13:30:57.244] Warning: 13 resqml20.obj_WellboreTrajectoryRepresentation [2024-11-15 13:30:57.244] Warning: 1,886 TOTAL`