Wellbore Domain Services merge requestshttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/merge_requests2023-08-17T11:20:27Zhttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/merge_requests/317Review on dataframe serializer2023-08-17T11:20:27ZYannickReview on dataframe serializerupon team agreement:
- remove V3 & alpha V2 support JSON 'columns' format on write bulk APIs
- JSON 'columns' still supported on V3 & alpha V2 on read bulk APIs
- on legacy V2 read/write bulk APIs, supports unchanged but no convert axes...upon team agreement:
- remove V3 & alpha V2 support JSON 'columns' format on write bulk APIs
- JSON 'columns' still supported on V3 & alpha V2 on read bulk APIs
- on legacy V2 read/write bulk APIs, supports unchanged but no convert axes from Pandas.
- no try to infer date time type from JSON. This was already the case on V3 & alpha V2. This change impacts only legacy V2 but mainly fix unwanted index & column alteration. Date time passed using JSON will pass stored as-it in plain string. Parquet format must be used for full typed date time index.M10 - Release 0.13https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/merge_requests/316Use app as build image name2023-08-17T11:20:29ZAlexandre VincentUse app as build image nameM10 - Release 0.13Alexandre VincentAlexandre Vincenthttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/merge_requests/315parameterize docker image app_name for push and pull to/from central acrs2023-08-17T11:20:30ZAlexandre Vincentparameterize docker image app_name for push and pull to/from central acrsParameterizing pipeline templates for reusabilityParameterizing pipeline templates for reusabilityM10 - Release 0.13Alexandre VincentAlexandre Vincenthttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/merge_requests/312remove obsolete alpha feature setting2023-08-17T11:20:33ZAlexandre Vincentremove obsolete alpha feature settingThis cleans up the code, by removing an obsolete setting in the service config and its tests.This cleans up the code, by removing an obsolete setting in the service config and its tests.M10 - Release 0.13Alexandre VincentAlexandre Vincenthttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/merge_requests/311Bug: 500 returned instead of 404 when calling /data with describe2023-08-17T11:20:35Zfabian serinBug: 500 returned instead of 404 when calling /data with describeBefore the fix : in case of describe (read stats) when the blob is not found OSError exception was raised and catched (as Exception) by wdms which is ansewring a 500.
With the fix the OSError is properly catched and a bulknotfound is ret...Before the fix : in case of describe (read stats) when the blob is not found OSError exception was raised and catched (as Exception) by wdms which is ansewring a 500.
With the fix the OSError is properly catched and a bulknotfound is returned (404)M10 - Release 0.13https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/merge_requests/309change entity id hash function for local storage2023-08-17T11:20:36ZJeremie Hallalchange entity id hash function for local storagein order to persist bulk data the hash function needs to be changed to give the same result between instances of the servicein order to persist bulk data the hash function needs to be changed to give the same result between instances of the serviceM10 - Release 0.13YannickYannickhttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/merge_requests/308module to gather dataframe validation functions2023-08-17T11:20:38ZYannickmodule to gather dataframe validation functionsgather dataframe validation functions + have them stateless without side effects.gather dataframe validation functions + have them stateless without side effects.M10 - Release 0.13https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/merge_requests/307Update dask version (vulnerability issue in current version)2023-08-17T11:20:39Zfabian serinUpdate dask version (vulnerability issue in current version)M10 - Release 0.13https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/merge_requests/306[bulk storage refactoring]Add module for blob path manipulation2023-08-17T11:20:41ZJeremie Hallal[bulk storage refactoring]Add module for blob path manipulation[Refactoring task] add module for path manipulation of blob files[Refactoring task] add module for path manipulation of blob filesM10 - Release 0.13YannickJeremie HallalVictor DosdaYannickhttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/merge_requests/305Extract utils from dask storage2023-08-17T11:20:42ZYannickExtract utils from dask storageCode clean upCode clean upM10 - Release 0.13https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/merge_requests/304Update Dockerfile - use latest debain image as base2023-08-17T11:20:44ZVincent RondotUpdate Dockerfile - use latest debain image as baseUse latests debian base image, this should fix a few known vulnerabilities.Use latests debian base image, this should fix a few known vulnerabilities.M10 - Release 0.13https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/merge_requests/303Improves the error message and the stack trace in case of error 500 when call...2023-08-17T11:20:45ZChristophe LallementImproves the error message and the stack trace in case of error 500 when calling GET /ddms/v2/trajectories/{trajectoryid}/dataM10 - Release 0.13Christophe LallementChristophe Lallementhttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/merge_requests/302Fix - 500 error "[none] not in index" when reading bulk data2023-08-17T11:20:47ZCyril MonmoutonFix - 500 error "[none] not in index" when reading bulk dataM10 - Release 0.13https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/merge_requests/301Review bulk uri2023-08-17T11:20:49ZYannickReview bulk uriReview bulk URI to be explicit and centralized management.
Especially, it's used to differentiate storage engine. A meaningless prefix constant usage were found in multiple places in the code with some confusion between bulk id and bulk ...Review bulk URI to be explicit and centralized management.
Especially, it's used to differentiate storage engine. A meaningless prefix constant usage were found in multiple places in the code with some confusion between bulk id and bulk URI making quite hard to follow.
This MR introduce a new class `BulkURI` that encapsulates encode/decoding the URI value. It contains 2 attributes:
- bulkd id
- storage engine version
Storage engine version encapsulates its specific prefix value used in URI value. This prefix is no longer exposed and not meant to be used directly.
Storage engine version on [construction](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/blob/review_bulk_uri/app/bulk_persistence/bulk_uri.py#L72), [check](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/blob/review_bulk_uri/app/bulk_persistence/bulk_uri.py#L72) is made explicit.
Usage examples:
- ctor from URI string value:
`bulk_uri = BulkURI.decode(uri_str)`
- ctor explicit given a bulk_id and a storage version:
`bulk_uri = BulkURI(bulk_id=bulk_id_value, version=BulkStorageVersion_V1)`
- ctor explict using class method:
`bulk_uri = BulkURI.from_bulk_storage_V1(bulk_id_value)`
- encode to URI string value:
`uri_str: str = bulk_uri.encode()`
- check which storage engine version is:
`bulk_uri.storage_version == BulkStorageVersion_V0`
`bulk_uri.is_bulk_storage_V0()`M10 - Release 0.13https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/merge_requests/299add missing unit tests2023-08-17T11:20:50ZChristophe Lallementadd missing unit testsM10 - Release 0.13Christophe LallementChristophe Lallementhttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/merge_requests/298Updated fossa notice2023-08-17T11:20:52ZLuc YriarteUpdated fossa noticeM10 - Release 0.13https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/merge_requests/297fix aws deployment2023-08-17T11:20:53ZYunhua Koglinfix aws deploymentM10 - Release 0.13Yunhua KoglinYunhua Koglinhttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/merge_requests/296Update to log get groups test2023-08-17T11:20:55ZVictor DosdaUpdate to log get groups testM10 - Release 0.13Victor DosdaVictor Dosdahttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/merge_requests/295Delete Wellbore DDMS kickoff with R3 team.mp42023-08-17T11:20:56ZVincent RondotDelete Wellbore DDMS kickoff with R3 team.mp4Delete Wellbore DDMS kickoff with R3 team.mp4
(moved to https://community.opengroup.org/osdu/documentation/-/tree/master/platform/api/Wellbore-DDMS/media)Delete Wellbore DDMS kickoff with R3 team.mp4
(moved to https://community.opengroup.org/osdu/documentation/-/tree/master/platform/api/Wellbore-DDMS/media)M10 - Release 0.13https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/merge_requests/294Draft: GONRG-3276 add GCP readme2021-11-09T15:21:45ZAleksandr Spivakov (EPAM)Draft: GONRG-3276 add GCP readmeAdd GCP readme with entitlements groups infoAdd GCP readme with entitlements groups infoM10 - Release 0.13Siarhei Khaletski (EPAM)Siarhei Khaletski (EPAM)