Wellbore Domain Services merge requestshttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/merge_requests2021-11-18T14:03:31Zhttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/merge_requests/314parameterize docker image app_name for push and pull to/from central acrs2021-11-18T14:03:31ZAlexandre Vincentparameterize docker image app_name for push and pull to/from central acrsParameterizing pipeline templates for reusabilityParameterizing pipeline templates for reusabilityAlexandre VincentAlexandre Vincenthttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/merge_requests/175remove unnecessary parametrize cases in integration tests "multiple_sessions_...2021-11-18T15:49:31ZYannickremove unnecessary parametrize cases in integration tests "multiple_sessions_in_parallel"Motivation:
These 2 integration tests are related to session management. It checks everything work as expected when multiple sessions are alive in parallel on same record.
These tests were parametrize to check over different data format ...Motivation:
These 2 integration tests are related to session management. It checks everything work as expected when multiple sessions are alive in parallel on same record.
These tests were parametrize to check over different data format (JSON/parquet) and entity types (WellLog, Trajectory, Log) and then leads to 6 distinct combinations data format/entity type each (so 12 tests in total). Testing all these combination doesn't provide any plus value since it only on session management really matters. Moreover there are dedicated tests to check on all entity types and format.
Since these tests are quite long to execute (between 30s and 20s on dev env), it adds 5 minutes to complete the full integration test set without providing value.
The change here is to only keep one occurrence of each saving 4-5 minutes on pipeline per environment.https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/merge_requests/333Draft: Tuning default probe settings2021-12-08T09:36:31ZAlexandre VincentDraft: Tuning default probe settingsAlexandre VincentAlexandre Vincenthttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/merge_requests/300Draft: Full dask work2021-12-17T11:15:51ZYannickDraft: Full dask workImplementation done for write Data only. Read data WIP.Implementation done for write Data only. Read data WIP.https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/merge_requests/356Draft: add TracingRoute class to store path in request.state2022-01-04T15:15:24ZAlexandre VincentDraft: add TracingRoute class to store path in request.stateAlexandre VincentAlexandre Vincenthttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/merge_requests/365limit the number of columns per chunk2022-01-11T13:58:36ZJeremie Hallallimit the number of columns per chunkset max limit for the number of columns per chunk when sending dataset max limit for the number of columns per chunk when sending datahttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/merge_requests/361Draft: upgrading fastapi to 0.69.02022-01-11T17:09:27ZAlexandre VincentDraft: upgrading fastapi to 0.69.0Alexandre VincentAlexandre Vincenthttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/merge_requests/341Welllog data consistency2022-01-12T15:28:46ZChristophe LallementWelllog data consistencyChristophe LallementChristophe Lallementhttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/merge_requests/373GONRG-4172: wellbore-ddms python test suite changed to basic2022-01-18T15:46:20ZAliaksandr Lubouski (EPAM)GONRG-4172: wellbore-ddms python test suite changed to basicGONRG-4172: wellbore-ddms python test suite changed to basicGONRG-4172: wellbore-ddms python test suite changed to basichttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/merge_requests/388Draft: Well id regex check take 22022-02-03T13:32:18ZAlexandre VincentDraft: Well id regex check take 2Alexandre VincentAlexandre Vincenthttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/merge_requests/390Draft: read bulk some code clean up + usage of regex2022-02-07T10:28:18ZYannickDraft: read bulk some code clean up + usage of regexThe goal is to do some clean up around get bulk data workflow in order to ease further work (especially the move of serialization inside Dask worker). Mainly function/method/variable renaming and type hint/comment addition but there are ...The goal is to do some clean up around get bulk data workflow in order to ease further work (especially the move of serialization inside Dask worker). Mainly function/method/variable renaming and type hint/comment addition but there are also the following changes:
- bulk filter parsing done using a [regex](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/blob/read_bulk_code_clean_up/app/model/model_chunking.py#L76) instead of manual [parsing](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/blob/master/app/model/filter.py#L22). It also allows to filter if the column name contains ':' by enclosing it in double quotation marks (e.g. `"Sensor:0":lt:1` `"Sensor:180":gt:42`).
- bulk filter regex defined at [query param declaration](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/blob/read_bulk_code_clean_up/app/model/model_chunking.py#L53).
- bulk filter operators defined only once as [an enum](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/blob/read_bulk_code_clean_up/app/model/filter.py#L7) instead of multiple times.
- [explicit class](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/blob/read_bulk_code_clean_up/app/model/filter.py#L7) instead of [raw dict](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/blob/master/app/model/filter.py#L27).
- [explicit model](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/blob/read_bulk_code_clean_up/app/model/model_chunking.py#L105) for describe response instead of [raw dict](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/blob/master/app/routers/bulk/utils.py#L288).
renaming/move:
- `GetDataParams` => `GetDataQueryParameters`
- `_process_request_v1` => `load_dataframe_storage_V1` + moved from bulk_routes.py to utils.py beside others bulk existing utils functions.
- `process_params` => `filter_dataframe`
- `DataFrameRender.df_render` => split into `DataFrameRender.describe` & `DataFrameRender.render`.
- `get_size` => `get_row_count` to avoid misleading (see [pandas.Dataframe.size](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.size.html) <=> number of values)
- `filter_test.py` moved from "routers" folder to "model" folderhttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/merge_requests/401Draft: Welllog consistency dask2022-02-15T16:48:06ZChristophe LallementDraft: Welllog consistency daskChristophe LallementChristophe Lallementhttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/merge_requests/368Draft: welllog data consistency2022-02-15T16:48:47ZChristophe LallementDraft: welllog data consistencyChristophe LallementChristophe Lallementhttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/merge_requests/414Draft: correct osdu service name2022-02-17T00:11:38ZElizabeth HalperDraft: correct osdu service namehttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/merge_requests/417fossa NOTICE2022-02-17T16:05:23ZLuc Yriartefossa NOTICEhttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/merge_requests/415Draft: add x-user-id2022-02-24T18:28:45ZElizabeth HalperDraft: add x-user-idhttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/merge_requests/431Draft: test rbac bugfix2022-02-24T22:32:37ZElizabeth HalperDraft: test rbac bugfixtest to run pipeline againtest to run pipeline againhttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/merge_requests/399Draft: Well id regex check take 32022-02-25T08:54:09ZAlexandre VincentDraft: Well id regex check take 3Alexandre VincentAlexandre Vincenthttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/merge_requests/385Draft: refactor chunking fixtures2022-02-25T10:14:22ZAlexandre VincentDraft: refactor chunking fixturesfix tests impacted by new fixturesfix tests impacted by new fixturesAlexandre VincentAlexandre Vincenthttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/merge_requests/380Draft: add well id regex type and url path check2022-02-25T10:15:08ZAlexandre VincentDraft: add well id regex type and url path checkAlexandre VincentAlexandre Vincent