seismic-dms-service merge requestshttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/merge_requests2022-08-15T18:32:31Zhttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/merge_requests/482Cherry-pick 'feat: added list endpoint as post' into release/0.162022-08-15T18:32:31ZDavid Diederichd.diederich@opengroup.orgCherry-pick 'feat: added list endpoint as post' into release/0.16Original MR: !436Original MR: !436M13 - Release 0.16David Diederichd.diederich@opengroup.orgDavid Diederichd.diederich@opengroup.orghttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/merge_requests/481Cherry-pick 'ci: update alpine version to fix busybox vulnerability' into rel...2022-08-12T21:35:35ZDavid Diederichd.diederich@opengroup.orgCherry-pick 'ci: update alpine version to fix busybox vulnerability' into release/0.16Original MR: !444Original MR: !444M13 - Release 0.16David Diederichd.diederich@opengroup.orgDavid Diederichd.diederich@opengroup.orghttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/merge_requests/479fix: (azure) forcing usage old key model when using a single partition model ...2022-08-15T16:22:58ZDiego Moltenifix: (azure) forcing usage old key model when using a single partition model db in azure comsosforcing usage old key model when using a single partition model db in azure comsosforcing usage old key model when using a single partition model db in azure comsosM13 - Release 0.16Diego MolteniDiego Moltenihttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/merge_requests/475fix: fix the enforce key based on db model2022-08-12T07:19:42ZDiego Moltenifix: fix the enforce key based on db model fix the enforce key based on db model fix the enforce key based on db modelM13 - Release 0.16Diego MolteniDiego Moltenihttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/merge_requests/468Cherry-pick 'ci: fixed pipeline condition' into release/0.162022-08-11T03:27:40ZDavid Diederichd.diederich@opengroup.orgCherry-pick 'ci: fixed pipeline condition' into release/0.16Original MR: !464Original MR: !464M13 - Release 0.16David Diederichd.diederich@opengroup.orgDavid Diederichd.diederich@opengroup.orghttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/merge_requests/467fix: Anthos logger, MinIO list objects, and search Datasets by gtags2022-08-17T09:01:02ZYan Sushchynski (EPAM)fix: Anthos logger, MinIO list objects, and search Datasets by gtagsFix two issues:
1. log4js doesn't write logs in GCP, so it was changed to usual `console`
1. Different versions of MinIO return different responses for `ListObjects` with and without `Contents` field, so we added handling such a situati...Fix two issues:
1. log4js doesn't write logs in GCP, so it was changed to usual `console`
1. Different versions of MinIO return different responses for `ListObjects` with and without `Contents` field, so we added handling such a situation.
2. Add searching in JSON arrays in the DatabaseM13 - Release 0.16https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/merge_requests/466Draft: Fix anthos logger minio2022-08-09T10:07:53ZYan Sushchynski (EPAM)Draft: Fix anthos logger minioFix two issues:
1. log4js doesn't write logs in GCP, so it was changed to usual `console`
2. Different versions of MinIO return different responses for `ListObjects` with and without `Contents` field, so we added handling such a situation.Fix two issues:
1. log4js doesn't write logs in GCP, so it was changed to usual `console`
2. Different versions of MinIO return different responses for `ListObjects` with and without `Contents` field, so we added handling such a situation.M13 - Release 0.16https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/merge_requests/464ci: fixed pipeline condition2022-08-16T14:50:45ZDaniel Perezci: fixed pipeline conditionM13 - Release 0.16https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/merge_requests/458feat: unit test implementation for filemetadata2022-08-04T13:43:14ZPanagiotis Kraniotisfeat: unit test implementation for filemetadataImpl of unit tests for filemetadataImpl of unit tests for filemetadataM13 - Release 0.16Duo ChenPanagiotis KraniotisDuo Chenhttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/merge_requests/455fix: defined the model for file-metadata record field in the azure sidecar2022-08-03T14:02:56ZDiego Moltenifix: defined the model for file-metadata record field in the azure sidecarThe filemetadata model was modeled as generic object in the sidecar and this was not allowing the conversion in the service.
this MR will provide the data model for the file-metadata field for the dataset record.The filemetadata model was modeled as generic object in the sidecar and this was not allowing the conversion in the service.
this MR will provide the data model for the file-metadata field for the dataset record.M13 - Release 0.16Diego MolteniDiego Moltenihttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/merge_requests/450ci: migrate ubuntu and node images to use MCR for seismic sdms and filemetadata2022-08-18T23:43:44ZMarija Dukicci: migrate ubuntu and node images to use MCR for seismic sdms and filemetadata# Overview
- Copied [Dockerfile](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/blob/master/app/filemetadata/Dockerfile) for Seismic Filemetadata to **devops/azur...# Overview
- Copied [Dockerfile](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/blob/master/app/filemetadata/Dockerfile) for Seismic Filemetadata to **devops/azure** location and updated `FROM` in new Dockerfile to use ubuntu image from Microsoft Container Registry (this is to avoid other cloud providers using MCR)
- Copied [runtime.Dockerfile](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/blob/master/app/sdms/docker/runtime.Dockerfile) for Seismic SDMS to **devops/azure** location and updated `FROM` in new Dockerfile to use node:16-alpine image from Microsoft Container Registry (this is to avoid other cloud providers using MCR)
- Updated **filemetadata_azure_containerize** job in pipeline to build image from new Dockerfile location
- Updated **sdms_azure_containerize** job in pipeline to build image from new Dockerfile location
- ADO work item: [Securing the containers supply chain - Seismic](https://dev.azure.com/OpenEnergyPlatform/Open%20Energy%20Platform/_workitems/edit/7918)
# Testing
- Local testing has passed successfully and new image is being used
- **filemetadata_azure_containerize** step is pulling _ubuntu:20.04_ image from MCR in MR pipeline
![image](/uploads/9b09260045683ddca74c79e23a00576f/image.png)
- **sdms_azure_containerize** step is pulling _node:14-alpine_ image from MCR in MR pipeline
![image](/uploads/b6dcff935597bc410f48b0b846469c9c/image.png)
- Filemetadata and SDMS Azure integration tests have passed successfully in MR pipeline
- SDMS ibm and gcp deployment is failing due to invalid or expired token (known issue)
- Additional testing will be done on Azure side with build pipeline against **m12-master** branchM13 - Release 0.16Marija DukicMarija Dukichttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/merge_requests/449ci: GONRG-5336 - add var for env id2022-07-29T08:09:38ZAliaksandr Ramanovich (EPAM)ci: GONRG-5336 - add var for env idadd var to define env id in gcp seismic configmapadd var to define env id in gcp seismic configmapM13 - Release 0.16Aliaksandr Ramanovich (EPAM)Aliaksandr Ramanovich (EPAM)https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/merge_requests/446feat: Remove favicon because it is not licensed2022-07-27T19:37:50ZDuo Chenfeat: Remove favicon because it is not licensedRemove favicon because it is not licensed.Remove favicon because it is not licensed.M13 - Release 0.16https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/merge_requests/445fix: updated sidecar to support directories list in a subproject2022-07-28T12:47:25ZDiego Moltenifix: updated sidecar to support directories list in a subprojectThe sidecar was working by querying and converting full datasets records.
In case the directory list is required, the query selects only the "path" filed of the record and, because of this selection, the result cannot be converted.
This ...The sidecar was working by querying and converting full datasets records.
In case the directory list is required, the query selects only the "path" filed of the record and, because of this selection, the result cannot be converted.
This problem has been fixed by adding a specific endpoint in the sidecar that is called from the path selection query from SDMS.M13 - Release 0.16Diego MolteniDiego Moltenihttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/merge_requests/444ci: update alpine version to fix busybox vulnerability2023-05-11T12:38:37ZYifei Xuci: update alpine version to fix busybox vulnerabilityM13 - Release 0.16Yifei XuYifei Xuhttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/merge_requests/442fix: list sub-projects returns datasets from other sub-projects2022-07-25T14:44:37ZDiego Moltenifix: list sub-projects returns datasets from other sub-projectsReplaced the generic LIKE rule with a RegExMatch in the SQL query and URI Encode it before sending it to the sidecar.
The SQL query select datasets in a subproject using the rule "id LIKE sp-<tenant_name>-<subproject_name>-%" In case a ...Replaced the generic LIKE rule with a RegExMatch in the SQL query and URI Encode it before sending it to the sidecar.
The SQL query select datasets in a subproject using the rule "id LIKE sp-<tenant_name>-<subproject_name>-%" In case a subproject contains a "-" in the name, it can be wrongly selected.
For example:
tenant_name = "opendes"
subproject_name_01= "sandbox"
subproject_name_02 = "sandbox-test"
To select datasets in "sandbox" using the rule "id LIKE sp-opendes-sandbox-%" also wrongly select dataset in "sandbox-test"
replacing the SQL LIKE rule with the RegexMath fixes the issueM13 - Release 0.16Diego MolteniDiego Moltenihttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/merge_requests/439feat: add data partition id logging2022-07-28T14:58:28ZKonstantin Khottchenkovfeat: add data partition id loggingM13 - Release 0.16Konstantin KhottchenkovKonstantin Khottchenkovhttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/merge_requests/438ci: adding files for cicd pipeline2022-07-29T07:57:37ZDaniel Perezci: adding files for cicd pipelineAdding files to be used in cicd pipeline.Adding files to be used in cicd pipeline.M13 - Release 0.16Daniel PerezDaniel Perezhttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/merge_requests/437fix: refactor vars to camelcase2022-07-29T07:56:22ZKrisztian Molnar (EPAM)fix: refactor vars to camelcaseM13 - Release 0.16Krisztian Molnar (EPAM)Krisztian Molnar (EPAM)https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/merge_requests/436feat: added list endpoint as post2022-09-15T23:50:06ZDiego Moltenifeat: added list endpoint as post- The dataset list endpoint has been cloned and updated from a GET to a POST method.
- This endpoint will allow clients to send continuation-token of big size (avoid 414 Too Large Url).
- The Azure sidecar interface and implementation h...- The dataset list endpoint has been cloned and updated from a GET to a POST method.
- This endpoint will allow clients to send continuation-token of big size (avoid 414 Too Large Url).
- The Azure sidecar interface and implementation has been internally updated to implement the POST method only.
- The documentation has been updated.M13 - Release 0.16Diego MolteniDiego Molteni