seismic-dms-service merge requestshttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/merge_requests2023-09-06T11:58:43Zhttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/merge_requests/882feat: API endpoint to retrieve dataset sizes2023-09-06T11:58:43ZIzabela Kulakowskafeat: API endpoint to retrieve dataset sizes## Type of change
* [ ] Bug Fix
* [x] Feature
* [ ] Pipeline
* [ ] Test
* [ ] Documentation
## Does this introduce a change in the core logic?
* [x] No
* [ ] Yes
## Does this introduce a change in the cloud provider implementation, i...## Type of change
* [ ] Bug Fix
* [x] Feature
* [ ] Pipeline
* [ ] Test
* [ ] Documentation
## Does this introduce a change in the core logic?
* [x] No
* [ ] Yes
## Does this introduce a change in the cloud provider implementation, if so which cloud?
* [ ] AWS
* [ ] Anthos
* [x] Azure
* [ ] GCP
* [ ] IBM
## Does this follow [conventional commits spec](https://www.conventionalcommits.org/en/v1.0.0/#summary)?
* [ ] No
* [x] Yes
## Have you set the target Milestone?
* [ ] No
* [x] Yes
## Have you set the no-detached-pipeline label?
* [ ] No
* [x] Yes
## Updates description?
> More information in the ADR: https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/issues/108M21 - Release 0.24Izabela KulakowskaIzabela Kulakowskahttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/merge_requests/793feat: autogenerate swagger UI2023-07-07T16:55:13ZNova Laderfeat: autogenerate swagger UI## Type of change
- [ ] Bug Fix
- [X] Feature
- [ ] Pipeline
- [ ] Test
- [X] Documentation
## Does this introduce a change in the core logic?
- [X] No
- [ ] Yes
## Does this introduce a change in the cloud provider implementation, if ...## Type of change
- [ ] Bug Fix
- [X] Feature
- [ ] Pipeline
- [ ] Test
- [X] Documentation
## Does this introduce a change in the core logic?
- [X] No
- [ ] Yes
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [ ] Anthos
- [ ] Azure
- [ ] GCP
- [ ] IBM
## Does this follow [conventional commits spec](https://www.conventionalcommits.org/en/v1.0.0/#summary)?
- [ ] No
- [X] Yes
## Have you set the target Milestone?
- [ ] No
- [X] Yes
## Have you set the no-detached-pipeline label?
- [ ] No
- [X] Yes
## Updates description?
adds automatic document generation build scriptM20 - Release 0.23Nova LaderNova Laderhttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/merge_requests/792feat: autogenerate swagger UI2023-07-07T14:55:42ZNova Laderfeat: autogenerate swagger UI## Type of change
- [ ] Bug Fix
- [X] Feature
- [ ] Pipeline
- [ ] Test
- [X] Documentation
## Does this introduce a change in the core logic?
- [X] No
- [ ] Yes
## Does this introduce a change in the cloud provider implementation, if ...## Type of change
- [ ] Bug Fix
- [X] Feature
- [ ] Pipeline
- [ ] Test
- [X] Documentation
## Does this introduce a change in the core logic?
- [X] No
- [ ] Yes
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [ ] Anthos
- [ ] Azure
- [ ] GCP
- [ ] IBM
## Does this follow [conventional commits spec](https://www.conventionalcommits.org/en/v1.0.0/#summary)?
- [ ] No
- [X] Yes
## Have you set the target Milestone?
- [ ] No
- [X] Yes
## Have you set the no-detached-pipeline label?
- [ ] No
- [X] Yes
## Updates description?
added document autogeneratorM20 - Release 0.23Nova LaderNova Laderhttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/merge_requests/767feat: autogenerate swagger ui2023-07-07T15:22:58ZNova Laderfeat: autogenerate swagger ui## Type of change
- [ ] Bug Fix
- [X] Feature
- [ ] Pipeline
- [ ] Test
## Does this introduce a change in the core logic?
- [X] No
- [ ] Yes
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ...## Type of change
- [ ] Bug Fix
- [X] Feature
- [ ] Pipeline
- [ ] Test
## Does this introduce a change in the core logic?
- [X] No
- [ ] Yes
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [ ] Azure
- [ ] GCP
- [ ] IBM
## Does this follow [conventional commits spec](https://www.conventionalcommits.org/en/v1.0.0/#summary)?
- [ ] No
- [X] Yes
## Have you set the target Milestone?
- [ ] No
- [X] Yes
## Have you set the no-detached-pipeline label?
- [ ] No
- [X] Yes
## Updates description?
> Update this part with a description of your changesM19 - Release 0.22Nova LaderNova Laderhttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/merge_requests/41Feat/aws server fix2023-08-18T10:26:46ZRucha DeshpandeFeat/aws server fix- AWS Implementation
- Server support for SSL- AWS Implementation
- Server support for SSLM5 - Release 0.8Rucha DeshpandeDiego MolteniRucha Deshpandehttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/merge_requests/40Feat/aws ss impl2023-08-18T10:26:47ZRucha DeshpandeFeat/aws ss implM5 - Release 0.8Rucha DeshpandeRucha Deshpandehttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/merge_requests/843feat: Azure CosmosDB sidecar - update QueryRequestOptions2023-08-18T14:58:59ZIzabela Kulakowskafeat: Azure CosmosDB sidecar - update QueryRequestOptions## Type of change
- [ ] Bug Fix
- [x] Feature
- [ ] Pipeline
- [ ] Test
- [ ] Documentation
## Does this introduce a change in the core logic?
- [x] No
- [ ] Yes
## Does this introduce a change in the cloud provider implementation, if ...## Type of change
- [ ] Bug Fix
- [x] Feature
- [ ] Pipeline
- [ ] Test
- [ ] Documentation
## Does this introduce a change in the core logic?
- [x] No
- [ ] Yes
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [ ] Anthos
- [x] Azure
- [ ] GCP
- [ ] IBM
## Does this follow [conventional commits spec](https://www.conventionalcommits.org/en/v1.0.0/#summary)?
- [ ] No
- [x] Yes
## Have you set the target Milestone?
- [x] No
- [ ] Yes
## Have you set the no-detached-pipeline label?
- [x] No
- [ ] Yes
## Updates description?
Updating QueryRequestOptions in .Net CosmosDB sidecar for better performance of read queries.M20 - Release 0.23Izabela KulakowskaIzabela Kulakowskahttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/merge_requests/700feat: azure dataset exist getMulti2023-04-20T13:16:21ZRashaad Grayfeat: azure dataset exist getMulti## Type of change
- [ ] Bug Fix
- [X] Feature
## Does this introduce a change in the core logic?
- [X] No
- [ ] Yes
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [ ] Azure
- [ ] GC...## Type of change
- [ ] Bug Fix
- [X] Feature
## Does this introduce a change in the core logic?
- [X] No
- [ ] Yes
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [ ] Azure
- [ ] GCP
- [ ] IBM
## Does this follow [conventional commits spec](https://www.conventionalcommits.org/en/v1.0.0/#summary)?
- [ ] No
- [X] Yes
## Updates description?
> New feature changes how the azure_dataset_exist endpoint confirm that multiple datasets exist within cosmosdb with the function getMultiM18 - Release 0.21Rashaad GrayRashaad Grayhttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/merge_requests/252feat: caller headers forwarded to the outbound traffic2021-11-09T14:45:21ZDiego Moltenifeat: caller headers forwarded to the outbound trafficEnabled called header propagation on services outbound traffic. default forwarded fields are ([ref](https://www.npmjs.com/package/hpropagate)):
- x-correlation-id
- x-variant-id
- x-feature-flags
- x-request-id
- x-b3-traceid
- x-b3-spa...Enabled called header propagation on services outbound traffic. default forwarded fields are ([ref](https://www.npmjs.com/package/hpropagate)):
- x-correlation-id
- x-variant-id
- x-feature-flags
- x-request-id
- x-b3-traceid
- x-b3-spanid
- x-b3-parentspanid
- x-b3-sampled
- x-b3-flags
- x-ot-span-context
The default forward configuration can be changed, by specifying the list of headers to forward in the env variable **CALLER_FORWARD_HEADERS** as comma-separated strings list, no spacesM10 - Release 0.13Diego MolteniDiego Moltenihttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/merge_requests/968feat: change @cloudant/cloudant dependency by @ibm-cloud/cloudant2024-01-16T17:12:30ZIsha Kumarifeat: change @cloudant/cloudant dependency by @ibm-cloud/cloudantfeat: change @cloudant/cloudant dependency by @ibm-cloud/cloudantfeat: change @cloudant/cloudant dependency by @ibm-cloud/cloudantM22 - Release 0.25Isha KumariIsha Kumarihttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/merge_requests/962feat: change sdms v4 dependency url to kubernetes in-cluster url2023-12-05T09:59:03ZKajal Sawalefeat: change sdms v4 dependency url to kubernetes in-cluster url## Type of change
* [ ] Bug Fix
* [x] Feature
* [ ] Pipeline
* [ ] Test
* [ ] Documentation
## Does this introduce a change in the core logic?
* [ ] No
* [x] Yes
## Does this introduce a change in the cloud provider implementation, i...## Type of change
* [ ] Bug Fix
* [x] Feature
* [ ] Pipeline
* [ ] Test
* [ ] Documentation
## Does this introduce a change in the core logic?
* [ ] No
* [x] Yes
## Does this introduce a change in the cloud provider implementation, if so which cloud?
* [ ] AWS
* [ ] Anthos
* [ ] Azure
* [ ] GCP
* [ ] IBM
## Does this follow [conventional commits spec](https://www.conventionalcommits.org/en/v1.0.0/#summary)?
* [ ] No
* [x] Yes
## Have you set the target Milestone?
* [ ] No
* [x] Yes
## Have you set the no-detached-pipeline label?
* [ ] No
* [x] Yes
## Updates description?
> Used new environment variables inorder to use kube incluster dnsM22 - Release 0.25Kajal SawaleKajal Sawalehttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/merge_requests/899feat: code enhanced and pre-hook added2023-09-14T21:06:07ZDiego Moltenifeat: code enhanced and pre-hook added## Type of change
- [ ] Bug Fix
- [ ] Feature
- [ ] Pipeline
- [ ] Test
- [ ] Documentation
- [X] Enhancement
## Does this introduce a change in the core logic?
- [X] No
- [ ] Yes
## Does this introduce a change in the cloud provider i...## Type of change
- [ ] Bug Fix
- [ ] Feature
- [ ] Pipeline
- [ ] Test
- [ ] Documentation
- [X] Enhancement
## Does this introduce a change in the core logic?
- [X] No
- [ ] Yes
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [ ] Anthos
- [ ] Azure
- [ ] GCP
- [ ] IBM
## Does this follow [conventional commits spec](https://www.conventionalcommits.org/en/v1.0.0/#summary)?
- [ ] No
- [X] Yes
## Have you set the target Milestone?
- [ ] No
- [X] Yes
## Have you set the no-detached-pipeline label?
- [ ] No
- [X] Yes
## Updates description?
- added code linter
- added code style
- added code speller
- added markdown linter
- added git pre-hook
- fixed spell and linting issue in code
- fixed the cli generator
- added main README
```text
-------------------------------------------------------
1 - spelling errors check
-------------------------------------------------------
> sdms@4.0.0 spell
> cspell --no-progress .
CSpell: Files checked: 107, Issues found: 0 in 0 files
-------------------------------------------------------
2 - markdown linting
-------------------------------------------------------
> sdms@4.0.0 markdown-lint
> markdownlint-cli2 **/*.md #node_modules
markdownlint-cli2 v0.9.2 (markdownlint v0.30.0)
Finding: **/*.md !node_modules
Linting: 3 file(s)
Summary: 0 error(s)
-------------------------------------------------------
3 - code linting
-------------------------------------------------------
> sdms@4.0.0 lint
> eslint --ignore-path .eslintignore --ext .js,.ts .
-------------------------------------------------------
4 - code formatting
-------------------------------------------------------
> sdms@4.0.0 prettier
> prettier -c src/**/*.ts
Checking formatting...
All matched files use Prettier code style!
-------------------------------------------------------
5 - validate commit message
-------------------------------------------------------
commit message ok
-------------------------------------------------------
```M21 - Release 0.24Daniel PerezDaniel Perezhttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/merge_requests/1004feat: compute size service implementation2023-12-14T14:15:49ZIzabela Kulakowskafeat: compute size service implementation## Type of change
- [ ] Bug Fix
- [x] Feature
- [ ] Pipeline
- [ ] Test
- [ ] Documentation
## Does this introduce a change in the core logic?
- [ ] No
- [x] Yes
## Does this introduce a change in the cloud provider implementation, if ...## Type of change
- [ ] Bug Fix
- [x] Feature
- [ ] Pipeline
- [ ] Test
- [ ] Documentation
## Does this introduce a change in the core logic?
- [ ] No
- [x] Yes
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [ ] Anthos
- [x] Azure
- [ ] GCP
- [ ] IBM
## Does this follow [conventional commits spec](https://www.conventionalcommits.org/en/v1.0.0/#summary)?
- [ ] No
- [x] Yes
## Have you set the target Milestone?
- [ ] No
- [x] Yes
## Have you set the no-detached-pipeline label?
- [ ] No
- [x] Yes
## Updates description?
This MR contains an implementation of ComputeSize background service (Azure), that is responsible for syncing the size property between Azure Storage and datasets metadata.
The compute size operation is triggered on dataset being closed (using PATCH API endpoint 'close'). Then the message is sent to a Azure Storage queue and picked up from a queue by the ComputeSize service.
The ComputeSize service fetches blobs for the dataset, sums up their sizes and compares the size with the one sent in the message queue.
If they are not the same then it locks the dataset and performs a partial update on the dataset metadata record.
E2E tests will be raised in the next MR (after updating infrastructure in [helm-charts-azure](https://community.opengroup.org/osdu/platform/deployment-and-operations/helm-charts-azure) repo).M22 - Release 0.25Izabela KulakowskaIzabela Kulakowskahttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/merge_requests/324feat: control logs for azure via env settings2022-02-09T11:15:11ZDiego Moltenifeat: control logs for azure via env settingscontrol logs for azure via env var settingscontrol logs for azure via env var settingsM11 - Release 0.14https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/merge_requests/342feat: create a default subproject whenever a2022-03-15T14:42:58ZVarunkumar Manoharfeat: create a default subproject whenever aCreate a `default` subproject whenever a tenant is registeredCreate a `default` subproject whenever a tenant is registeredM11 - Release 0.14https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/merge_requests/1005feat: Dataset list improvements2023-11-21T09:42:14ZMaggie Salakfeat: Dataset list improvements## Type of change
- [x] Bug Fix
- [x] Feature
- [ ] Pipeline
- [ ] Test
- [ ] Documentation
## Does this introduce a change in the core logic?
- [ ] No
- [x] Yes
## Does this introduce a change in the cloud provider implementation, if ...## Type of change
- [x] Bug Fix
- [x] Feature
- [ ] Pipeline
- [ ] Test
- [ ] Documentation
## Does this introduce a change in the core logic?
- [ ] No
- [x] Yes
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [ ] Anthos
- [x] Azure
- [ ] GCP
- [ ] IBM
## Does this follow [conventional commits spec](https://www.conventionalcommits.org/en/v1.0.0/#summary)?
- [ ] No
- [x] Yes
## Have you set the target Milestone?
- [x] No
- [ ] Yes
## Have you set the no-detached-pipeline label?
- [ ] No
- [x] Yes
## Updates description?
Improvements and fixes for dataset retrieval from Cosmos DB:
- Updates the Sidecar implementation of dataset retrieval to return exactly the requested number of records from Cosmos DB, if available.
- Fix the utility/ls implementation to only return datasets in the root folder in specified path is '/'M22 - Release 0.25Maggie SalakMaggie Salakhttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/merge_requests/733feat: dataset size performance boost2023-07-13T15:02:37ZJason Wangfeat: dataset size performance boost## Type of change
- [ ] Bug Fix
- [X] Feature
- [ ] Pipeline
- [ ] Test
## Does this introduce a change in the core logic?
- [X] No
- [ ] Yes
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ...## Type of change
- [ ] Bug Fix
- [X] Feature
- [ ] Pipeline
- [ ] Test
## Does this introduce a change in the core logic?
- [X] No
- [ ] Yes
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [X] Azure
- [ ] GCP
- [ ] IBM
## Does this follow [conventional commits spec](https://www.conventionalcommits.org/en/v1.0.0/#summary)?
- [X] No
- [ ] Yes
## Have you set the target Milestone?
- [X] No
- [ ] Yes
## Have you set the no-detached-pipeline label?
- [X] No
- [ ] Yes
## Updates description?
> Update this part with a description of your changesM19 - Release 0.22Jason WangJason Wanghttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/merge_requests/565feat: disable schema properties format check in SDMS-V42022-11-30T13:12:47ZDiego Moltenifeat: disable schema properties format check in SDMS-V4disable schema properties format check in SDMS-V4
these can be re-enabled via env configs ```export ENABLE_SCHEMA_PROPERTIES_FORMAT_VALIDATION="true"```disable schema properties format check in SDMS-V4
these can be re-enabled via env configs ```export ENABLE_SCHEMA_PROPERTIES_FORMAT_VALIDATION="true"```M16 - Release 0.19Diego MolteniDiego Moltenihttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/merge_requests/1079feat: enable dataset for bulkDelete2024-02-13T20:34:56ZRashaad Grayfeat: enable dataset for bulkDelete## Type of change
- [ ] Bug Fix
- [X] Feature
- [ ] Pipeline
- [ ] Test
- [ ] Documentation
## Does this introduce a change in the core logic?
- [X] No
- [ ] Yes
## Does this introduce a change in the cloud provider implementation, if ...## Type of change
- [ ] Bug Fix
- [X] Feature
- [ ] Pipeline
- [ ] Test
- [ ] Documentation
## Does this introduce a change in the core logic?
- [X] No
- [ ] Yes
## Does this introduce a change in the cloud provider implementation, if so which cloud?
- [ ] AWS
- [ ] Anthos
- [ ] Azure
- [ ] GCP
- [ ] IBM
## Does this follow [conventional commits spec](https://www.conventionalcommits.org/en/v1.0.0/#summary)?
- [ ] No
- [X] Yes
## Have you set the target Milestone?
- [ ] No
- [X] Yes
## Have you set the no-detached-pipeline label?
- [ ] No
- [X] Yes
## Updates description?
> Enable use of datasetPath for bulkDelete and sync sidecar changesM23 - Release 0.26Rashaad GrayRashaad Grayhttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/merge_requests/125feat: enable dataset level ACL based authorization - Part 32021-07-07T11:54:39ZVarunkumar Manoharfeat: enable dataset level ACL based authorization - Part 3This merge request accomplishes the following:
1. If the subproject access policy is set to `uniform` then datagroups based acl authorization remains the same
2. Else if the subproject access policy is set to `dataset` then the operatio...This merge request accomplishes the following:
1. If the subproject access policy is set to `uniform` then datagroups based acl authorization remains the same
2. Else if the subproject access policy is set to `dataset` then the operations at the dataset level will now use the ACL groups at the dataset level for authorization checks
3. User addition at the dataset level
4. Copy utility endpoint to enforce dataset level ACL based authorization
5. Updates to the `{{SVC_URL}}/utility/gcs-access-token` endpoint enabling the generation of Google Cloud Storage access tokens at the object prefix level (specific folder level).
Previously it was only possible to generate access tokens only at the bucket level. A bucket could contain multiple folders. `Note this functionality is currently available only for Google Cloud`
7 unit test fixesM7 - Release 0.10