seismic-dms-service issueshttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/issues2023-03-27T19:35:26Zhttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/issues/1Delete dataset API does not delete COS (Blob Storage) object2023-03-27T19:35:26ZWalter DDelete dataset API does not delete COS (Blob Storage) objectThe delete dataset API of seismic-store-service, calls the storage service POST delete record API. This API deletes the object from COS(Blob Storage) belonging to the dataset. However, the COS object is available even though the response...The delete dataset API of seismic-store-service, calls the storage service POST delete record API. This API deletes the object from COS(Blob Storage) belonging to the dataset. However, the COS object is available even though the response is 204 No Content. We realize that storage service POST delete is just doing soft delete. We wanted to confirm if this is the expected behavior.ethiraj krishnamanaiduethiraj krishnamanaiduhttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/issues/2upgrade typescript package to be 4.1.32023-09-07T15:08:59ZYunhua Koglinupgrade typescript package to be 4.1.3Hello,
We would like to use aws-sdk package version 3 (the new release) for aws implementation. However, it requires typescript package version upgraded to be 4.1.3.
With this upgrade, it will also require to remove asyn with abstract...Hello,
We would like to use aws-sdk package version 3 (the new release) for aws implementation. However, it requires typescript package version upgraded to be 4.1.3.
With this upgrade, it will also require to remove asyn with abstract modifier. https://stackoverflow.com/questions/65355164/error-ts1243-async-modifier-cannot-be-used-with-abstract-modifier.
Please let us know if these changes will be fine.
Regards;
YunhuaJoeRucha DeshpandeDiego MolteniYunhua KoglinJoehttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/issues/3Documentation of latest code changes2023-03-28T04:07:18ZWalter DDocumentation of latest code changesHi Team.
Hope everyone is safe and doing well.
The latest code check-in has lot of changes from addition
1. cloud.ts
2. trace.ts
to deletion of files (iam.ts) along with several updated files.
Also, the folder structure has changed ...Hi Team.
Hope everyone is safe and doing well.
The latest code check-in has lot of changes from addition
1. cloud.ts
2. trace.ts
to deletion of files (iam.ts) along with several updated files.
Also, the folder structure has changed
1. /config and /swaggerdocs are not present.
2. config.ts is moved into /cloud
Is it possible to provide us with document containing the important changes resulting in changes to CSP implementation? Thank you.Diego MolteniDiego Moltenihttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/issues/4GCP specfic naming conventions2023-03-27T19:32:16ZRucha DeshpandeGCP specfic naming conventionsThere are many GCP specific names used in the models:
such as gcpid, gcp_bucket etc.
There is also an API called /api/v3/utility/gcs-access-token.
The code should be re-visited to remove any CSP specific naming used.There are many GCP specific names used in the models:
such as gcpid, gcp_bucket etc.
There is also an API called /api/v3/utility/gcs-access-token.
The code should be re-visited to remove any CSP specific naming used.Rucha DeshpandeDiego MolteniRucha Deshpandehttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/issues/5Purpose of 'esd' field in tenant model2021-06-09T12:31:20ZRucha DeshpandePurpose of 'esd' field in tenant modelThe 'esd' field seems specific to GCP. The core code uses this field to determine partition ID for authorization calls. What is the purpose of this field for other CSPs?
The datapartitionID should be from the request header.The 'esd' field seems specific to GCP. The core code uses this field to determine partition ID for authorization calls. What is the purpose of this field for other CSPs?
The datapartitionID should be from the request header.Rucha DeshpandeDiego MolteniRucha Deshpandehttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/issues/6e2e tests - postman collection does not include tenant registration2023-05-02T19:11:18ZRucha Deshpandee2e tests - postman collection does not include tenant registrationThe postman collection used for e2e tests does not include the tenant creation which is required for e2e tests.The postman collection used for e2e tests does not include the tenant creation which is required for e2e tests.Rucha DeshpandeDiego MolteniRucha Deshpandehttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/issues/7Data Ecosystem Service Host refactoring2021-02-11T16:29:58ZAnuj GuptaData Ecosystem Service Host refactoringThere is a variable DES_SERVICE_HOST at below code which assumes that all the data ecosystem services (storage, entitlement, legal) should have same base url which may not be the case in ibm implementation and also reduces the flexibilit...There is a variable DES_SERVICE_HOST at below code which assumes that all the data ecosystem services (storage, entitlement, legal) should have same base url which may not be the case in ibm implementation and also reduces the flexibility to orchestrate services for debugging.
https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/blob/master/src/cloud/config.ts#L156
As of now we have raised below MR but may have to make change in core for above said issue.
https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/merge_requests/1
cc: @dsouzawalterDiego MolteniDiego Moltenihttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/issues/8Integration tests assume a valid legal tag exists2023-03-27T19:30:52ZRucha DeshpandeIntegration tests assume a valid legal tag existsIf the FEATURE FLAG for legal is set to true, it checks the validity of the legal tag passed in 'ltag'.If the FEATURE FLAG for legal is set to true, it checks the validity of the legal tag passed in 'ltag'.Rucha DeshpandeDiego MolteniRucha Deshpandehttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/issues/9E2E test case issues2021-03-12T06:32:11ZWalter DE2E test case issuesHey @DiegoMolteni, hope all good. As discussed on the call on Friday, attaching list of errors from the E2E testing. I've I assigned this issue to you as not sure whom you wanted it to be assigned. Please let me know the name and will ma...Hey @DiegoMolteni, hope all good. As discussed on the call on Friday, attaching list of errors from the E2E testing. I've I assigned this issue to you as not sure whom you wanted it to be assigned. Please let me know the name and will make the change accordingly. Thank you.
- [x] GET SUBPROJECT GET Validate Subproject Details | AssertionError: expected 11 to deeply equal 7 .Why 7 fields and not 11?
- [x] GET DATASET CHECK CTAG Body matches string | AssertionError: expected 'false' to deeply equal 'true'. Getting false as response
- [ ] POST DATASET SIZES Body matches string | AssertionError: expected '[-1,-1,-1,-1,-1]' to include '[-1,-1,-1,-1,1021]' check with our code
- [ ] PUT DATASET READONLY LOCK WRITE Status code is 400 Bad Request | AssertionError: expected response to have status code 400 but got 423. Read a data locked for writing | AssertionError: expected '[seismic-store-service] The dataset opendes/subproject0/testdata is locked for read' to include 'cannot be locked' . check code, also, maybe test issue
- [ ] GET DATASET GET +SEISMICMETA (FROM PATCH)
Check seismicmetadata | TypeError: Cannot read property 'coordinates' of undefined
- could be test error
- [ ] GET SEARCH BY SINGLE TAG
a. Tagged Data List | AssertionError: expected 0 to deeply equal 2
*check previous test cases
b. Find tagged data in list | TypeError: Cannot read property 'path' of undefined
- [ ] GET SEARCH BY MULTIPLE TAG
a. Data List | AssertionError: expected 0 to deeply equal 2
b. Find the tagged data in list | TypeError: Cannot read property 'path' of undefined
- [ ] GET CHECK ADDED TAG
gtag is added | AssertionError: expected undefined to deeply equal 'tag1'
- [ ] GET SEARCH BY ADDED TAG
Tagged Data List | AssertionError: expected 0 to deeply equal 3
Find tagged data in list | TypeError: Cannot read property 'path' of undefined
- [ ] GET SEARCH BY NEW TAG
Tagged Data List | AssertionError: expected 0 to deeply equal 1
Find tagged data in list | TypeError: Cannot read property 'path' of undefined
- [ ] POST UPLOAD NEW TO GCS
Status code is 200 | AssertionError: expected response to have status code 200 but got 401
- [ ] GET SEARCH BY NEW TAG
Tagged Data List | AssertionError: expected 0 to deeply equal 1
Find tagged data in list | TypeError: Cannot read property 'path' of undefined GET DOWNLOAD NEW FROM GCS
Status code is 200 | AssertionError: expected response to have status code 200 but got 401
check crc32c code | AssertionError: expected undefined to deeply equal null
- [ ] GET SEARCH BY NEW TAG
Tagged Data List | AssertionError: expected 0 to deeply equal 1
Find tagged data in list | TypeError: Cannot read property 'path' of undefined PUT USER ADD
Status code is 200 | AssertionError: expected response to have status code 200 but got 400
- [ ] GET SEARCH BY NEW TAG
Tagged Data List | AssertionError: expected 0 to deeply equal 1
Find tagged data in list | TypeError: Cannot read property 'path' of undefined DELETE USER REMOVE
Status code is 200 | AssertionError: expected response to have status code 200 but got 400
- [ ] GET SEARCH BY NEW TAG
Tagged Data List | AssertionError: expected 0 to deeply equal 1
Find tagged data in list | TypeError: Cannot read property 'path' of undefined GET LIST TRUSTED
Validate fake email not in trusted app List | AssertionError: expected 0 to be above 0
[E2E_Test_Cases_Errors.txt](/uploads/04b40790521dc53160e2204616b2d7c9/E2E_Test_Cases_Errors.txt)Diego MolteniDiego Moltenihttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/issues/10E2E test issue2021-02-23T17:51:46ZRucha DeshpandeE2E test issueDATASET LIST AFTER DELETE
expects the list returned to be of length 6..
The collection posts only datasets and deletes one. In this case, the expected value must be 2.DATASET LIST AFTER DELETE
expects the list returned to be of length 6..
The collection posts only datasets and deletes one. In this case, the expected value must be 2.Rucha DeshpandeRucha Deshpandehttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/issues/11e2e tests: imptoken collection should be optional for CSPs that do not have i...2021-02-23T04:29:18ZRucha Deshpandee2e tests: imptoken collection should be optional for CSPs that do not have impersonation token implementedimptoken collection should be optional for CSPs that do not have impersonation token implementedimptoken collection should be optional for CSPs that do not have impersonation token implementedRucha DeshpandeDiego MolteniRucha Deshpandehttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/issues/12e2e test collection: clean up of collection using gcs urls2021-02-23T17:48:51ZRucha Deshpandee2e test collection: clean up of collection using gcs urlsThere are some requests in the collection that use 'googleapis' url to test.
These need to be cleaned up or changed to env vars.There are some requests in the collection that use 'googleapis' url to test.
These need to be cleaned up or changed to env vars.Rucha DeshpandeDiego MolteniRucha Deshpandehttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/issues/13Enable CI CD pipeline for IBM siesmic store svc2021-02-11T06:36:25ZAnuj GuptaEnable CI CD pipeline for IBM siesmic store svcEnable CI CD pipeline for IBMEnable CI CD pipeline for IBMAnuj GuptaAnuj Guptahttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/issues/14Validating REGION against hardcoded region names instead of dynamic provider-...2021-02-23T15:31:17ZBrady Spiva [AWS]Validating REGION against hardcoded region names instead of dynamic provider-specific regions## Observed behavior
When attempting to create subprojects, the `storage_location` parameter is restricted to a hardcoded list of values (GCP values?).
These values are hardcoded here, in `services/subproject/parser.ts`:
https://commun...## Observed behavior
When attempting to create subprojects, the `storage_location` parameter is restricted to a hardcoded list of values (GCP values?).
These values are hardcoded here, in `services/subproject/parser.ts`:
https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/blob/master/src/services/subproject/parser.ts#L25-31
When attempting to pass an AWS-specific region, this is the error code received:
> [400] [seismic-store-service] The storage_location body field US-EAST-1 is not valid. It must be one of ASIA, EU, US, NORTHAMERICA-NORTHEAST1, US-CENTRAL1, US-EAST1, US-EAST4, US-WEST1, SOUTHAMERICA-EAST1, EUROPE-WEST1, EUROPE-WEST2, EUROPE-WEST3, EUROPE-WEST4, ASIA-EAST1, ASIA-NORTHEAST1, ASIA-SOUTH1, ASIA-SOUTHEAST1, AUSTRALIA-SOUTHEAST1
## Expected behavior
Subproject creation should validate the user-provided region against cloud provider's regions dynamically.
## Potential solutions
This might already be addressed in @DiegoMolteni's upcoming updates to make the Seismic service more abstracted and generic. If not, can we work to abstract `parser.ts` to have cloud provider specific implementations?Rucha DeshpandeDiego MolteniBrady Spiva [AWS]Yunhua KoglinRucha Deshpandehttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/issues/15502 Gateway Error on calling Redis get and set functions2021-02-23T04:27:47ZWalter D502 Gateway Error on calling Redis get and set functionsHi @DiegoMolteni
We have been getting the 502 Gateway Error for some APIs. One of the APIs is Create Subproject API. On debugging the code the error is thrown on the following 2 lines in compliance.ts file in the create subproject flow:...Hi @DiegoMolteni
We have been getting the 502 Gateway Error for some APIs. One of the APIs is Create Subproject API. On debugging the code the error is thrown on the following 2 lines in compliance.ts file in the create subproject flow:
1. await this._cache.set(ltag, results.invalidLegalTags.length === 0);
2. await this._cache.set(ltag, results.invalidLegalTags.length === 0);
Interestingly, this is happening on the DEV environment. I've not encountered the issue in my local. Have you faced this error or have a clue on what the problem could be? Any help would be appreciated. Thank you.M1 - Release 0.1Diego MolteniDaniel PerezDiego Molteni2021-02-26https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/issues/16Standardizing the OAUTH2.0 JWT payload2021-03-04T22:50:05ZBrady Spiva [AWS]Standardizing the OAUTH2.0 JWT payload## Observed behavior
The service assumes an "email" attribute is present in the JWT payload, and attempts to get the "email" attribute. See here in [/src/shared/utils.ts](https://community.opengroup.org/osdu/platform/domain-data-mgmt-ser...## Observed behavior
The service assumes an "email" attribute is present in the JWT payload, and attempts to get the "email" attribute. See here in [/src/shared/utils.ts](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/blob/master/src/shared/utils.ts#L38). There is not a standardized naming convention for this attribute, it could be "email", or "username", etc.
When using the sdutil to upload data with the `--idtoken=` parameter, the service silently handles an error, and returns a cryptic response: `'created_by'`
## Expected behavior
The service should send a request to the `/userInfo/` OAUTH2.0 endpoint to determine what this custom email attribute is, and then get it from the JWT dynamically to avoid naming conflicts between cloud providers and identity provider services.
## Potential solutions
Here is an existing implementation of using this flow to first discover the custom email attribute, and then get it from the JWT payload: https://community.opengroup.org/osdu/platform/system/lib/cloud/aws/os-core-lib-aws/-/blob/master/src/main/java/org/opengroup/osdu/core/aws/entitlements/Authorizer.java#L121https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/issues/17Tenants - usage of Partition Service2023-03-27T19:30:13ZRucha DeshpandeTenants - usage of Partition ServiceSince we are using a new Tenant model in this service there are new tenant related APIs.
Can we not use the existing Partition Service APIs?Since we are using a new Tenant model in this service there are new tenant related APIs.
Can we not use the existing Partition Service APIs?Rucha DeshpandeDiego MolteniRucha Deshpandehttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/issues/18Dataset with seimsic metadata fails due to updates in R3 data definitions in ...2023-03-27T19:29:25ZRucha DeshpandeDataset with seimsic metadata fails due to updates in R3 data definitions in Storage ServicePosting a dataset with seismic metadata that is to be stored as a Storage record fails.
Seismic DMS service needs to be updated to work with R3 Data Definitions.
See issue:
https://community.opengroup.org/osdu/platform/system/storage/-/i...Posting a dataset with seismic metadata that is to be stored as a Storage record fails.
Seismic DMS service needs to be updated to work with R3 Data Definitions.
See issue:
https://community.opengroup.org/osdu/platform/system/storage/-/issues/44Rucha DeshpandeDiego MolteniRucha Deshpandehttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/issues/19e2e test: getGCSAccessToken API validates token type2021-02-23T04:26:36ZRucha Deshpandee2e test: getGCSAccessToken API validates token typeThe getGCSAccessToken API test script validates the return token type to be 'Bearer'.
For AWS the return type could be 'STS token'.
This validation should be removedThe getGCSAccessToken API test script validates the return token type to be 'Bearer'.
For AWS the return type could be 'STS token'.
This validation should be removedRucha DeshpandeDiego MolteniRucha Deshpandehttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/issues/20e2e test: Subproject Get New - checks return child length2021-02-23T04:24:39ZRucha Deshpandee2e test: Subproject Get New - checks return child lengthSubproject get new test script validates response child length. It is hardcoded to 8.
The length of the response can vary per CSP.
pm.expect(child.length).to.eql(8);Subproject get new test script validates response child length. It is hardcoded to 8.
The length of the response can vary per CSP.
pm.expect(child.length).to.eql(8);Rucha DeshpandeDiego MolteniRucha Deshpande