seismic-dms-service issueshttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/issues2024-02-29T12:14:56Zhttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/issues/129DATASET SELECT LS POST: while putting invalid characters in select it is givi...2024-02-29T12:14:56ZIsha KumariDATASET SELECT LS POST: while putting invalid characters in select it is giving response code 200. it should give 400 DATASET SELECT LS POST: while putting invalid characters in selectit is giving response code 200. it should give 400 DATASET SELECT LS POST: while putting invalid characters in selectit is giving response code 200. it should give 400https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/issues/109Unsupported Feature in Dataset LS Get Endpoint Causing Test Failures on AWS a...2023-09-06T20:02:06ZPratiksha ShedgeUnsupported Feature in Dataset LS Get Endpoint Causing Test Failures on AWS and AnthosA new feature has been introduced for the dataset LS get endpoint, comprising the Search (to select a single SQL-like search parameter) and Select (to choose multiple fields for retrieval) query parameters. The API is expected to return ...A new feature has been introduced for the dataset LS get endpoint, comprising the Search (to select a single SQL-like search parameter) and Select (to choose multiple fields for retrieval) query parameters. The API is expected to return a list of datasets based on the search and select query parameters. However, AWS and Anthos do not support this new feature for this endpoint, leading to test failures during pipeline runs.
Pipeline runs:
AWS: https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/jobs/2200880
Anthos: https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/jobs/2200882https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/issues/99Include aws region in dataset information for AWS Seismic DDMS data2024-02-26T21:52:49ZMichaelInclude aws region in dataset information for AWS Seismic DDMS dataWhen using sdapi to retreive seismic ddms data coming from AWS, a user needs to first set the AWS_REGION environment variable (see ticket https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/s...When using sdapi to retreive seismic ddms data coming from AWS, a user needs to first set the AWS_REGION environment variable (see ticket https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-cpp-lib/-/issues/21).
To better handle this use case, the get dataset service `/dataset/tenant/{tenantid}/subproject/{subproject}/dataset/{datasetid}` should provide information regarding the aws region if the dataset is stored in s3 storage.https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/issues/98Pagination not supported by IBM and AWS for DATASET LIST (POST) endpoint2023-04-05T14:28:58ZPratiksha ShedgePagination not supported by IBM and AWS for DATASET LIST (POST) endpointA new API has been added as DATASET LIST (POST) endpoint which supports pagination. This API should return the list of datasets and nextPageCursor to get the next list of datasets. However, IBM and AWS do not support pagination for this ...A new API has been added as DATASET LIST (POST) endpoint which supports pagination. This API should return the list of datasets and nextPageCursor to get the next list of datasets. However, IBM and AWS do not support pagination for this endpoint, which causes the pagination tests to fail during pipeline runs.
Pipeline runs:
IBM-https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/jobs/1823012
AWS-https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/jobs/1842803https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/issues/89Implement dataset storage for AWS2023-09-27T13:19:38ZSacha BrantsImplement dataset storage for AWShttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/issues/68Utility LS endpoint doesn't work for directories2023-03-24T19:11:22ZKonstantin KhottchenkovUtility LS endpoint doesn't work for directoriesNew test scenario was added for UTILITY LS endpoint. The feature of filtering the output for only datasets, only folders or both datasets and folders was added and tested.
The result of tests shows that use of "wmode" parameter with valu...New test scenario was added for UTILITY LS endpoint. The feature of filtering the output for only datasets, only folders or both datasets and folders was added and tested.
The result of tests shows that use of "wmode" parameter with values "dirs" and "all" that filter response to receive only names of directories or both datasets and directories correspondingly fails for AWS and ANTHOS. We couldn't check if Google also affected because Google environment is broken at all. Thus this tests were disabled for mentioned CSPs.
[Pipeline run](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/jobs/1458328)https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/issues/59What are some of the variables in tests/e2e/postman_env.json and where does o...2022-06-16T09:21:25ZKamlesh TodaiWhat are some of the variables in tests/e2e/postman_env.json and where does one find the values for them?Looking at the following file
https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/blob/master/app/sdms/tests/e2e/postman_env.json
I am not sure what these variables a...Looking at the following file
https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/blob/master/app/sdms/tests/e2e/postman_env.json
I am not sure what these variables are and where does one get the values for them
the list of the variables that need clarification and also where to get to get the values for them?
**SVC_API_KEY**
**STOKEN** is it id_token, access_token or refresh_token?
**DE_APP_KEY**
**VCS_Provider**M12 - Release 0.15https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/issues/58For Tenant there is no endpoint that can be used to list all the available te...2023-03-24T19:22:43ZKamlesh TodaiFor Tenant there is no endpoint that can be used to list all the available tenantsThere should be a way to list all the tenants to which the user has access. At present, there is no way to do that. If one had created the tenant in the past and cannot remember the name, then there is no way to find that name.There should be a way to list all the tenants to which the user has access. At present, there is no way to do that. If one had created the tenant in the past and cannot remember the name, then there is no way to find that name.https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/issues/51duplicate aws yml into seismic service gitlab cicd and cause staled info2023-03-24T19:33:50ZYunhua Koglinduplicate aws yml into seismic service gitlab cicd and cause staled infoCurrently seismic service gitlab cicd duplicates code from https://community.opengroup.org/osdu/platform/ci-cd-pipelines/-/blob/master/cloud-providers/aws-global.yml and when we deploy in different stack, this staled info will fail.Currently seismic service gitlab cicd duplicates code from https://community.opengroup.org/osdu/platform/ci-cd-pipelines/-/blob/master/cloud-providers/aws-global.yml and when we deploy in different stack, this staled info will fail.Daniel PerezDaniel Perezhttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/issues/33Inconsistent behavior across CSPs - while Registering/creating seismic-dms te...2023-03-27T19:24:28ZKamlesh TodaiInconsistent behavior across CSPs - while Registering/creating seismic-dms tenantPOST .../tenant​/{tenantid} Register a seismic-dms tenant. The API document says Required roles: seistore.system.admin
Is this role granted to the average user/tester or does it require a user with elevated permissions?
What I ...POST .../tenant​/{tenantid} Register a seismic-dms tenant. The API document says Required roles: seistore.system.admin
Is this role granted to the average user/tester or does it require a user with elevated permissions?
What I am noticing is that in
**For AWS**,
I am able to register the new seismic-dms tenant, where else
**For IBM and Azure**,
I am not able to do that. I get the message - [seismic-store-service] The tenant name must match the data partition ID 'opendes'
**For GCP** -
**I have to pass the id_token instead of access_token** and when I do that I get the message - [entitlement-service] 401 - "{\"code\":401,\"reason\":\"Obtaining user info issue\",\"message\":\"Not valid token provided\"}" - appears that when I pass the id_token, the entitlement API is running into problem and if I pass the access_token then I get Unexpected token k in JSON at position 0
@ChrisZhang @manishk @anujgupta @Kateryna_Kurach @Wibben @sachahttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/issues/27AWS seismic store service ci cd testing yaml needs to be in ci cd repo2023-03-27T19:26:25ZDaniel PerezAWS seismic store service ci cd testing yaml needs to be in ci cd repoI have noticed that gitlab yaml for AWS testing has been included in seismic store service (https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/tree/master/devops/aws...I have noticed that gitlab yaml for AWS testing has been included in seismic store service (https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/tree/master/devops/aws), this file needs to be in CI CD repo https://community.opengroup.org/osdu/platform/ci-cd-pipelines/-/tree/master/
Please also follow standard and integrate in same yaml inside of cloud providers as we do for other providers https://community.opengroup.org/osdu/platform/ci-cd-pipelines/-/tree/master/cloud-providersRucha DeshpandeRucha Deshpandehttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/issues/25Code linting AWS2021-03-23T16:43:04ZDiego MolteniCode linting AWScode linting to apply on aws code.
``$ tslint -c tslint.json 'src/cloud/providers/aws/**/*.ts'``code linting to apply on aws code.
``$ tslint -c tslint.json 'src/cloud/providers/aws/**/*.ts'``Rucha DeshpandeYunhua KoglinRucha Deshpandehttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/issues/23createQuery and createKey - generalize structure2023-03-27T19:27:48ZRucha DeshpandecreateQuery and createKey - generalize structureThe following 2 methods
createQuery(namespace: string, kind: string): IJournalQueryModel;
createKey(specs: any): object;
The structure of the parameter should be abstracted to be s
AWS wants to be able to pass information such as
{...The following 2 methods
createQuery(namespace: string, kind: string): IJournalQueryModel;
createKey(specs: any): object;
The structure of the parameter should be abstracted to be s
AWS wants to be able to pass information such as
{
table_name:
tenant_name
subproject_name
..etc
}
of type 'any'.
This is required for AWS,as we are restricted to parse and use the 'Namespace', 'kind' which does not work in all scenarios for the models we have.Rucha DeshpandeDiego MolteniRucha Deshpandehttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/issues/22e2e test script needs to run from repository root only2023-03-27T19:28:43ZRucha Deshpandee2e test script needs to run from repository root onlyThe run-e2e-tests.sh script has the following check. This will not work in internal pipelines where the distribution folder structure is different.
if [ ! -f "tsconfig.json" ]; then
printf "\n%s\n" "[ERROR] The script must be cal...The run-e2e-tests.sh script has the following check. This will not work in internal pipelines where the distribution folder structure is different.
if [ ! -f "tsconfig.json" ]; then
printf "\n%s\n" "[ERROR] The script must be called from the project root directory."
exit 1
fiRucha DeshpandeDiego MolteniRucha Deshpandehttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/issues/21e2e test - Dataset exist / Dataset sizes validates a hardcoded list2021-03-04T22:50:06ZRucha Deshpandee2e test - Dataset exist / Dataset sizes validates a hardcoded listThis test validates a hardcoded list. Must change to environment variables OR the test script must be updated.
{
"datasets":[
"/async/dsx01",
"/a/b/c/dsx02",
"async/dsx03",
"test/dsx01",
"{{path01}}/{{dataset01}}"
]
}
Simil...This test validates a hardcoded list. Must change to environment variables OR the test script must be updated.
{
"datasets":[
"/async/dsx01",
"/a/b/c/dsx02",
"async/dsx03",
"test/dsx01",
"{{path01}}/{{dataset01}}"
]
}
Similarly, the Dataset Sizes test also validates the sizes for a hardcoded listRucha DeshpandeDiego MolteniRucha Deshpandehttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/issues/20e2e test: Subproject Get New - checks return child length2021-02-23T04:24:39ZRucha Deshpandee2e test: Subproject Get New - checks return child lengthSubproject get new test script validates response child length. It is hardcoded to 8.
The length of the response can vary per CSP.
pm.expect(child.length).to.eql(8);Subproject get new test script validates response child length. It is hardcoded to 8.
The length of the response can vary per CSP.
pm.expect(child.length).to.eql(8);Rucha DeshpandeDiego MolteniRucha Deshpandehttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/issues/19e2e test: getGCSAccessToken API validates token type2021-02-23T04:26:36ZRucha Deshpandee2e test: getGCSAccessToken API validates token typeThe getGCSAccessToken API test script validates the return token type to be 'Bearer'.
For AWS the return type could be 'STS token'.
This validation should be removedThe getGCSAccessToken API test script validates the return token type to be 'Bearer'.
For AWS the return type could be 'STS token'.
This validation should be removedRucha DeshpandeDiego MolteniRucha Deshpandehttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/issues/18Dataset with seimsic metadata fails due to updates in R3 data definitions in ...2023-03-27T19:29:25ZRucha DeshpandeDataset with seimsic metadata fails due to updates in R3 data definitions in Storage ServicePosting a dataset with seismic metadata that is to be stored as a Storage record fails.
Seismic DMS service needs to be updated to work with R3 Data Definitions.
See issue:
https://community.opengroup.org/osdu/platform/system/storage/-/i...Posting a dataset with seismic metadata that is to be stored as a Storage record fails.
Seismic DMS service needs to be updated to work with R3 Data Definitions.
See issue:
https://community.opengroup.org/osdu/platform/system/storage/-/issues/44Rucha DeshpandeDiego MolteniRucha Deshpandehttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/issues/17Tenants - usage of Partition Service2023-03-27T19:30:13ZRucha DeshpandeTenants - usage of Partition ServiceSince we are using a new Tenant model in this service there are new tenant related APIs.
Can we not use the existing Partition Service APIs?Since we are using a new Tenant model in this service there are new tenant related APIs.
Can we not use the existing Partition Service APIs?Rucha DeshpandeDiego MolteniRucha Deshpandehttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/issues/16Standardizing the OAUTH2.0 JWT payload2021-03-04T22:50:05ZBrady Spiva [AWS]Standardizing the OAUTH2.0 JWT payload## Observed behavior
The service assumes an "email" attribute is present in the JWT payload, and attempts to get the "email" attribute. See here in [/src/shared/utils.ts](https://community.opengroup.org/osdu/platform/domain-data-mgmt-ser...## Observed behavior
The service assumes an "email" attribute is present in the JWT payload, and attempts to get the "email" attribute. See here in [/src/shared/utils.ts](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/blob/master/src/shared/utils.ts#L38). There is not a standardized naming convention for this attribute, it could be "email", or "username", etc.
When using the sdutil to upload data with the `--idtoken=` parameter, the service silently handles an error, and returns a cryptic response: `'created_by'`
## Expected behavior
The service should send a request to the `/userInfo/` OAUTH2.0 endpoint to determine what this custom email attribute is, and then get it from the JWT dynamically to avoid naming conflicts between cloud providers and identity provider services.
## Potential solutions
Here is an existing implementation of using this flow to first discover the custom email attribute, and then get it from the JWT payload: https://community.opengroup.org/osdu/platform/system/lib/cloud/aws/os-core-lib-aws/-/blob/master/src/main/java/org/opengroup/osdu/core/aws/entitlements/Authorizer.java#L121