OSDU Software issueshttps://community.opengroup.org/groups/osdu/-/issues2024-02-09T14:30:12Zhttps://community.opengroup.org/osdu/platform/system/project-and-workflow/-/issues/24F1: Set up Docker Compose with main service and dependencies2024-02-09T14:30:12ZKat PisaniecF1: Set up Docker Compose with main service and dependencies1. **Description**: Create a docker-compose file that allows the developers to deploy and run the application in a local machine for development purposes.
2. **Acceptance criteria**: a docker compose file can be used to run a container w...1. **Description**: Create a docker-compose file that allows the developers to deploy and run the application in a local machine for development purposes.
2. **Acceptance criteria**: a docker compose file can be used to run a container with the application and reflect code changes in real time.
3. **Testing scenarios**:
- Run docker-compose up and the image of the application is deployed
- Change the code in the local machine, and see the effect in the running application
4. **Tech Notes**: N/ASantiago Ortiz [EPAM]santiago_ortiz@epam.comSantiago Ortiz [EPAM]santiago_ortiz@epam.comhttps://community.opengroup.org/osdu/platform/system/project-and-workflow/-/issues/23F1: Set up the service Docker image build2024-02-22T03:56:24ZKat PisaniecF1: Set up the service Docker image build1. **Description**: Create a Docker image of the application that can be deployed in a local container for testing and use in CI/CD pipelines.
2. **Acceptance criteria**: A Dockerfile that could be deployed in a local container and run t...1. **Description**: Create a Docker image of the application that can be deployed in a local container for testing and use in CI/CD pipelines.
2. **Acceptance criteria**: A Dockerfile that could be deployed in a local container and run the application successfully.
3. **Testing scenarios**:
- The Dockerfile can be passed to the docker build command to create an image of the application.
- The image of the application can be deployed in a container
4. **Tech Notes**:
The image must be built taking into account the following documentation
- [Blazing fast Python Docker builds with Poetry](https://medium.com/@albertazzir/blazing-fast-python-docker-builds-with-poetry-a78a66f5aed0)
- [Package a poetry project in a docker container for production](https://dev.to/farcellier/package-a-poetry-project-in-a-docker-container-for-production-3b4m)Santiago Ortiz [EPAM]santiago_ortiz@epam.comSantiago Ortiz [EPAM]santiago_ortiz@epam.comhttps://community.opengroup.org/osdu/platform/system/project-and-workflow/-/issues/22F1: Set up the application's local launch instructions (for developers)2024-02-09T14:28:17ZKat PisaniecF1: Set up the application's local launch instructions (for developers)1. **Description**: Include a README file with step-by-step instructions for setting up the environment and running the application on a local machine for development and debugging purposes.
2. **Acceptance criteria**: A README file cont...1. **Description**: Include a README file with step-by-step instructions for setting up the environment and running the application on a local machine for development and debugging purposes.
2. **Acceptance criteria**: A README file containing the following sections can be used to run the applications locally:
"installation", "runnint tests", "Run the application"
3. **Testing scenarios**:
- The application can be deployed locally
- Tests suites can be run locally
- The application can be run and debugged locally
4. **Tech Notes**: N/ASantiago Ortiz [EPAM]santiago_ortiz@epam.comSantiago Ortiz [EPAM]santiago_ortiz@epam.comhttps://community.opengroup.org/osdu/platform/system/reference/schema-upgrade/-/issues/22Service permission2024-02-15T14:34:37ZMark ChanceService permissionImplement a client of Policy Service which will enforce user permissions to run an upgrade.
Perhaps add a "test" option where the upgrade runs and does everything but commit the new resource?Implement a client of Policy Service which will enforce user permissions to run an upgrade.
Perhaps add a "test" option where the upgrade runs and does everything but commit the new resource?M23 - Release 0.26https://community.opengroup.org/osdu/platform/system/project-and-workflow/-/issues/21F1: Enable and configure code linters/checkers2024-02-09T14:27:28ZKat PisaniecF1: Enable and configure code linters/checkersDescription
---
We want to have linters, checkers and validations to enforce certain coding conventions into the repository.
Acceptance criteria
---
We include tools like `pre-commit`, code formatters, code linters.
Modify the reposi...Description
---
We want to have linters, checkers and validations to enforce certain coding conventions into the repository.
Acceptance criteria
---
We include tools like `pre-commit`, code formatters, code linters.
Modify the repository's documentation to have [this template structure](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/rock-and-fluid-sample/rafs-ddms-services/-/blob/main/README.md?ref_type=heads).
Include a part in the documentation that explains how we are going to be using `pre-commit` git hooks to enforce coding styles, and checks before making any commits into the codebase, and therefore minimizing code smells, code errors, bugs, code that doesn't follow established conventions.
Testing scenarios
---
Technical notes
---
- [Linting in VS Code](https://code.visualstudio.com/docs/python/linting)Carlos ColinCarlos Colinhttps://community.opengroup.org/osdu/platform/system/project-and-workflow/-/issues/20F1: Configure logging2024-02-09T14:28:54ZKat PisaniecF1: Configure loggingDescription
---
As developers we want to enable logging within the application so we can have a clear trace of every request and potential problems that arise when the application is subject to a load.
Acceptance criteria
---
We have ...Description
---
As developers we want to enable logging within the application so we can have a clear trace of every request and potential problems that arise when the application is subject to a load.
Acceptance criteria
---
We have enabled logging locally, and that logging can be extended to perform logging on any form or external service.
Testing scenarios
---
Tech Notes
---Carlos ColinCarlos Colinhttps://community.opengroup.org/osdu/platform/system/project-and-workflow/-/issues/19F1: Configure app settings2024-02-09T14:29:32ZKat PisaniecF1: Configure app settingsDescription
---
As developers we want to have modules that configure application's settings for all the environments that we will be running the app on.
Acceptance criteria
---
We have at least a base class of settings, which most lik...Description
---
As developers we want to have modules that configure application's settings for all the environments that we will be running the app on.
Acceptance criteria
---
We have at least a base class of settings, which most likely will be for production, and all the other settings will derive from it to modify it to suit the respective environments' needs.
Testing scenarios
---
Tech Notes
---Carlos ColinCarlos Colinhttps://community.opengroup.org/osdu/platform/system/project-and-workflow/-/issues/18F1: Set up the project structure and Python frameworks2024-02-09T14:26:11ZKat PisaniecF1: Set up the project structure and Python frameworksDescription
---
Set up the project trying to follow best practices.
Acceptance criteria
---
Have an initial repository ready for development and next stories.
Testing scenarios
---
Technical notes
---
We can follow some standards u...Description
---
Set up the project trying to follow best practices.
Acceptance criteria
---
Have an initial repository ready for development and next stories.
Testing scenarios
---
Technical notes
---
We can follow some standards used in [Rock & Fluid Samples repository](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/rock-and-fluid-sample/rafs-ddms-services/-/tree/main?ref_type=heads).Carlos ColinCarlos Colinhttps://community.opengroup.org/osdu/platform/system/reference/schema-upgrade/-/issues/20Log and error handling2024-03-19T15:15:08ZMark ChanceLog and error handlingM23 - Release 0.26Vikas Hoode [BP]vikas.hoode@bp.comVikas Hoode [BP]vikas.hoode@bp.comhttps://community.opengroup.org/osdu/platform/system/project-and-workflow/-/issues/17F1: Set up testing for CI/CD2024-02-09T14:26:28ZKat PisaniecF1: Set up testing for CI/CDDescription
---
As developers we want to be able to know whether the tests are passing or not when an automated pipeline runs as we push code to the feature branches.
Acceptance criteria
---
The stage for testing runs correctly with t...Description
---
As developers we want to be able to know whether the tests are passing or not when an automated pipeline runs as we push code to the feature branches.
Acceptance criteria
---
The stage for testing runs correctly with the required dependencies and fails when an error is thrown, otherwise, it will continue with the pipeline.
Testing scenarios
---
Technical notes
---Carlos ColinCarlos Colinhttps://community.opengroup.org/osdu/platform/system/project-and-workflow/-/issues/16F1: Set up CI/CD pipelines2024-02-09T14:28:35ZKat PisaniecF1: Set up CI/CD pipelineshttps://community.opengroup.org/osdu/platform/system/project-and-workflow/-/issues/15Alignment on Jupiter notebook between Mateusz and Ros, Carlos & Santiago2024-02-09T11:38:55ZKat PisaniecAlignment on Jupiter notebook between Mateusz and Ros, Carlos & Santiago1. **Description**: As a dev team we want to have shared understanding of the goals of MVP 1 as well as align on proposed form.
2. **Acceptance criteria**:
3. **Testing scenarios**: N/A
4. **Tech Notes**: Proposed solution is creation of...1. **Description**: As a dev team we want to have shared understanding of the goals of MVP 1 as well as align on proposed form.
2. **Acceptance criteria**:
3. **Testing scenarios**: N/A
4. **Tech Notes**: Proposed solution is creation of Jupyter notebook along with complimentary toolingMateusz RuszczykMateusz Ruszczykhttps://community.opengroup.org/osdu/platform/system/project-and-workflow/-/issues/14Overview session for the PWS team on how to connect to environment2024-02-09T11:38:31ZKat PisaniecOverview session for the PWS team on how to connect to environment1. **Description**:
2. **Acceptance criteria**:
3. **Testing scenarios**:
4. **Tech Notes**:1. **Description**:
2. **Acceptance criteria**:
3. **Testing scenarios**:
4. **Tech Notes**:harvey gillharvey gillhttps://community.opengroup.org/osdu/platform/system/project-and-workflow/-/issues/13AWS guide to using OSDU2024-02-09T11:39:39ZKat PisaniecAWS guide to using OSDU1. **Description**:
2. **Acceptance criteria**:
3. **Testing scenarios**:
4. **Tech Notes**:1. **Description**:
2. **Acceptance criteria**:
3. **Testing scenarios**:
4. **Tech Notes**:harvey gillharvey gillhttps://community.opengroup.org/osdu/platform/pre-shipping/-/issues/686AZURE M22 - Core Services - Unable to convert Unit without use of persistable...2024-02-15T13:55:29ZJuliana Fernandesjuliana.fernandes@iesbrazil.com.brAZURE M22 - Core Services - Unable to convert Unit without use of persistable referenceHello,
I`m trying to complete Core Services - FoR (Unit Conversion) without use of persistableReference but when I search the record the unit is not converted. So far I've:
- Collection: [Azure_OSDU_Smoke_Tests.postman_collection.json]...Hello,
I`m trying to complete Core Services - FoR (Unit Conversion) without use of persistableReference but when I search the record the unit is not converted. So far I've:
- Collection: [Azure_OSDU_Smoke_Tests.postman_collection.json](https://community.opengroup.org/osdu/platform/pre-shipping/-/blob/main/R3-M22/Azure-M22/Services/Smoke-tests/Azure_OSDU_Smoke_Tests.postman_collection.json?ref_type=heads)
- Environment: [Verify_Preshipping_New_TeamA-E.postman_environment.json](https://community.opengroup.org/osdu/platform/pre-shipping/-/blob/main/R3-M22/Azure-M22/Environment/Verify_Preshipping_New_TeamA-E.postman_environment.json?ref_type=heads)
## 1. Create a Record
PUT https://{{STORAGE_HOST}}/records/
**BODY CURL VERSION**
```
curl --location --request PUT 'https://osdu-ship.msft-osdu-test.org/api/storage/v2/records/' \
--header 'data-partition-id: opendes' \
--header 'Accept: application/json' \
--header 'Authorization: Bearer <Token>' \
--header 'Content-Type: application/json' \
--data-raw '[
{
"acl": {
"owners": [
"data.default.owners@opendes.contoso.com"
],
"viewers": [
"data.default.viewers@opendes.contoso.com"
]
},
"data": {
"DefaultVerticalMeasurementID": "Rotary Table",
"FacilityEvents": [
{
"EffectiveDateTime": "2002-05-12T00:00:00",
"FacilityEventTypeID": "opendes:reference-data--FacilityEventType:SPUD:"
},
{
"EffectiveDateTime": "2002-06-07T00:00:00",
"FacilityEventTypeID": "opendes:reference-data--FacilityEventType:DRILLING%20FINISH:"
}
],
"FacilityID": "A14-02-test",
"FacilityOperators": [
{
"FacilityOperatorOrganisationID": "opendes:master-data--Organisation:NAM:"
}
],
"FacilityStates": [
{
"FacilityStateTypeID": "opendes:reference-data--FacilityStateType:Abandoned:"
}
],
"FacilityTypeID": "opendes:reference-data--FacilityType:A14-02:",
"GeoContexts": [
{
"GeoPoliticalEntityID": "opendes:master-data--GeoPoliticalEntity:Netherlands:",
"GeoTypeID": "opendes:reference-data--GeoPoliticalEntityType:Country:"
},
{
"GeoPoliticalEntityID": "opendes:master-data--GeoPoliticalEntity:A14:",
"GeoTypeID": "opendes:reference-data--GeoPoliticalEntityType:BlockID:"
}
],
"NameAliases": [
{
"AliasName": "A14-02-test",
"AliasNameTypeID": "opendes:reference-data--AliasNameType:WELL_NAME:"
},
{
"AliasName": "8577-test",
"AliasNameTypeID": "opendes:reference-data--AliasNameType:UWI:"
}
],
"OperatingEnvironmentID": "opendes:reference-data--OperatingEnvironment:OFF:",
"Source": "TNO",
"SpatialLocation": {
"Wgs84Coordinates": {
"features": [
{
"geometry": {
"coordinates": [
3.39488769,
55.32443529
],
"type": "Point"
},
"properties": {},
"type": "Feature"
}
],
"type": "FeatureCollection"
}
},
"VerticalMeasurements": [
{
"VerticalCRSID": "opendes:reference-data--CoordinateReferenceSystem:MSL:",
"VerticalMeasurement": 39.2,
"VerticalMeasurementID": "Rotary Table",
"VerticalMeasurementPathID": "opendes:reference-data--VerticalMeasurementPath:Elevation:",
"VerticalMeasurementUnitOfMeasureID": "opendes:reference-data--UnitOfMeasure:M:"
}
]
},
"id": "opendes:master-data--Well:JFA29JANAZURE",
"kind": "osdu:wks:master-data--Well:1.0.0",
"legal": {
"legaltags": [
"opendes-Test-Legal-Tag-4375119"
],
"otherRelevantDataCountries": [
"US"
],
"status": "compliant"
},
"meta": [
{
"kind": "Unit",
"name": "ft",
"unitOfMeasureID": "opendes:reference-data--UnitOfMeasure:ft:",
"propertyNames": [
"VerticalMeasurements[].VerticalMeasurement"
]
}
],
"version": 0
}
]'
```
**BODY**
```json
[
{
"acl": {
"owners": [
"{{New_OwnerDataGroup}}@{{data-partition-id}}{{domain}}"
],
"viewers": [
"{{New_ViewerDataGroup}}@{{data-partition-id}}{{domain}}"
]
},
"data": {
"DefaultVerticalMeasurementID": "Rotary Table",
"FacilityEvents": [
{
"EffectiveDateTime": "2002-05-12T00:00:00",
"FacilityEventTypeID": "{{data-partition-id}}:reference-data--FacilityEventType:SPUD:"
},
{
"EffectiveDateTime": "2002-06-07T00:00:00",
"FacilityEventTypeID": "{{data-partition-id}}:reference-data--FacilityEventType:DRILLING%20FINISH:"
}
],
"FacilityID": "A14-02-test",
"FacilityOperators": [
{
"FacilityOperatorOrganisationID": "{{data-partition-id}}:master-data--Organisation:NAM:"
}
],
"FacilityStates": [
{
"FacilityStateTypeID": "{{data-partition-id}}:reference-data--FacilityStateType:Abandoned:"
}
],
"FacilityTypeID": "{{data-partition-id}}:reference-data--FacilityType:A14-02:",
"GeoContexts": [
{
"GeoPoliticalEntityID": "{{data-partition-id}}:master-data--GeoPoliticalEntity:Netherlands:",
"GeoTypeID": "{{data-partition-id}}:reference-data--GeoPoliticalEntityType:Country:"
},
{
"GeoPoliticalEntityID": "{{data-partition-id}}:master-data--GeoPoliticalEntity:A14:",
"GeoTypeID": "{{data-partition-id}}:reference-data--GeoPoliticalEntityType:BlockID:"
}
],
"NameAliases": [
{
"AliasName": "A14-02-test",
"AliasNameTypeID": "{{data-partition-id}}:reference-data--AliasNameType:WELL_NAME:"
},
{
"AliasName": "8577-test",
"AliasNameTypeID": "{{data-partition-id}}:reference-data--AliasNameType:UWI:"
}
],
"OperatingEnvironmentID": "{{data-partition-id}}:reference-data--OperatingEnvironment:OFF:",
"Source": "TNO",
"SpatialLocation": {
"Wgs84Coordinates": {
"features": [
{
"geometry": {
"coordinates": [
3.39488769,
55.32443529
],
"type": "Point"
},
"properties": {},
"type": "Feature"
}
],
"type": "FeatureCollection"
}
},
"VerticalMeasurements": [
{
"VerticalCRSID": "{{data-partition-id}}:reference-data--CoordinateReferenceSystem:MSL:",
"VerticalMeasurement": 39.2,
"VerticalMeasurementID": "Rotary Table",
"VerticalMeasurementPathID": "{{data-partition-id}}:reference-data--VerticalMeasurementPath:Elevation:",
"VerticalMeasurementUnitOfMeasureID": "{{data-partition-id}}:reference-data--UnitOfMeasure:M:"
}
]
},
"id": "{{data-partition-id}}:master-data--Well:JFA29JANAZURE",
"kind": "osdu:wks:master-data--Well:1.0.0",
"legal": {
"legaltags": [
"{{LegalTagNameExists}}"
],
"otherRelevantDataCountries": [
"US"
],
"status": "compliant"
},
"meta": [
{
"kind": "Unit",
"name": "ft",
"unitOfMeasureID": "{{data-partition-id}}:reference-data--UnitOfMeasure:ft:",
"propertyNames": [
"VerticalMeasurements[].VerticalMeasurement"
]
}
],
"version": 0
}
]
```
**RESULTS**
```json
{
"recordCount": 1,
"recordIds": [
"opendes:master-data--Well:JFA29JANAZURE"
],
"skippedRecordIds": [],
"recordIdVersions": [
"opendes:master-data--Well:JFA29JANAZURE:1706539464473285"
]
}
```
## 2. Search - Retrieve the record
POST https://{{SEARCH_HOST}}/query
**BODY CURL VERSION**
```
curl --location 'https://osdu-ship.msft-osdu-test.org/api/search/v2/query' \
--header 'Authorization: Bearer <Token>' \
--header 'data-partition-id: opendes' \
--header 'Content-Type: application/json' \
--data '{
"kind": "osdu:wks:master-data--Well:*.*.*",
"query": "id: \"opendes:master-data--Well:JFA29JANAZURE\" "
}
'
```
**BODY**
```json
{
"kind": "osdu:wks:master-data--Well:*.*.*",
"query": "id: \"opendes:master-data--Well:JFA29JANAZURE\" "
}
```
**RESULTS**
```json
{
"results": [
{
"data": {
"FacilityStates": [
{
"FacilityStateTypeID": "opendes:reference-data--FacilityStateType:Abandoned:"
}
],
"VirtualProperties.DefaultLocation.QuantitativeAccuracyBandID": null,
"ResourceLifecycleStatus": null,
"DefaultVerticalMeasurementID": "Rotary Table",
"SpatialLocation.Wgs84Coordinates": {
"geometries": [
{
"coordinates": [
3.39488769,
55.32443529
],
"type": "point"
}
],
"type": "geometrycollection"
},
"ResourceCurationStatus": null,
"TechnicalAssuranceID": null,
"VirtualProperties.DefaultLocation.SpatialGeometryTypeID": null,
"FacilityName": null,
"Source": "TNO",
"FacilityID": "A14-02-test",
"VirtualProperties.DefaultName": null,
"VerticalMeasurements": [
{
"WellboreTVDTrajectoryID": null,
"VerticalCRSID": "opendes:reference-data--CoordinateReferenceSystem:MSL:",
"VerticalReferenceID": null,
"VerticalMeasurementSourceID": null,
"VerticalMeasurementID": "Rotary Table",
"VerticalMeasurementPathID": "opendes:reference-data--VerticalMeasurementPath:Elevation:",
"VerticalMeasurement": 39.2,
"VerticalMeasurementTypeID": null,
"VerticalMeasurementDescription": null,
"VerticalMeasurementUnitOfMeasureID": "opendes:reference-data--UnitOfMeasure:M:",
"VerticalReferenceEntityID": null
}
],
"VirtualProperties.DefaultLocation.CoordinateQualityCheckPerformedBy": null,
"VersionCreationReason": null,
"ResourceSecurityClassification": null,
"InterestTypeID": null,
"DataSourceOrganisationID": null,
"SpatialLocation.SpatialParameterTypeID": null,
"ExistenceKind": null,
"SpatialLocation.CoordinateQualityCheckPerformedBy": null,
"FacilityOperators": [
{
"FacilityOperatorID": null,
"FacilityOperatorOrganisationID": "opendes:master-data--Organisation:NAM:"
}
],
"UWI-JFA": null,
"FacilityTypeID": "opendes:reference-data--FacilityType:A14-02:",
"NameAliases": [
{
"AliasName": "A14-02-test",
"AliasNameTypeID": "opendes:reference-data--AliasNameType:WELL_NAME:",
"DefinitionOrganisationID": null
},
{
"AliasName": "8577-test",
"AliasNameTypeID": "opendes:reference-data--AliasNameType:UWI:",
"DefinitionOrganisationID": null
}
],
"DefaultVerticalCRSID": null,
"FacilityEvents": [
{
"EffectiveDateTime": "2002-05-12T00:00:00+0000",
"FacilityEventTypeID": "opendes:reference-data--FacilityEventType:SPUD:"
},
{
"EffectiveDateTime": "2002-06-07T00:00:00+0000",
"FacilityEventTypeID": "opendes:reference-data--FacilityEventType:DRILLING%20FINISH:"
}
],
"VirtualProperties.DefaultLocation.IsDecimated": false,
"TechnicalAssuranceTypeID": null,
"GeoContexts": [
{
"BasinID": null,
"FieldID": null,
"PlayID": null,
"GeoPoliticalEntityID": "opendes:master-data--GeoPoliticalEntity:Netherlands:",
"GeoTypeID": "opendes:reference-data--GeoPoliticalEntityType:Country:",
"ProspectID": null
},
{
"BasinID": null,
"FieldID": null,
"PlayID": null,
"GeoPoliticalEntityID": "opendes:master-data--GeoPoliticalEntity:A14:",
"GeoTypeID": "opendes:reference-data--GeoPoliticalEntityType:BlockID:",
"ProspectID": null
}
],
"CurrentOperatorID": null,
"SpatialLocation.QualitativeSpatialAccuracyTypeID": null,
"SpatialLocation.SpatialGeometryTypeID": null,
"OperatingEnvironmentID": "opendes:reference-data--OperatingEnvironment:OFF:",
"VirtualProperties.DefaultLocation.SpatialParameterTypeID": null,
"ResourceHomeRegionID": null,
"VirtualProperties.DefaultLocation.QualitativeSpatialAccuracyTypeID": null,
"InitialOperatorID": null,
"VirtualProperties.DefaultLocation.Wgs84Coordinates": {
"geometries": [
{
"coordinates": [
3.39488769,
55.32443529
],
"type": "point"
}
],
"type": "geometrycollection"
},
"SpatialLocation.QuantitativeAccuracyBandID": null
},
"kind": "osdu:wks:master-data--Well:1.0.0",
"source": "wks",
"acl": {
"viewers": [
"data.default.viewers@opendes.contoso.com"
],
"owners": [
"data.default.owners@opendes.contoso.com"
]
},
"type": "master-data--Well",
"version": 1706539464473285,
"tags": {
"normalizedKind": "osdu:wks:master-data--Well:1"
},
"createTime": "2024-01-29T14:44:25.103Z",
"authority": "osdu",
"namespace": "osdu:wks",
"legal": {
"legaltags": [
"opendes-Test-Legal-Tag-4375119"
],
"otherRelevantDataCountries": [
"US"
],
"status": "compliant"
},
"createUser": "preshipping@azureglobal1.onmicrosoft.com",
"id": "opendes:master-data--Well:JFA29JANAZURE"
}
],
"aggregations": null,
"phraseSuggestions": [],
"totalCount": 1
}
```
The **"VerticalMeasurement": 39.2** is not converted. Can you help me with that?
Regards,
Juliana FernandesM22 - Release 0.25VidyaDharani LokamVidyaDharani Lokamhttps://community.opengroup.org/osdu/platform/pre-shipping/-/issues/684AZURE M22 - Core Services - Augmented Index2024-02-06T13:59:29ZJuliana Fernandesjuliana.fernandes@iesbrazil.com.brAZURE M22 - Core Services - Augmented IndexHello,
I'm trying to Reindex the Records but getting response 404 Not Found.
So far what I've node is:
- Collection: [Azure_OSDU_Smoke_Tests.postman_collection.json](https://community.opengroup.org/osdu/platform/pre-shipping/-/blob/ma...Hello,
I'm trying to Reindex the Records but getting response 404 Not Found.
So far what I've node is:
- Collection: [Azure_OSDU_Smoke_Tests.postman_collection.json](https://community.opengroup.org/osdu/platform/pre-shipping/-/blob/main/R3-M22/Azure-M22/Services/Smoke-tests/Azure_OSDU_Smoke_Tests.postman_collection.json?ref_type=heads)
- Environment: [Verify_Preshipping_New_TeamA-E.postman_environment.json
](https://community.opengroup.org/osdu/platform/pre-shipping/-/blob/main/R3-M22/Azure-M22/Environment/Verify_Preshipping_New_TeamA-E.postman_environment.json?ref_type=heads)
## 1. Create Recrod Using Storage Service
PUT https://{{STORAGE_HOST}}/records/
**BODY CURL VERSION**
```
curl --location --request PUT 'https://osdu-ship.msft-osdu-test.org/api/storage/v2/records/' \
--header 'data-partition-id: opendes' \
--header 'Accept: application/json' \
--header 'Authorization: Bearer <Token>' \
--header 'Content-Type: application/json' \
--data-raw '[
{
"acl": {
"owners": [
"data.default.owners@opendes.contoso.com"
],
"viewers": [
"data.default.viewers@opendes.contoso.com"
]
},
"data": {
"Source": "Juliana",
"PlayID": "01292024-JFA",
"PlayName": "JFA29JAN",
"PlayDescription": "Creating a record do Augmented Index",
"NameAliases": [
{
"AliasName": "SFA-ID01292024",
"AliasNameTypeID": "opendes:reference-data--AliasNameType:RegulatoryIdentifier:"
}
]
},
"id": "opendes:master-data--Play:Play-JFA29JAN",
"kind": "osdu:wks:master-data--Play:1.0.0",
"legal": {
"legaltags": [
"opendes-Test-Legal-Tag-4375119"
],
"otherRelevantDataCountries": [
"US"
],
"status": "compliant"
},
"meta": [],
"version": 0
}
]
'
```
**BODY**
```json
[
{
"acl": {
"owners": [
"{{New_OwnerDataGroup}}@{{data-partition-id}}{{domain}}"
],
"viewers": [
"{{New_ViewerDataGroup}}@{{data-partition-id}}{{domain}}"
]
},
"data": {
"Source": "Juliana",
"PlayID": "01292024-JFA",
"PlayName": "JFA29JAN",
"PlayDescription": "Creating a record do Augmented Index",
"NameAliases": [
{
"AliasName": "SFA-ID01292024",
"AliasNameTypeID": "{{data-partition-id}}:reference-data--AliasNameType:RegulatoryIdentifier:"
}
]
},
"id": "{{data-partition-id}}:master-data--Play:Play-JFA29JAN",
"kind": "osdu:wks:master-data--Play:1.0.0",
"legal": {
"legaltags": [
"{{LegalTagNameExists}}"
],
"otherRelevantDataCountries": [
"US"
],
"status": "compliant"
},
"meta": [],
"version": 0
}
]
```
**RESPONSE**
```json
{
"recordCount": 1,
"recordIds": [
"opendes:master-data--Play:Play-JFA29JAN"
],
"skippedRecordIds": [],
"recordIdVersions": [
"opendes:master-data--Play:Play-JFA29JAN:1706530251825746"
]
}
```
## 2. Retrieve Record using Storage
GET https://{{STORAGE_HOST}}/records/{{recordID}}
**BODY CURL VERSION**
```
curl --location 'https://osdu-ship.msft-osdu-test.org/api/storage/v2/records/opendes:master-data--Play:Play-JFA29JAN' \
--header 'data-partition-id: opendes' \
--header 'Authorization: Bearer <Token>'
```
**RESPONSE**
```json
{
"data": {
"Source": "Juliana",
"PlayID": "01292024-JFA",
"PlayName": "JFA29JAN",
"PlayDescription": "Creating a record do Augmented Index",
"NameAliases": [
{
"AliasName": "SFA-ID01292024",
"AliasNameTypeID": "opendes:reference-data--AliasNameType:RegulatoryIdentifier:"
}
]
},
"meta": [],
"id": "opendes:master-data--Play:Play-JFA29JAN",
"version": 1706530251825746,
"kind": "osdu:wks:master-data--Play:1.0.0",
"acl": {
"viewers": [
"data.default.viewers@opendes.contoso.com"
],
"owners": [
"data.default.owners@opendes.contoso.com"
]
},
"legal": {
"legaltags": [
"opendes-Test-Legal-Tag-4375119"
],
"otherRelevantDataCountries": [
"US"
],
"status": "compliant"
},
"createUser": "preshipping@azureglobal1.onmicrosoft.com",
"createTime": "2024-01-29T12:10:52.772Z"
}
```
## 3. Retrieve Record using Search
POST https://{{SEARCH_HOST}}/query
**BODY CURL VERSION**
```
curl --location 'https://osdu-ship.msft-osdu-test.org/api/search/v2/query' \
--header 'Authorization: Bearer <Token>' \
--header 'data-partition-id: opendes' \
--header 'Content-Type: application/json' \
--data '{
"kind": "osdu:wks:master-data--Play:1.0.0",
"query": "id: \"opendes:master-data--Play:Play-JFA29JAN\" "
}
'
```
**BODY**
```json
{
"kind": "osdu:wks:master-data--Play:1.0.0",
"query": "id: \"opendes:master-data--Play:Play-JFA29JAN\" "
}
```
**RESPONSE**
```json
{
"results": [
{
"data": {
"VirtualProperties.DefaultLocation.QuantitativeAccuracyBandID": null,
"ResourceLifecycleStatus": null,
"ResourceCurationStatus": null,
"TechnicalAssuranceID": null,
"PlayID": "01292024-JFA",
"VirtualProperties.DefaultLocation.SpatialGeometryTypeID": null,
"Source": "Juliana",
"VirtualProperties.DefaultName": "JFA29JAN",
"PlayDescription": "Creating a record do Augmented Index",
"VersionCreationReason": null,
"VirtualProperties.DefaultLocation.CoordinateQualityCheckPerformedBy": null,
"ResourceSecurityClassification": null,
"SpatialLocation.SpatialParameterTypeID": null,
"ExistenceKind": null,
"SpatialLocation.CoordinateQualityCheckPerformedBy": null,
"NameAliases": [
{
"AliasName": "SFA-ID01292024",
"AliasNameTypeID": "opendes:reference-data--AliasNameType:RegulatoryIdentifier:",
"DefinitionOrganisationID": null
}
],
"PlayTypeID": null,
"DCCustomRegulatoryID": "SFA-ID01292024",
"TechnicalAssuranceTypeID": null,
"PlayName": "JFA29JAN",
"SpatialLocation.QualitativeSpatialAccuracyTypeID": null,
"SpatialLocation.SpatialGeometryTypeID": null,
"VirtualProperties.DefaultLocation.SpatialParameterTypeID": null,
"ResourceHomeRegionID": null,
"VirtualProperties.DefaultLocation.QualitativeSpatialAccuracyTypeID": null,
"SpatialLocation.QuantitativeAccuracyBandID": null
},
"kind": "osdu:wks:master-data--Play:1.0.0",
"source": "wks",
"acl": {
"viewers": [
"data.default.viewers@opendes.contoso.com"
],
"owners": [
"data.default.owners@opendes.contoso.com"
]
},
"type": "master-data--Play",
"version": 1706530251825746,
"tags": {
"normalizedKind": "osdu:wks:master-data--Play:1"
},
"createTime": "2024-01-29T12:10:52.772Z",
"authority": "osdu",
"namespace": "osdu:wks",
"legal": {
"legaltags": [
"opendes-Test-Legal-Tag-4375119"
],
"otherRelevantDataCountries": [
"US"
],
"status": "compliant"
},
"createUser": "preshipping@azureglobal1.onmicrosoft.com",
"id": "opendes:master-data--Play:Play-JFA29JAN"
}
],
"aggregations": null,
"phraseSuggestions": [],
"totalCount": 1
}
```
## 4. Create Reference data for Augmented Index
PUT https://{{STORAGE_HOST}}/records/
**BODY CURL VERSION**
```
curl --location --request PUT 'https://osdu-ship.msft-osdu-test.org/api/storage/v2/records/' \
--header 'data-partition-id: opendes' \
--header 'Accept: application/json' \
--header 'Authorization: Bearer <Token>' \
--header 'Content-Type: application/json' \
--data-raw '[
{
"acl": {
"owners": [
"data.default.owners@opendes.contoso.com"
],
"viewers": [
"data.default.viewers@opendes.contoso.com"
]
},
"legal": {
"legaltags": [
"opendes-Test-Legal-Tag-4375119"
],
"otherRelevantDataCountries": [
"US"
],
"status": "compliant"
},
"meta": [],
"data": {
"Name": "Play-IndexPropertyPathConfiguration",
"Description": "valid for all master-data--Play kinds for major version 1.",
"Code": "osdu:wks:master-data--Play:1.",
"AttributionAuthority": "OSDU",
"Configurations": [
{
"Name": "DCCustomRegulatoryID",
"Policy": "ExtractFirstMatch",
"Paths": [
{
"ValueExtraction": {
"RelatedConditionMatches": [
"opendes:reference-data--AliasNameType:RegulatoryIdentifier:"
],
"RelatedConditionProperty": "data.NameAliases[].AliasNameTypeID",
"ValuePath": "data.NameAliases[].AliasName"
}
}
],
"UseCase": "As a user I want to discover and match Play records by their custom field value."
}
]
},
"id": "opendes:reference-data--IndexPropertyPathConfiguration:wks:master-data--Play:1.",
"kind": "osdu:wks:reference-data--IndexPropertyPathConfiguration:1.0.0",
"version": 0
}
]'
```
**BODY**
```json
[
{
"acl": {
"owners": [
"{{New_OwnerDataGroup}}@{{data-partition-id}}{{domain}}"
],
"viewers": [
"{{New_ViewerDataGroup}}@{{data-partition-id}}{{domain}}"
]
},
"legal": {
"legaltags": [
"{{LegalTagNameExists}}"
],
"otherRelevantDataCountries": [
"US"
],
"status": "compliant"
},
"meta": [],
"data": {
"Name": "Play-IndexPropertyPathConfiguration",
"Description": "valid for all master-data--Play kinds for major version 1.",
"Code": "osdu:wks:master-data--Play:1.",
"AttributionAuthority": "OSDU",
"Configurations": [
{
"Name": "DCCustomRegulatoryID",
"Policy": "ExtractFirstMatch",
"Paths": [
{
"ValueExtraction": {
"RelatedConditionMatches": [
"{{data-partition-id}}:reference-data--AliasNameType:RegulatoryIdentifier:"
],
"RelatedConditionProperty": "data.NameAliases[].AliasNameTypeID",
"ValuePath": "data.NameAliases[].AliasName"
}
}
],
"UseCase": "As a user I want to discover and match Play records by their custom field value."
}
]
},
"id": "{{data-partition-id}}:reference-data--IndexPropertyPathConfiguration:wks:master-data--Play:1.",
"kind": "osdu:wks:reference-data--IndexPropertyPathConfiguration:1.0.0",
"version": 0
}
]
```
**RESPONSE**
```json
{
"recordCount": 1,
"recordIds": [
"opendes:reference-data--IndexPropertyPathConfiguration:wks:master-data--Play:1."
],
"skippedRecordIds": [],
"recordIdVersions": [
"opendes:reference-data--IndexPropertyPathConfiguration:wks:master-data--Play:1.:1706530435891902"
]
}
```
## 5. Reindex Records
POST https://{{INDEXER_HOST}}/reindex?force_clean=true
**BODY CURL VERSION**
```
curl --location 'https://osdu-ship.msft-osdu-test.org/api/indexer/v2/ /reindex?force_clean=true' \
--header 'Content-Type: application/json' \
--header 'data-partition-id: opendes' \
--header 'Authorization: Bearer <Token>' \
--data '{
"kind": "osdu:wks:master-data--Play:1.0.0"
}'
```
**BODY**
```json
{
"kind": "osdu:wks:master-data--Play:1.0.0"
}
```
**RESPONSE**
```json
{
"timestamp": "2024-01-29T12:17:35.896+00:00",
"status": 404,
"error": "Not Found",
"path": "/api/indexer/v2/%20/reindex"
}
```
Regards,M22 - Release 0.25https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/226Building from source2024-01-29T08:38:09ZVasilii SinkevichBuilding from sourceHi,
I am trying to build the static library from the source code and as a first step I did compile dynamically as instructed and noticed that Wavelet compression is not supported in the resulting library - just checked with OpenVDS::Is...Hi,
I am trying to build the static library from the source code and as a first step I did compile dynamically as instructed and noticed that Wavelet compression is not supported in the resulting library - just checked with OpenVDS::IsCompressionMethodSupported(OpenVDS::CompressionMethod::Wavelet).
Is it how it is supposed to be or I did something wrong?
Does it mean that both compression/decompression is not supported or decompression will still be working?
Thanks you,
Vasiliihttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/225DmsDataset class is using resources that have been moved from2024-01-29T13:47:01ZDeepa KumariDmsDataset class is using resources that have been moved fromThe vector containing response is moved here: https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/blob/master/src/OpenVDS/IO/DmsIoFactories/DmsIoManagerFactory.cpp#L200
https://community.opengroup....The vector containing response is moved here: https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/blob/master/src/OpenVDS/IO/DmsIoFactories/DmsIoManagerFactory.cpp#L200
https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/blob/master/src/OpenVDS/IO/DmsIoFactories/DmsIoManagerFactory.cpp#L223
The same resource which was already moved from, is being used here:
https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/blob/master/src/OpenVDS/IO/DmsIoFactories/DmsIoManagerFactory.cpp#L204
and https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/blob/master/src/OpenVDS/IO/DmsIoFactories/DmsIoManagerFactory.cpp#L227
Resources that have been moved from should not be used, that is the objective of move. More ref: https://en.cppreference.com/w/cpp/utility/moveM23 - Release 0.26Deepa KumariDeepa Kumarihttps://community.opengroup.org/osdu/platform/security-and-compliance/legal/-/issues/70Compliance API loops through all tenants but fetches tenant countries for the...2024-01-30T14:21:36ZRucha DeshpandeCompliance API loops through all tenants but fetches tenant countries for the tenant in the RequestInfoContext:
The compliance trigger calls the BASE_URL/api/legal/v1/jobs/updateLegalTagStatus API.
As is the norm with OSDU APIs, the API needs to be called with a certain data-partition-id. Since the compliance API loops through all tenant...Context:
The compliance trigger calls the BASE_URL/api/legal/v1/jobs/updateLegalTagStatus API.
As is the norm with OSDU APIs, the API needs to be called with a certain data-partition-id. Since the compliance API loops through all tenants (see [here](https://community.opengroup.org/osdu/platform/security-and-compliance/legal/-/blob/master/legal-core/src/main/java/org/opengroup/osdu/legal/controller/LegalTagStatusJobController.java?ref_type=heads)), this API is called with any random data-partition-id.
Problem:
The [LegalTagCountriesService](https://community.opengroup.org/osdu/platform/security-and-compliance/legal/-/blob/master/legal-core/src/main/java/org/opengroup/osdu/legal/countries/LegalTagCountriesService.java?ref_type=heads) has a @PostConstruct setup step where it reads all the tenant countries from provider specific storage but always uses the tenant from requestInfo object (the tenant with which the compliance API is called) which remains constant. This results in an inconsistency which may result in fetching of the wrong Legal_COO.json
ultimately resulting in this error:
`legal.app: Changing state of osdu-brazil-tag1 from Valid to Invalid. Errors found in legaltag Invalid country of origin set. It should match one of the ISO alpha 2 codes and be a country with no restriction on data residency. Found: BR. {correlation-id=8fbdd6f4-3bd8-4e7a-9c0e-655033806699, data-partition-id=int-test-csv-parser}`M22 - Release 0.25https://community.opengroup.org/osdu/platform/security-and-compliance/policy/-/issues/126Update Policy Service API version2024-01-25T16:17:12ZShane HutchinsUpdate Policy Service API versionMake Policy Service API version to be 1.0.0 and not include any other identifiers (including v, milestone, build information, etc).
The Build information is still available in description and info API.
Requested by @chadMake Policy Service API version to be 1.0.0 and not include any other identifiers (including v, milestone, build information, etc).
The Build information is still available in description and info API.
Requested by @chadM23 - Release 0.26Shane HutchinsShane Hutchins