OSDU Software issueshttps://community.opengroup.org/groups/osdu/-/issues2024-01-01T08:47:32Zhttps://community.opengroup.org/osdu/platform/system/storage/-/issues/181GET: /records/{recordID}/{version} - ERROR 5002024-01-01T08:47:32ZSiarhei Khaletski (EPAM)GET: /records/{recordID}/{version} - ERROR 500**Context**
GET: /records/{recordID}/{version} fails with error 500 if an invalid version is provided (see the attachment)
We noticed an odd behavior of the service:
List of existing versions of the following record: `opendes:work-pro...**Context**
GET: /records/{recordID}/{version} fails with error 500 if an invalid version is provided (see the attachment)
We noticed an odd behavior of the service:
List of existing versions of the following record: `opendes:work-product-component--SamplesAnalysis:e9f02f48f43149a8b69606ff7597f391`
![image](/uploads/3d75fd80a57f5558c7d0eb00a4d795eb/image.png)
If request unexisting version `1` - status error 500
![image](/uploads/d3dc228f70263bd24ff7d09975baa63c/image.png)
Meanwhile, if request unexisting version `1234` - status 404
![image](/uploads/e82da89c3673b643aaa26845f0eb0c81/image.png)
**Azure GLab Logs**
![image](/uploads/8d54b1addcbc1835b4ea3c90135072b6/image.png)
**Expected Behavior**
404 - status codeM22 - Release 0.25Siarhei Khaletski (EPAM)Chad LeongSiarhei Khaletski (EPAM)https://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/issues/156listallworkflow API does not return Version of services/DAG deployed.2023-10-23T08:06:15Zvinisha krishnalistallworkflow API does not return Version of services/DAG deployed.The following API returns Version 1 which seems to be hardcoded. We need to get the version of services/DAGs deployed. With ADME we do not have visibility of which version is needed.
{base_url}/solutions/data-flow/apis/workflow-service...The following API returns Version 1 which seems to be hardcoded. We need to get the version of services/DAGs deployed. With ADME we do not have visibility of which version is needed.
{base_url}/solutions/data-flow/apis/workflow-service#/Workflow/listAllWorkflow
![image](/uploads/c99bde240f7cd2276d899e3f05f7cd73/image.png)https://community.opengroup.org/osdu/platform/pre-shipping/-/issues/589Azure M19 - Audit and Metric (Platform KPI) - results in restricted site2023-08-16T08:40:30ZDebasis ChatterjeeAzure M19 - Audit and Metric (Platform KPI) - results in restricted siteWhen testing this Platform KPI, I am shown a site where I do not have privilege. So, I cannot check what results are generated by this API.
In addition, there is a hint of using a vendor proprietary system.
POST https://osdu-ship.msft-...When testing this Platform KPI, I am shown a site where I do not have privilege. So, I cannot check what results are generated by this API.
In addition, there is a hint of using a vendor proprietary system.
POST https://osdu-ship.msft-osdu-test.org/api/aam/azure/subscriptions/{{subscription_id}}/{{resource_group_id}}/{{vm_name}}
Body
```
{
"metric_id":"Percentage CPU",
"timespan":"2023-08-03T06:25:00Z/2023-08-07T18:41:00Z",
"aggregation":"Total",
"interval":"PT1H"
}
```
Response
```
{
"cost": 6495,
"interval": "PT1H",
"namespace": "Microsoft.Compute/virtualMachines",
"resourceregion": "eastus",
"timespan": "2023-08-03T06:25:00Z/2023-08-07T18:41:00Z",
"value": [
{
"displayDescription": "The percentage of allocated compute units that are currently in use by the Virtual Machine(s)",
"errorCode": "Success",
"id": "/subscriptions/7c052588-ead2-45c9-9346-5b156a157bd1/resourceGroups/osdu-gcz-naresh-rg/providers/Microsoft.Compute/virtualMachines/osdu-gcz-dev-vm/providers/Microsoft.Insights/metrics/Percentage CPU",
"name": {
"localizedValue": "Percentage CPU",
"value": "Percentage CPU"
},
"timeseries": [],
"type": "Microsoft.Insights/metrics",
"unit": "Percent"
}
]
}
```Mohd Asad ShaikhMohd Asad Shaikhhttps://community.opengroup.org/osdu/platform/pre-shipping/-/issues/583M19: GC EDS Ingestion Error, some of the expected Reference Data is missing, ...2024-01-08T16:18:59ZAnkit GoyalM19: GC EDS Ingestion Error, some of the expected Reference Data is missing, while testing EDS and adding Connected Source Registry Entry.DAG: Osdu_ingest R3 manifest processing with providing integrity
XCom
KeyValuesaved_record_ids{}skipped_ids{'provide_manifest_integrity_task': [{'id': 'm19:master-data--ConnectedSourceRegistryEntry:AWSPreship-AGAUG102023', 'kind': 'osdu...DAG: Osdu_ingest R3 manifest processing with providing integrity
XCom
KeyValuesaved_record_ids{}skipped_ids{'provide_manifest_integrity_task': [{'id': 'm19:master-data--ConnectedSourceRegistryEntry:AWSPreship-AGAUG102023', 'kind': 'osdu:wks:master-data--ConnectedSourceRegistryEntry:1.2.0', 'reason': 'Missing parents: {SRN: m19:master-data--Organisation:AWS-PRESHIP-AGAUG102023, SRN: m19:master-data--ActivityTemplate:CSRE-ActivityTemplate-AGAUG102023, SRN: m19:reference-data--OAuth2FlowType:RefreshTokenKeyName}'}]}
@chad @debasisc @todaiksDzmitry Malkevich (EPAM)Jeyakumar DevarajuluDzmitry Malkevich (EPAM)https://community.opengroup.org/osdu/platform/system/storage/-/issues/180Unable to nullify a non-system attribute from DateTime value to null or empty...2023-08-22T10:20:49ZShubhankar SrivastavaUnable to nullify a non-system attribute from DateTime value to null or empty value using Storage serviceTo support a business use case, a user would need to update an existing attribute (with data type as date time) residing under data { } section of a **work-product-component** schema from a valid DateTime value (e.g.- 2023-08-10T00:00:00...To support a business use case, a user would need to update an existing attribute (with data type as date time) residing under data { } section of a **work-product-component** schema from a valid DateTime value (e.g.- 2023-08-10T00:00:00+0000) to "null" or an empty string (""). But when this transaction is attempted and executed via STORAGE service, the value of the attribute remains unchanged even after a successful execution (HTTP status code - 200). STORAGE service should allow users to register an empty/null value for DateTime attribute.
Please note that the attribute "DateSubmitted" does not belong to the list of System Properties like "createTime" or "modifyTime" and might not be used for auditing purposes.
1. "kind": "shell:wks:work-product-component--LQCWebSheet:1.0.0"
2. Example record:
{
"data": {
"ApprovalStatusTypeID": "osdu:reference-data--LQCApprovalStatusType:Submitted:",
"Source": "shell",
"Name": null,
"IsBonus": false,
"LoggingInterpreter": null,
"FinalDeliveryDuration": 1.0,
"WebSheetName": "Test_LWD_Websheet_Edit_Approver_Request_v2",
"LastUpdatedPPEmail": null,
"ApproverEmail": "NewApprover1.Nayak@shell.com",
"WellboreID": "osdu:master-data--Wellbore:BDLQCGOM2_1_WB2:", "
"OperationalComment": "Test_LWD_Websheet_Edit_Approver_v2_Operational_Comments",
"ApproverComment": null,
"SourceApplication": "Created in LQC WebSheets",
"SubmitterName": "Sujith.Submitter@shell.com",
"IsApprovalStatusReset": true,
"DateSubmitted": "2023-06-05T07:56:19.914485+0000"
},
"kind": "shell:wks:work-product-component--LQCWebSheet:1.0.0",
"source": "wks",
"acl": {
"viewers": [
"data.default.viewers@osdu.shell.com"
],
"owners": [
"data.default.owners@osdu.shell.com"
]
},
"type": "work-product-component--LQCWebSheet",
"version": 1686283555925808,
"tags": {
"normalizedKind": "shell:wks:work-product-component--LQCWebSheet:1"
},
"modifyUser": "Monalisa.Mohapatra@shell.com",
"modifyTime": "2023-06-09T04:05:56.083Z",
"createTime": "2022-12-15T11:26:58.940Z",
"authority": "shell",
"namespace": "shell:wks",
"legal": {
"legaltags": [
"osdu-shell-lqc-dataset-testing"
],
"otherRelevantDataCountries": [
"US"
],
"status": "compliant"
},
"createUser": "Labanyendu.Nayak@shell.com",
"id": "osdu:work-product-component--LQCWebSheet:62008"
}
3. Target attribute - "data.DateSubmitted"https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/issues/78Anthos/Baremetal. (NoSuchKey) when calling "get welllog data"2023-10-04T16:46:26ZYan Sushchynski (EPAM)Anthos/Baremetal. (NoSuchKey) when calling "get welllog data"Hello,
Postman Environment: https://community.opengroup.org/osdu/platform/pre-shipping/-/blob/main/R3-M19/QA_Artifacts_M19/envFilesAndCollections/envFiles/OSDU%20R3%20M19%20RI%20Pre-ship.postman_environment.json Postman Collection: htt...Hello,
Postman Environment: https://community.opengroup.org/osdu/platform/pre-shipping/-/blob/main/R3-M19/QA_Artifacts_M19/envFilesAndCollections/envFiles/OSDU%20R3%20M19%20RI%20Pre-ship.postman_environment.json Postman Collection: https://community.opengroup.org/osdu/platform/pre-shipping/-/blob/main/R3-M19/QA_Artifacts_M19/envFilesAndCollections/Wellbore%20DDMS%20CI-CD%20v3.0.postman_collection.json.
Steps to reproduce:
1. Create a WellLog
2. Post the WellLog data
3. Get the WellLog data.
The logs show that when we post the well log data, a new fodler and a parquet file are created:
```log
DEBUG:Sending http request: <AWSPreparedRequest stream_output=False, method=PUT, url=https://s3.ref.gcp.gnrg-osdu.projects.epam.com/wellbore/logstore-osdu/9ee8ed74df9b8efb695f376771eea3e707b66753/bulk/2c0429ad-b4a1-4a70-a17e-bb08cc245f3f/data/0_4_1691662228355.e70a959cea89c6147785c7fa57cde5be8b6dc250.parquet
```
And then, when we want to get the data, it attempts to get an absent `bulk_catalog.json`:
```log
DEBUG:Sending http request: <AWSPreparedRequest stream_output=True, method=GET, url=https://s3.ref.gcp.gnrg-osdu.projects.epam.com/wellbore/logstore-osdu/9ee8ed74df9b8efb695f376771eea3e707b66753/bulk/2c0429ad-b4a1-4a70-a17e-bb08cc245f3f/data/bulk_catalog.json
```
Linked issue: https://community.opengroup.org/osdu/platform/pre-shipping/-/issues/568M19 - Release 0.22YannickYannickhttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/wellbore/wellbore-domain-services/-/issues/77Use of DDMSDatasets construct for linking "catalog" and "content"2024-01-08T17:59:03ZDebasis ChatterjeeUse of DDMSDatasets construct for linking "catalog" and "content"When testing in M19/Preship/Azure, I noticed that it is still the old style of linking from WellLog work-product component to parquet.
```
"ExtensionProperties": {
"step": {
"unitKey": "ft",
...When testing in M19/Preship/Azure, I noticed that it is still the old style of linking from WellLog work-product component to parquet.
```
"ExtensionProperties": {
"step": {
"unitKey": "ft",
"value": 0.1
},
"dateModified": "2013-03-22T11:16:03Z",
"wdms": {
"bulkURI": "urn:wdms-1:uuid:5ed7e9ef-b94e-4ba0-9b4b-c9acf03edb99"
}
}
```
whereas the current approach seems to be using DDMSDatasets.
See here
https://community.opengroup.org/osdu/data/data-definitions/-/blob/master/Examples/work-product-component/WellLog.1.4.0.json
```
"DDMSDatasets": [
"urn://wddms-3/uuid:20840361-adc0-4842-999b-5639bd07bb38",
"eml://rddms-1/dataspace('demo/Volve')/resqml20.obj_ContinuousProperty(1615d8d2-2a2d-482c-885e-14225b89e90c)"
],
```
Please advise when it is planned to switch.
cc @chadhttps://community.opengroup.org/osdu/platform/pre-shipping/-/issues/577M19 Azure GCZ- Unable to access registered GCZ service in AGOL2023-08-18T07:55:40ZEkta SinghM19 Azure GCZ- Unable to access registered GCZ service in AGOLSteps to replicate.
In AGOL register the item https://osdu-gcz.msft-osdu-test.org/ignite-provider/gcz/FeatureServer/1 .
A new item will be created with a unique item id. Now open this newly created item in a webmap .
Result - item will f...Steps to replicate.
In AGOL register the item https://osdu-gcz.msft-osdu-test.org/ignite-provider/gcz/FeatureServer/1 .
A new item will be created with a unique item id. Now open this newly created item in a webmap .
Result - item will fail to load
Note- Upon checking the url again of the registered item we find the service urls is getting saved ashttps://osdu-gcz.msft-osdu-test.org/ignite-provider/gcz/FeatureServer instead of https://osdu-gcz.msft-osdu-test.org/ignite-provider/gcz/FeatureServer/1M19 - Release 0.22Levi RemingtonLevi Remingtonhttps://community.opengroup.org/osdu/platform/pre-shipping/-/issues/576In IBM R3 M19 Pre-ship environment CRS Conversion API V3 failing for all the...2023-08-25T06:06:46ZKamlesh TodaiIn IBM R3 M19 Pre-ship environment CRS Conversion API V3 failing for all the available endpoints except infoIt appears that all the CRS reference data may not be loaded or available as expected.
Convert trajectory stations
```
curl --location 'https://cpd-osdu.apps.ibmosdu-preship.lndu.p1.openshiftapps.com/osdu-crs-conversion/api/crs/convert...It appears that all the CRS reference data may not be loaded or available as expected.
Convert trajectory stations
```
curl --location 'https://cpd-osdu.apps.ibmosdu-preship.lndu.p1.openshiftapps.com/osdu-crs-conversion/api/crs/converter/v3/convertTrajectory' \
--header 'data-partition-id: opendes' \
--header 'Content-Type: application/json' \
--header 'Accept: application/json' \
--header 'Authorization: Bearer eyJhbGciOi...Truncated...Ia1Rq0xTDcC-NQ' \
--data '{
"trajectoryCRS": "opendes:reference-data--CoordinateReferenceSystem:BoundProjected:EPSG::23032_EPSG::1612:",
"azimuthReference": "TN",
"unitZ": "opendes:reference-data--UnitOfMeasure:m:",
"referencePoint": {
"x": 400000,
"y": 6500000,
"z": 0
},
"inputKind": "MD_Incl_Azim",
"interpolate": true,
"inputStations": [
{
"md": 1000,
"azimuth": 100,
"inclination": 10
},
{
"md": 2000,
"azimuth": 200,
"inclination": 200
}
],
"method": "AzimuthalEquidistant"
}'
Response 400 bad Request
{
"code": 400,
"message": "record not found: opendes:reference-data--CoordinateReferenceSystem:BoundProjected:EPSG::23032_EPSG::1612/"
}
```
Convert Bin Grid
```
curl --location 'https://cpd-osdu.apps.ibmosdu-preship.lndu.p1.openshiftapps.com/osdu-crs-conversion/api/crs/converter/v3/convertBinGrid' \
--header 'data-partition-id: opendes' \
--header 'Content-Type: application/json' \
--header 'Accept: application/json' \
--header 'Authorization: Bearer eyJhbGciOi...Truncated...Ia1Rq0xTDcC-NQ' \
--data '{
"toCRS": "opendes:reference-data--CoordinateReferenceSystem:BoundProjected:EPSG::32064_EPSG::15851:",
"inBinGrid": {
"CoveragePercent": 1.0,
"ABCDBinGridSpatialLocation": {
"AsIngestedCoordinates": {
"type": "AnyCrsFeatureCollection",
"CoordinateReferenceSystemID": "opendes:reference-data--CoordinateReferenceSystem:Projected:EPSG::32615:",
"features": [
{
"type": "AnyCrsFeature",
"properties": {
"Kind": "osdu:wks:AbstractGeoJson.PropertiesBinGridCorners:1.0.0",
"PointProperties": [
{
"Inline": 1,
"Crossline": 1000
}
]
},
"geometry": {
"type": "AnyCrsPoint",
"coordinates": [
500000.00,
3000000.00
]
}
},
{
"type": "AnyCrsFeature",
"properties": {
"Kind": "osdu:wks:AbstractGeoJson.PropertiesBinGridCorners:1.0.0",
"PointProperties": [
{
"Inline": 1,
"Crossline": 2000
}
]
},
"geometry": {
"type": "AnyCrsPoint",
"coordinates": [
500000.00,
3100000.00
]
}
},
{
"type": "AnyCrsFeature",
"properties": {
"Kind": "osdu:wks:AbstractGeoJson.PropertiesBinGridCorners:1.0.0",
"PointProperties": [
{
"Inline": 101,
"Crossline": 1000
}
]
},
"geometry": {
"type": "AnyCrsPoint",
"coordinates": [
600000.00,
3000000.00
]
}
},
{
"type": "AnyCrsFeature",
"properties": {
"Kind": "osdu:wks:AbstractGeoJson.PropertiesBinGridCorners:1.0.0",
"PointProperties": [
{
"Inline": 101,
"Crossline": 2000
}
]
},
"geometry": {
"type": "AnyCrsPoint",
"coordinates": [
600000.00,
3100000.00
]
}
}
]
}
},
"P6ScaleFactorOfBinGrid": 1,
"P6BinNodeIncrementOnIaxis": 1,
"P6BinNodeIncrementOnJaxis": 1
}
}'
Response 400 bad request
{
"code": 400,
"message": "record not found: opendes:reference-data--CoordinateReferenceSystem:BoundProjected:EPSG::32064_EPSG::15851/"
}
```Anuj Guptavikas ranaAnuj Guptahttps://community.opengroup.org/osdu/platform/pre-shipping/-/issues/575IBM R3 M19 environment having issue with Search and or indexer service.2023-08-11T09:53:55ZKamlesh TodaiIBM R3 M19 environment having issue with Search and or indexer service.After ingesting the data using manifest ingestion (Osdu_ingest), one can retrieve the data using storage API, but cannot retrieve it using the Search API.
Even after waiting for more then 15 minutes.
Using Storage API
```
curl --locat...After ingesting the data using manifest ingestion (Osdu_ingest), one can retrieve the data using storage API, but cannot retrieve it using the Search API.
Even after waiting for more then 15 minutes.
Using Storage API
```
curl --location 'https://cpd-osdu.apps.ibmosdu-preship.lndu.p1.openshiftapps.com/osdu-storage/api/storage/v2/records/opendes:master-data--GeoPoliticalEntity:Limburg' \
--header 'data-partition-id: opendes' \
--header 'Authorization: Bearer eyJhbGciOi...Truncated...3k3p_JLqg'
Response 200 OK
{
"data": {
"GeoPoliticalEntityName": "Limburg",
"Source": "TNO",
"VirtualProperties.DefaultName": "Limburg"
},
"meta": null,
"modifyUser": "osdu-bvt@osdu.opengroup.org",
"modifyTime": "2023-08-08T13:40:37.481Z",
"id": "opendes:master-data--GeoPoliticalEntity:Limburg",
"version": 1691502037052901,
"kind": "osdu:wks:master-data--GeoPoliticalEntity:1.0.0",
"acl": {
"viewers": [
"data.default.viewers@opendes.ibm.com"
],
"owners": [
"data.default.owners@opendes.ibm.com"
]
},
"legal": {
"legaltags": [
"opendes-R3FullManifest-Legal-Tag-Test7791699"
],
"otherRelevantDataCountries": [
"US"
],
"status": "compliant"
},
"createUser": "osdu-bvt@osdu.opengroup.org",
"createTime": "2023-08-04T20:51:59.686Z"
}
```
But when one performs the search using Search API, the results returned are zero for the count.
```
curl --location 'https://cpd-osdu.apps.ibmosdu-preship.lndu.p1.openshiftapps.com/osdu-search/api/search/v2/query' \
--header 'Content-Type: application/json' \
--header 'data-partition-id: opendes' \
--header 'Authorization: Bearer eyJhbGciOi...Truncated...3p_JLqg' \
--data '{
"kind": "osdu:wks:master-data--GeoPoliticalEntity:1.0.0",
"query" : "*",
"limit" : 500,
"returnedFields" : ["id"]
}'
Response 200 OK
{
"results": [],
"aggregations": [],
"totalCount": 0
}
```
Does not matter what search is being performed, the total count returned is zero.
@chad @debasisc @davidglassAnuj Guptavikas ranaAnuj Guptahttps://community.opengroup.org/osdu/platform/pre-shipping/-/issues/574Azure M19 - Search results not returning expected fields2023-08-10T15:52:40ZJuliana Fernandesjuliana.fernandes@iesbrazil.com.brAzure M19 - Search results not returning expected fieldsHi,
I'm trying to perform the search query for Unit Conversion but I`m not getting what I'm expecting.
What I'm doing is:
POST
```
curl --location 'https://osdu-ship.msft-osdu-test.org/api/search/v2/query' \
```
Response
```json
{
...Hi,
I'm trying to perform the search query for Unit Conversion but I`m not getting what I'm expecting.
What I'm doing is:
POST
```
curl --location 'https://osdu-ship.msft-osdu-test.org/api/search/v2/query' \
```
Response
```json
{
"results": [
{
"kind": "osdu:wks:master-data--SeismicAcquisitionSurvey:1.0.0",
"source": "wks",
"acl": {
"viewers": [
"data.default.viewers@opendes.contoso.com"
],
"owners": [
"data.default.owners@opendes.contoso.com"
]
},
"type": "master-data--SeismicAcquisitionSurvey",
"version": 1686905173380617,
"tags": {
"normalizedKind": "osdu:wks:master-data--SeismicAcquisitionSurvey:1"
},
"modifyUser": "preshipping@azureglobal1.onmicrosoft.com",
"modifyTime": "2023-06-16T08:46:13.991Z",
"createTime": "2023-06-15T13:18:03.831Z",
"authority": "osdu",
"namespace": "osdu:wks",
"legal": {
"legaltags": [
"opendes-R3FullManifest-Legal-Tag-Test6917654"
],
"otherRelevantDataCountries": [
"US"
],
"status": "compliant"
},
"createUser": "preshipping@azureglobal1.onmicrosoft.com",
"id": "opendes:master-data--SeismicAcquisitionSurvey:ST0202R08_UC_JFA_Jun152023"
}
],
"aggregations": null,
"totalCount": 1
}
```
What I'm expecting as response is something like:
```json
{
"results": [
{
"data": {
"Operator": null,
"SourceArrayCount": 2,
"ProjectName": null,
"ShotpointIncrementDistance": 7.62,
"VirtualProperties.DefaultLocation.QuantitativeAccuracyBandID": null,
"ResourceLifecycleStatus": null,
"ProjectID": null,
"CableCount": 4,
"CableLength": 1828.8000000000002,
"ResourceCurationStatus": null,
"TechnicalAssuranceID": null,
"VirtualProperties.DefaultLocation.SpatialGeometryTypeID": null,
"Source": null,
"SourceArraySeparationDistance": 100.0,
"VirtualProperties.DefaultName": null,
"CableSpacingDistance": 30.48,
"VirtualProperties.DefaultLocation.CoordinateQualityCheckPerformedBy": null,
"VersionCreationReason": null,
"ResourceSecurityClassification": null,
"SpatialLocation.SpatialParameterTypeID": null,
"ExistenceKind": null,
"SpatialLocation.CoordinateQualityCheckPerformedBy": null,
"VesselNames": [
"GECO ANGLER"
],
"MaxOffsetDistance": 1828.8000000000002,
"TechnicalAssuranceTypeID": null,
"Purpose": "Acquisition for Volve",
"RecordLength": 10.200000000000001,
"FoldCount": 120,
"SpatialLocation.QualitativeSpatialAccuracyTypeID": null,
"SpatialLocation.SpatialGeometryTypeID": null,
"OperatingEnvironmentID": "opendes:reference-data--OperatingEnvironment:Offshore:",
"VirtualProperties.DefaultLocation.SpatialParameterTypeID": null,
"ResourceHomeRegionID": null,
"SeismicGeometryTypeID": "opendes:reference-data--SeismicGeometryType:3D:",
"VirtualProperties.DefaultLocation.QualitativeSpatialAccuracyTypeID": null,
"SampleInterval": 4.0,
"EnergySourceTypeID": "opendes:reference-data--SeismicEnergySourceType:Airgun:",
"SpatialLocation.QuantitativeAccuracyBandID": null
},
"kind": "osdu:wks:master-data--SeismicAcquisitionSurvey:1.0.0",
"source": "wks",
"acl": {
"viewers": [
"data.default.viewers@opendes.contoso.com"
],
"owners": [
"data.default.owners@opendes.contoso.com"
]
},
"type": "master-data--SeismicAcquisitionSurvey",
"version": 1687473559876188,
"tags": {
"normalizedKind": "osdu:wks:master-data--SeismicAcquisitionSurvey:1"
},
"modifyUser": "preshipping@azureglobal1.onmicrosoft.com",
"modifyTime": "2023-06-22T22:39:20.957Z",
"createTime": "2023-06-22T19:35:49.945Z",
"authority": "osdu",
"namespace": "osdu:wks",
"legal": {
"legaltags": [
"opendes-Test-Legal-Tag-9727379"
],
"otherRelevantDataCountries": [
"US"
],
"status": "compliant"
},
"createUser": "preshipping@azureglobal1.onmicrosoft.com",
"id": "opendes:master-data--SeismicAcquisitionSurvey:ST0202R08_UC_JFA_Jun152023_DB22Jun"
}
],
"aggregations": null,
"totalCount": 1
}
```https://community.opengroup.org/osdu/platform/security-and-compliance/legal/-/issues/44Incorrect validation error message for updating LegalTags - HV000028: Unexpec...2023-11-22T06:58:00ZChad LeongIncorrect validation error message for updating LegalTags - HV000028: Unexpected exception during isValid call# Summary
If you are updating legal tags with invalid request body, the error message is incorrect.
```PUT https://osdu.bm-preship.gcp.gnrg-osdu.projects.epam.com/api/legal/v1/legaltags/```
Request body:
```json
{
"name": "opende...# Summary
If you are updating legal tags with invalid request body, the error message is incorrect.
```PUT https://osdu.bm-preship.gcp.gnrg-osdu.projects.epam.com/api/legal/v1/legaltags/```
Request body:
```json
{
"name": "opendes-Test-Legal-Tag-chad-123456",
"description": "updated desc 2",
"properties": {
"contractId": "123456",
"countryOfOrigin": [
"US",
"CA"
],
"dataType": "Third Party Data",
"exportClassification": "EAR99",
"originator": "Schlumberger",
"personalData": "No Personal Data",
"securityClassification": "Private",
"expirationDate": "2023-07-31"
}
}
```
# Expected Behavior
Response
```json
{
"code": 400,
"reason": "Validation error.",
"message": "The request body is invalid."
}
```
## Actual Behavior
Response
```json
{
"code": 400,
"reason": "Validation error.",
"message": "HV000028: Unexpected exception during isValid call."
}
```
This is an example of valid request body
```json
{
"name": "osdu-Test-Legal-Tag-chad-123456",
"description": "Legal Tag added for Well",
"contractId": "chad-AE33334",
"expirationDate": "2100-12-21",
"extensionProperties": {
"test_attr": "chad-test"
}
}
```https://community.opengroup.org/osdu/platform/pre-shipping/-/issues/565While performing ingestion to check for FoR for CRS conversion there is a sch...2023-08-07T12:43:05ZKamlesh TodaiWhile performing ingestion to check for FoR for CRS conversion there is a schema validation error which which is not understood.When using collection [Manifest_Based_Ingestion_Osdu_ingest_CI-CD_v2.0](https://community.opengroup.org/osdu/platform/pre-shipping/-/blob/main/R3-M19/QA_Artifacts_M19/envFilesAndCollections/Manifest_Based_Ingestion_Osdu_ingest_CI-CD_v2....When using collection [Manifest_Based_Ingestion_Osdu_ingest_CI-CD_v2.0](https://community.opengroup.org/osdu/platform/pre-shipping/-/blob/main/R3-M19/QA_Artifacts_M19/envFilesAndCollections/Manifest_Based_Ingestion_Osdu_ingest_CI-CD_v2.0.postman_collection.json) and "folder Integrity Check FoR for CRS Conversion" to test,
one gets the failure message from Osdu_ingest
complaining about the schema validation which is not clear. (This collection works in other cloud provider's (Azure and GC) environments
```
File "/home/airflow/.local/lib/python3.8/site-packages/jsonschema/validators.py", line 353, in validate
raise error
jsonschema.exceptions.ValidationError: 'acl' is a required property
Failed validating 'required' in schema:
{'$id': 'https://schema.osdu.opengroup.org/json/master-data/Well.1.0.0.json',
'$schema': 'http://json-schema.org/draft-07/schema#',
'additionalProperties': False,
'definitions': {'osdu:wks:AbstractAccessControlList:1.0.0': {'$id': 'https://schema.osdu.opengroup.org/json/abstract/AbstractAccessControlList.1.0.0.json',
'$schema': 'http://json-schema.org/draft-07/schema#',
'additionalProperties': False,
'description': 'The '
'access '
'control '
'tags '
'associated '
'with '
'this '
'entity. '
'This '
'structure '
'is '
'included '
'by '
'the '
'SystemProperties '
'"acl", '
'which '
'is '
'part '
'of '
'all '
'OSDU '
'records. '
'Not '
'extensible.',
```
**More details can be found by looking at the logs of run id: 1d22e6b1-29aa-4960-b0dd-ea3744dd1866 OR following the following link**
[http://airflow-web-osdu.apps.ibmosdu-preship.lndu.p1.openshiftapps.com/log?dag_id=Osdu_ingest&task_id=validate_manifest_schema_task&execution_date=2023-08-04T19%3A07%3A31.213150%2B00%3A00](http://airflow-web-osdu.apps.ibmosdu-preship.lndu.p1.openshiftapps.com/log?dag_id=Osdu_ingest&task_id=validate_manifest_schema_task&execution_date=2023-08-04T21%3A48%3A04.758668%2B00%3A00)
**Please Note the CRS**
```postman_json
{{data-partition-id}}:reference-data--CoordinateReferenceSystem:23031024 is missing from this instance
```
**Payload while triggering the DAG**
```
curl --location 'https://cpd-osdu.apps.ibmosdu-preship.lndu.p1.openshiftapps.com/osdu-workflow/api/workflow/v1/workflow/Osdu_ingest/workflowRun' \
--header 'data-partition-id: opendes' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer eyJhbGciOi...Truncated...2NgrgQQCk8qWg' \
--data-raw '{
"runId": "f3aaac37-1366-4205-a389-e84659b0c6b1",
"executionContext": {
"Payload": {
"AppKey": "test-app",
"data-partition-id": "opendes"
},
"manifest": {
"kind": "osdu:wks:master-data--Well:1.0.0",
"ReferenceData": [ ],
"MasterData": [
{
"id": "opendes:master-data--Well:818188-FoR-CRS",
"kind": "osdu:wks:master-data--Well:1.0.0",
"acl": {
"owners": [
"data.default.owners@opendes.ibm.com"
],
"viewers": [
"data.default.viewers@opendes.ibm.com"
]
},
"legal": {
"legaltags": [
"opendes-Manifest-Ingestion-Legal-Tag"
],
"otherRelevantDataCountries": [
"US"
],
"status": "compliant"
},
"meta": [
{
"kind": "Unit",
"name": "m",
"persistableReference": "{\"abcd\":{\"a\":0.0,\"b\":1.0,\"c\":1.0,\"d\":0.0},\"symbol\":\"m\",\"baseMeasurement\":{\"ancestry\":\"L\",\"type\":\"UM\"},\"type\":\"UAD\"}",
"unitOfMeasureID": "opendes:reference-data--UnitOfMeasure:m:",
"propertyNames": [
"VerticalMeasurements[].VerticalMeasurement"
]
}
],
"data": {
"SpatialLocation": {
"AsIngestedCoordinates": {
"type": "AnyCrsFeatureCollection",
"CoordinateReferenceSystemID": "opendes:reference-data--CoordinateReferenceSystem:23031024:",
"persistableReferenceCrs": "{\"authCode\":{\"auth\":\"OSDU\",\"code\":\"23031024\"},\"lateBoundCRS\":{\"authCode\":{\"auth\":\"EPSG\",\"code\":\"23031\"},\"name\":\"ED_1950_UTM_Zone_31N\",\"type\":\"LBC\",\"ver\":\"PE_10_3_1\",\"wkt\":\"PROJCS[\\\"ED_1950_UTM_Zone_31N\\\",GEOGCS[\\\"GCS_European_1950\\\",DATUM[\\\"D_European_1950\\\",SPHEROID[\\\"International_1924\\\",6378388.0,297.0]],PRIMEM[\\\"Greenwich\\\",0.0],UNIT[\\\"Degree\\\",0.0174532925199433]],PROJECTION[\\\"Transverse_Mercator\\\"],PARAMETER[\\\"False_Easting\\\",500000.0],PARAMETER[\\\"False_Northing\\\",0.0],PARAMETER[\\\"Central_Meridian\\\",3.0],PARAMETER[\\\"Scale_Factor\\\",0.9996],PARAMETER[\\\"Latitude_Of_Origin\\\",0.0],UNIT[\\\"Meter\\\",1.0],AUTHORITY[\\\"EPSG\\\",23031]]\"},\"name\":\"ED50 * EPSG-Nor S62 2001 / UTM zone 31N [23031,1613]\",\"singleCT\":{\"authCode\":{\"auth\":\"EPSG\",\"code\":\"1613\"},\"name\":\"ED_1950_To_WGS_1984_24\",\"type\":\"ST\",\"ver\":\"PE_10_3_1\",\"wkt\":\"GEOGTRAN[\\\"ED_1950_To_WGS_1984_24\\\",GEOGCS[\\\"GCS_European_1950\\\",DATUM[\\\"D_European_1950\\\",SPHEROID[\\\"International_1924\\\",6378388.0,297.0]],PRIMEM[\\\"Greenwich\\\",0.0],UNIT[\\\"Degree\\\",0.0174532925199433]],GEOGCS[\\\"GCS_WGS_1984\\\",DATUM[\\\"D_WGS_1984\\\",SPHEROID[\\\"WGS_1984\\\",6378137.0,298.257223563]],PRIMEM[\\\"Greenwich\\\",0.0],UNIT[\\\"Degree\\\",0.0174532925199433]],METHOD[\\\"Position_Vector\\\"],PARAMETER[\\\"X_Axis_Translation\\\",-90.365],PARAMETER[\\\"Y_Axis_Translation\\\",-101.13],PARAMETER[\\\"Z_Axis_Translation\\\",-123.384],PARAMETER[\\\"X_Axis_Rotation\\\",0.333],PARAMETER[\\\"Y_Axis_Rotation\\\",0.077],PARAMETER[\\\"Z_Axis_Rotation\\\",0.894],PARAMETER[\\\"Scale_Difference\\\",1.994],AUTHORITY[\\\"EPSG\\\",1613]]\"},\"type\":\"EBC\",\"ver\":\"PE_10_3_1\"}",
"features": [
{
"type": "AnyCrsFeature",
"geometry": {
"type": "AnyCrsPoint",
"coordinates": [
700113.0,
5757315.0
]
},
"properties": {}
}
]
}
},
"FacilityTypeID": "opendes:reference-data--FacilityType:Wellbore:",
"FacilityName": "1001",
"SequenceNumber": 1,
"VerticalMeasurements": [
{
"VerticalMeasurementID": "TD-Original",
"VerticalMeasurement": 662.0,
"VerticalMeasurementTypeID": "opendes:reference-data--VerticalMeasurementType:TD:",
"VerticalMeasurementPathID": "opendes:reference-data--VerticalMeasurementPath:MD:",
"VerticalReferenceID": "Measured_From"
},
{
"VerticalMeasurementID": "TVD",
"VerticalMeasurement": 662.0,
"VerticalMeasurementTypeID": "opendes:reference-data--VerticalMeasurementType:TD:",
"VerticalMeasurementPathID": "opendes:reference-data--VerticalMeasurementPath:TVD:",
"VerticalReferenceID": "Measured_From"
},
{
"VerticalMeasurementID": "Measured_From",
"VerticalMeasurement": 11.01,
"VerticalMeasurementTypeID": "opendes:reference-data--VerticalMeasurementType:RT:",
"VerticalMeasurementPathID": "opendes:reference-data--VerticalMeasurementPath:ELEV:"
}
],
"DefaultVerticalMeasurementID": "Measured_From",
"ProjectedBottomHoleLocation": {
"AsIngestedCoordinates": {
"type": "AnyCrsFeatureCollection",
"CoordinateReferenceSystemID": "opendes:reference-data--CoordinateReferenceSystem:23031024:",
"persistableReferenceCrs": "{\"authCode\":{\"auth\":\"OSDU\",\"code\":\"23031024\"},\"lateBoundCRS\":{\"authCode\":{\"auth\":\"EPSG\",\"code\":\"23031\"},\"name\":\"ED_1950_UTM_Zone_31N\",\"type\":\"LBC\",\"ver\":\"PE_10_3_1\",\"wkt\":\"PROJCS[\\\"ED_1950_UTM_Zone_31N\\\",GEOGCS[\\\"GCS_European_1950\\\",DATUM[\\\"D_European_1950\\\",SPHEROID[\\\"International_1924\\\",6378388.0,297.0]],PRIMEM[\\\"Greenwich\\\",0.0],UNIT[\\\"Degree\\\",0.0174532925199433]],PROJECTION[\\\"Transverse_Mercator\\\"],PARAMETER[\\\"False_Easting\\\",500000.0],PARAMETER[\\\"False_Northing\\\",0.0],PARAMETER[\\\"Central_Meridian\\\",3.0],PARAMETER[\\\"Scale_Factor\\\",0.9996],PARAMETER[\\\"Latitude_Of_Origin\\\",0.0],UNIT[\\\"Meter\\\",1.0],AUTHORITY[\\\"EPSG\\\",23031]]\"},\"name\":\"ED50 * EPSG-Nor S62 2001 / UTM zone 31N [23031,1613]\",\"singleCT\":{\"authCode\":{\"auth\":\"EPSG\",\"code\":\"1613\"},\"name\":\"ED_1950_To_WGS_1984_24\",\"type\":\"ST\",\"ver\":\"PE_10_3_1\",\"wkt\":\"GEOGTRAN[\\\"ED_1950_To_WGS_1984_24\\\",GEOGCS[\\\"GCS_European_1950\\\",DATUM[\\\"D_European_1950\\\",SPHEROID[\\\"International_1924\\\",6378388.0,297.0]],PRIMEM[\\\"Greenwich\\\",0.0],UNIT[\\\"Degree\\\",0.0174532925199433]],GEOGCS[\\\"GCS_WGS_1984\\\",DATUM[\\\"D_WGS_1984\\\",SPHEROID[\\\"WGS_1984\\\",6378137.0,298.257223563]],PRIMEM[\\\"Greenwich\\\",0.0],UNIT[\\\"Degree\\\",0.0174532925199433]],METHOD[\\\"Position_Vector\\\"],PARAMETER[\\\"X_Axis_Translation\\\",-90.365],PARAMETER[\\\"Y_Axis_Translation\\\",-101.13],PARAMETER[\\\"Z_Axis_Translation\\\",-123.384],PARAMETER[\\\"X_Axis_Rotation\\\",0.333],PARAMETER[\\\"Y_Axis_Rotation\\\",0.077],PARAMETER[\\\"Z_Axis_Rotation\\\",0.894],PARAMETER[\\\"Scale_Difference\\\",1.994],AUTHORITY[\\\"EPSG\\\",1613]]\"},\"type\":\"EBC\",\"ver\":\"PE_10_3_1\"}",
"features": [
{
"type": "AnyCrsFeature",
"geometry": {
"type": "AnyCrsPoint",
"coordinates": [
700113.0,
5757315.0
]
},
"properties": {}
}
]
}
}
}
}
]
}
}
}'
```Anuj Guptavikas ranaAnuj Guptahttps://community.opengroup.org/osdu/platform/pre-shipping/-/issues/564In IBM R3 M19 environment some of the reference data for Entity type UNIT is ...2023-08-24T10:32:13ZKamlesh TodaiIn IBM R3 M19 environment some of the reference data for Entity type UNIT is missing.Following UNIT reference data for Entity reference-data--UnitOfMeasure is missing.
```plaintext
'Missing parents: {SRN: opendes:reference-data--UnitOfMeasure:in,
SRN: opendes:reference-data--UnitOfMeasure:gAPI,
...Following UNIT reference data for Entity reference-data--UnitOfMeasure is missing.
```plaintext
'Missing parents: {SRN: opendes:reference-data--UnitOfMeasure:in,
SRN: opendes:reference-data--UnitOfMeasure:gAPI,
SRN: opendes:reference-data--UnitOfMeasure:cC,
SRN: opendes:dataset--File.Generic:999283442886,
SRN: opendes:reference-data--UnitOfMeasure:psi,
SRN: opendes:reference-data--UnitOfMeasure:ohm.m,
SRN: opendes:reference-data--UnitOfMeasure:g%2Fcm3,
SRN: opendes:reference-data--UnitOfMeasure:h,
SRN: opendes:reference-data--UnitOfMeasure:m3%2Fm3,
SRN: opendes:reference-data--UnitOfMeasure:m%2Fh,
SRN: opendes:reference-data--UnitOfMeasure:degC}'
```Anuj Guptavikas ranaAnuj Guptahttps://community.opengroup.org/osdu/platform/pre-shipping/-/issues/563Energistics XML parser DAG not able to execute successfully. Fails while cr...2023-08-16T10:04:39ZKamlesh TodaiEnergistics XML parser DAG not able to execute successfully. Fails while creating new pod for the oepratorWhen one looks at the DAG runs there has not been any successful run. It is failing while creating new pod.
```plaintext
[2023-08-04 20:00:50,255] {pod_launcher.py:128} WARNING - Pod not yet started: witsml-parser-task.10c656bb27934c3c9...When one looks at the DAG runs there has not been any successful run. It is failing while creating new pod.
```plaintext
[2023-08-04 20:00:50,255] {pod_launcher.py:128} WARNING - Pod not yet started: witsml-parser-task.10c656bb27934c3c90951ea67d10e8a0
[2023-08-04 20:00:50,272] {taskinstance.py:1463} ERROR - Task failed with exception
Traceback (most recent call last):
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py", line 367, in execute
final_state, remote_pod, result = self.create_new_pod_for_operator(labels, launcher)
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py", line 520, in create_new_pod_for_operator
launcher.start_pod(self.pod, startup_timeout=self.startup_timeout_seconds)
File "/home/airflow/.local/lib/python3.8/site-packages/tenacity/__init__.py", line 329, in wrapped_f
return self.call(f, *args, **kw)
File "/home/airflow/.local/lib/python3.8/site-packages/tenacity/__init__.py", line 409, in call
do = self.iter(retry_state=retry_state)
File "/home/airflow/.local/lib/python3.8/site-packages/tenacity/__init__.py", line 356, in iter
return fut.result()
File "/usr/local/lib/python3.8/concurrent/futures/_base.py", line 437, in result
return self.__get_result()
File "/usr/local/lib/python3.8/concurrent/futures/_base.py", line 389, in __get_result
raise self._exception
File "/home/airflow/.local/lib/python3.8/site-packages/tenacity/__init__.py", line 412, in call
result = fn(*args, **kwargs)
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/providers/cncf/kubernetes/utils/pod_launcher.py", line 131, in start_pod
raise AirflowException("Pod took too long to start")
airflow.exceptions.AirflowException: Pod took too long to start
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/models/taskinstance.py", line 1165, in _run_raw_task
self._prepare_and_execute_task_with_callbacks(context, task)
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/models/taskinstance.py", line 1283, in _prepare_and_execute_task_with_callbacks
result = self._execute_task(context, task_copy)
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/models/taskinstance.py", line 1313, in _execute_task
result = task_copy.execute(context=context)
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py", line 374, in execute
raise AirflowException(f'Pod Launching failed: {ex}')
airflow.exceptions.AirflowException: Pod Launching failed: Pod took too long to start
[2023-08-04 20:00:50,273] {taskinstance.py:1506} INFO - Marking task as FAILED. dag_id=Energistics_xml_ingest, task_id=witsml_parser_task, execution_date=20230804T195547, start_date=20230804T195549, end_date=20230804T200050
[2023-08-04 20:00:50,318] {local_task_job.py:151} INFO - Task exited with return code 1
[2023-08-04 20:00:50,339] {local_task_job.py:261} INFO - 0 downstream tasks scheduled from follow-on schedule check
```Anuj Guptavikas ranaAnuj Guptahttps://community.opengroup.org/osdu/platform/pre-shipping/-/issues/562CSV parser DAG on IBM R3 M19 environment is failing. Issue seems to be config...2023-08-22T18:30:48ZKamlesh TodaiCSV parser DAG on IBM R3 M19 environment is failing. Issue seems to be configuration and or resource allocationWhen one looks at the DAG runs for CSV parser, there is not a single run that has been successful.
The log indicates that there is some issue in dict - undefined (may be setup configuration issue)
```
[2023-08-04 19:00:31,847] {standa...When one looks at the DAG runs for CSV parser, there is not a single run that has been successful.
The log indicates that there is some issue in dict - undefined (may be setup configuration issue)
```
[2023-08-04 19:00:31,847] {standard_task_runner.py:77} INFO - Job 986: Subtask csv-parser
[2023-08-04 19:00:32,162] {logging_mixin.py:109} INFO - Running <TaskInstance: csv-parser-dag.csv-parser 2023-08-04T18:55:28.858482+00:00 [running]> on host ***-worker-0.***-worker.osdu.svc.cluster.local
[2023-08-04 19:00:32,194] {taskinstance.py:1463} ERROR - Task failed with exception
Traceback (most recent call last):
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/models/taskinstance.py", line 1165, in _run_raw_task
self._prepare_and_execute_task_with_callbacks(context, task)
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/models/taskinstance.py", line 1246, in _prepare_and_execute_task_with_callbacks
self.render_templates(context=context)
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/models/taskinstance.py", line 1755, in render_templates
self.task.render_template_fields(context)
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/models/baseoperator.py", line 997, in render_template_fields
self._do_render_template_fields(self, self.template_fields, context, jinja_env, set())
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/models/baseoperator.py", line 1010, in _do_render_template_fields
rendered_content = self.render_template(content, context, jinja_env, seen_oids)
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/models/baseoperator.py", line 1061, in render_template
return [self.render_template(element, context, jinja_env) for element in content]
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/models/baseoperator.py", line 1061, in <listcomp>
return [self.render_template(element, context, jinja_env) for element in content]
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/models/baseoperator.py", line 1047, in render_template
return jinja_env.from_string(content).render(**context)
File "/home/airflow/.local/lib/python3.8/site-packages/jinja2/environment.py", line 1090, in render
self.environment.handle_exception()
File "/home/airflow/.local/lib/python3.8/site-packages/jinja2/environment.py", line 832, in handle_exception
reraise(*rewrite_traceback_stack(source=source))
File "/home/airflow/.local/lib/python3.8/site-packages/jinja2/_compat.py", line 28, in reraise
raise value.with_traceback(tb)
File "<template>", line 1, in top-level template code
jinja2.exceptions.UndefinedError: 'dict object' has no attribute 'correlationId'
[2023-08-04 19:00:32,197] {taskinstance.py:1506} INFO - Marking task as FAILED. dag_id=csv-parser-dag, task_id=csv-parser, execution_date=20230804T185528, start_date=20230804T190031, end_date=20230804T190032
```Anuj Guptavikas ranaAnuj Guptahttps://community.opengroup.org/osdu/platform/system/schema-service/-/issues/133Restrict high limit value on get schema endpoint2023-08-07T09:54:59ZAbhishek Kumar (SLB)Restrict high limit value on get schema endpointAt present there is no upper limit put by the api on the limit parameter to get the list of Schema Info.
The default value is 100 when limit is not explicitly provided, likewise, there should be some upper bound to restrict abuse case fo...At present there is no upper limit put by the api on the limit parameter to get the list of Schema Info.
The default value is 100 when limit is not explicitly provided, likewise, there should be some upper bound to restrict abuse case for this endpoint.https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-gcp-provisioning/-/issues/25docker-desktop Installation issue?2023-08-16T16:26:21ZShane Hutchinsdocker-desktop Installation issue?- Docker Desktop: 4.21.1 (114176)
- istioctl: 1.18.2
- OS/Arch: darwin/arm64
custom-values:
- domain: "localhost"
- useHttps: false
- limitsEnabled: false
- useInternalServerUrl: true
`% kubectl get po|egrep "Error|CrashLoopBackOff"`
<...- Docker Desktop: 4.21.1 (114176)
- istioctl: 1.18.2
- OS/Arch: darwin/arm64
custom-values:
- domain: "localhost"
- useHttps: false
- limitsEnabled: false
- useInternalServerUrl: true
`% kubectl get po|egrep "Error|CrashLoopBackOff"`
<details><summary>Click to expand</summary>
`airflow-bootstrap-deployment-78959b9768-b426s 0/1 CrashLoopBackOff 11 (80s ago) 58m
elastic-bootstrap-deployment-75444774c7-f64km 0/1 CrashLoopBackOff 12 (4m23s ago) 58m
elasticsearch-0 0/1 CrashLoopBackOff 13 (4m54s ago) 58m
entitlements-7c78fffcc9-cr6xf 1/2 CrashLoopBackOff 9 (2m30s ago) 58m
entitlements-bootstrap-58566c8847-45k47 0/1 CrashLoopBackOff 12 (2m12s ago) 58m
file-84864855f9-wfk74 1/2 CrashLoopBackOff 8 (4m3s ago) 57m
indexer-869f9fd455-pswx2 1/2 CrashLoopBackOff 10 (4m28s ago) 58m
keycloak-bootstrap-deployment-76cf5d84bc-kb4d2 0/1 CrashLoopBackOff 11 (2m43s ago) 57m
notification-575f8db75c-2g75f 0/1 CrashLoopBackOff 9 (3m43s ago) 57m
partition-bootstrap-7856b465f9-lbgwt 1/2 CrashLoopBackOff 13 (3m51s ago) 58m
rabbitmq-bootstrap-deployment-bb7f6c6fd-zzdx4 0/1 CrashLoopBackOff 11 (4m30s ago) 58m
schema-bootstrap-7ff4dd7d5f-qgv7z 0/1 CrashLoopBackOff 12 (2m33s ago) 57m
storage-8658c8d47d-bxd7l 0/1 CrashLoopBackOff 9 (3m8s ago) 57m
wellbore-68b8c4799b-slmrf 0/1 CrashLoopBackOff 20 (4m6s ago) 57m
workflow-bootstrap-859dc856c6-4lfp6 0/1 CrashLoopBackOff 12 (116s ago) 57m`
</details>
`% helm install -f custom-values.yaml osdu-baremetal oci://community.opengroup.org:5555/osdu/platform/deployment-and-operations/infra-gcp-provisioning/gc-helm/osdu-gc-baremetal`
<details><summary>Click to expand</summary>
Pulled: community.opengroup.org:5555/osdu/platform/deployment-and-operations/infra-gcp-provisioning/gc-helm/osdu-gc-baremetal:0.22.2
Digest: sha256:76f7b528ba2d8266ce8fa9274a93fe47240f294cfa492d9d7c0dc6dbfc63364e
W0803 14:23:07.067208 96416 warnings.go:70] configured AuthorizationPolicy will deny all traffic to TCP ports under its scope due to the use of only HTTP attributes in a DENY rule; it is recommended to explicitly specify the port
W0803 14:23:07.073368 96416 warnings.go:70] configured AuthorizationPolicy will deny all traffic to TCP ports under its scope due to the use of only HTTP attributes in a DENY rule; it is recommended to explicitly specify the port
NAME: osdu-baremetal
LAST DEPLOYED: Thu Aug 3 14:22:54 2023
NAMESPACE: default
STATUS: deployed
REVISION: 1
TEST SUITE: None
</details>Dzmitry Malkevich (EPAM)Dzmitry Malkevich (EPAM)https://community.opengroup.org/osdu/platform/pre-shipping/-/issues/559Cannot trigger the DAG Osdu_ingest_by_reference to execute manifest ingestion...2023-08-08T09:43:28ZKamlesh TodaiCannot trigger the DAG Osdu_ingest_by_reference to execute manifest ingestion by Reference in IBM R3 M19 environmentCannot trigger the DAG Osdu_ingest_by_reference to execute manifest ingestion by Reference in IBM R3 M19 environment
Request
curl --location 'https://cpd-osdu.apps.ibmosdu-preship.lndu.p1.openshiftapps.com/osdu-workflow/api/workflow/v1...Cannot trigger the DAG Osdu_ingest_by_reference to execute manifest ingestion by Reference in IBM R3 M19 environment
Request
curl --location 'https://cpd-osdu.apps.ibmosdu-preship.lndu.p1.openshiftapps.com/osdu-workflow/api/workflow/v1/workflow/Osdu_ingest_by_reference/workflowRun' \
\--header 'data-partition-id: opendes' \
\--header 'Content-Type: application/json' \
\--header 'Authorization: Bearer eyJhbGciOiJSUzI1NiIsIn...JjQhFlbeJb0gg' \
\--data '{ "executionContext": { "Payload": { "AppKey": "test-app", "data-partition-id": "opendes" }, "manifest": "opendes:dataset--File.Generic:5bb42ca88eca49da90c48272601fca68" } }'
Response
{ "code": 500, "reason": "could not create addressopendes-status-event-default-topic", "message": "io.netty.util.internal.ReferenceCountUpdater.setInitialValue(Lio/netty/util/ReferenceCounted;)V" }
@anujgupta @chad @debasisc @davidglassvikas ranavikas ranahttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/issues/105Symbol(Id) in the documents should be removed2023-08-29T15:08:38ZLaura DamianSymbol(Id) in the documents should be removedRight now each document in the database contain a Symbol(id) entry which contains information on the name and the partition key.
This information is duplicated in the document's name and its id of the documents.
Removing this information...Right now each document in the database contain a Symbol(id) entry which contains information on the name and the partition key.
This information is duplicated in the document's name and its id of the documents.
Removing this information would have a few benefits:
- keep the cost down, as it adds up to the cost of storage
- improve the performance (even marginally) of both inserts and reads, but it could add up for big numbers
- simplify a potential future migration to a different partition key(s).Diego MolteniDiego Molteni