OSDU Software issueshttps://community.opengroup.org/groups/osdu/-/issues2022-12-02T12:22:49Zhttps://community.opengroup.org/osdu/platform/deployment-and-operations/helm-charts-azure/-/issues/15Error installing Helm Chart for OSDU Istio2022-12-02T12:22:49ZFabien BosquetError installing Helm Chart for OSDU Istiobase on the actual instruction on master banch (commit f29171ef63af43a4574dd9b06f5b9edb52a401c8) in
[Helm Chart for OSDU Istio on Azure](https://community.opengroup.org/osdu/platform/deployment-and-operations/helm-charts-azure/-/blob/mas...base on the actual instruction on master banch (commit f29171ef63af43a4574dd9b06f5b9edb52a401c8) in
[Helm Chart for OSDU Istio on Azure](https://community.opengroup.org/osdu/platform/deployment-and-operations/helm-charts-azure/-/blob/master/osdu-istio/README.md)
I am having the following error message when installing osdu-istio
`bash> helm upgrade -i osdu-istio osdu-istio/ -n $NAMESPACE -f osdu-istio/values.yaml -f osdu_istio_custom_values.yaml`
```
Release "osdu-istio" does not exist. Installing it now.
Error: failed post-install: unable to build kubernetes object for post-install hook osdu-istio/templates/istio-appgw-health-vs.yaml: error validating "": error validating data: ValidationError(VirtualService.spec.http[0]): unknown field "directResponse" in io.istio.networking.v1alpha3.VirtualService.spec.http
```
Then the `istio-system` service stays in `pending` state
`bash> kubectl get svc -n istio-system istio-ingressgateway`
```
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
istio-ingressgateway LoadBalancer 10.0.13.20 <pending> 15021:31191/TCP,80:31018/TCP,443:32622/TCP 158m
```https://community.opengroup.org/osdu/ui/data-loading/wellbore-ddms-data-loader/-/issues/38Trajectory data loader from WITSML format2022-12-01T21:17:04ZDebasis ChatterjeeTrajectory data loader from WITSML formatWITSML Parser handles only catalog data.
The "ask" here is to populate station data (depth, md, inclination angle, Azimuth...) into Wellbore DDMS.
Support of V2.0 and V1.4.WITSML Parser handles only catalog data.
The "ask" here is to populate station data (depth, md, inclination angle, Azimuth...) into Wellbore DDMS.
Support of V2.0 and V1.4.https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/reservoir/open-etp-server/-/issues/29creating data space with custom data argument --xdata from the cmd line does ...2022-12-01T17:36:03Zalexandre hugotcreating data space with custom data argument --xdata from the cmd line does not do anythingcreate a new data space from the command line with --xdata "{\"key\":\"value\"}" does not add customData to the data spacecreate a new data space from the command line with --xdata "{\"key\":\"value\"}" does not add customData to the data spaceM15 - Release 0.18alexandre hugotalexandre hugothttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/reservoir/open-etp-server/-/issues/27ETP client fails when delegation is enabled2022-12-01T17:34:26ZNur SheikhETP client fails when delegation is enabledWith delegation='' or delegation='${ENTITLEMENTS_URL}' option, it is not working as expected (even when preship user was added in the correct groups) (See [comment in MR](https://community.opengroup.org/osdu/platform/domain-data-mgmt-ser...With delegation='' or delegation='${ENTITLEMENTS_URL}' option, it is not working as expected (even when preship user was added in the correct groups) (See [comment in MR](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/reservoir/open-etp-server/-/merge_requests/52#note_161574) for more details on logs)
If delegation it is removed, the open-etp-server it is working fine (issue with delegation and authentication issue on c++ code), this is likely to be a related issue: https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/reservoir/open-etp-server/-/issues/21
However, there is a different issue in master about connection reset (Not sure if we need to introduce delays in each test to fix it [log output from pipeline](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/reservoir/open-etp-server/-/jobs/1483193#L308))
Open-etp-client it is dependent on this delegation auth method, therefore, if delegation it is enabled, open-etp-client will fail, therefore auth delegation must be fixed first in open-etp-server.https://community.opengroup.org/osdu/platform/system/lib/core/os-core-common/-/issues/60Changing info logs into debug logs2022-12-01T15:14:39ZNeha KhandelwalChanging info logs into debug logsIn the OSDU Storage service, we are incurring large costs due to info logs created every time a request is sent to the Entitlements service. These logs are being created in the class HttpClientHandler.java (line 110) and are of the forma...In the OSDU Storage service, we are incurring large costs due to info logs created every time a request is sent to the Entitlements service. These logs are being created in the class HttpClientHandler.java (line 110) and are of the format "method: %s | latency: %s | url: %s | correlation id: %s". Such logs can be turned into debug logs to save on costs. There are two reasons why this change is viable: the Entitlements service creates request logs whenever a request is received, and info logs will still be created in Storage if any error occurs with the request. If needed, the logs can be switched back to info logs in the future.Neha KhandelwalNeha Khandelwalhttps://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/issues/74osdu:wks:dataset--FileCollection.Bluware.OpenVDS:1.0.02022-12-01T09:02:52ZSacha Brantsosdu:wks:dataset--FileCollection.Bluware.OpenVDS:1.0.0https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/issues/73osdu:wks:dataset--FileCollection.Slb.OpenZGY:1.0.02022-12-01T09:02:04ZSacha Brantsosdu:wks:dataset--FileCollection.Slb.OpenZGY:1.0.0https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/issues/72osdu:wks:dataset--FileCollection.SEGY:1.0.02022-12-01T09:01:58ZSacha Brantsosdu:wks:dataset--FileCollection.SEGY:1.0.0https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/issues/71osdu:wks:dataset--FileCollection.Generic:1.0.02022-12-01T09:01:40ZSacha Brantsosdu:wks:dataset--FileCollection.Generic:1.0.0https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/open-vds/-/issues/161World coordinates when cdp is not defined2022-11-30T13:39:02ZAlena ChaikouskayaWorld coordinates when cdp is not definedWe believe that this case is unlikely to happen in real files, but wanted to point that out anyway.
When file has no cdp information, asking for information in World coordinates returns data in Annotation coordinate system. If transfor...We believe that this case is unlikely to happen in real files, but wanted to point that out anyway.
When file has no cdp information, asking for information in World coordinates returns data in Annotation coordinate system. If transformation with missing World coordinates is impossible, I would have expected some error message along the way.
SEGY [without_cdp.segy](/uploads/2af802cd954648a22c6a8b03c6241f54/without_cdp.segy):
```
spec.samples = [4, 8]
spec.ilines = [3, 4, 5]
spec.xlines = [10, 11]
DelayRecordingTime: 5,
```
openvds:
```
auto transformer = OpenVDS::IJKCoordinateTransformer(layout);
auto annotation = transformer.IJKIndexToAnnotation({0, 0, 0})
auto world = transformer.IJKIndexToWorld({ 0, 0, 0 });
```
Result:
```
Annotation: 3 10 5
World: 3 10 -5
```
World coordinates created from transformed I/J are same as annotation, which is misleading.
World coordinate created from transformed K is `-Time` (also in files with cdp), which seems strange, though somewhat understandable.https://community.opengroup.org/osdu/platform/system/storage/-/issues/152Upgrade azure-storage SDK2022-11-28T14:39:21ZNur SheikhUpgrade azure-storage SDKIn storage service we are using the azure-storage sdk 8.6.5 from com.microsoft.azure package which is too old and not having much support. It iis advisable to use the latest sdk for com.azure package.In storage service we are using the azure-storage sdk 8.6.5 from com.microsoft.azure package which is too old and not having much support. It iis advisable to use the latest sdk for com.azure package.https://community.opengroup.org/osdu/platform/system/file/-/issues/63Preloadfilepath & ExtensionProperties removed from file Metadata API2022-11-28T14:10:32Zivar SoerheimPreloadfilepath & ExtensionProperties removed from file Metadata APIDuring ingestion of file metadata under /files/metadata using POST command the Preloadfilepath and ExtensionProperties are not persisted when returning the record post ingest.
This seems like strange behaviour to me. I would like to ei...During ingestion of file metadata under /files/metadata using POST command the Preloadfilepath and ExtensionProperties are not persisted when returning the record post ingest.
This seems like strange behaviour to me. I would like to either understand why this happens, or extend the file metadata api so these properties are not removed.
This is the workflow:
1. Get Signed URL for upload
2. Upload file using signed URL
3. Upload file metadata using file api (this returns ID of created record and can be searched in storage)
4. Refer to this ID when creating well log record or any other record
The problem with this workflow is that:
- PreloadFilePath and ExtensionProperties are removed from the record during metadata uploadhttps://community.opengroup.org/osdu/platform/pre-shipping/-/issues/393SEGY-to-VDS DAG fails on Azure2022-11-28T10:15:45ZRaman SinghSEGY-to-VDS DAG fails on AzureAs seen in the attached log, the DAG reports failure. The SEGY file is from Volve dataset.
[AirFlow_log_segy_to_vds.txt](/uploads/41b61e7fbf84e37f067883e11af8c136/AirFlow_log_segy_to_vds.txt)As seen in the attached log, the DAG reports failure. The SEGY file is from Volve dataset.
[AirFlow_log_segy_to_vds.txt](/uploads/41b61e7fbf84e37f067883e11af8c136/AirFlow_log_segy_to_vds.txt)M14 - Release 0.17https://community.opengroup.org/osdu/platform/system/schema-service/-/issues/45Schema is getting created even if invalid def ref is passed in payload2022-11-24T12:50:10ZTsvetelina IvanovaSchema is getting created even if invalid def ref is passed in payloadWhen there is an internal `$ref` value in schema service, for example, “$ref”: “#/definitions/example”, but the “example” is missing in the `definitions` section we should throw an exception, because this should be invalid input. Current...When there is an internal `$ref` value in schema service, for example, “$ref”: “#/definitions/example”, but the “example” is missing in the `definitions` section we should throw an exception, because this should be invalid input. Currently, we allow schema to be created even if the definition for an internal reference is missing.
State after the fix - this should not be allowed, because 'person' is missing in the `definitions` section:
```
{
"$schema": "http://json-schema.org/draft-07/schema#",
"x-os-lifecycle-state": "published",
"description": "The entity well.",
"title": "Well",
"type": "object",
"definitions": { },
"properties": {
"person": {
"description": "Information for person",
"title": "Information for person",
"$ref": "#/definitions/person"
},
"locationOriginalCRS": {
"description": "The well's original location as AnyCrsFeatureCollection - a structure similar to but distinct from GeoJSON.",
"title": "Original CRS Location",
"$ref": "os:wks:anyCrsFeatureCollection.1.0"
},
"locationWGS84": {
"description": "The well's location as GeoJSON FeatureCollection.",
"title": "WGS84 Location",
"$ref": "https://geojson.org/schema/FeatureCollection.json",
"example": {
"features": [
{
"geometry": {
"coordinates": [
-92.11569999999999,
29.8823,
153.4779442519685
],
"type": "Point"
},
"type": "Feature",
"properties": {
"name": "Newton 2-31"
}
}
],
"type": "FeatureCollection"
}
}
}
}
```https://community.opengroup.org/osdu/platform/system/partition/-/issues/11Upgrade dependencies to resolve security vulnerabilities2022-11-24T12:49:06ZRostislav Vatolinvatolinrp@gmail.comUpgrade dependencies to resolve security vulnerabilitiesPartition service has dependencies with critical bugs and vulnerabilities:
Updating:
guava, because it has security vulnerabilities: [CVE-2020-8908](https://nvd.nist.gov/vuln/detail/CVE-2020-8908)
bom for netty because of: [CVE-2021-2...Partition service has dependencies with critical bugs and vulnerabilities:
Updating:
guava, because it has security vulnerabilities: [CVE-2020-8908](https://nvd.nist.gov/vuln/detail/CVE-2020-8908)
bom for netty because of: [CVE-2021-21290](https://nvd.nist.gov/vuln/detail/CVE-2021-21290) and [CVE-2021-21409](https://nvd.nist.gov/vuln/detail/CVE-2021-21409) and [CVE-2021-21295](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-21295)
Upgrading version of spring: [WS-2016-7107](https://www.whitesourcesoftware.com/vulnerability-database/WS-2016-7107) and [WS-2020-0293](https://www.whitesourcesoftware.com/vulnerability-database/WS-2020-0293) and [CVE-2020-5421](https://nvd.nist.gov/vuln/detail/CVE-2020-5421)
Fixes were applied in common libraries.https://community.opengroup.org/osdu/platform/system/partition/-/issues/10Customized readiness check API2022-11-24T12:48:15ZMingyang ZhuCustomized readiness check APIWe'd like to use spring boot built-in actuator health endpoint for the partition service health check API, and implement a customized health indicator.
Partition service implements the cache layer, different cloud provider's implementat...We'd like to use spring boot built-in actuator health endpoint for the partition service health check API, and implement a customized health indicator.
Partition service implements the cache layer, different cloud provider's implementation implements the cache differently. It will be good to make sure the cache infrastructure and connection are ready before the pod serves the traffic.
To achieve this, the service will implement the custom health indicator, and get a dummy key from the cache instance. No matter it is memcache, redis, it expects no exception to claim the pod is ready to serve the traffic. This health check can be enabled or disabled by the configuration.Mingyang ZhuMingyang Zhuhttps://community.opengroup.org/osdu/platform/system/home/-/issues/79API Documentation -Dataset services2022-11-24T12:47:08ZDebasis ChatterjeeAPI Documentation -Dataset servicesChecked here from Platform. "Dataset" is not mentioned as Core Services.
https://community.opengroup.org/osdu/platform/system/home
And here, it says "To be added".
https://community.opengroup.org/osdu/documentation/-/wikis/Core-Servic...Checked here from Platform. "Dataset" is not mentioned as Core Services.
https://community.opengroup.org/osdu/platform/system/home
And here, it says "To be added".
https://community.opengroup.org/osdu/documentation/-/wikis/Core-Services-Overviewhttps://community.opengroup.org/osdu/platform/system/search-service/-/issues/28Kinds that are not partition-specific are not discoverable via Search using w...2022-11-24T12:44:27ZGary MurphyKinds that are not partition-specific are not discoverable via Search using wildcards<b>Summary</b>
In the context of Schema Service providing a schema and thus a "kind" for a Storage Record, a bug has been found that prevents records from being found via Search wildcard queries on "kind" when the record "kind" does not ...<b>Summary</b>
In the context of Schema Service providing a schema and thus a "kind" for a Storage Record, a bug has been found that prevents records from being found via Search wildcard queries on "kind" when the record "kind" does not start with the data partition name where it is resident. This is important for the very common "kind" queries applications need to perform to find relevant records.<br><br>
<b>Details</b>
Here is a specific scenario that has been observed by SLB testing:
1) records are ingested with two kinds: a) osdu:wks:master-data--Wellbore:1.0.0 and b) partition1:wks:master-data--Wellbore:1.0.0.
2) the above records (1201 count for each kind) can be found in Storage and directly via search when kind queries such as: "kind": osdu:\*:\*:\* and "kind": partition1:\*:\*:\* are used. So the records appear to be indexed and are searchable.
3) When wildcard queries using the first field are used, however, only the partition-based kind records are found. In this case, the query "kind":*:wks:*:* finds *only* the "partition1" records -- 1201 vs. the 2402 records that are expected.https://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/issues/6Boilerplate happening in the Partition interfacing.2022-11-24T12:42:48ZKomal MakkarBoilerplate happening in the Partition interfacing.https://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/blob/master/src/main/java/org/opengroup/osdu/azure/partition/PartitionServiceEventGridClient.java#L15
check how the Partition info can be abstracte...https://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/blob/master/src/main/java/org/opengroup/osdu/azure/partition/PartitionServiceEventGridClient.java#L15
check how the Partition info can be abstracted well. As of now, the boilerplate code is making a problem.Komal MakkarKomal Makkarhttps://community.opengroup.org/osdu/platform/system/schema-service/-/issues/37R3 Schema service - Documentation link is broken2022-11-24T12:39:29ZDebasis ChatterjeeR3 Schema service - Documentation link is brokenStarted from the following link.
`https://community.opengroup.org/osdu/documentation/-/wikis/Core-Services-Overview`
Clicked on "Notes" in Documentation column. The link is broken.
Please check.
And while on the subject, please menti...Started from the following link.
`https://community.opengroup.org/osdu/documentation/-/wikis/Core-Services-Overview`
Clicked on "Notes" in Documentation column. The link is broken.
Please check.
And while on the subject, please mention overlap (if any) with some functions of Storage service.
At a first glance, it appears that some of the functions from Storage may have been deprecated?
Perhaps a clear explanation of the difference/relationship between schema (from Schema-service) and storage schema (from Storage service)?
`POST https://{{domain}}/api/schema-service/v1/schema`
and
`POST https://{{domain}}/api/storage/v2/schemas`
`https://community.opengroup.org/osdu/platform/system/storage/-/blob/master/docs/tutorial/StorageService.md`
Example -
`GET {{schema_api_url}}/schema?limit=10000`
and
`GET /api/storage/v2/query/kinds`