Documentation issueshttps://community.opengroup.org/osdu/documentation/-/issues2023-06-06T20:29:36Zhttps://community.opengroup.org/osdu/documentation/-/issues/94Core services overview page needs to be updated to reflect the latest API doc2023-06-06T20:29:36ZChad LeongCore services overview page needs to be updated to reflect the latest API docCore services overview [page](https://community.opengroup.org/osdu/documentation/-/wikis/Core-Services-Overview) needs to be updated to reflect the latest API doc as per https://community.opengroup.org/osdu/platform/pre-shipping/-/issues...Core services overview [page](https://community.opengroup.org/osdu/documentation/-/wikis/Core-Services-Overview) needs to be updated to reflect the latest API doc as per https://community.opengroup.org/osdu/platform/pre-shipping/-/issues/480 .
We need to update to maintain both static and dynamic page.Debasis ChatterjeeChad LeongDebasis Chatterjeehttps://community.opengroup.org/osdu/documentation/-/issues/93flip order of CRS Convert and Catlog2023-05-22T22:03:25ZBert Kampesflip order of CRS Convert and Catlog@chad, a few comments. Very minor, but you may be able to make the improvements :
[ ] for the CRS services it makes sense to first list Catalog, then Convert. At the same time, the swagger calls it "CRS Convert" and this page "Conver...@chad, a few comments. Very minor, but you may be able to make the improvements :
[ ] for the CRS services it makes sense to first list Catalog, then Convert. At the same time, the swagger calls it "CRS Convert" and this page "Conversion". I would change it to "CRS Convert" on this page. Perhaps you just want to close this issue without action, but if it is not a trouble when you are editing the page, I suggest it is better to flip the rows in the table around.
[ ] For the Geospatial Consumption Zone, it says "code repo". You may want to change this to "Geospatial Consumption Zone" to be consistent with others.
[ ] perhaps more important, the CRS Convert and Catolog services v2 are deprecated for use. The swagger pages/docs don't really seem to indicated that well. Would it be possible to remove v2 or add "(deprecated for use)" after the services? The thing to remember perhaps is that particularly for the CRS Catalog service it is a complete break with the past.https://community.opengroup.org/osdu/documentation/-/issues/92Create Marker - Unclear error message2022-04-05T08:37:53ZDebasis ChatterjeeCreate Marker - Unclear error message/ddms/v3/wellboremarkersets
Agreed, there may be problem in JSON payload. But the following error message does not give much clue.
(I am using sample collection as provided by AWS Support. cc @fhoueto.amz)
```
{
"detail": [
.../ddms/v3/wellboremarkersets
Agreed, there may be problem in JSON payload. But the following error message does not give much clue.
(I am using sample collection as provided by AWS Support. cc @fhoueto.amz)
```
{
"detail": [
{
"loc": [
"body"
],
"msg": "value is not a valid list",
"type": "type_error.list"
}
]
}
```
See the payload here.
[AWS-R3M10-Patch-Wellbore-DDMS-Marker-error.txt](/uploads/cd3bd9b390bf368a1db94229c3749202/AWS-R3M10-Patch-Wellbore-DDMS-Marker-error.txt)https://community.opengroup.org/osdu/documentation/-/issues/91Workflow service - Swagger style documentation2023-05-22T22:05:14ZDebasis ChatterjeeWorkflow service - Swagger style documentationPlease check this site
https://community.opengroup.org/osdu/documentation/-/wikis/Core-Services-Overview
It is missing documentation for common services such as -
POST {{osduonaws_base_url}}/api/workflow/v1/workflow/Osdu_ingest/workflo...Please check this site
https://community.opengroup.org/osdu/documentation/-/wikis/Core-Services-Overview
It is missing documentation for common services such as -
POST {{osduonaws_base_url}}/api/workflow/v1/workflow/Osdu_ingest/workflowRun
AWS - osduonaws_base_url = https://r3m9.preshiptesting.osdu.aws
GET https://{{WORKFLOW_HOST}}/info
GCP - WORKFLOW_HOST = preship.gcp.gnrg-osdu.projects.epam.com/api/workflow
cc - @ChrisZhang and @npickus for informationhttps://community.opengroup.org/osdu/documentation/-/issues/90Storage service documentation - Adding information on records insert limit [D...2021-10-26T10:09:35ZNaufal Mohamed NooriStorage service documentation - Adding information on records insert limit [Document Enhancement]Please see the following link:
https://community.opengroup.org/osdu/platform/system/storage/-/blob/master/docs/tutorial/StorageService.md
We dont have any information related to the limit of record insertion using storage service. I ha...Please see the following link:
https://community.opengroup.org/osdu/platform/system/storage/-/blob/master/docs/tutorial/StorageService.md
We dont have any information related to the limit of record insertion using storage service. I have tried from my end and seeing this message:
PUT: BASE_URL/api/storage/v2/records
```
{
"code": 400,
"reason": "Validation error.",
"message": "createOrUpdateRecords.records: Up to 500 records can be ingested at a time"
}
```
It will be beneficial if the documentation shows this limit.
cc: @debasischttps://community.opengroup.org/osdu/documentation/-/issues/89R3M7 - Authentication information2021-08-15T17:52:24ZDebasis ChatterjeeR3M7 - Authentication informationHi Aliaksandr,
I am checking this site. URL includes R2, but I presume that it is valid for R3. Right?
https://community.opengroup.org/osdu/documentation/-/wikis/Releases/R2.0/GCP/GCP-Pre-Prod-Onboarding-Documentation
I also have an e...Hi Aliaksandr,
I am checking this site. URL includes R2, but I presume that it is valid for R3. Right?
https://community.opengroup.org/osdu/documentation/-/wikis/Releases/R2.0/GCP/GCP-Pre-Prod-Onboarding-Documentation
I also have an earlier email from Maksimelyan.
My questions –
1. Username – presumably use my existing username (debasis_chatterjee@osdu-gcp.go3-nrg.projects.epam.com) that I used prior to M7. Right?
2. Instruction says – client_secret will be sent by separate email. Should I use what was provided to me prior to M7?
3. Where is clear step by step instruction as to how one should authenticate in Postman?
4. We also need separate Smoke test collection for running standard Core services API (ex: Search, Legal, Entitlements..).
5. Can you confirm if my username is privileged to perform Seismic DDMS steps?
Please check and advise.
Thank you
Debasis
cc - @maksimelyan_tamashevichhttps://community.opengroup.org/osdu/documentation/-/issues/88Show "permissions required" for all services in documentation in consistent m...2021-06-28T21:11:49ZDebasis ChatterjeeShow "permissions required" for all services in documentation in consistent mannerExample from Search service -
https://community.opengroup.org/osdu/platform/system/search-service/-/blob/master/docs/tutorial/SearchService.md
Scroll at the very end. It shows you that minimum permissions required is "users.osdu.viewer...Example from Search service -
https://community.opengroup.org/osdu/platform/system/search-service/-/blob/master/docs/tutorial/SearchService.md
Scroll at the very end. It shows you that minimum permissions required is "users.osdu.viewers".
Else, when someone gets 401 (Unauthorized) error from a service, he/she does not have much clue to figure out which role/group is missed from his/her user access credentials.
GET {{osduonaws_base_url}}/api/entitlements/v1/groupshttps://community.opengroup.org/osdu/documentation/-/issues/87R3 - Unit Service documentation2021-06-29T05:32:00ZDebasis ChatterjeeR3 - Unit Service documentationCan you please provide some high level overview, by using simple example of unit conversion?
Such as feet -> meters.
Temperature is also good candidate as this will show sue of multiplier and added value.
Such as from centigrade and fa...Can you please provide some high level overview, by using simple example of unit conversion?
Such as feet -> meters.
Temperature is also good candidate as this will show sue of multiplier and added value.
Such as from centigrade and farenheit.
As far as I see now, the documentation lists the API calls with brief explanation of each.https://community.opengroup.org/osdu/documentation/-/issues/86Allowed kind in storage create storage api2021-06-28T21:10:29ZRupali KombadeAllowed kind in storage create storage apiWhere can I get information about allowed kind in storage create schema API. if I want to specify 'array of objects' how to specify it?
Consider the data format is as below:
data {
name: "area name",
country: "xyz",
wells:[
name: "we...Where can I get information about allowed kind in storage create schema API. if I want to specify 'array of objects' how to specify it?
Consider the data format is as below:
data {
name: "area name",
country: "xyz",
wells:[
name: "well1",
type: "planning",
coordinates: {
latitute:12345,
longitude: 1235
}
]
}
to make above data searchable I need to connect it with storage schema. When I tried to create schema using below service, its giving me error that - **Schema item 'wells' has an invalid data type 'object'**, cay you please suggest how to add array or list in schema kind for wells array?
api/storage/v2/schemas
{
"kind": "common:wks:welldata:1.0.0",
"schema": [
{
"path": "name",
"kind": "string"
},
{
"path": "country",
"kind": "string"
},
{
"path": "wells",
"kind": "object"
}
]
}https://community.opengroup.org/osdu/documentation/-/issues/85Storage service documentation - using the term "dataset-name" for explanation...2021-06-28T21:08:24ZDebasis ChatterjeeStorage service documentation - using the term "dataset-name" for explanation of "kind"Please see this link
https://community.opengroup.org/osdu/platform/system/storage/-/blob/master/docs/tutorial/StorageService.md
Explains a few basic concepts like "id", "kind".
**kind**: *(mandatory)* Kind of data being ingested. Must...Please see this link
https://community.opengroup.org/osdu/platform/system/storage/-/blob/master/docs/tutorial/StorageService.md
Explains a few basic concepts like "id", "kind".
**kind**: *(mandatory)* Kind of data being ingested. Must follow the naming convention: `{Data-Partition-Id}:{dataset-name}:{record-type}:{version}`.
The problem is using the term "**dataset**" in second component of "kind".
As you know, right now the term "dataset" is used in a different way (Data Definition such as for File.Generic, File.Collection and also new Services).
Can you plan to make suitable corrections?
Sample "kind" from json files in wiki of Data Loading team.
`"kind": "osdu:wks:work-product-component--WellLog:1.0.0",`
Thank youhttps://community.opengroup.org/osdu/documentation/-/issues/84Ingestion Workflow - provide an option to run integrity test on previous load...2021-06-28T22:14:54ZDebasis ChatterjeeIngestion Workflow - provide an option to run integrity test on previous loaded data (with integrity test option disabled)This option may allow the Data Manager/Loader to run integrity test on previous loaded data (a set of records), and result may show which ones are still invalid.
For the valid ones (ex: reference check is now valid as suitable "referen...This option may allow the Data Manager/Loader to run integrity test on previous loaded data (a set of records), and result may show which ones are still invalid.
For the valid ones (ex: reference check is now valid as suitable "referenced data" now exists), the system property can be reset as the record is no longer invalid from integrity perspective.
Please refer to this issue.
https://gitlab.opengroup.org/osdu/subcommittees/data-def/work-products/schema/-/issues/203https://community.opengroup.org/osdu/documentation/-/issues/83Ingestion workflow - provide an option for "Dry run" simply for validation ch...2021-06-28T22:13:19ZDebasis ChatterjeeIngestion workflow - provide an option for "Dry run" simply for validation checksThis option can be helpful to alert the Data Loader of issues in input data, simply from integrity tests.
Then he/she can take corrective action up front and then attempt to actually load the large data set.This option can be helpful to alert the Data Loader of issues in input data, simply from integrity tests.
Then he/she can take corrective action up front and then attempt to actually load the large data set.https://community.opengroup.org/osdu/documentation/-/issues/82Ingestion workflow - provide option to enable, disable validation checks (ref...2021-03-19T11:38:38ZDebasis ChatterjeeIngestion workflow - provide option to enable, disable validation checks (referenced information)Starting with recent version of "Ingestion Workflow", we see that integrity checks are enabled. This is very useful.
However, please provide option of setting true/false for "integrity checks" to the end user (Data Loader) who may or ma...Starting with recent version of "Ingestion Workflow", we see that integrity checks are enabled. This is very useful.
However, please provide option of setting true/false for "integrity checks" to the end user (Data Loader) who may or may not have required skills to tweak python code inside the DAG. Default can be left as "True".
Excerpt from Airflow Log (GCP environment) is shown below for quick reference -
```
[2021-03-18 17:45:08,504] {base_task_runner.py:113} INFO - Job 27967: Subtask provide_manifest_integrity_task [2021-03-18 17:45:08,503]
{validate_referential_integrity.py:156} DEBUG - Extracted reference ids:
['osdu:reference-data--AliasNameType:WELL_NAME',
'osdu:reference-data--VerticalMeasurementPath:DEPTH_DATUM_ELEV',
'osdu:reference-data--ResourceSecurityClassification:Public',
'osdu:reference-data--FacilityEventType:SPUD_DATE',
'osdu:reference-data--FacilityType:WELLBLABLA',
'osdu:master-data--Organisation:HESS']
```
In this example, all checks failed as the environment lacked standard Reference values at the time of this run. Else, I would only expect one reference check to fail (FacilityType = "WELLBLABLA" instead of "WELL").
```
[2021-03-18 17:45:44,405] {base_task_runner.py:113} INFO - Job 27967: Subtask provide_manifest_integrity_task [2021-03-18 17:45:44,405]
{validate_referential_integrity.py:177} WARNING - The next ids are absent in the system:
['osdu:reference-data--FacilityType:WELLBLABLA',
'osdu:reference-data--FacilityEventType:SPUD_DATE',
'osdu:reference-data--ResourceSecurityClassification:Public',
'osdu:reference-data--VerticalMeasurementPath:DEPTH_DATUM_ELEV',
'osdu:master-data--Organisation:HESS',
'osdu:reference-data--AliasNameType:WELL_NAME']
[2021-03-18 17:45:44,413] {base_task_runner.py:113} INFO - Job 27967: Subtask provide_manifest_integrity_task [2021-03-18 17:45:44,411]
{validate_referential_integrity.py:231} WARNING - Resource with kind odesprod:wks:master-data--Well:1.0.0 was rejected
```https://community.opengroup.org/osdu/documentation/-/issues/81R3 Entitlement Service - Update content to OSDU context, fix broken link2021-06-28T22:10:58ZDebasis ChatterjeeR3 Entitlement Service - Update content to OSDU context, fix broken linkThe content seems very Schlumberger-oriented. Please change to general context, and in line with upcoming Admin UI.
The link below is broken.
`All the API's can be found [here](/osdu/documentation/-/wikis/solutions/dataecosystem/apis)`...The content seems very Schlumberger-oriented. Please change to general context, and in line with upcoming Admin UI.
The link below is broken.
`All the API's can be found [here](/osdu/documentation/-/wikis/solutions/dataecosystem/apis)`
Also provide link to Swagger documentation as in other services.
Also, I did not see clear documentation about any service to "get details of a group".https://community.opengroup.org/osdu/documentation/-/issues/80Capture "Processing application" as a new field over and above usual audit tr...2021-07-06T15:30:37ZDebasis ChatterjeeCapture "Processing application" as a new field over and above usual audit trail fields (created-by, created-on-date)As we start to see many alternatives for loading data such as "Ingestion workflow", "CSV Parser", "WITSML parser", "LAS parser", this will be useful addition. First we need suitable field common for Work-product-component, Master data, R...As we start to see many alternatives for loading data such as "Ingestion workflow", "CSV Parser", "WITSML parser", "LAS parser", this will be useful addition. First we need suitable field common for Work-product-component, Master data, Reference data.
This information will help us to troubleshoot if we see any problem with ingested data.
Copying to @alan.henson and @james.pipe for awareness.https://community.opengroup.org/osdu/documentation/-/issues/79R3 API documentation - Storage service2021-06-28T21:00:52ZDebasis ChatterjeeR3 API documentation - Storage service![R3-core-Storage-service](/uploads/771723f9250969752e7386e773b01e1c/R3-core-Storage-service.PNG)
Start with this page -
`https://community.opengroup.org/osdu/documentation/-/wikis/Core-Services-Overview`
Click on "How to" for "Storage...![R3-core-Storage-service](/uploads/771723f9250969752e7386e773b01e1c/R3-core-Storage-service.PNG)
Start with this page -
`https://community.opengroup.org/osdu/documentation/-/wikis/Core-Services-Overview`
Click on "How to" for "Storage service". Takes us to
`https://community.opengroup.org/osdu/platform/system/storage/-/blob/master/docs/tutorial/StorageService.md`
Please provide some recommendation about "addition of record into Document store" - as to end users may now leverage "Ingestion workflow" (DAG based approach) for creation of Master data, Reference data and also the chain wp-wpc-dataset/file.
I refer to this -
`https://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/blob/master/README.md#post-v1workflowworkflow_nameworkflowrun`
Also provide similar clarification regarding apparent overlap between "Schema" features documented here with the new Schema service (yet to be documented properly).
In section on "Partition", this mentions DELFI. And also here is an opportunity to provide a good lead to the new "Partition service".https://community.opengroup.org/osdu/documentation/-/issues/78R3 Ingestion - alternatives2021-06-28T21:40:35ZDebasis ChatterjeeR3 Ingestion - alternativesAs per recent discussion with @alan.henson , I would like to request a clear statement in this page stating which alternative is recommended and will be fully supported by 4 CSPs at the time of R3 release.
![R3-Ingestion-alternatives](/...As per recent discussion with @alan.henson , I would like to request a clear statement in this page stating which alternative is recommended and will be fully supported by 4 CSPs at the time of R3 release.
![R3-Ingestion-alternatives](/uploads/6149088d7608bb90e404abe47a3f2482/R3-Ingestion-alternatives.PNG)
The following wiki page mentioned two entries under "Ingestion".
`https://community.opengroup.org/osdu/documentation/-/wikis/Core-Services-Overview`
Thank youhttps://community.opengroup.org/osdu/documentation/-/issues/77R3 File service - "Notes" link is broken2021-06-28T20:59:20ZDebasis ChatterjeeR3 File service - "Notes" link is broken![Core-service-File](/uploads/1d5ed33d77b8300ad9f1ef8caabeed0e/Core-service-File.PNG)
Please check this link and fix as needed.
Thank you![Core-service-File](/uploads/1d5ed33d77b8300ad9f1ef8caabeed0e/Core-service-File.PNG)
Please check this link and fix as needed.
Thank youhttps://community.opengroup.org/osdu/documentation/-/issues/76R3 Search Service - typo error for message when resource does not exist2021-06-29T15:19:44ZDebasis ChatterjeeR3 Search Service - typo error for message when resource does not existSee this message -
POST {{base_url}}/api/search/v2/query
Body
```
{
"kind": "opendes:wks:master-data--wellbore:1.0.0",
"query": "*"
}
```
Response -
```
{
"code": 404,
"reason": "Not Found",
"message": "Resource yo...See this message -
POST {{base_url}}/api/search/v2/query
Body
```
{
"kind": "opendes:wks:master-data--wellbore:1.0.0",
"query": "*"
}
```
Response -
```
{
"code": 404,
"reason": "Not Found",
"message": "Resource you are trying to find does not exists"
}
```
Also note that the error message is the same when user mistypes entity name (ex: Wellbore1 instead of Wellbore) and when entity is empty i.e. no data loaded yet.
I have earlier verified using Schema service that this entity exists -
```
{
"schemaIdentity": {
"authority": "opendes",
"source": "wks",
"entityType": "master-data--Wellbore",
"schemaVersionMajor": 1,
"schemaVersionMinor": 0,
"schemaVersionPatch": 0,
"id": "opendes:wks:master-data--Wellbore:1.0.0"
},
"createdBy": "e09bb2a5-337f-48a7-8f11-13f939b8a6ee",
"dateCreated": "2021-02-18T03:53:57.738+00:00",
"status": "DEVELOPMENT",
"scope": "SHARED"
},
```https://community.opengroup.org/osdu/documentation/-/issues/75R3 Storage service - Documentation - explanation of "kind"2021-02-26T22:38:17ZDebasis ChatterjeeR3 Storage service - Documentation - explanation of "kind"Please check the following.
`https://community.opengroup.org/osdu/platform/system/storage/-/blob/master/docs/tutorial/StorageService.md#create-schema`
**kind**: *(mandatory)* Kind of data being ingested. Must follow the naming conventi...Please check the following.
`https://community.opengroup.org/osdu/platform/system/storage/-/blob/master/docs/tutorial/StorageService.md#create-schema`
**kind**: *(mandatory)* Kind of data being ingested. Must follow the naming convention: `{Data-Partition-Id}:{dataset-name}:{record-type}:{version}`.
Suggest use "schema" rather than "dataset" for the second part. In R3, the term "Dataset" is used separately to refer to File, Collection of files etc.
See example of "kind" from GCP R3-M3 environment.
> "kind": "odesprod:wks:work-product-component--WellboreMarkerSet:1.0.0",