GC M22 - Core Services - Storage Patching - Final Search is not returning "data" fields

Hello,

I was trying to perform the Storage Patching for the core services and in the final search I'm not getting the "data" fields to confirm my patch. When I search with record I get everything right. I'm sending all my actions for this, but the problem is in the 6. Check the Updated Record Using Search Service

1. Create legal tag

POST https://{{LEGAL_HOST}}/legaltags

curl --location 'https://preship.gcp.gnrg-osdu.projects.epam.com/api/legal/v1/legaltags' \
--header 'Content-Type: application/json' \
--header 'data-partition-id: m19' \
--header 'Authorization: Bearer <Token>' \
--data '{
    "name": "m19-Test-Legal-Tag-2587616",
    "description": "Legal Tag added for Well",
    "properties": {
        "contractId": "123456",
        "countryOfOrigin": [
            "US",
            "CA"
        ],
        "dataType": "Third Party Data",
        "exportClassification": "EAR99",
        "originator": "Schlumberger",
        "personalData": "No Personal Data",
        "securityClassification": "Private",
        "expirationDate": "2025-12-25"
    }
}'

Response

{
    "name": "m19-Test-Legal-Tag-2587616",
    "description": "Legal Tag added for Well",
    "properties": {
        "countryOfOrigin": [
            "US",
            "CA"
        ],
        "contractId": "123456",
        "expirationDate": "2025-12-25",
        "originator": "Schlumberger",
        "dataType": "Third Party Data",
        "securityClassification": "Private",
        "personalData": "No Personal Data",
        "exportClassification": "EAR99"
    }
}

2. Create New Record

Pay Attention!!!

  • In order to get success in this step I needed to replace the variable "record_id" to master-data--well.

  • The "Kind" in the Body request is not defined correctly and when you search the id generated you will get nothing in return. You will see "kind": "{{data-partition-id}}:osdu:{{record_id}}:{{schema_version}}" (makinkg the values explicit you have: "kind": "m19:osdu:master-data--Well:0.2.1"). First I noticed the {{schema_version}} is defined as 0.2.1 and the search does not recognize this version. After run some tests the kind that you have a successfully return on search is {{authority}}:{{schemaSource}}:master-data--Well:1.0.0 (making the values explicit you have: osdu:wks:master-data--Well:1.0.0)

PUT https://{{STORAGE_HOST}}/records/

BODY CURL VERSION

curl --location --request PUT 'https://preship.gcp.gnrg-osdu.projects.epam.com/api/storage/v2/records' \
--header 'data-partition-id: m19' \
--header 'Accept: application/json' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer <Token>' \
--data-raw '[
  {
    "acl": {
      "owners": [
        "data.default.owners@m19.group"
      ],
      "viewers": [
        "data.default.viewers@m19.group"
      ]
    },
    "data": {
        "FacilityID": "Reconcavo-Basin-01182024",
        "FacilityName": "4-EP-2-BA",
        "FacilityTypeID": "m19:reference-data--FacilityType:BA02-3:",
        "Source": "ANP"
      },
    "id": "m19:master-data--Well:18-Jan-2024-Storage-Patch-Test",
    "kind": "osdu:wks:master-data--well:1.0.0",
    "legal": {
      "legaltags": [
        "m19-Test-Legal-Tag-2587616"
      ],
      "otherRelevantDataCountries": [
        "US"
      ],
      "status": "compliant"
    },
    "meta": [
      {}
    ],
    "version": 0
  }
]

'

BODY

[
  {
    "acl": {
      "owners": [
        "{{New_OwnerDataGroup}}@{{data-partition-id}}{{domain}}"
      ],
      "viewers": [
        "{{New_ViewerDataGroup}}@{{data-partition-id}}{{domain}}"
      ]
    },
    "data": {
        "FacilityID": "Reconcavo-Basin-01182024",
        "FacilityName": "4-EP-2-BA",
        "FacilityTypeID": "m19:reference-data--FacilityType:BA02-3:",
        "Source": "ANP"
      },
    "id": "{{data-partition-id}}:master-data--Well:18-Jan-2024-Storage-Patch-Test",
    "kind": "{{authority}}:{{schemaSource}}:master-data--well:1.0.0",
    "legal": {
      "legaltags": [
        "{{LegalTagNameExists}}"
      ],
      "otherRelevantDataCountries": [
        "US"
      ],
      "status": "compliant"
    },
    "meta": [
      {}
    ],
    "version": 0
  }
]

RESPONSE

{
    "recordCount": 1,
    "recordIds": [
        "m19:master-data--Well:18-Jan-2024-Storage-Patch-Test"
    ],
    "skippedRecordIds": [],
    "recordIdVersions": [
        "m19:master-data--Well:18-Jan-2024-Storage-Patch-Test:1705583153907823"
    ]
}

3. Search the Record Using Storage Service

GET https://{{STORAGE_HOST}}/records/m19:master-data--Well:18-Jan-2024-Storage-Patch-Test

BODY CURL VERSION

curl --location 'https://preship.gcp.gnrg-osdu.projects.epam.com/api/storage/v2/records/m19:master-data--Well:18-Jan-2024-Storage-Patch-Test' \
--header 'data-partition-id: m19' \
--header 'Authorization: Bearer <Token>'

RESPONSE

{
    "data": {
        "FacilityID": "Reconcavo-Basin-01182024",
        "FacilityName": "4-EP-2-BA",
        "FacilityTypeID": "m19:reference-data--FacilityType:BA02-3:",
        "Source": "ANP"
    },
    "meta": [
        {}
    ],
    "id": "m19:master-data--Well:18-Jan-2024-Storage-Patch-Test",
    "version": 1705583153907823,
    "kind": "osdu:wks:master-data--well:1.0.0",
    "acl": {
        "viewers": [
            "data.default.viewers@m19.group"
        ],
        "owners": [
            "data.default.owners@m19.group"
        ]
    },
    "legal": {
        "legaltags": [
            "m19-Test-Legal-Tag-2587616"
        ],
        "otherRelevantDataCountries": [
            "US"
        ],
        "status": "compliant"
    },
    "createUser": "preshipping_test_user_m19@gcp.gnrg-osdu.projects.epam.com",
    "createTime": "2024-01-18T13:05:54.538Z"
}

4. PATCH #1 - Replace Operation in the Record

Additional informations to complete this task

  • In this following thest we want to change the value of the field data.FacilityID from "m19:reference-data--FacilityType:BA02-3:" to "REC18012023 - Revised on 18 Jan by JFA for GC" and remove the field Source.
  • If you don't have a PATCH Request you can duplicate the PUT (Store Records) Request and make the necessary changes.
  • Once you have the PATCH Request you need to make a change in the "Headers":
  1. Disable or change the Key "Content-Type" value from "application/json" to "application/json-patch+json";
  2. If the Key "Content-Type" is not available for changes just disable the key with the value "application/json" and create a new one with the name "Content-Type" and the value "application/json-patch+json";
  3. Disable the key "Accept" if its value is "application/json";
  4. Save the request with the changes.

PATCH https://{{STORAGE_HOST}}/records

BODY CURL VERSION

curl --location --request PATCH 'https://preship.gcp.gnrg-osdu.projects.epam.com/api/storage/v2/records' \
--header 'data-partition-id: m19' \
--header 'Content-Type: application/json-patch+json' \
--header 'Authorization: Bearer <Token>' \
--data '{
    "query": {
        "ids": ["m19:master-data--Well:18-Jan-2024-Storage-Patch-Test"]
    },
    "ops": [
      {
        "op": "replace",
        "path": "/data/FacilityID",
        "value": "REC18012023 - Revised on 18 Jan by JFA for GC"
      },
      { 
        "op": "remove", 
        "path": "/data/Source"
      }
    ]
}'

BODY

{
    "query": {
        "ids": ["m19:master-data--Well:18-Jan-2024-Storage-Patch-Test"]
    },
    "ops": [
      {
        "op": "replace",
        "path": "/data/FacilityID",
        "value": "REC18012023 - Revised on 18 Jan by JFA for GC"
      },
      { 
        "op": "remove", 
        "path": "/data/Source"
      }
    ]
}

RESPONSE

{
    "recordCount": 1,
    "recordIds": [
        "m19:master-data--Well:18-Jan-2024-Storage-Patch-Test:1705589468017161"
    ],
    "notFoundRecordIds": [],
    "failedRecordIds": [],
    "errors": []
}

5. Check the Updated Record Using Storage Service

GET https://{{STORAGE_HOST}}/records/m19:master-data--Well:18-Jan-2024-Storage-Patch-Test

BODY

curl --location 'https://preship.gcp.gnrg-osdu.projects.epam.com/api/storage/v2/records/m19:master-data--Well:18-Jan-2024-Storage-Patch-Test' \
--header 'data-partition-id: m19' \
--header 'Authorization: Bearer <Token>'

RESPONSE

{
    "data": {
        "FacilityID": "REC18012023 - Revised on 18 Jan by JFA for GC",
        "FacilityName": "4-EP-2-BA",
        "FacilityTypeID": "m19:reference-data--FacilityType:BA02-3:"
    },
    "meta": [
        {}
    ],
    "modifyUser": "preshipping_test_user_m19@gcp.gnrg-osdu.projects.epam.com",
    "modifyTime": "2024-01-18T14:51:08.115Z",
    "id": "m19:master-data--Well:18-Jan-2024-Storage-Patch-Test",
    "version": 1705589468017161,
    "kind": "osdu:wks:master-data--well:1.0.0",
    "acl": {
        "viewers": [
            "data.default.viewers@m19.group"
        ],
        "owners": [
            "data.default.owners@m19.group"
        ]
    },
    "legal": {
        "legaltags": [
            "m19-Test-Legal-Tag-2587616"
        ],
        "otherRelevantDataCountries": [
            "US"
        ],
        "status": "compliant"
    },
    "createUser": "preshipping_test_user_m19@gcp.gnrg-osdu.projects.epam.com",
    "createTime": "2024-01-18T13:05:54.538Z"
}

6. Check the Updated Record Using Search Service

POST https://{{SEARCH_HOST}}/query

BODY

curl --location 'https://preship.gcp.gnrg-osdu.projects.epam.com/api/search/v2/query' \
--header 'Authorization: <Token>' \
--header 'data-partition-id: m19' \
--header 'Content-Type: application/json' \
--data '{
   "kind": "osdu:wks:master-data--Well:1.0.0",
   "query": "id:\"m19:master-data--Well:18-Jan-2024-Storage-Patch-Test\""
}'

RESPONSE

{
    "results": [
        {
            "kind": "osdu:wks:master-data--well:1.0.0",
            "source": "wks",
            "acl": {
                "viewers": [
                    "data.default.viewers@m19.group"
                ],
                "owners": [
                    "data.default.owners@m19.group"
                ]
            },
            "type": "master-data--well",
            "version": 1705589468017161,
            "tags": {
                "normalizedKind": "osdu:wks:master-data--well:1"
            },
            "modifyUser": "preshipping_test_user_m19@gcp.gnrg-osdu.projects.epam.com",
            "modifyTime": "2024-01-18T14:51:08.115Z",
            "createTime": "2024-01-18T13:05:54.538Z",
            "authority": "osdu",
            "namespace": "osdu:wks",
            "legal": {
                "legaltags": [
                    "m19-Test-Legal-Tag-2587616"
                ],
                "otherRelevantDataCountries": [
                    "US"
                ],
                "status": "compliant"
            },
            "createUser": "preshipping_test_user_m19@gcp.gnrg-osdu.projects.epam.com",
            "id": "m19:master-data--Well:18-Jan-2024-Storage-Patch-Test"
        }
    ],
    "aggregations": null,
    "phraseSuggestions": [],
    "totalCount": 1
}

Regards,

Juliana Fernandes