Skip to content

#539 Implement Reset to Initial/New Config" API and Services for Transformer

Rostislav Dublin (EPAM) requested to merge 539-reset-to-initial-or-custom-config into 536-cron-validator-and-more

#539 (closed) Implement "Reset to Initial/New Config" API and Services for Transformer

Type of change

  • Feature

Please provide link to gitlab issue or ADR(Architecture Decision Record)
Closes #539 (closed)

Does this introduce a change in the core logic?

  • YES

Does this introduce a change in the cloud provider implementation, if so which cloud?

  • AWS
  • Azure
  • GCP
  • IBM

Does this introduce a breaking change?

  • NO

What is the current behavior?

  • The Transformer service does not support resetting configurations automatically. Admins must manually update the configurations and caches without the ability to revert to the initial setup or apply a new configuration.

What is the new/expected behavior?

  • The service now supports resetting to the initial configuration or applying a new custom configuration via the /admin/transformerConfig/reset API.
  • The default behavior resets the configuration to its original state as used during the first start.
  • Optionally, admins can provide a new configuration file in JSON, YAML, or properties format, which will be validated and applied.
  • Kind cache configurations and transformer configuration are reset within a single transaction.
  • Logging and validation mechanisms are in place to handle both successful and failed reset operations.
  • Swagger and OpenAPI documentation now includes the new method /admin/transformerConfig/reset, which accepts one of three supported configuration data formats (JSON, YAML, properties) or, if no configuration is provided, reverts the service to the initial configuration.

Have you added/updated Unit Tests and Integration Tests?

  • Unit tests cover the default reset scenario and applying a custom configuration. Transactional behavior and failure cases are also tested.

Testing scenarios:

Example 1: POST Request (JSON, Initial Reset)

curl -L -X POST 'http://localhost:8080/gcz/transformer/admin/transformerConfig/reset' \
-H 'data-partition-id: opendes' \
-H 'Authorization: Bearer <your_bearer_token>' \
-d ''

What to test:

  • Test the default reset behavior.
  • Ensure all kinds are updated based on the projected configuration, and any outdated kinds are removed.
  • Verify the transformer configuration is reset.

Example 2: POST Request (YAML, Custom Reset)

curl -L 'http://localhost:8080/gcz/transformer/admin/transformerConfig/reset' \
-H 'data-partition-id: opendes' \
-H 'Content-Type: application/x-yaml' \
-H 'Authorization: Bearer <your_bearer_token>' \
-d 'application:
  kinds:
  - {chronExpr: 0 0 8 * * SAT, cacheName: Well_1.0.0, name: '\''osdu:wks:master-data--Well:1.0.0'\''}
  - {chronExpr: 0 0 8 * * SAT, cacheName: Well_1.2.0, name: '\''osdu:wks:master-data--Well:1.2.0'\''}
  - {chronExpr: 0 0 8 * * SAT, cacheName: Well_1.1.0, name: '\''osdu:wks:master-data--Well:1.1.0'\''}
  - {chronExpr: 0 0 9 * * SAT, cacheName: Wellbore_1.0.0, name: '\''osdu:wks:master-data--Wellbore:1.0.0'\''}
  dataPartitionId: opendes
  loadInBatch: true
  batchSize: 2345
  supportedDataTypes: master-data--Well,master-data--Wellbore,master-data--SeismicAcquisitionSurvey'

What to test:

  • Send a custom YAML configuration for kind cache and transformer reset.
  • Verify that all specified kinds are upserted and that the new transformer configuration is applied.

Example 3: POST Request (JSON, Custom Reset)

curl -L 'http://localhost:8080/gcz/transformer/admin/transformerConfig/reset' \
-H 'data-partition-id: opendes' \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer <your_bearer_token>' \
-d '{
  "application": {
    "dataPartitionId": "opendes",
    "batchSize": 1234,
    "supportedDataTypes": [
      "master-data--Well",
      "master-data--Wellbore",
      "master-data--SeismicAcquisitionSurvey"
    ],
    "kinds": [
      {
        "chronExpr": "0 0 8 * * SAT",
        "cacheName": "Well_1.0.0",
        "name": "osdu:wks:master-data--Well:1.0.0"
      }
    ]
  }
}'

What to test:

  • Send a JSON-based custom configuration.
  • Test the insertion and update of the kinds and transformer configurations based on the given input.

Example 4: POST Request (Properties Format)

curl -L 'http://localhost:8080/gcz/transformer/admin/transformerConfig/reset' \
-H 'data-partition-id: opendes' \
-H 'Content-Type: text/plain' \
-H 'Authorization: Bearer <your_bearer_token>' \
-d 'application.batchSize=4321
application.dataPartitionId=opendes
application.supportedDataTypes[0]=master-data--Well
application.supportedDataTypes[1]=master-data--Wellbore
application.supportedDataTypes[2]=master-data--SeismicAcquisitionSurvey
application.kinds[0].name=osdu:wks:master-data--Well:1.0.0
application.kinds[0].cacheName=Well
application.kinds[0].cronExpr="0 0 8 * * SAT"'

What to test:

  • Send a configuration in properties format.
  • Verify the reset process using this format and ensure kinds and transformer configurations are processed correctly.

Any other useful information

  • This feature introduces the ability to reset both the kind cache configurations and transformer configuration in one atomic transaction, ensuring data consistency.
  • The reset operation is validated, and proper logs are generated for both successful and failed operations.
  • Supported input formats: JSON, YAML, and properties.

Merge request reports