Skip to content

#465 #519 Introduce live/in-runtime config update capabilities for Transformer and Provider through internal Refactoring

Rostislav Dublin (EPAM) requested to merge config-externalization-poc into master

Type of change

  • Bug Fix
  • Feature

Resolves issue #465 (closed)

Does this introduce a change in the core logic?

  • [YES]

Does this introduce a change in the cloud provider implementation, if so which cloud?

  • AWS
  • Azure
  • GCP
  • IBM

Does this introduce a breaking change?

  • [YES]

What is the current behavior?

  • the Transformer app is configurable through the application.properties(yaml) files in deploy time
  • live updates in runtime are not possible, service restart and redeplyment is required
  • so, app reconfinfiguring requires maintance windows and create downtimes
  • the app is not ready for horizontal scaling, only one replica is supported
  • inside the app, scheduled and triggered on bootstrapping or through REST API cache updates may intersect/conflict
  • no Ambassador REST API methods for cache config CRUD operations
  • config files are internal to the app deployment location and cannot survive redeployments

What is the new/expected behavior?

  • KindCacheConfiguration store externalized and integrated with the Transformer app;
  • Implemented KindCacheConfigurations bootstrap from the application.properties to the externalized store;
  • Provided Ambassador API GET, PUT and DELETE methods for KindCache management
  • Implemented CRUD KindCacheConfiguration store items operations in runtime;
  • Implemented applying KindCacheConfiguration changes in runtime, when necessary:
    • Create, Replace and Delete KindCaches in Ignite
    • Trigger complete KindCache updates on sufficient config changes
    • Interrupt running KindCache update processes
    • Schedule, Reschedule and Unschedule KindCaches incremental update tasks
  • The apps scalability improved:
    • Shrink & Grow: Multiple replicas may be added and removed with 0 outage
    • Orchestration: no duplicated task executions, no conflicts among replicas
    • Load-balancing: tasks are fairly balanced among replicas
  • The apps instance internal multiprocessing effectiveness improved:
    • Event-driven command delivery
    • Queue-based command distribution
    • Asynchronous command execution
  • Persistence improved:
    • Implemented optional persistent mode even for the “local” profile
      • Allowed testing app post-restart recovery even locally
      • Even KindCacheConfiguration store made persistent
  • Improved KindCacheUpdate execution due to:
    • Excluded chance of the same KindCache update collisions
      • Queue with deduplication all types of updates (complete and incremental)
      • Execute all types of updates by the same executor pool
      • Switch to the extendable executor pool type, let it shrink and grow
    • Queued update requests supersession:
      • a request for an incremental update is superseded by a new request for a full update
      • a request for a full update is not superseded by any new request
      • an existing request of the same update type is never superseded
  • Improved KindCacheUpdate scheduling:
    • When the app restarts it recreates all the cache update tasks

image

Have you added/updated Unit Tests and Integration Tests?

  • existing tests: YES, all remediated and green
  • new tests: NOT YET

Any other useful information

Closes #520

Edited by Rostislav Dublin (EPAM)

Merge request reports