Does this introduce a change in the cloud provider implementation, if so which cloud?
AWS
Azure
Google Cloud
IBM
Does this introduce a breaking change?
[YES]
What is the new/expected behavior?
Third-party subscriber subscribes/unsubscribes for specific OSDU events notifications via the Register service REST API
Register service publishes registration/unregistration info via the register-subscriber-control topic/exchange
Notification service instances process single (statically defined) subscription/queue to obtain registration info changes
Notification service instances persists/reads registration info in the shared database/cache.
OSDU services (Storage, Schema etc.) publish their events through corresponding topics/exchanges (records-changed, schema-changed etc.)
Notification service instances concurrently process single subscription/queue, statically defined per each original topic/exchange
Notification service publishes N (N is the number of third-party subscribtions) outgoing messages via the corresponding *-notification topic/exchange (example: records-changed-notification for records changed events). Each outgoing event is the original event enriched with the destination information (subscriber id). The original event is properly acknowledged upon end of processing.
RabbitMQ: *-notification exchange should be a delayed exchange (type x-delayed-message), which requires special RabbitMQ rabbitmq_delayed_message_exchange plugin installed
RabbitMQ: the outgoing event contains no x-delay header to be processed w/o time delay
Notification service instances concurrently process single subscription/queue, statically defined per each outgoing topic/exchange
Notification service collects corresponding subscription info from the DB (4) or distributed cache and sends notification (HTTP call to the endpoint) to the third-party subscriber. In case of successful call the outgoing message is properly acknowledged. In case of notification failure (third-party endpoint not available for instance) the following should be performed to provide notification delivery retry logic:
GCP PubSub: the outgoing message is NACK-ed and then re-delivered after configured back-off time in accordance with Retry policy
RabbitMQ: the outgoing message is ACK-ed and re-published (7) in the the corresponding *-notification exchange with x-delay and x-retries headers to be re-processed with time delay. If x-retries already present in the processed message header it's value should be incremented. If x-retries value equals to configured limit of retries, the message is not re-published. In further implementations it may be routed to dead letter queue (out of scope for now)
