Skip to content

(GONRG-3831) GCP Notification: OQM mapper

Rostislav Dublin (EPAM) requested to merge gcp-oqm into master

Type of change

  • Bug Fix
  • Feature

Does this introduce a change in the core logic?

  • [NO]

Does this introduce a change in the cloud provider implementation, if so which cloud?

  • AWS
  • Azure
  • GCP
  • IBM

Does this introduce a breaking change?

  • [YES/NO]

What is the current behavior?

Notification works with Message Broker directly

What is the new/expected behavior?

Notification will work with MQ through mapper/driver abstraction

Have you added/updated Unit Tests and Integration Tests?

yes

Any other useful information

Features of implementation

This is a universal solution created using EPAM OQM mapper technology. It allows you to work with various implementations of message brokers.

Limitations of the current version

In the current version, the mapper is equipped with several drivers to the message brokers:

  • Google PubSub;
  • RabbitMQ

Extensibility

To use any other message broker, implement a driver for it. With an extensible set of drivers, the solution is unrestrictedly universal and portable without modification to the main code.

Mapper tuning mechanisms

This service uses specific implementations of DestinationResolvers based on the tenant information provided by the OSDU Partition service. A total of 2 resolvers are implemented, which are divided into two groups:

for universal technologies:
  • for RabbitMQ: mappers/oqm/MqTenantOqmDestinationResolver.java
Their algorithms are as follows:
  • incoming Destination carries data-partition-id
  • resolver accesses the Partition service and gets PartitionInfo
  • from PartitionInfo resolver retrieves properties for the connection: URL, username, password etc.
  • resolver creates a data source, connects to the resource, remembers the datasource
  • resolver gives the datasource to the mapper in the Resolution object
for native Google Cloud technologies:
  • for PubSub: mappers/oqm/PsTenantOqmDestinationResolver.java
Their algorithms are similar,

Except that they do not receive special properties from the Partition service for connection, because the location of the resources is unambiguously known - they are in the GCP project. And credentials are also not needed - access to data is made on behalf of the Google Identity SA under which the service itself is launched. Therefore, resolver takes only the value of the projectId property from PartitionInfo and uses it to connect to a resource in the corresponding GCP project.

Edited by Rostislav Dublin (EPAM)

Merge request reports