Commit cbf398f0 authored by Hema Vishnu Pola [Microsoft]'s avatar Hema Vishnu Pola [Microsoft]
Browse files

Merge branch 'users/komakkar/SOPTopic' into 'master'

Adding playbook for Topic Creation and Consumption

See merge request !62
parents b4c2c413 4f29e942
Pipeline #32637 failed with stages
in 23 minutes and 21 seconds
# SOPs for Creating a new Topic and Adding subscriber for it.
### Introduction
Notification service is invoked by PubSub called [Event Grid](https://docs.microsoft.com/en-us/azure/event-grid/).
This document describes how to create a topic, publish to it and listen to it.
### How to create the Topic and start using it.
#### 1. Create Event Grid Topic.
Following this will enable creation of topic in all data partitions.
1. Create a Topic. [Reference](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/blob/master/infra/templates/osdu-r3-mvp/data_partition/main.tf#L301)
2. Assign the Service Principals of service the role "EventGrid EventSubscription Contributor". [Reference](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/merge_requests/32)
3. Identify the fully qualified topic name and it's alias. The fully qualified topic name is the one you see on the Azure portal. An example is "osdu-env-dp1env-as12-grid-recordstopic". The alias could be recordstopic. Make sure the alias ends with topic. Invalid topic names will be "testtopic1", "test_topic", "testtopics" etc.
4. Create secrets in Common Resources Key Vault that helps your service identify the event grid topic end point. [Reference](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/merge_requests/30/diffs)
Make sure you name them in the format "eventgrid-\<alias\>" and "eventgrid-\<alias\>-accesskey". For example, if the topic alias is "testtopic", the keys should look like "eventgrid-testtopic-accesskey" and "eventgrid-testtopic".
#### 2. Make partition service aware of the new topic.
Partition service maintains the configuration for each infrastructure component.
For all the tenants, add the new topic information. [Reference](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/merge_requests/101)
This is to be done per partition per deployment.
#### 3. Make Register service aware of the new Topic
1. Add the topic alias to [topics.json](https://community.opengroup.org/osdu/platform/system/register/-/blob/master/provider/register-azure/src/main/resources/topics.json). Make sure you fill in the values of description, state and examples to ensure that the get Topic API contract is not broken.
#### 4. Publish to Event Grid Topic.
Using core-lib-azure [Event Grid facade](https://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/blob/master/src/main/java/org/opengroup/osdu/azure/eventgrid/EventGridTopicStore.java), you can publish the events. [Reference](https://community.opengroup.org/osdu/platform/system/storage/-/blob/master/provider/storage-azure/src/main/java/org/opengroup/osdu/storage/provider/azure/MessageBusImpl.java#L67)
#### 5. Steps to Subscribe
1. Register your subscriber with Register Service. You will need to expose two endpoints
__Get Endpoint__: for a Handshake/challenge response. [Reference](https://community.opengroup.org/osdu/platform/system/register/-/blob/master/register-core/src/main/java/org/opengroup/osdu/register/api/test/SubscriberListenerApi.java#L52)
__Post Endpoint__: for receiving notifications. [Reference](https://community.opengroup.org/osdu/platform/system/register/-/blob/master/register-core/src/main/java/org/opengroup/osdu/register/api/test/SubscriberListenerApi.java#L65)
2. For onboarding a service, you have to initiate a handshake, per data partition.
3. The authZ mechanism for the Get endpoint, isn't done by entitlement service.The secret used during the handshake should be used by the service to authorize the jwt.
### Security compliance
1. Please refrain from printing fully qualified topic name in the logs.
### FAQs
1. __Event Grid is push endpoint. I have need for a pull mechanism, how can I achieve it?__
Notification service is not equiped for pull mechanism yet. You can publish to pull based products like Service Bus.
In case you want both pull and push subscribers, notification service can be an interface for only push (Event Grid).
A subscriber to that Event Grid Topic could be a Service Bus Topic. This Service Bus Topic in turn can have multiple
pull subscribers. [Reference](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/merge_requests/97/diffs)
2. __Does notification service provide filtering capability?__
Not today.
3. __When should I create a topic vs using an existing one?__
If the nature of messages and subscribers overlap completely with an existing topic, re-use it.
4. __Are there different Topics for OSDU Platform and it's consumers?__
The need is getting evaluated.
5. __The topics.json isn't scalable model__
We are evaluating the need for the bringing up a store.
\ No newline at end of file
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment