SOP_FOR_TOPIC_CREATION.md

# SOPs for Creating a new Topic and Adding subscriber for it. 

### Introduction
Notification service is invoked by PubSub called [Event Grid](https://docs.microsoft.com/en-us/azure/event-grid/).
This document describes how to create a topic, publish to it and listen to it. 

### How to create the Topic and start using it.
#### 1. Create Event Grid Topic. 
Following this will enable creation of topic in all data partitions. 
1. Create a Topic. [Reference](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/blob/master/infra/templates/osdu-r3-mvp/data_partition/main.tf#L301)
2. Assign the Service Principals of the service the role of subscription contributor. [Reference](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/merge_requests/32)
3. Create secrets in Common Resources Key Vault that helps your service identify the event grid topic end point. [Reference](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/merge_requests/30/diffs)
Make sure you name them in the fashion  "eventgrid-<nameoftopic>topic" and "eventgrid-<nameoftopic>topic-accesskey"

#### 2. Make partition service aware of the new topic.
Partition service maintains the configuration for each infrastructure component. 

For all the tenants, add the new topic information. [Reference](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/merge_requests/101)
This is to be done per deployment. 

#### 3. Make Register service aware of the new Topic 
1. Add the topic alias to [topics.json](https://community.opengroup.org/osdu/platform/system/register/-/blob/master/provider/register-azure/src/main/resources/topics.json). If your key is named "eventgrid-testtopic", alias will be "testtopic". Make sure you fill in the values of description, state and examples to ensure that the get Topic API contract is not broken. 


Note: The topics.json file mechanism will change soon. 

#### 4. Publish to Event Grid Topic.
Use the Event Grid Store to publish the message. [Reference](https://community.opengroup.org/osdu/platform/system/storage/-/blob/master/provider/storage-azure/src/main/java/org/opengroup/osdu/storage/provider/azure/MessageBusImpl.java#L67)

#### 5. Steps to Subscribe
1. Register your subscriber with Register Service. You will need to expose two endpoints
	
	__Get Endpoint__: for a Handshake/challenge response. [Reference](https://community.opengroup.org/osdu/platform/system/register/-/blob/master/register-core/src/main/java/org/opengroup/osdu/register/api/test/SubscriberListenerApi.java#L52)
    
    __Post Endpoint__: for receiving notifications. [Reference](https://community.opengroup.org/osdu/platform/system/register/-/blob/master/register-core/src/main/java/org/opengroup/osdu/register/api/test/SubscriberListenerApi.java#L65)

2. Initiate a handshake for each tenant, to register service. Which means for onboarding a service, you have to initiate a handshake, per data partition.
3. Whitelist the endpoints from Istio. Make sure you are not calling entitlements service for the same. 
 
### Security compliance 
1. Please refrain from printing fully qualified topic name in the logs.  
 
### FAQs
1. __Event Grid is push endpoint. I have need for a pull mechanism, how can I achieve it?__
Notification service is not equiped for pull mechanism yet. You can publish to pull based products like Service Bus. 
In case you want both pull and push subscribers, notification service can be an interface for only push (Event Grid). 
A subscriber to that Event Grid Topic could be a Service Bus Topic. This Service Bus Topic in turn can have multiple 
pull subscribers. [Reference](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/merge_requests/97/diffs)

2. __Does notification service provide filtering capability?__
Not today. 

3. __When should I create a topic vs using an existing one?__
If the nature of messages and subscribers overlap completely with an existing topic, re-use it.

4. __Are there different Topics for OSDU Platform and it's consumers?__
The need is getting evaluated.