SOP_FOR_TOPIC_CREATION.md 4.14 KB
Newer Older
Komal Makkar's avatar
Komal Makkar committed
1
2
# SOPs for Creating a new Topic and Adding subscriber for it. 

Komal Makkar's avatar
Komal Makkar committed
3
### Introduction
Komal Makkar's avatar
Komal Makkar committed
4
Notification service is invoked by PubSub called [Event Grid](https://docs.microsoft.com/en-us/azure/event-grid/).
Komal Makkar's avatar
Komal Makkar committed
5
This document describes how to create a topic, publish to it and listen to it. 
Komal Makkar's avatar
Komal Makkar committed
6

Komal Makkar's avatar
Komal Makkar committed
7
8
### How to create the Topic and start using it.
#### 1. Create Event Grid Topic. 
Komal Makkar's avatar
Komal Makkar committed
9
Following this will enable creation of topic in all data partitions. 
Komal Makkar's avatar
Komal Makkar committed
10
11
12
13
14
1. Create a Topic. [Reference](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/blob/master/infra/templates/osdu-r3-mvp/data_partition/main.tf#L301)
2. Assign the Service Principals of the service the role of subscription contributor. [Reference](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/merge_requests/32)
3. Create secrets in Common Resources Key Vault that helps your service identify the event grid topic end point. [Reference](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/merge_requests/30/diffs)
Make sure you name them in the fashion  "eventgrid-<nameoftopic>topic" and "eventgrid-<nameoftopic>topic-accesskey"

Komal Makkar's avatar
Komal Makkar committed
15
16
#### 2. Make partition service aware of the new topic.
Partition service maintains the configuration for each infrastructure component. 
Komal Makkar's avatar
Komal Makkar committed
17

Komal Makkar's avatar
Komal Makkar committed
18
For all the tenants, add the new topic information. [Reference](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/merge_requests/101)
Komal Makkar's avatar
Komal Makkar committed
19
20
This is to be done per deployment. 

Komal Makkar's avatar
Komal Makkar committed
21
22
23
#### 3. Make Register service aware of the new Topic 
1. Add the topic alias to [topics.json](https://community.opengroup.org/osdu/platform/system/register/-/blob/master/provider/register-azure/src/main/resources/topics.json). If your key is named "eventgrid-testtopic", alias will be "testtopic". Make sure you fill in the values of description, state and examples to ensure that the get Topic API contract is not broken. 

Komal Makkar's avatar
Komal Makkar committed
24

Komal Makkar's avatar
Komal Makkar committed
25
26
27
Note: The topics.json file mechanism will change soon. 

#### 4. Publish to Event Grid Topic.
Komal Makkar's avatar
Komal Makkar committed
28
29
Use the Event Grid Store to publish the message. [Reference](https://community.opengroup.org/osdu/platform/system/storage/-/blob/master/provider/storage-azure/src/main/java/org/opengroup/osdu/storage/provider/azure/MessageBusImpl.java#L67)

Komal Makkar's avatar
Komal Makkar committed
30
#### 5. Steps to Subscribe
Komal Makkar's avatar
Komal Makkar committed
31
32
33
34
35
36
37
38
39
1. Register your subscriber with Register Service. You will need to expose two endpoints
	
	__Get Endpoint__: for a Handshake/challenge response. [Reference](https://community.opengroup.org/osdu/platform/system/register/-/blob/master/register-core/src/main/java/org/opengroup/osdu/register/api/test/SubscriberListenerApi.java#L52)
    
    __Post Endpoint__: for receiving notifications. [Reference](https://community.opengroup.org/osdu/platform/system/register/-/blob/master/register-core/src/main/java/org/opengroup/osdu/register/api/test/SubscriberListenerApi.java#L65)

2. Initiate a handshake for each tenant, to register service. Which means for onboarding a service, you have to initiate a handshake, per data partition.
3. Whitelist the endpoints from Istio. Make sure you are not calling entitlements service for the same. 
 
Komal Makkar's avatar
Komal Makkar committed
40
41
42
43
### Security compliance 
1. Please refrain from printing fully qualified topic name in the logs.  
 
### FAQs
Komal Makkar's avatar
Komal Makkar committed
44
45
46
47
48
49
1. __Event Grid is push endpoint. I have need for a pull mechanism, how can I achieve it?__
Notification service is not equiped for pull mechanism yet. You can publish to pull based products like Service Bus. 
In case you want both pull and push subscribers, notification service can be an interface for only push (Event Grid). 
A subscriber to that Event Grid Topic could be a Service Bus Topic. This Service Bus Topic in turn can have multiple 
pull subscribers. [Reference](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/merge_requests/97/diffs)

Komal Makkar's avatar
Komal Makkar committed
50
2. __Does notification service provide filtering capability?__
Komal Makkar's avatar
Komal Makkar committed
51
52
53
Not today. 

3. __When should I create a topic vs using an existing one?__
Komal Makkar's avatar
Komal Makkar committed
54
55
56
57
If the nature of messages and subscribers overlap completely with an existing topic, re-use it.

4. __Are there different Topics for OSDU Platform and it's consumers?__
The need is getting evaluated.