diff --git a/provider/notification-azure/GUIDELINES_FOR_USING_NOTIFICATION.md b/provider/notification-azure/GUIDELINES_FOR_USING_NOTIFICATION.md new file mode 100644 index 0000000000000000000000000000000000000000..db3b3b47550309abe646e98622f181771096a962 --- /dev/null +++ b/provider/notification-azure/GUIDELINES_FOR_USING_NOTIFICATION.md @@ -0,0 +1,42 @@ +## Introduction +The document enumerates the scenarios service is designed for, limitations of the service and onboarding guide. + + +## When to use Notification Service? +TBD + +## How to use Notification Service. +#### Prerequisite +Please verify that the topic you want to use exists. If now, follow [this](https://community.opengroup.org/osdu/platform/system/notification/-/blob/master/provider/notification-azure/PLAYBOOK_FOR_TOPIC_CREATION.md) guide to create one. + +#### 1. Publish to Event Grid Topic. +Using core-lib-azure [Event Grid facade](https://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/blob/master/src/main/java/org/opengroup/osdu/azure/eventgrid/EventGridTopicStore.java), you can publish the events. [Reference](https://community.opengroup.org/osdu/platform/system/storage/-/blob/master/provider/storage-azure/src/main/java/org/opengroup/osdu/storage/provider/azure/MessageBusImpl.java#L67) + +#### 2. Steps to Subscribe +1. Register your subscriber with Register Service. You will need to expose two endpoints + + __Get Endpoint__: for a Handshake/challenge response. [Reference](https://community.opengroup.org/osdu/platform/system/register/-/blob/master/register-core/src/main/java/org/opengroup/osdu/register/api/test/SubscriberListenerApi.java#L52) + + __Post Endpoint__: for receiving notifications. [Reference](https://community.opengroup.org/osdu/platform/system/register/-/blob/master/register-core/src/main/java/org/opengroup/osdu/register/api/test/SubscriberListenerApi.java#L65) + +2. For onboarding a service, you have to initiate a handshake, per data partition. +3. The authZ mechanism for the Get endpoint, isn't done by entitlement service.The secret used during the handshake should be used by the service to authorize the jwt. + +### Security compliance +1. Please refrain from printing fully qualified topic name in the logs. + +### FAQs +1. __Event Grid is push endpoint. I have need for a pull mechanism, how can I achieve it?__ +Notification service is not equiped for pull mechanism yet. You can publish to pull based products like Service Bus. +In case you want both pull and push subscribers, notification service can be an interface for only push (Event Grid). +A subscriber to that Event Grid Topic could be a Service Bus Topic. This Service Bus Topic in turn can have multiple +pull subscribers. [Reference](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/merge_requests/97/diffs) + +2. __Does notification service provide filtering capability?__ +Not today. + +3. __When should I create a topic vs using an existing one?__ +If the nature of messages and subscribers overlap completely with an existing topic, re-use it. + +4. __The topics.json isn't scalable model__ +We are evaluating the need for the bringing up a store. diff --git a/provider/notification-azure/PLAYBOOK_FOR_TOPIC_CREATION.md b/provider/notification-azure/PLAYBOOK_FOR_TOPIC_CREATION.md new file mode 100644 index 0000000000000000000000000000000000000000..3602867b445bffc7477fec8c084852ca9cc24d74 --- /dev/null +++ b/provider/notification-azure/PLAYBOOK_FOR_TOPIC_CREATION.md @@ -0,0 +1,23 @@ +# SOPs for Creating a new Topic and Adding subscriber for it. + +### Introduction +Notification service is invoked by PubSub called [Event Grid](https://docs.microsoft.com/en-us/azure/event-grid/). +This document describes how to create an Event Grid Topic in OSDU universe, before you can start consuming it. + +### How to create the Topic +#### 1. Create Event Grid Topic on infrastructure. +Following this will enable creation of topic in all data partitions. +1. Create a Topic via terraform. [Reference](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/blob/master/infra/templates/osdu-r3-mvp/data_partition/main.tf#L301) +2. Assign the Service Principals of service the role "EventGrid EventSubscription Contributor". [Reference](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/merge_requests/32) +3. Identify the fully qualified topic name and it's alias. The fully qualified topic name is the one you see on the Azure portal. An example is "osdu-env-dp1env-as12-grid-recordstopic". The alias could be recordstopic. Make sure the alias ends with topic. Invalid topic names will be "testtopic1", "test_topic", "testtopics" etc. +4. Create secrets in Common Resources Key Vault that helps your service identify the event grid topic end point. [Reference](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/merge_requests/30/diffs) +Make sure you name them in the format "eventgrid-\<alias\>" and "eventgrid-\<alias\>-accesskey". For example, if the topic alias is "testtopic", the keys should look like "eventgrid-testtopic-accesskey" and "eventgrid-testtopic". + +#### 2. Make partition service aware of the new topic. +Partition service maintains the configuration for each infrastructure component. + +For all the tenants, add the new topic information. [Reference](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/merge_requests/101) +This is to be done per partition per deployment. + +#### 3. Make Register service aware of the new Topic, if the consumers are outside the bounds of OSDU. +1. Add the topic alias to [topics.json](https://community.opengroup.org/osdu/platform/system/register/-/blob/master/provider/register-azure/src/main/resources/topics.json). Make sure you fill in the values of description, state and examples to ensure that the get Topic API contract is not broken. diff --git a/provider/notification-azure/SOP_FOR_TOPIC_CREATION.md b/provider/notification-azure/SOP_FOR_TOPIC_CREATION.md deleted file mode 100644 index af075c165be9b00e438a861b0cd7d60b1b24c91d..0000000000000000000000000000000000000000 --- a/provider/notification-azure/SOP_FOR_TOPIC_CREATION.md +++ /dev/null @@ -1,58 +0,0 @@ -# SOPs for Creating a new Topic and Adding subscriber for it. - -### Introduction -Notification service is invoked by PubSub called [Event Grid](https://docs.microsoft.com/en-us/azure/event-grid/). -This document describes how to create a topic, publish to it and listen to it. - -### How to create the Topic and start using it. -#### 1. Create Event Grid Topic. -Following this will enable creation of topic in all data partitions. -1. Create a Topic. [Reference](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/blob/master/infra/templates/osdu-r3-mvp/data_partition/main.tf#L301) -2. Assign the Service Principals of service the role "EventGrid EventSubscription Contributor". [Reference](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/merge_requests/32) -3. Identify the fully qualified topic name and it's alias. The fully qualified topic name is the one you see on the Azure portal. An example is "osdu-env-dp1env-as12-grid-recordstopic". The alias could be recordstopic. Make sure the alias ends with topic. Invalid topic names will be "testtopic1", "test_topic", "testtopics" etc. -4. Create secrets in Common Resources Key Vault that helps your service identify the event grid topic end point. [Reference](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/merge_requests/30/diffs) -Make sure you name them in the format "eventgrid-\<alias\>" and "eventgrid-\<alias\>-accesskey". For example, if the topic alias is "testtopic", the keys should look like "eventgrid-testtopic-accesskey" and "eventgrid-testtopic". - -#### 2. Make partition service aware of the new topic. -Partition service maintains the configuration for each infrastructure component. - -For all the tenants, add the new topic information. [Reference](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/merge_requests/101) -This is to be done per partition per deployment. - -#### 3. Make Register service aware of the new Topic -1. Add the topic alias to [topics.json](https://community.opengroup.org/osdu/platform/system/register/-/blob/master/provider/register-azure/src/main/resources/topics.json). Make sure you fill in the values of description, state and examples to ensure that the get Topic API contract is not broken. - -#### 4. Publish to Event Grid Topic. -Using core-lib-azure [Event Grid facade](https://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/blob/master/src/main/java/org/opengroup/osdu/azure/eventgrid/EventGridTopicStore.java), you can publish the events. [Reference](https://community.opengroup.org/osdu/platform/system/storage/-/blob/master/provider/storage-azure/src/main/java/org/opengroup/osdu/storage/provider/azure/MessageBusImpl.java#L67) - -#### 5. Steps to Subscribe -1. Register your subscriber with Register Service. You will need to expose two endpoints - - __Get Endpoint__: for a Handshake/challenge response. [Reference](https://community.opengroup.org/osdu/platform/system/register/-/blob/master/register-core/src/main/java/org/opengroup/osdu/register/api/test/SubscriberListenerApi.java#L52) - - __Post Endpoint__: for receiving notifications. [Reference](https://community.opengroup.org/osdu/platform/system/register/-/blob/master/register-core/src/main/java/org/opengroup/osdu/register/api/test/SubscriberListenerApi.java#L65) - -2. For onboarding a service, you have to initiate a handshake, per data partition. -3. The authZ mechanism for the Get endpoint, isn't done by entitlement service.The secret used during the handshake should be used by the service to authorize the jwt. - -### Security compliance -1. Please refrain from printing fully qualified topic name in the logs. - -### FAQs -1. __Event Grid is push endpoint. I have need for a pull mechanism, how can I achieve it?__ -Notification service is not equiped for pull mechanism yet. You can publish to pull based products like Service Bus. -In case you want both pull and push subscribers, notification service can be an interface for only push (Event Grid). -A subscriber to that Event Grid Topic could be a Service Bus Topic. This Service Bus Topic in turn can have multiple -pull subscribers. [Reference](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/merge_requests/97/diffs) - -2. __Does notification service provide filtering capability?__ -Not today. - -3. __When should I create a topic vs using an existing one?__ -If the nature of messages and subscribers overlap completely with an existing topic, re-use it. - -4. __Are there different Topics for OSDU Platform and it's consumers?__ -The need is getting evaluated. - -5. __The topics.json isn't scalable model__ -We are evaluating the need for the bringing up a store. \ No newline at end of file