infra-azure-provisioning issueshttps://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/issues2021-06-14T04:26:39Zhttps://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/issues/53WKS Service Onboarding2021-06-14T04:26:39Zharshit aggarwalWKS Service Onboarding**Service name**: `WKS Service`
The following steps must be completed for a service to onboard with OSDU on Azure. Additionally, please add the `Service Onboarding` tag to this issue when it is created.
For more information, visit our ...**Service name**: `WKS Service`
The following steps must be completed for a service to onboard with OSDU on Azure. Additionally, please add the `Service Onboarding` tag to this issue when it is created.
For more information, visit our service onboarding documentation [here](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/blob/master/docs/service-onboarding.md).
## Steps:
**Infrastructure and Initial Requirements**
- [x] Add any additional Azure cloud infrastructure (Cosmos containers, Storage containers, fileshares, etc.) to the Terraform template. [Link](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/tree/master/infra/templates/osdu-r3-mvp). Note that if the infrastructure is a part of the data-partition template, you may need to add secrets to the keyvault that are partition specific; if doing so, update the createPartition REST request to include the keys that you have added so they are accessible in service code. [Link](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/blob/master/tools/rest/partition.http#L48)
- [x] Create an ingress point for the service. [Link](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/blob/master/charts/osdu-common/templates/appgw-ingress.yaml)
- [x] Add any test data that is required for the service integration tests. [Link](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/tree/master/tools/test_data)
- [x] Update `upload-data.py` to upload any new test data files you created. [Link](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/blob/master/tools/test_data/upload-data.py).
- [x] Update the integration tester with any entitlements required to test the service. [Link](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/blob/master/tools/test_data/user_info_1.json)
- [x] Add in any new secrets that the service needs to run. [Link](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/blob/master/charts/osdu-common/templates/kv-secrets.yaml)
- [x] Create environment variable script to generate .yaml files to be used with Intellij [EnvFile](https://plugins.jetbrains.com/plugin/7861-envfile) plugin and .envrc files to be used with [direnv](https://direnv.net/). [Link](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/tree/master/tools/variables)
**Gitlab Code and Documentation**
- [x] Complete the service code such that it passes all integration tests locally. There is some documentation on starting off implementing an Azure provider. [Link](./gitlab-service-readme-template.md)
- [x] Create helm charts for service. The charts for each service are located in the `devops/azure` directory. You can look at charts from other services as a model. The charts will be nearly identical except for the different environment variables, values, etc each service needs to run. [Link](./gitlab-service-guide.md)
- [x] Implement Istio for the service if this has not already been done. Here is an example MR that shows what steps are required. [Link](https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/64)
- [x] Create an Istio auth policy in the `devops/azure/chart/templates` directory. Here is an example of an Istio auth policy that is generic and can be used by other services. [Link](https://community.opengroup.org/osdu/platform/system/storage/-/blob/master/devops/azure/chart/templates/azure-istio-auth-policy.yaml)
- [x] Add any variables that are required for the service integration tests to the Azure CI-CD file. [Link](https://community.opengroup.org/osdu/platform/ci-cd-pipelines/-/blob/master/cloud-providers/azure.yml)
- [x] Verify that the README for the Azure provider correctly and clearly describes how to run and test the service. There is a README template to help. [Link](./gitlab-service-readme-template.md)
- [x] Push any changes and verify that the Gitlab pipeline is passing in master.
**Development and Demo Azure Devops Pipelines**
- [x] Create development ADO pipeline at `devops/azure/development-pipeline.yml` in the service repo.
- [x] Verify development pipeline passes in ADO.
- [x] Create Demo ADO pipeline at `devops/azure/pipeline.yml` in the service repo.
- [x] Verify demo pipeline is passing in ADO.
**User Documentation**
- [x] Add the service to the mirror pipeline instructions. [Link](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/blob/master/docs/code-mirroring.md)
- [x] Add the service to the manual deployment instructions. [Link](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/tree/master/charts)
- [x] Add any required variables to the already existing variable group instructions for automated deployment. You should know if any variables need to be added to existing variable groups from creating the development and demo pipelines. [Link](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/blob/master/docs/service-automation.md#create-osdu-service-libraries)
- [x] Add a variable group `Azure Service Release - $SERVICE_NAME` to the documentation. You should know what values to set for this variable group from creating the development and demo pipelines. [Link](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/blob/master/docs/service-automation.md#create-osdu-service-libraries)
- [x] Add a step for creating the service pipeline at the bottom of the service-automation page. [Link](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/blob/master/docs/service-automation.md#create-osdu-service-libraries)
- [x] Create a rest script with sample calls to the service for users. [Link](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/tree/master/tools/rest) [This Step is not applicable for WKS]January - 21harshit aggarwalharshit aggarwalhttps://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/issues/43Ingestion Workflow service onboarding2021-06-14T04:26:38ZKishore BattulaIngestion Workflow service onboarding**Service name**: `Ingestion Workflow`
The following steps must be completed for a service to onboard with OSDU on Azure. Additionally, please add the `Service Onboarding` tag to this issue when it is created.
For more information, vis...**Service name**: `Ingestion Workflow`
The following steps must be completed for a service to onboard with OSDU on Azure. Additionally, please add the `Service Onboarding` tag to this issue when it is created.
For more information, visit our service onboarding documentation [here](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/blob/master/docs/service-onboarding.md).
## Steps:
**Infrastructure and Initial Requirements**
- [x] Add any additional Azure cloud infrastructure (Cosmos containers, Storage containers, fileshares, etc.) to the Terraform template. [Link](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/tree/master/infra/templates/osdu-r3-mvp). Note that if the infrastructure is a part of the data-partition template, you may need to add secrets to the keyvault that are partition specific; if doing so, update the createPartition REST request to include the keys that you have added so they are accessible in service code. [Link](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/blob/master/tools/rest/partition.http#L48)
- [x] Create an ingress point for the service. [Link](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/blob/master/charts/osdu-common/templates/appgw-ingress.yaml)
- [x] Add any test data that is required for the service integration tests. [Link](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/tree/master/tools/test_data)
- [x] Update `upload-data.py` to upload any new test data files you created. [Link](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/blob/master/tools/test_data/upload-data.py).
- [x] Update the integration tester with any entitlements required to test the service. [Link](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/blob/master/tools/test_data/user_info_1.json)
- [x] Add in any new secrets that the service needs to run. [Link](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/blob/master/charts/osdu-common/templates/kv-secrets.yaml)
- [x] Create environment variable script to generate .yaml files to be used with Intellij [EnvFile](https://plugins.jetbrains.com/plugin/7861-envfile) plugin and .envrc files to be used with [direnv](https://direnv.net/). [Link](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/tree/master/tools/variables)
**Gitlab Code and Documentation**
- [x] Complete the service code such that it passes all integration tests locally. There is some documentation on starting off implementing an Azure provider. [Link](./gitlab-service-readme-template.md)
- [x] Create helm charts for service. The charts for each service are located in the `devops/azure` directory. You can look at charts from other services as a model. The charts will be nearly identical except for the different environment variables, values, etc each service needs to run. [Link](./gitlab-service-guide.md)
- [x] Implement Istio for the service if this has not already been done. Here is an example MR that shows what steps are required. [Link](https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/64)
- [x] Create an Istio auth policy in the `devops/azure/chart/templates` directory. Here is an example of an Istio auth policy that is generic and can be used by other services. [Link](https://community.opengroup.org/osdu/platform/system/storage/-/blob/master/devops/azure/chart/templates/azure-istio-auth-policy.yaml)
- [x] Add any variables that are required for the service integration tests to the Azure CI-CD file. [Link](https://community.opengroup.org/osdu/platform/ci-cd-pipelines/-/blob/master/cloud-providers/azure.yml)
- [x] Verify that the README for the Azure provider correctly and clearly describes how to run and test the service. There is a README template to help. [Link](./gitlab-service-readme-template.md)
- [x] Push any changes and verify that the Gitlab pipeline is passing in master.
**Development and Demo Azure Devops Pipelines**
- [x] Create development ADO pipeline at `devops/azure/development-pipeline.yml` in the service repo.
- [x] Verify development pipeline passes in ADO.
- [x] Create Demo ADO pipeline at `devops/azure/pipeline.yml` in the service repo.
- [x] Verify demo pipeline is passing in ADO.
**User Documentation**
- [x] Add the service to the mirror pipeline instructions. [Link](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/blob/master/docs/code-mirroring.md)
- [x] Add the service to the manual deployment instructions. [Link](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/tree/master/charts)
- [x] Add any required variables to the already existing variable group instructions for automated deployment. You should know if any variables need to be added to existing variable groups from creating the development and demo pipelines. [Link](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/blob/master/docs/service-automation.md#create-osdu-service-libraries)
- [x] Add a variable group `Azure Service Release - $SERVICE_NAME` to the documentation. You should know what values to set for this variable group from creating the development and demo pipelines. [Link](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/blob/master/docs/service-automation.md#create-osdu-service-libraries)
- [x] Add a step for creating the service pipeline at the bottom of the service-automation page. [Link](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/blob/master/docs/service-automation.md#create-osdu-service-libraries)
- [x] Create a rest script with sample calls to the service for users. [Link](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/tree/master/tools/rest)January - 21https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/issues/135Create variables for manifest ingestion and fix version of python libraries2021-03-12T16:41:21ZKishore BattulaCreate variables for manifest ingestion and fix version of python libraries1. Manifest ingestion need to connect to different services in OSDU platform. Make these available through airflow variables.
2. Currently the extraPip packages use latest version of python library. Freeze them at a working library so th...1. Manifest ingestion need to connect to different services in OSDU platform. Make these available through airflow variables.
2. Currently the extraPip packages use latest version of python library. Freeze them at a working library so that we have consistent working ingestion across environments and customers.M4 - Release 0.7 - removehttps://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/issues/130BUG: STORAGE_QUERY_RECORD_HOST is incorrect in indexer-service.sh2021-03-02T02:02:29ZAman VermaBUG: STORAGE_QUERY_RECORD_HOST is incorrect in indexer-service.shThe Indexer-service.sh container incorrect URL for storage query records API. Example:
https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/blob/master/tools/variables/indexer-service.sh#L70...The Indexer-service.sh container incorrect URL for storage query records API. Example:
https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/blob/master/tools/variables/indexer-service.sh#L70
**FIX**
STORAGE_QUERY_RECORD_HOST should be set to `"https://${ENV_HOST}/api/storage/v2/query/records"`M4 - Release 0.7 - removeAman VermaAman Vermahttps://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/issues/129Bug: Yaml indentation issue for Istio EnvoyFilter2021-03-02T14:01:25ZHema Vishnu Pola [Microsoft]Bug: Yaml indentation issue for Istio EnvoyFilter### Overview
Envoy Filter is being added to Istio for extracting the required claims from the token and propagate in the headers to the service for required Authorization in the system.
### Issue
The filter has been merged but has an ind...### Overview
Envoy Filter is being added to Istio for extracting the required claims from the token and propagate in the headers to the service for required Authorization in the system.
### Issue
The filter has been merged but has an indentation issue, which is causing the deployment pipelines to fail.M4 - Release 0.7 - removeHema Vishnu Pola [Microsoft]Hema Vishnu Pola [Microsoft]https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/issues/127Elastic Search - Version 72022-09-15T23:49:37ZDaniel SchollElastic Search - Version 7Currently the Infrastructure Version for Elastic Search was Version 6. This has now been changed to Version 7 and updated in the Search Service.
https://community.opengroup.org/osdu/platform/system/search-service/-/merge_requests/70
D...Currently the Infrastructure Version for Elastic Search was Version 6. This has now been changed to Version 7 and updated in the Search Service.
https://community.opengroup.org/osdu/platform/system/search-service/-/merge_requests/70
Documentation Changes need to occur where Version 6 was noted.M4 - Release 0.7 - removeDaniel SchollDaniel Schollhttps://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/issues/126Feature Change - Add support to pass JAVA_OPTS to java command in DockerFile2021-06-14T04:26:42ZKrishna Nikhil VedurumudiFeature Change - Add support to pass JAVA_OPTS to java command in DockerFileCurrently there is no way to be able to pass JAVA_OPTS to the service. Essential JVM parameters such as InitialRAMPercentage, MaxRAMPercentage and several other options can be configured using JAVA_OPTS env variable.
Add the required s...Currently there is no way to be able to pass JAVA_OPTS to the service. Essential JVM parameters such as InitialRAMPercentage, MaxRAMPercentage and several other options can be configured using JAVA_OPTS env variable.
Add the required support in the Dockerfile that is used to run OSDU services.M4 - Release 0.7 - removeKrishna Nikhil VedurumudiKrishna Nikhil Vedurumudihttps://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/issues/125Feature Change - Enable Manifest Ingestion Capability2021-06-14T04:26:42ZDaniel SchollFeature Change - Enable Manifest Ingestion CapabilityManifest DAG Ingestion Capability Exists by leveraging Airflow for DAGs and Workflow Service then for executing the DAG. This capability needs to be enabled and validated.Manifest DAG Ingestion Capability Exists by leveraging Airflow for DAGs and Workflow Service then for executing the DAG. This capability needs to be enabled and validated.M4 - Release 0.7 - removeDaniel SchollDaniel Schollhttps://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/issues/124Bug - SDMS Pipeline Documentation Does Not Work2021-06-14T04:26:42ZJasonBug - SDMS Pipeline Documentation Does Not WorkI am going through the process of installing SDMS using ADO pipelines. There are some issues in the documentation that need to be resolved for this service to be deployable
- When creating the library variable group, there are variables...I am going through the process of installing SDMS using ADO pipelines. There are some issues in the documentation that need to be resolved for this service to be deployable
- When creating the library variable group, there are variables defined with periods in them (`utest.mount.dir` and ` utest.runtime.image`). I'm not sure how you tested the code, but I don't think this is allowed and was throwing errors for me.
- SDMS has a custom build stage. Similar to the deploy stage used by other services, it references the ADO variable group for the service. The issue is that the naming convention other services are using is `${{ provider.name }} Service Release - ${{ parameters.serviceName }}` and SDMS is using `R3MVP - ${{ provider.name }} Service Release - ${{ parameters.serviceName }}`. The SDMS variable group name reference ought to be changed I think. See [here](https://community.opengroup.org/osdu/platform/domain-data-mgmt-services/seismic/seismic-dms-suite/seismic-store-service/-/blob/master/devops/azure/template/build-stage.yml#L9) for where this is in SDMS code and [here](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/blob/master/devops/deploy-stages.yml#L43) for where it is for code other service pipelines reference.M4 - Release 0.7 - removeDaniel PerezDaniel Perezhttps://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/issues/123Arch Change - Ingestion Workflow Cosmos Collections2022-08-23T10:47:32ZAalekh JainArch Change - Ingestion Workflow Cosmos CollectionsNeed to create new collections for R3 ingestion workflow, as there is a change in partition key semantics
Collections to be created
* WorkflowTasksSharingInfoV2- (Partition key - /partitionKey)
The above collection should support larg...Need to create new collections for R3 ingestion workflow, as there is a change in partition key semantics
Collections to be created
* WorkflowTasksSharingInfoV2- (Partition key - /partitionKey)
The above collection should support large size partition keysM4 - Release 0.7 - removeAalekh JainAalekh Jainhttps://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/issues/113ADR - Bring your own HTTPS Certificate2021-06-14T04:26:42ZDaniel SchollADR - Bring your own HTTPS CertificateProposed ADR for Bring your own Certificate -- Open for Discussion
Need to move to Approve and get sign off from Arch VTeam then decide if this is going to be committed to M4.
[ADR 0002](https://community.opengroup.org/osdu/platform/de...Proposed ADR for Bring your own Certificate -- Open for Discussion
Need to move to Approve and get sign off from Arch VTeam then decide if this is going to be committed to M4.
[ADR 0002](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/blob/master/docs/architecture/decisions/0002-security-item-bring-your-own-certificate.md)M4 - Release 0.7 - removeMANISH KUMARMANISH KUMARhttps://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/issues/81Entitlements V2 Service Onboarding2021-06-14T04:26:40ZRostislav Vatolinvatolinrp@gmail.comEntitlements V2 Service Onboarding**Service name**: `Entitlements`
The following steps must be completed for a service to onboard with OSDU on Azure. Additionally, please add the `Service Onboarding` tag to this issue when it is created.
For more information, visit our...**Service name**: `Entitlements`
The following steps must be completed for a service to onboard with OSDU on Azure. Additionally, please add the `Service Onboarding` tag to this issue when it is created.
For more information, visit our service onboarding documentation [here](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/blob/master/docs/service-onboarding.md).
## Steps:
**Infrastructure and Initial Requirements**
* [x] Add any additional Azure cloud infrastructure (Cosmos containers, Storage containers, fileshares, etc.) to the Terraform template. [Link](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/tree/master/infra/templates/osdu-r3-mvp). Note that if the infrastructure is a part of the data-partition template, you may need to add secrets to the keyvault that are partition specific; if doing so, update the createPartition REST request to include the keys that you have added so they are accessible in service code. [Link](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/blob/master/tools/rest/partition.http#L48)
* [x] Create an ingress point for the service. [Link](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/blob/master/charts/osdu-common/templates/appgw-ingress.yaml)
* [x] Add any test data that is required for the service integration tests. [Link](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/tree/master/tools/test_data)
* [x] Update `upload-data.py` to upload any new test data files you created. [Link](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/blob/master/tools/test_data/upload-data.py).
* [x] Update the integration tester with any entitlements required to test the service. [Link](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/blob/master/tools/test_data/user_info_1.json)
* [x] Add in any new secrets that the service needs to run. [Link](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/blob/master/charts/osdu-common/templates/kv-secrets.yaml)
* [x] Create environment variable script to generate .yaml files to be used with Intellij [EnvFile](https://plugins.jetbrains.com/plugin/7861-envfile) plugin and .envrc files to be used with [direnv](https://direnv.net/). [Link](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/tree/master/tools/variables)
**Gitlab Code and Documentation**
* [x] Complete the service code such that it passes all integration tests locally. There is some documentation on starting off implementing an Azure provider. [Link](/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/blob/master/./gitlab-service-readme-template.md)
* [x] Create helm charts for service. The charts for each service are located in the `devops/azure` directory. You can look at charts from other services as a model. The charts will be nearly identical except for the different environment variables, values, etc each service needs to run. [Link](/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/blob/master/./gitlab-service-guide.md)
* [x] Implement Istio for the service if this has not already been done. Here is an example MR that shows what steps are required. [Link](https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/64)
* [x] Create an Istio auth policy in the `devops/azure/chart/templates` directory. Here is an example of an Istio auth policy that is generic and can be used by other services. [Link](https://community.opengroup.org/osdu/platform/system/storage/-/blob/master/devops/azure/chart/templates/azure-istio-auth-policy.yaml)
* [x] Add any variables that are required for the service integration tests to the Azure CI-CD file. [Link](https://community.opengroup.org/osdu/platform/ci-cd-pipelines/-/blob/master/cloud-providers/azure.yml)
* [x] Verify that the README for the Azure provider correctly and clearly describes how to run and test the service. There is a README template to help. [Link](/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/blob/master/./gitlab-service-readme-template.md)
* [x] Push any changes and verify that the Gitlab pipeline is passing in master.
**Development and Demo Azure Devops Pipelines**
* [x] Create development ADO pipeline at `devops/azure/development-pipeline.yml` in the service repo.
* [x] Verify development pipeline passes in ADO.
* [x] Create Demo ADO pipeline at `devops/azure/pipeline.yml` in the service repo.
* [x] Verify demo pipeline is passing in ADO.
**User Documentation**
* [x] Add the service to the mirror pipeline instructions. [Link](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/blob/master/docs/code-mirroring.md)
* [x] Add the service to the manual deployment instructions. [Link](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/tree/master/charts)
* [x] Add any required variables to the already existing variable group instructions for automated deployment. You should know if any variables need to be added to existing variable groups from creating the development and demo pipelines. [Link](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/blob/master/docs/service-automation.md#create-osdu-service-libraries)
* [x] Add a variable group `Azure Service Release - $SERVICE_NAME` to the documentation. You should know what values to set for this variable group from creating the development and demo pipelines. [Link](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/blob/master/docs/service-automation.md#create-osdu-service-libraries)
* [x] Add a step for creating the service pipeline at the bottom of the service-automation page. [Link](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/blob/master/docs/service-automation.md#create-osdu-service-libraries)
* [x] Create a rest script with sample calls to the service for users. [Link](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/tree/master/tools/rest)
## Setup:
1. Create an empty repo `entitlements`
2. Add a variable into `Mirror Variables`
> ADO_ORGANIZATION and ADO_PROJECT should be your actual names.
| Variable | Value |
|----------|-------|
| ENTITLEMENTS_REPO | `https://dev.azure.com/${ADO_ORGANIZATION}/$ADO_PROJECT/_git/entitlements` |
3. Edit the Mirror Pipeline and add the task
```
- task: swellaby.mirror-git-repository.mirror-git-repository-vsts-task.mirror-git-repository-vsts-task@1
displayName: 'entitlements'
inputs:
sourceGitRepositoryUri: 'https://community.opengroup.org/osdu/platform/security-and-compliance/entitlements.git'
destinationGitRepositoryUri: '$(ENTITLEMENTS_REPO)'
destinationGitRepositoryPersonalAccessToken: $(ACCESS_TOKEN)
```
4. Run the Mirror Pipeline
5. Create a Variable Group `Azure Service Release - entitlements` with the variables:
| Variable | Value |
|----------|-------|
| MAVEN_DEPLOY_POM_FILE_PATH | `drop/provider/entitlements-v2-azure` |
| MAVEN_INTEGRATION_TEST_OPTIONS | `-DargLine=""` |
| MAVEN_INTEGRATION_TEST_POM_FILE_PATH | `drop/deploy/testing/entitlements-v2-test-azure/pom.xml` |
| SERVICE_RESOURCE_NAME | `$(ENTITLEMENTS_SERVICE_NAME)` |
6. Create a Pipeline `service-entitlements` against the Repo `entitlements-service` for file `/devops/azure/pipeline.yml`
7. Execute the PipelineM4 - Release 0.7 - removeRostislav Vatolinvatolinrp@gmail.comRostislav Vatolinvatolinrp@gmail.comhttps://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/issues/149Feature Change - Service Resources - Make auto-scale parameters of App Gatewa...2021-04-12T18:17:53ZKrishna Nikhil VedurumudiFeature Change - Service Resources - Make auto-scale parameters of App Gateway configurableThe default min and max instances for App Gateway are 2 and 10 respectively.
For environments that take high volume of loads, these configurations are not sufficient.
Provide a way to give inputs for min and max instances for App Gat...The default min and max instances for App Gateway are 2 and 10 respectively.
For environments that take high volume of loads, these configurations are not sufficient.
Provide a way to give inputs for min and max instances for App Gateway so that they can be configured as needed.
**More details**
Observed 5xx during performance testing.
![AppGW-Backend-5xx](/uploads/3c4a5f268ab5b513452411d183ab289d/AppGW-Backend-5xx.PNG)
Which was caused by a dip in health probes of all the services.
![AppGW-HealthyHost-Count](/uploads/9cfda38378c0045cf74cbe193701987c/AppGW-HealthyHost-Count.PNG)
The issue is root caused to App Gateway not having enough instances to be able to handle the load.
![AppGW-Capacity-Units](/uploads/98aec41343df012f38f8fe7eac45b732/AppGW-Capacity-Units.PNG)
Making auto-scale params configurable, gives the environment owners the flexibility to adjust auto scaling parameters as required.M5 - Release 0.8 - removeKrishna Nikhil VedurumudiKrishna Nikhil Vedurumudihttps://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/issues/143Feature Change - Enable Entitlements V2 Service2021-04-22T23:08:53ZMANISH KUMARFeature Change - Enable Entitlements V2 ServiceThis is tracking issue for cutover of following services to use Entitlements V2:
1. Indexer-Service
2. Search-Service
3. Storage
Following MRs:
1. https://community.opengroup.org/osdu/platform/security-and-compliance/legal/-/merge_reque...This is tracking issue for cutover of following services to use Entitlements V2:
1. Indexer-Service
2. Search-Service
3. Storage
Following MRs:
1. https://community.opengroup.org/osdu/platform/security-and-compliance/legal/-/merge_requests/101
2. https://community.opengroup.org/osdu/platform/system/storage/-/merge_requests/155
3. https://community.opengroup.org/osdu/platform/system/indexer-service/-/merge_requests/115
4. https://community.opengroup.org/osdu/platform/system/search-service/-/merge_requests/84
5. https://community.opengroup.org/osdu/platform/system/schema-service/-/merge_requests/91
6. https://community.opengroup.org/osdu/platform/system/file/-/merge_requests/104
7. https://community.opengroup.org/osdu/platform/system/register/-/merge_requests/76
8 https://community.opengroup.org/osdu/platform/system/notification/-/merge_requests/66
9. https://community.opengroup.org/osdu/platform/system/reference/unit-service/-/merge_requests/75
10. https://community.opengroup.org/osdu/platform/system/reference/crs-catalog-service/-/merge_requests/41
11. https://community.opengroup.org/osdu/platform/system/reference/crs-conversion-service/-/merge_requests/41
12. https://community.opengroup.org/osdu/platform/data-flow/ingestion/ingestion-workflow/-/merge_requests/96M5 - Release 0.8 - removeDaniel SchollMANISH KUMARDaniel Schollhttps://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/issues/137Close Release - Drop Release 0.72021-03-12T03:37:49ZDaniel SchollClose Release - Drop Release 0.7Milestone 4 Release -- App Version 0.7Milestone 4 Release -- App Version 0.7M5 - Release 0.8 - removeDaniel SchollDaniel Schollhttps://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/issues/121Feature Change - Add new environment template to bring up terraform resources...2021-06-14T04:26:42ZVibhuti Sharma [Microsoft]Feature Change - Add new environment template to bring up terraform resources related to Monitoring and AlertingResources related to monitoring and alerting cannot be brought up by existing templates. A new template is needed which will execute the terraform modules associated with these resources.
## Acceptance Criteria
1. ADR must be opened an...Resources related to monitoring and alerting cannot be brought up by existing templates. A new template is needed which will execute the terraform modules associated with these resources.
## Acceptance Criteria
1. ADR must be opened and approved for this change.
2. Necessary infrastructure and helm charts should be created or updated.
3. New and existing unit and integration tests should pass.
4. Process Documentation should exist to provide instructions on how to use or enable this feature.M5 - Release 0.8 - removeVibhuti Sharma [Microsoft]Vibhuti Sharma [Microsoft]https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/issues/115Feature Change - Enable a Custom HTTPS Certificate for use2021-06-14T04:26:42ZDaniel SchollFeature Change - Enable a Custom HTTPS Certificate for useThe current implementation of OSDU on Azure utilizes an auto provisioned HTTPS certificate by using cert-manager.
As a user of OSDU I should be able to utilize a custom certificate of my choosing to handle https ingress on my deploymen...The current implementation of OSDU on Azure utilizes an auto provisioned HTTPS certificate by using cert-manager.
As a user of OSDU I should be able to utilize a custom certificate of my choosing to handle https ingress on my deployment of OSDU.
Acceptance Criteria
---
1. A design decision should be made on the best way to handle this feature.
2. Necessary infrastructure and helm charts should be created or updated as necessary.
3. Any unit and integration tests should pass.
4. Process Documentation should exist to provide instructions on how to use or enable this feature.M5 - Release 0.8 - removeAnkit Sharma [Microsoft]Ankit Sharma [Microsoft]https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/issues/114ADR - Implement Java Service Base Container2021-06-14T04:26:42ZDaniel SchollADR - Implement Java Service Base ContainerProposed ADR for Implement Java Service Base Container -- Open for Discussion
Need to move to Approve and get sign off from Arch VTeam then decide if this is going to be committed to M4.
[ADR 0003](https://community.opengroup.org/osdu/...Proposed ADR for Implement Java Service Base Container -- Open for Discussion
Need to move to Approve and get sign off from Arch VTeam then decide if this is going to be committed to M4.
[ADR 0003](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/blob/master/docs/architecture/decisions/0003-security-item-implement-java-service-base-container.md)M5 - Release 0.8 - removeMANISH KUMARMANISH KUMARhttps://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/issues/104Operationalize - Back Up and Restore for Storage Account and CosmosDB Account2021-06-14T04:26:41ZKomal MakkarOperationalize - Back Up and Restore for Storage Account and CosmosDB AccountThis issue is to track all the action items that will help facilitate BackUp and Restore Feature.
**1. Enable delete locks for the following:**
- [x] Storage Accounts.
- [x] Cosmos DB Accounts.
**2. Restore Configuration**
- [x] Feasi...This issue is to track all the action items that will help facilitate BackUp and Restore Feature.
**1. Enable delete locks for the following:**
- [x] Storage Accounts.
- [x] Cosmos DB Accounts.
**2. Restore Configuration**
- [x] Feasibility study. Further action Items will be listed after the approach is decided.M5 - Release 0.8 - removeKomal MakkarKomal Makkarhttps://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/issues/170BUG: Schema service update elastic for any new schema add2022-08-23T11:19:15ZMANISH KUMARBUG: Schema service update elastic for any new schema addSchema service has a bug that it doesn't update indices for new schema insertion which is being fixed by this issue.
The changes are merged using [MR](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-p...Schema service has a bug that it doesn't update indices for new schema insertion which is being fixed by this issue.
The changes are merged using [MR](https://community.opengroup.org/osdu/platform/deployment-and-operations/infra-azure-provisioning/-/merge_requests/268).M6 - Release 0.9 - removeAbhishek Kumar (SLB)Abhishek Kumar (SLB)