OSDU Software issueshttps://community.opengroup.org/groups/osdu/-/issues2021-06-16T22:18:51Zhttps://community.opengroup.org/osdu/platform/deployment-and-operations/multi-region-deployment/multi-region/-/issues/9[Multi-Region Deployment] : Horizon 1 : GCP Simple Scenarios2021-06-16T22:18:51ZStephen Whitley (Invited Expert)[Multi-Region Deployment] : Horizon 1 : GCP Simple ScenariosSee [Design Review](https://community.opengroup.org/osdu/platform/deployment-and-operations/home/-/wikis/multi-region/Design/R3-Horizon1) for additional Details
**Simple Scenarios**
- [ ] replicate everything in the storage service (me...See [Design Review](https://community.opengroup.org/osdu/platform/deployment-and-operations/home/-/wikis/multi-region/Design/R3-Horizon1) for additional Details
**Simple Scenarios**
- [ ] replicate everything in the storage service (metadata + entitlements) between them and
- [ ] on user's search, leverage the delivery service to serve the requested record remotely
- [ ] share identity and entitlements (for the same end user) among the regionsM1 - Release 0.1Ferris ArgyleDmitriy RudkoBrian KirklandFerris Argylehttps://community.opengroup.org/osdu/platform/deployment-and-operations/multi-region-deployment/multi-region/-/issues/8[Multi-Region Deployment] : Horizon 1 : Open Shift Simple Scenarios2021-06-16T22:18:51ZStephen Whitley (Invited Expert)[Multi-Region Deployment] : Horizon 1 : Open Shift Simple ScenariosSee [Design Review](https://community.opengroup.org/osdu/platform/deployment-and-operations/home/-/wikis/multi-region/Design/R3-Horizon1) for additional Details
**Simple Scenarios**
- [ ] replicate everything in the storage service (me...See [Design Review](https://community.opengroup.org/osdu/platform/deployment-and-operations/home/-/wikis/multi-region/Design/R3-Horizon1) for additional Details
**Simple Scenarios**
- [ ] replicate everything in the storage service (metadata + entitlements) between them and
- [ ] on user's search, leverage the delivery service to serve the requested record remotely
- [ ] share identity and entitlements (for the same end user) among the regionsM1 - Release 0.1Wladmir FrazaoBrian KirklandWladmir Frazaohttps://community.opengroup.org/osdu/platform/deployment-and-operations/multi-region-deployment/multi-region/-/issues/7[Multi-Region Deployment] : Horizon 1 : Azure Simple Scenarios2021-06-16T22:18:52ZStephen Whitley (Invited Expert)[Multi-Region Deployment] : Horizon 1 : Azure Simple ScenariosSee [Design Review](https://community.opengroup.org/osdu/platform/deployment-and-operations/home/-/wikis/multi-region/Design/R3-Horizon1) for additional Details
**Simple Scenarios**
- [ ] replicate everything in the storage service (me...See [Design Review](https://community.opengroup.org/osdu/platform/deployment-and-operations/home/-/wikis/multi-region/Design/R3-Horizon1) for additional Details
**Simple Scenarios**
- [ ] replicate everything in the storage service (metadata + entitlements) between them and
- [ ] on user's search, leverage the delivery service to serve the requested record remotely
- [ ] share identity and entitlements (for the same end user) among the regionsM1 - Release 0.1Dania Kodeih (Microsoft)Brian KirklandDania Kodeih (Microsoft)https://community.opengroup.org/osdu/platform/deployment-and-operations/multi-region-deployment/multi-region/-/issues/6[Multi-Region Deployment] : Horizon 1 : Simple Data Replication Scenarios2021-06-16T22:18:53ZStephen Whitley (Invited Expert)[Multi-Region Deployment] : Horizon 1 : Simple Data Replication ScenariosSee [Design Review](https://community.opengroup.org/osdu/platform/deployment-and-operations/home/-/wikis/multi-region/Design/R3-Horizon1) for additional Details
**Simple Scenarios**
Within all regions in an operator's OSDU instance, be...See [Design Review](https://community.opengroup.org/osdu/platform/deployment-and-operations/home/-/wikis/multi-region/Design/R3-Horizon1) for additional Details
**Simple Scenarios**
Within all regions in an operator's OSDU instance, be able to:
- [ ] replicate everything in the storage service (metadata + entitlements) between the regions and
- [ ] on user's search, leverage the delivery service to serve the requested dataset remotely
- [ ] share identity, entitlements (for the same end user), and legality tags among the regions
Acceptable functional limitations:
- no support for embargoed data
- no support for enforcing data residency restrictions when replicating metadata/dataM1 - Release 0.1Brian KirklandGregBrian Kirklandhttps://community.opengroup.org/osdu/platform/deployment-and-operations/multi-region-deployment/multi-region/-/issues/5[Multi-Region Deployment] : Horizon 1 : GCP Configuration2021-06-16T22:18:54ZStephen Whitley (Invited Expert)[Multi-Region Deployment] : Horizon 1 : GCP ConfigurationSee [Design Review](https://community.opengroup.org/osdu/platform/deployment-and-operations/home/-/wikis/multi-region/Design/R3-Horizon1) for additional Details
**System Configuration**
- [ ] stand-up multiple region deployments for an...See [Design Review](https://community.opengroup.org/osdu/platform/deployment-and-operations/home/-/wikis/multi-region/Design/R3-Horizon1) for additional Details
**System Configuration**
- [ ] stand-up multiple region deployments for an OSDU instance
- [ ] compatible deployment of core services to these deployments
- [ ] demonstrate observability (health and status) for these deploymentsM1 - Release 0.1Stephen Whitley (Invited Expert)Ferris ArgyleDmitriy RudkoBrian KirklandStephen Whitley (Invited Expert)https://community.opengroup.org/osdu/platform/deployment-and-operations/multi-region-deployment/multi-region/-/issues/4[Multi-Region Deployment] : Horizon 1 : Open Shift Configuration2021-06-16T22:18:55ZStephen Whitley (Invited Expert)[Multi-Region Deployment] : Horizon 1 : Open Shift ConfigurationSee [Design Review](https://community.opengroup.org/osdu/platform/deployment-and-operations/home/-/wikis/multi-region/Design/R3-Horizon1) for additional Details
**System Configuration**
- [ ] stand-up multiple region deployments for an...See [Design Review](https://community.opengroup.org/osdu/platform/deployment-and-operations/home/-/wikis/multi-region/Design/R3-Horizon1) for additional Details
**System Configuration**
- [ ] stand-up multiple region deployments for an OSDU instance
- [ ] compatible deployment of core services to these deployments
- [ ] demonstrate observability (health and status) for these deploymentsM1 - Release 0.1Stephen Whitley (Invited Expert)Wladmir FrazaoBrian KirklandStephen Whitley (Invited Expert)https://community.opengroup.org/osdu/platform/deployment-and-operations/multi-region-deployment/multi-region/-/issues/3[Multi-Region Deployment] : Horizon 1 : Azure Configuration2021-06-16T22:18:56ZStephen Whitley (Invited Expert)[Multi-Region Deployment] : Horizon 1 : Azure ConfigurationSee [Design Review](https://community.opengroup.org/osdu/platform/deployment-and-operations/home/-/wikis/multi-region/Design/R3-Horizon1) for additional Details
**System Configuration**
- [ ] able to provision and stand-up multiple reg...See [Design Review](https://community.opengroup.org/osdu/platform/deployment-and-operations/home/-/wikis/multi-region/Design/R3-Horizon1) for additional Details
**System Configuration**
- [ ] able to provision and stand-up multiple region deployments for an OSDU instance
- [ ] repeatable deployment capability for all OSDU core services to each regional deployment
- [ ] complete set of OSDU services deployed to all regions
- [ ] region registry in place to manage which regions deployments make up an OSDU federated instance
- [ ] routing in place to direct end-user / application calls to the nearest service instances
- [ ] automation tests in place to report pass/fail for fully functional OSDU services in each region
- [ ] demonstrate observability (health and status) for these deployments
- [ ] dashboard or CLI to query service availability and notify operators on failureM1 - Release 0.1Stephen Whitley (Invited Expert)Dania Kodeih (Microsoft)Brian KirklandStephen Whitley (Invited Expert)2021-02-26https://community.opengroup.org/osdu/platform/deployment-and-operations/multi-region-deployment/multi-region/-/issues/2[Multi-Region Deployment] : Horizon 1 : AWS Configuration2022-08-23T10:47:25ZStephen Whitley (Invited Expert)[Multi-Region Deployment] : Horizon 1 : AWS ConfigurationSee [Design Review](https://community.opengroup.org/osdu/platform/deployment-and-operations/home/-/wikis/multi-region/Design/R3-Horizon1) for additional Details
**System Configuration**
- [ ] able to provision and stand-up multiple reg...See [Design Review](https://community.opengroup.org/osdu/platform/deployment-and-operations/home/-/wikis/multi-region/Design/R3-Horizon1) for additional Details
**System Configuration**
- [ ] able to provision and stand-up multiple region deployments for an OSDU instance
- [ ] repeatable deployment capability for all OSDU core services to each regional deployment
- [ ] complete set of OSDU services deployed to all regions
- [ ] region registry in place to manage which regions deployments make up an OSDU federated instance
- [ ] routing in place to direct end-user / application calls to the nearest service instances
- [ ] automation tests in place to report pass/fail for fully functional OSDU services in each region
- [ ] demonstrate observability (health and status) for these deployments
- [ ] dashboard or CLI to query service availability and notify operators on failureM1 - Release 0.1Stephen Whitley (Invited Expert)Brian KirklandGregStephen Whitley (Invited Expert)2020-11-30https://community.opengroup.org/osdu/platform/system/register/-/issues/14[Register Service] TopicsRepository in register-core retrieves topics from a ...2021-06-16T22:18:17ZRucha Deshpande[Register Service] TopicsRepository in register-core retrieves topics from a json fileIn it's current state the `GET api/register/v1/topics` API retrieves topics from a json file which it reads from the classpath.
Code can be found here:
[TopicsRepository](https://community.opengroup.org/osdu/platform/system/register/-/...In it's current state the `GET api/register/v1/topics` API retrieves topics from a json file which it reads from the classpath.
Code can be found here:
[TopicsRepository](https://community.opengroup.org/osdu/platform/system/register/-/blob/master/register-core/src/main/java/org/opengroup/osdu/register/subscriber/persistence/TopicsRepository.java)
TopicsRepository should be moved outside core code.
We need some clarity on who would be creating the topics and in that case would we need to add PUT and DELETE APIs for TopicsRepository?M1 - Release 0.1ethiraj krishnamanaiduJoeRucha DeshpandeMatt Wiseethiraj krishnamanaiduhttps://community.opengroup.org/osdu/platform/system/schema-service/-/issues/22[Schema Service] Schema Integration tests use "mvn verify" instead of "mvn test"2020-10-09T14:37:51ZRucha Deshpande[Schema Service] Schema Integration tests use "mvn verify" instead of "mvn test"
The schema service currently uses "mvn verify" to run integration tests. For consistency, "mvn test" should be used instead so that integration tests always run as part of the test phase.
*Search service uses cucumber for integration t...
The schema service currently uses "mvn verify" to run integration tests. For consistency, "mvn test" should be used instead so that integration tests always run as part of the test phase.
*Search service uses cucumber for integration tests and uses "mvn test" as well.M1 - Release 0.1ethiraj krishnamanaiduJoeRucha DeshpandeMatt Wiseethiraj krishnamanaiduhttps://community.opengroup.org/osdu/platform/deployment-and-operations/multi-region-deployment/multi-region/-/issues/1[Multi-Region Deployment] Support OSDU Multi-Region Deployment : Requirements2021-06-16T22:18:58ZStephen Whitley (Invited Expert)[Multi-Region Deployment] Support OSDU Multi-Region Deployment : Requirements**Background:**
Subsurface exploration and production is a global enterprise, hence most operating companies have a need for subsurface data availability across multiple geographic regions. Strict latency, data sovereignty, or local regu...**Background:**
Subsurface exploration and production is a global enterprise, hence most operating companies have a need for subsurface data availability across multiple geographic regions. Strict latency, data sovereignty, or local regulatory/legislative requirements may result in the need for numerous distributed, high-availability OSDU regions. Sharing and managing data across multiple distributed installations is a core capability for OSDU.
An OSDU Region is defined as an independent and highly available geographical deployment of the full OSDU design. It provides full functionality within a given region and there is no need for deploying multiple OSDU environments for high availability or disaster recovery requirements. Strict latency, data sovereignty or local legislation requirements may result into the need for additional OSDU deployments. OSDU deployments are preferably public cloud based. In situation where in-country OSDU deployment is required and no public cloud region is available, physical edge capabilities from the public cloud provider will be used. In any case, the exposed OSDU APIs deployed in the different OSDU regions will provide identical functionality.
**Objective:**
- This requirement is to ensure OSDU data platform can support highly available multi-region/geographical deployment.
**Business Case:**
1. Adopting company has subsurface professionals interacting with OSDU around the world. All expect optimal performance and high availability when interacting with data and applications.
2. Adopting company has stringent availability/up-time and fault tolerance demands on the data and applications hosted in OSDU.
3. Adopting company have disaster recovery and business continuity requirements that need to be fulfilled. A multi-region deployment is essential to the goal of maximum redundancy and resilience to catastrophic failure.
4. Regulatory restrictions governing the adopting company's activities mandate regional data never travels beyond the region's geographic borders.
**Summary of Requirements**
OSDU must enable OSDU Using Companies to stand-up and operate multiple concurrent, homogeneous instances of OSDU in different geographic regions. Each OSDU deployment instance is stand-alone/independent, but all are federated to provide global access to the underlying subsurface data hosted and delivered via the OSDU Data Platform in each region. OSDU shall deliver users no more than approx. 40 millisecond latency between the end user and the OSDU co-resident applications and services; therefore, multiple regional deployments shall have globally replicated metadata and regionally originated and stored actual data files (datasets, etc.), including raw data, processed data and interpreted data.
**Requirements**
*General Requirements*
1. Using companies shall be able to deploy any number of independent, complete, and homogeneous instances of OSDU in different geographic regions.
2. Each regional deployment includes identical functionalities in the main architectural components: platform services, loading/enrichment/consumption data services, search engine, graph and NoSQL databases, and object stores.
3. Each deployment is 'stand-alone': applications, services and data can be independently deployed, accessed, and utilized by real-world users within the hosting region.
4. Each regional deployment can access the shared 'System of Record' i.e., the data platform including all the globally replicated metadata and reference data.
5. Stand up of a new deployment may require administrative capabilities residing in another regional deployment
*Administration Requirements*
1. One region shall be designated as the 'Admin Region' which provides administrative functions allowing configuration, monitoring, and control of all regional instances.
2. OSDU shall provide the ability to designate and configure one and only one deployment as an 'Admin Region'.
3. Once an OSDU deployment is designated as the 'Admin Region', all administrative API calls throughout all OSDU regional deployments are re-rerouted to the administrative services hosted in the Admin Region
4. From the ‘Admin Region’, the health and availability of other OSDU regional deployments can be accessed and logged.
5. Administrative entitlements shall be defined to govern which users/roles can access which administrative capabilities.
*Infrastructure Requirements*
1. One region (not necessarily the 'Admin Region') can be designated a 'Central Region' in order to host the development and deployment infrastructure (dev/test environments, CI/CD tooling, etc.)
2. Using companies shall be able to install and configure development + deployment infrastructure to a regional instance for development, acceptance test, and CI/CD activities.
3. Deployment infrastructure accommodates setup of the OSDU deployment instances (platform services, identity provider, data platform, search capabilities, etc.) It also must accommodate deployment of applications and services from the OSDU marketplace as well as Using Company proprietary works. All regional deployments are homogeneous and the deployment infrastructure accommodates making new regional deployments appear similar to pre-existing deployments.
4. Using companies shall be able to setup and configure connectivity between an OSDU instance and Operator's on-premise data center(s)
5. Each OSDU deployment must allow connections to/from a co-hosted 'Transit VPC' which facilitates data connections between on-premise / in-country installations and OSDU
*Data Platform Requirements*
1. Replication of data and services are always subject to administratively-defined replication policies.
2. Replication policies are persisted and distributed as master reference data in the OSDU Data Platform.
3. Global access to metadata and underlying data records is governed by the entitlements’ policies / rules from the OSDU Data Platform.
4. All metadata is replicated across all nodes; at every node access to all metadata for that multi- region OSDU implementation. Single-cloud provider within an OSDU Using Company (i.e., replicating metadata between multiple cloud provider solutions is out of scope for R3).
5. Transactional data is replicated by exception, not by default. It is expected that the 'home' for data is well-defined and mapped between its geographical region and its usage. By exception and on-request, transactional data can be replicated to host regions to achieve efficient consumption.
6. Data is to be regionally bound and tagged. Data resides by default in the OSDU “home” region in which the loading/ingestion workflow occurred, which will be the region associated with the data from an earth location perspective. Since the describing metadata is replicated globally, any OSDU user can discern in which region the described data resides.
7. The OSDU DP Search API can be used to locate search results based on globally replicated master-data, reference-data, and metadata. As stated elsewhere in the requirements, metadata are not protected and a user accessing any OSDU regional deployment can see all metadata. However, this is only true for metadata and of course not for the data content files / datasets / etc. (raw, processed, interpreted, etc.).
8. Facilities shall be provided to enable on-demand replication of content data files from ‘home’ region to a ‘host’ region’: selected data can be transported via request from its home region to any other OSDU region. Subsequent changes, that is, new items of data that serve as successor versions or new derivations, are populated to the home region and, if requested, to the same ‘host region’.
*Standout Non-functional Requirements (**TBD**)*
1. High-availability: Are OSDU services and data always available to end users, despite failure conditions?
2. Performance: Does the distributed solution meet the in-region performance goals for the data platform?
3. Latency: How much latency is there between the creation of content and it's availability across all regions?
4. Security: How much does the solution increase the threat surface of the OSDU data platform?
5. Ease of Use: How easy is it configure and operate?
6. Observability: How easy is it to test, identify, and resolve incorrect behavior?
7. Maintainability: How much does the solution introduce complexity in development, deployment and operations of the data platform?
**Additional Use Cases**
1. Automated replication of metadata across all regions.
(applies to: work-product-metadata, work-product-component-metadata, associated file metadata)
* "Easy" - leverage CSP storage technologies
* Leads to 'eventual consistency' of search result indices which crawl the metadata at different intervals
2. Master-Reference Data - also needs to be automatically replicated across all regions (to enable context and validation for transactional data.)
3. On-demand request for a WP located in a different region:
* Can be requested by a user via search UI or by a microservice via API
* WP is copied to the requestor region
- Subject to E&O contract + replication policy. If the replication policy states the data cannot travel to the requestor region, then user is notified - confirm?
- Metadata has a 'locator' indicating the WP's home region
- Lineage preserved, but it's not a deep copy of the references
- WPCs are 'deep copied' to the requestor region with read-only access
* Any derivative WPs made in the requestor region are loaded/ingested back in the 'home' region as identified in the metadata 'locator'
- Lineage is updated as usual
- WPs are loaded/ingested in the 'home' region
4. Repartitioning (example: grow from 2-regions to 3-regions) - �be able to reallocate home partition boundaries via 'staging'
* Creating a new region -- provides instant accessibility to the 'common' partition.
* Administrative functions to be able to move/copy a partition between regions. Some using companies may require the ability to replicate a single partition across several regions. Each region has a replica with only the data applicable to its users / assets (geographic area) and can transfer changes of this data to the 'home' region. This allows the 'home' region to perform analysis on data that is up to date across the entire extent.
**Architecture Principles / Guidelines**
1. Synchronous cross-regional calls should be avoided when possible. Applications should use regional resources.
2. Embrace asynchronous systems and replication - high availability / deferred consistency
3. Leverage the principles of Isolation and Redundancy: a failure of any kind in one Region should not affect services running in another.
**OSDU Regions**
An OSDU Region is an independent and complete deployment of all infrastructure and application services required to provide OSDU Data Platform (DP) services. Whilst every OSDU Region will have a homogeneous DP service deployment, a few types of OSDU regions can be identified:
- “*OSDU Standard Hosted Region*”: This is the common type of OSDU Region and will host all data platform services, including security, ingest, logging and monitoring. Platform, application and workflow services will be deployed based on need in this standard OSDU Region.
- “*OSDU Central Region*”: One of the standard OSDU regions will be assigned as a central region and will be hosting the development, integration test and performance test environments and Continuous Integration (CI)/ Continuous Delivery (CD) services. If required, it may host a central logging service. It will also host global OSDU application.
- “*OSDU Admin Region*”: One of the standard OSDU regions will be assigned as an admin region and this is a configured parameter in the OSDU metadata, namely the 'Admin Management' parameters. Subsurface Master Data System (SMDS) Master and Reference Data as well as OSDU management parameters stored in SMDS can only be updated in this configured OSDU Admin Region. All Admin APIs are available in all OSDU regions, though the call will be rejected (or redirected) if the API is not hosted in the OSDU Admin Region.
- “*OSDU In-Country Region*”: When public cloud infrastructure cannot be leveraged to deploy an OSDU Region for data sovereignty or latency reasons, an in-country OSDU deployment takes place with the consideration of incurring least management and operating overhead. Public cloud edge capabilities will be leveraged to most possible extent to increase the consistency and lower the overhead. The closest “OSDU Standard Region” will be configured as a proxy for in-country region's Subsurface Work Product Services (SWPS) metadata represent files within the in-country OSDU Region and providing home region replication.
List of OSDU Regions, and the identification of the Central and Admin OSDU Region will be part of the SMDS Master Data and will be administered as part of the Admin service.M1 - Release 0.1Stephen Whitley (Invited Expert)Hrvoje MarkovicFerris ArgyleRaj KannanDania Kodeih (Microsoft)JoeAlan DonigerBrian KirklandStephen Whitley (Invited Expert)https://community.opengroup.org/osdu/platform/system/register/-/issues/13[Register Service] ServiceRole names are inconsistent with other services2021-06-16T22:18:18ZRucha Deshpande[Register Service] ServiceRole names are inconsistent with other servicesThe Service roles are defined as follows:
```
public final class ServiceRole {
public static final String VIEWER = "users.datalake.viewers";
public static final String EDITOR = "users.datalake.editors";
public static final S...The Service roles are defined as follows:
```
public final class ServiceRole {
public static final String VIEWER = "users.datalake.viewers";
public static final String EDITOR = "users.datalake.editors";
public static final String ADMIN = "users.datalake.admins";
public static final String OPS = "users.datalake.ops";
public static final String CRON = "cron.job";
}
```
- They are inconsistent with the other service role names. For example, storage has "service.storage.admin" etc
- Service roles in Register service are defined as 'User' roles.
- CRON role should be prefixed with the service name for better identification
Proposed change:
```
public final class ServiceRole {
public static final String VIEWER = "service.register.viewer";
public static final String EDITOR = "service.register.editor";
public static final String ADMIN = "service.register.admin";
public static final String OPS = "service.register.ops";
public static final String CRON = "service.register.cron";
}
```M1 - Release 0.1ethiraj krishnamanaiduJoeRucha DeshpandeMatt Wiseethiraj krishnamanaiduhttps://community.opengroup.org/osdu/platform/system/file/-/issues/13Complete CI/CD and Integration Tests for Azure2020-10-15T19:39:39ZDania Kodeih (Microsoft)Complete CI/CD and Integration Tests for AzureM1 - Release 0.1https://community.opengroup.org/osdu/platform/system/file/-/issues/12Complete Azure File Service CI/CD and Tests2020-09-16T20:28:29ZDania Kodeih (Microsoft)Complete Azure File Service CI/CD and TestsM1 - Release 0.1Dania Kodeih (Microsoft)Dania Kodeih (Microsoft)https://community.opengroup.org/osdu/platform/system/register/-/issues/12[Register Service] IGoogleServiceAccount is defined in register-core2021-06-16T22:18:19ZRucha Deshpande[Register Service] IGoogleServiceAccount is defined in register-coreIGoogleServiceAccount is defined in register-core. This forces the providers to define bean of type IGoogleServiceAccount. This interface should not exist in the core code.
```
package org.opengroup.osdu.register.utils;
public interface...IGoogleServiceAccount is defined in register-core. This forces the providers to define bean of type IGoogleServiceAccount. This interface should not exist in the core code.
```
package org.opengroup.osdu.register.utils;
public interface IGoogleServiceAccount {
String getIdToken(String keyString, String audience);
String getPrivateKeyId(String keyString);
}
```M1 - Release 0.1ethiraj krishnamanaiduJoeRucha DeshpandeMatt Wiseethiraj krishnamanaiduhttps://community.opengroup.org/osdu/platform/security-and-compliance/home/-/issues/43E&O Documentation Inputs2021-06-08T19:05:34ZKerry BlinstonE&O Documentation Inputs**Context and Scope**
Ahead of R3 release the broader E&O team needs to provide documentation explaining the options for enforcing entitlement. We will also need to provide test cases to enable certification. At the completion of the ...**Context and Scope**
Ahead of R3 release the broader E&O team needs to provide documentation explaining the options for enforcing entitlement. We will also need to provide test cases to enable certification. At the completion of the incubator we will need a handover of this information so we can proceed to create the documentation.
**Decision**
We will need the decision to be made about how the 'combined' approach to AuthZ is going to work. In simple terms what use cases can be satisfied by use of the legal tags and creation/population of the ACL's (the static approach), versus those that use the policy engine (the dynamic approach).
We also need to agree the resolve the current naming of the different services and ensure that they reflect the use/purpose and 'make sense' to the end-user
**Rationale**
Needed ahead of testing and release.
**Consequences**
We need this issue to be resolved so that we can complete the release documentation. It is also needed so that we can map the different entitlement use cases to the different possible approaches and judge the impact on performance of different decisions/mechanisms.
**Decision Criteria and Trade Offs**
- Performance
- UsabilityM1 - Release 0.1Kerry BlinstonKerry Blinston2020-09-30https://community.opengroup.org/osdu/platform/system/partition/-/issues/2Partition Service Should Support Updating Existing Secrets in a Partition2020-12-02T21:49:32ZMatt WisePartition Service Should Support Updating Existing Secrets in a Partition## Status
- [X] Proposed
- [ ] Trialing
- [ ] Under review
- [ ] Approved
- [ ] Retired
## Context & Scope
The Partition Service currently supports 3 API Operations.
1. Create Partition (POST)
2. Get Partition (GET)
3. Delete Partition...## Status
- [X] Proposed
- [ ] Trialing
- [ ] Under review
- [ ] Approved
- [ ] Retired
## Context & Scope
The Partition Service currently supports 3 API Operations.
1. Create Partition (POST)
2. Get Partition (GET)
3. Delete Partition (DELETE)
Currently, the only way to add new secrets or update existing ones to the KV store for a partition is to delete it and recreate it. This leads to extra API calls and risks missing putting back some data that was not meant to be touched.
## Decision
If this is implemented, KV stores will be simpler to update existing values on without having to delete existing data.
## Rationale
Partitions need to be updated with new secrets and edits to existing ones. In order to minimize the complexity of this operation for external callers, having a route to support this will simplify partition editing processes.
## Consequences
New API route in partition has to be supported/tested by all implementors of the serviceM1 - Release 0.1ethiraj krishnamanaiduDania Kodeih (Microsoft)JoeChris ZhangRucha DeshpandeMatt Wiseethiraj krishnamanaiduhttps://community.opengroup.org/osdu/platform/system/partition/-/issues/1Partition API routes are missing standard OSDU Roles2023-09-27T12:19:38ZMatt WisePartition API routes are missing standard OSDU RolesRoutes in the partition service use a custom authorization filter that does not use Roles, but rather hardcodes a specific account type 'DomainAdminServiceAccount'.
pre-auth block in API methods:
```java
@PreAuthorize("@authorizationFil...Routes in the partition service use a custom authorization filter that does not use Roles, but rather hardcodes a specific account type 'DomainAdminServiceAccount'.
pre-auth block in API methods:
```java
@PreAuthorize("@authorizationFilter.hasPermissions()")
```
Authorization filter has the following function:
```java
public boolean hasPermissions() {
return authorizationService.isDomainAdminServiceAccount();
}
```
**Should the partition service use standard roles rather than user types to be compliant with other services?**
Other services use the following:
```java
@PreAuthorize("@authorizationFilter.hasRole('" + DeliveryRole.VIEWER + "')")
```M1 - Release 0.1ethiraj krishnamanaiduJoeMatt Wiseethiraj krishnamanaiduhttps://community.opengroup.org/osdu/platform/system/storage/-/issues/32[Storage Service] Bulk ACL Integration tests do not cover revertObjectMetadat...2021-06-16T22:19:34ZRucha Deshpande[Storage Service] Bulk ACL Integration tests do not cover revertObjectMetadata functionalityThe Bulk ACL integration tests do not cover the revertObjectMetadata functionality.The Bulk ACL integration tests do not cover the revertObjectMetadata functionality.M1 - Release 0.1ethiraj krishnamanaiduJoeRucha DeshpandeMatt Wiseethiraj krishnamanaiduhttps://community.opengroup.org/osdu/platform/system/storage/-/issues/31[Storage Service] Integration tester ACL hardcoded for the bulk-acl integrati...2021-06-16T22:19:35ZRucha Deshpande[Storage Service] Integration tester ACL hardcoded for the bulk-acl integration testThe integration tester ACL used in the new bulk acl integration test is hardcoded in org.opengroup.osdu.storage.util\TestUtils.java.
This assumes the integration test user is already in that entitlements group.
` public static final S...The integration tester ACL used in the new bulk acl integration test is hardcoded in org.opengroup.osdu.storage.util\TestUtils.java.
This assumes the integration test user is already in that entitlements group.
` public static final String getIntegrationTesterAcl() {
return String.format("data.integration.test@%s", getAclSuffix());
}
`
An environment variable should be used instead.M1 - Release 0.1ethiraj krishnamanaiduJoeRucha DeshpandeMatt Wiseethiraj krishnamanaidu