Policy issueshttps://community.opengroup.org/osdu/platform/security-and-compliance/policy/-/issues2021-07-21T12:40:26Zhttps://community.opengroup.org/osdu/platform/security-and-compliance/policy/-/issues/16IBM Continuation of E&O workflow support2021-07-21T12:40:26Zjingdong sunIBM Continuation of E&O workflow supportM7 - Release 0.10Anuj GuptaShaonjingdong sunAnuj Guptahttps://community.opengroup.org/osdu/platform/security-and-compliance/policy/-/issues/10Operational readiness for policy service - bootstrap default policies2022-08-23T11:19:18ZHrvoje MarkovicOperational readiness for policy service - bootstrap default policiesInclude default policies in the bootstrap of the system that ensure the same behavior we have now.Include default policies in the bootstrap of the system that ensure the same behavior we have now.M7 - Release 0.10https://community.opengroup.org/osdu/platform/security-and-compliance/policy/-/issues/21Policy service bootstrapping doesn't work with Entitlements v22021-08-11T14:27:27ZKateryna Kurach (EPAM)Policy service bootstrapping doesn't work with Entitlements v2Step to reproduce:
1. Activate SuperAdmin Service Account datafier@osdu-cicd-epam.iam.gserviceaccount.com
2. Load environments required for bootstrap
export GCP_POLICY_SERVICE_URL=https://dev.osdu-gcp.go3-nrg.projects.epam.com/api/poli...Step to reproduce:
1. Activate SuperAdmin Service Account datafier@osdu-cicd-epam.iam.gserviceaccount.com
2. Load environments required for bootstrap
export GCP_POLICY_SERVICE_URL=https://dev.osdu-gcp.go3-nrg.projects.epam.com/api/policy/v1/policies
BEARER_TOKEN=`gcloud auth print-access-token`
export BEARER_TOKEN=$BEARER_TOKEN
export DATA_PARTITION=opendes
3. Run script
cd policy
python3 deployment/scripts/BootstrapDefaultPolicies.py -u $GCP_POLICY_SERVICE_URL
4. Get response:
(see screenshot 1)
![scr1](/uploads/365772bc2e2ab08ee1f6c36b4f4c4255/scr1.png)
Policy service running with next variables:
(see screenshot 2)
![scr2](/uploads/ed9071ebe50d0d2b31186aad5625301d/scr2.jpg)
Account datafier@osdu-cicd-epam.iam.gserviceaccount.com included in all groups ( including service.policy.admin & service.policy.user)
(see screenshot 3)
![scr3](/uploads/7784028f307ab2d815685ea2ef2ec65c/scr3.png)
Same situation with other SA. On Entitlements V2 we not see any errors or warnings during execute bootstrap request
Policy Service response that it healthy and online.
![scr4](/uploads/c2bb89b1140b6c853f5b5c075ef23e3d/scr4.png)M8 - Release 0.11https://community.opengroup.org/osdu/platform/security-and-compliance/policy/-/issues/20Fix Search return fields Issue (Blocker)2022-02-01T04:49:12ZAsh SathyaseelanFix Search return fields Issue (Blocker)M9 - Release 0.12Hrvoje MarkovicHrvoje Markovichttps://community.opengroup.org/osdu/platform/security-and-compliance/policy/-/issues/17Policy service fails to get legaltags' descriptions2022-02-01T04:42:02ZHanna KavalionakPolicy service fails to get legaltags' descriptionsThe Policy service requests the descriptions of all legaltags assigned to records.
The Legal service legaltags:batchRetrieve request is limited to 25 legaltags as input. The Search service can request policy evaluation for thousands of ...The Policy service requests the descriptions of all legaltags assigned to records.
The Legal service legaltags:batchRetrieve request is limited to 25 legaltags as input. The Search service can request policy evaluation for thousands of records. So, we can't use legaltag description info for writing policy rules.
Example:
Lets say that there are 100 file records (opendes partition is used) that we are going to search for:
query : _{
"kind": "opendes:wks:dataset--File.Generic:1.0.0",
"limit" :100
}_
Set of records: [test_data.txt](/uploads/9ff2e68215b35db4b6aaaf35f83b194a/test_data.txt)
The Legal service log:
```
AppException(error=AppError(code=400, reason=Validation failed., message=Validation failed., errors=null, debuggingInfo=null, originalException=org.springframework.web.bind.MethodArgumentNotValidException: Validation failed for argument [0] in public org.springframework.http.ResponseEntity<org.opengroup.osdu.legal.tags.dto.LegalTagDtos> org.opengroup.osdu.legal.api.LegalTagApi.getLegalTags(org.opengroup.osdu.legal.tags.dto.RequestLegalTags): [Field error in object 'requestLegalTags' on field 'names': rejected value [[opendes-public-usa-dataset-epam, opendes-public-usa-dataset-epam, opendes-public-usa-dataset-epam, opendes-public-usa-dataset-epam, opendes-public-usa-dataset-epam, opendes-public-usa-dataset-epam, opendes-public-usa-dataset-epam, opendes-public-usa-dataset-epam, opendes-public-usa-dataset-epam, opendes-public-usa-dataset-epam, opendes-public-usa-dataset-epam, opendes-public-usa-dataset-epam, opendes-public-usa-dataset-epam, opendes-public-usa-dataset-epam, opendes-public-usa-dataset-epam, opendes-public-usa-dataset-epam, opendes-public-usa-dataset-epam, opendes-public-usa-dataset-epam, opendes-public-usa-dataset-epam, opendes-public-usa-dataset-epam, opendes-public-usa-dataset-epam, opendes-public-usa-dataset-epam, opendes-public-usa-dataset-epam, opendes-public-usa-dataset-epam, opendes-public-usa-dataset-epam, opendes-public-usa-dataset-epam, opendes-public-usa-dataset-epam, opendes-public-usa-dataset-epam, opendes-public-usa-dataset-epam, opendes-public-usa-dataset-epam, opendes-public-usa-dataset-epam, opendes-public-usa-dataset-epam, opendes-public-usa-dataset-epam, opendes-public-usa-dataset-epam, opendes-public-usa-dataset-epam, opendes-public-usa-dataset-epam, opendes-public-usa-dataset-epam, opendes-public-usa-dataset-epam, opendes-public-usa-dataset-epam, opendes-public-usa-dataset-epam, opendes-public-usa-dataset-epam, opendes-public-usa-dataset-epam, opendes-public-usa-dataset-epam, opendes-public-usa-dataset-epam, opendes-public-usa-dataset-epam, opendes-public-usa-dataset-epam, opendes-public-usa-dataset-epam, opendes-public-usa-dataset-epam, opendes-public-usa-dataset-epam, opendes-public-usa-dataset-epam]]; codes [Size.requestLegalTags.names,Size.names,Size.java.util.List,Size]; arguments [org.springframework.context.support.DefaultMessageSourceResolvable: codes [requestLegalTags.names,names]; arguments []; default message [names],25,1]; default message [size must be between 1 and 25]] ), originalException=org.springframework.web.bind.MethodArgumentNotValidException: Validation failed for argument [0] in public org.springframework.http.ResponseEntity<org.opengroup.osdu.legal.tags.dto.LegalTagDtos>
```M9 - Release 0.12https://community.opengroup.org/osdu/platform/security-and-compliance/policy/-/issues/33Bundle POC - Evaluation for OPA based caching and direct call to OPA2022-02-01T05:14:47ZHrvoje MarkovicBundle POC - Evaluation for OPA based caching and direct call to OPAConduct a POC to evaluate a change in design to use OPA bundles and directly call OPA from storage. This should confirm that the new design resolves previously reported performance issues.Conduct a POC to evaluate a change in design to use OPA bundles and directly call OPA from storage. This should confirm that the new design resolves previously reported performance issues.M10 - Release 0.13https://community.opengroup.org/osdu/platform/security-and-compliance/policy/-/issues/7Operational readiness for policy service - caches for performance2022-02-01T04:38:12ZHrvoje MarkovicOperational readiness for policy service - caches for performanceReplicate entitlements groups and legal tags in OPA as documents to ensure the best performance. Cache responses in policy service to ensure best performance.Replicate entitlements groups and legal tags in OPA as documents to ensure the best performance. Cache responses in policy service to ensure best performance.M10 - Release 0.13Hrvoje MarkovicNitesh SelkariHrvoje Markovichttps://community.opengroup.org/osdu/platform/security-and-compliance/policy/-/issues/6Operational readiness for policy service - backup and restore2022-02-01T05:03:07ZHrvoje MarkovicOperational readiness for policy service - backup and restoreEnsure backup and restore of policies.
more info: https://community.opengroup.org/osdu/platform/security-and-compliance/policy/-/wikis/Persisting-policies-and-autoscalingEnsure backup and restore of policies.
more info: https://community.opengroup.org/osdu/platform/security-and-compliance/policy/-/wikis/Persisting-policies-and-autoscalingM10 - Release 0.13Hrvoje MarkovicHrvoje Markovichttps://community.opengroup.org/osdu/platform/security-and-compliance/policy/-/issues/4Operational readiness for policy service - performance and scalability tests2022-08-16T08:54:49ZHrvoje MarkovicOperational readiness for policy service - performance and scalability testsSet up performance and scalability tests for policy service integration.
They define acceptance criteria to switch to dynamic policies from current (acl only) based solution.Set up performance and scalability tests for policy service integration.
They define acceptance criteria to switch to dynamic policies from current (acl only) based solution.M10 - Release 0.13Hrvoje MarkovicDenis Karpenok (EPAM)Hrvoje Markovichttps://community.opengroup.org/osdu/platform/security-and-compliance/policy/-/issues/50Bundle file read/write SPI for CSP - GCP2022-06-08T12:15:15ZHrvoje MarkovicBundle file read/write SPI for CSP - GCPImplement SPI to read/write bundle file so that it can be manipulated through policy service.
Related to #25.Implement SPI to read/write bundle file so that it can be manipulated through policy service.
Related to #25.M11 - Release 0.14Siarhei Khaletski (EPAM)Siarhei Khaletski (EPAM)https://community.opengroup.org/osdu/platform/security-and-compliance/policy/-/issues/46Bundle bootstrapping for CSP - GCP2022-03-28T15:59:44ZHrvoje MarkovicBundle bootstrapping for CSP - GCPBootstrap the default bundle for dynamic policies.Bootstrap the default bundle for dynamic policies.M11 - Release 0.14https://community.opengroup.org/osdu/platform/security-and-compliance/policy/-/issues/41OPA bundle server for CSP - GCP2022-03-28T16:00:13ZHrvoje MarkovicOPA bundle server for CSP - GCPImplement OPA bundle server as described [here ](https://community.opengroup.org/osdu/platform/security-and-compliance/policy/-/wikis/Persisting-policies-and-autoscaling#required-changes).
Related to #25.Implement OPA bundle server as described [here ](https://community.opengroup.org/osdu/platform/security-and-compliance/policy/-/wikis/Persisting-policies-and-autoscaling#required-changes).
Related to #25.M11 - Release 0.14https://community.opengroup.org/osdu/platform/security-and-compliance/policy/-/issues/34New policies that use OPA requests and caching2022-11-02T13:29:41ZHrvoje MarkovicNew policies that use OPA requests and cachingProvide new policies that use OPA request and caching for entitlements groups and legal tags. Initial contribution is done here: https://community.opengroup.org/osdu/platform/security-and-compliance/policy/-/tree/add-dynamic-policy-examp...Provide new policies that use OPA request and caching for entitlements groups and legal tags. Initial contribution is done here: https://community.opengroup.org/osdu/platform/security-and-compliance/policy/-/tree/add-dynamic-policy-examples/policy-examples/load%20data%20dynamically%20POC
The scope is to create a data authz policy that supports all the data operations
-Create
-Update
-Delete
-Retrieve
- This needs to be integrated with Storage
- The interface of the contract should be validated by SDMS (doesnt need to be integrated)
- This should replicate the existing system behavior of entitlements and legal enforcement. Refer to the POC policies for reference in the link above
- As the system functional behavior doesnt change all integration tests should pass except where caching is applied and so causes a delay in updates to ACLs by 10 seconds. This is the only acceptable change in behavior
- We should validate performance has not dropped form before
Also
- We should think about how the implementation can be extended in the future when we have custom policies. A custom policy is one provided by a client of the system after osdu is deployed. This will create a distinction between system policies which is what we are defining here and what clients override them with.
This may mean a modular approach to creating policies so clients creating their own policies could re-use them. This will likely change as this feature becomes concrete but we should consider the problem in the design of the policies.M11 - Release 0.14Kelly ZhouKelly Zhouhttps://community.opengroup.org/osdu/platform/security-and-compliance/policy/-/issues/49Bundle file read/write SPI for CSP - AWS2022-11-02T13:44:32ZHrvoje MarkovicBundle file read/write SPI for CSP - AWSImplement SPI to read/write bundle file so that it can be manipulated through policy service.
Related to #25.Implement SPI to read/write bundle file so that it can be manipulated through policy service.
Related to #25.M12 - Release 0.15https://community.opengroup.org/osdu/platform/security-and-compliance/policy/-/issues/48Bundle file read/write SPI for CSP - Azure2022-06-23T19:08:13ZHrvoje MarkovicBundle file read/write SPI for CSP - AzureImplement SPI to read/write bundle file so that it can be manipulated through policy service.
Related to #25.Implement SPI to read/write bundle file so that it can be manipulated through policy service.
Related to #25.M12 - Release 0.15https://community.opengroup.org/osdu/platform/security-and-compliance/policy/-/issues/45Bundle bootstrapping for CSP - AWS2022-11-02T13:27:06ZHrvoje MarkovicBundle bootstrapping for CSP - AWSBootstrap the default bundle for dynamic policies.Bootstrap the default bundle for dynamic policies.M12 - Release 0.15https://community.opengroup.org/osdu/platform/security-and-compliance/policy/-/issues/44Bundle bootstrapping for CSP - Azure2022-06-23T19:09:50ZHrvoje MarkovicBundle bootstrapping for CSP - AzureBootstrap the default bundle for dynamic policies.Bootstrap the default bundle for dynamic policies.M12 - Release 0.15https://community.opengroup.org/osdu/platform/security-and-compliance/policy/-/issues/43Change storage to call OPA directly2022-09-30T12:18:44ZHrvoje MarkovicChange storage to call OPA directlyImplement change in storage so that OPA is called directly (not via policy service). Make this behavior default so that storage switches to dynamic policies.
Related to #33Implement change in storage so that OPA is called directly (not via policy service). Make this behavior default so that storage switches to dynamic policies.
Related to #33M12 - Release 0.15https://community.opengroup.org/osdu/platform/security-and-compliance/policy/-/issues/42OPA bundle server for CSP - IBM2022-10-03T16:02:12ZHrvoje MarkovicOPA bundle server for CSP - IBMImplement OPA bundle server as described [here ](https://community.opengroup.org/osdu/platform/security-and-compliance/policy/-/wikis/Persisting-policies-and-autoscaling#required-changes).
Related to #25.Implement OPA bundle server as described [here ](https://community.opengroup.org/osdu/platform/security-and-compliance/policy/-/wikis/Persisting-policies-and-autoscaling#required-changes).
Related to #25.M12 - Release 0.15https://community.opengroup.org/osdu/platform/security-and-compliance/policy/-/issues/39OPA bundle server for CSP - Azure2022-06-23T19:15:22ZHrvoje MarkovicOPA bundle server for CSP - AzureImplement OPA bundle server as described [here ](https://community.opengroup.org/osdu/platform/security-and-compliance/policy/-/wikis/Persisting-policies-and-autoscaling#required-changes).
Related to #25.Implement OPA bundle server as described [here ](https://community.opengroup.org/osdu/platform/security-and-compliance/policy/-/wikis/Persisting-policies-and-autoscaling#required-changes).
Related to #25.M12 - Release 0.15