... | ... | @@ -77,6 +77,37 @@ How to identify requesting services is an OSDU decision. If it comes as a header |
|
|
|
|
|
The 'Read' and 'Delete' AuthZ flow will come after the initial test on the create/update flow if it is successful as it requires being able to retrieve the storage records directly using just an ID and without the current AuthZ flow which will need more development effort
|
|
|
|
|
|
### Setup
|
|
|
We were using the new policies with a bundle server setup on an Azure OSDU deployment in one of our integration environments.
|
|
|
|
|
|
We ran 10 OPA pods, this was probably too high but we dont have monitoring around OPA and didnt want the number of instances to be a bottleneck.
|
|
|
|
|
|
We analyzed real traffic over a 7 day period with the integration wof OPA and the new policies and contrasted that with the same environment over different time periods over the previous weeks when entitleemnts and legal was in use directly.
|
|
|
|
|
|
We chose this method as we wanted to analyze the qualities with real traffic and the variety of request types and spikes we see in real usage from a variety of sources as opposed to a controlled load test that looked at only a couple of usage scenarios.
|
|
|
|
|
|
### Results
|
|
|
Below we show the usage statistics of the PUT Storage API over different time periods and compare that to when it was using OPA and the new policies.
|
|
|
|
|
|
|
|
|
| Time period | Requests | 4xx results | 5xx results | Availability| Latency ms 50% | Latency ms 95% | Latency ms 99% |
|
|
|
|--|--|--|--|--|--|--|--|
|
|
|
| **24th -30th Jan (OPA in use)** | 865,419 | 2696 | 426 | 99.95% | 226 | 5285 | 65389 |
|
|
|
| 17th-23rd Jan | 162,308 | 3242 | 360 | 99.78% | 541 | 20629 | 78986 |
|
|
|
| 10th-16th Jan | 127,920 | 11314 | 1291 | 98.99% | 662 | 37001 | 105362 |
|
|
|
| 13th-19th Dec | 516,277 | 308 | 58 | 99.99% | 1392 | 3563 | 35768 |
|
|
|
| 6th-12th Dec | 312,350 | 336 | 76 | 99.98% | 2501 | 12231 | 50720 |
|
|
|
| 8th-14th Nov | 2,190,926 | 58222| 3202 | 99.85% | 1431 | 17172 | 60561 |
|
|
|
|
|
|
Overall we can see that with OPA turned on we don't see any degradation in error rate or latencies compared to previous weeks, showing the solution does not appear to negatively impact the OSDU system.
|
|
|
|
|
|
|
|
|
### Recommendations
|
|
|
- Work needs to continue to validate the interface with SDMS and Well Planning services
|
|
|
- Work should continue to integrate the other data operations with storage and further measurements taken
|
|
|
- Work should be done to see if we can optimize the caching strategy for entitlements to use the standard 'cache-control' header
|
|
|
- Work needs to be done to see if OPA can automatically transform given headers it receives to be part of the input to minimize the coupling between it and calling services.
|
|
|
|
|
|
### Reference
|
|
|
|
|
|
- https://www.openpolicyagent.org/docs/latest/external-data/#summary
|
... | ... | |