... | ... | @@ -4,7 +4,7 @@ This will allow for better extensibility over time as we decouple the calling se |
|
|
|
|
|
The aim is to check the performance and scalability of the data authz workflow of this new design.
|
|
|
|
|
|
![image](uploads/512853e383f26277ac9264c7f3b185b2/image.png)
|
|
|
![n.drawio](uploads/d08896f915fb460d7e709150c1dff41c/n.drawio.png)
|
|
|
|
|
|
The sections in bold show the parts we will enable first. The other components are shown as an overview of what a complete solution would look like.
|
|
|
|
... | ... | @@ -82,10 +82,14 @@ We were using the new policies with a bundle server setup on an Azure OSDU deplo |
|
|
|
|
|
We ran 10 OPA pods, this was probably too high but we dont have monitoring around OPA and didnt want the number of instances to be a bottleneck.
|
|
|
|
|
|
We analyzed real traffic over a 7 day period with the integration wof OPA and the new policies and contrasted that with the same environment over different time periods over the previous weeks when entitleemnts and legal was in use directly.
|
|
|
We analyzed real traffic over a 7 day period with the integration wof OPA and the new policies and contrasted that with the same environment over different time periods over the previous weeks when entitlements and legal was in use directly.
|
|
|
|
|
|
We chose this method as we wanted to analyze the qualities with real traffic and the variety of request types and spikes we see in real usage from a variety of sources as opposed to a controlled load test that looked at only a couple of usage scenarios.
|
|
|
|
|
|
We chose to use the OPA caching directly rather than rely on HTTP cache-control as Entitlements service does not support this today. This could be an optimization added to improve the caching logic and potentially improve reliability/performance in the future as the cache-control header has support for more caching options e.g. cache while revalidate and error.
|
|
|
|
|
|
Also Storage service passes the standard OSDU headers x-user-id, data-partition id etc. directly as input to the OPA request. This increases the coupling between the two however we could not find a way for OPA to automatically transform these from the API request made to it. More work can be done here to see if the OPA server can set these automatically and reduce this coupling between calling servicesand the OPA server.
|
|
|
|
|
|
### Results
|
|
|
Below we show the usage statistics of the PUT Storage API over different time periods and compare that to when it was using OPA and the new policies.
|
|
|
|
... | ... | |