Policies: scalability and performance
- Under review
Context & Scope
When evaluating a policy engine, its scalability and performance are one of the critical trade-offs. We should seek to test the example policies with data volumes and complexities that represent what is considered 'normal' user scenarios.
We expect to support following volumes in an OSDU R3 data partition:
< 1e9 data items < 1000 of policies
We expect following performance: < 100 ms for policy evaluation when a typical request is made to storage or search in case of typical OSDU deployment.
We need to prove that not only does it work in the sense of enforcing policies but also that the performance is acceptable in normal usage.
Likely to be some consequences in terms of the number and complexity of policies that can be supported. Expect limitations on the granularity at which entitlement can be applied (without impacting performance)
When to revisit
Once criteria in the decision is not met.
Decision criteria and trade-offs
Granularity at which entitlement can be set versus performance Number and complexity of policies versus performance