Partition issueshttps://community.opengroup.org/osdu/platform/system/partition/-/issues2023-11-21T19:08:57Zhttps://community.opengroup.org/osdu/platform/system/partition/-/issues/35Add /liveness_check2023-11-21T19:08:57ZRiabokon Stanislav(EPAM)[GCP]Add /liveness_checkNeed to add the endpoint '/liveness_check' in order to verify the operational status of the Partition Service.Need to add the endpoint '/liveness_check' in order to verify the operational status of the Partition Service.M22 - Release 0.25Riabokon Stanislav(EPAM)[GCP]Riabokon Stanislav(EPAM)[GCP]https://community.opengroup.org/osdu/platform/system/partition/-/issues/33Use a Secret service for storing and fetching secrets and sensitive configura...2023-07-25T10:25:36ZRustam Lotsmanenko (EPAM)rustam_lotsmanenko@epam.comUse a Secret service for storing and fetching secrets and sensitive configurations.Rustam Lotsmanenko (EPAM)rustam_lotsmanenko@epam.comRustam Lotsmanenko (EPAM)rustam_lotsmanenko@epam.comhttps://community.opengroup.org/osdu/platform/system/partition/-/issues/31upgrade azure-storage SDK2023-01-18T21:31:34ZNur Sheikhupgrade azure-storage SDKIn partition service we are using the azure-storage sdk 8.6.5 from com.microsoft.azure package which is too old and not having much support. It iis advisable to use the latest sdk for com.azure package.In partition service we are using the azure-storage sdk 8.6.5 from com.microsoft.azure package which is too old and not having much support. It iis advisable to use the latest sdk for com.azure package.https://community.opengroup.org/osdu/platform/system/partition/-/issues/30Is there a solution to easily delete all the data related to a partition?2023-08-07T09:57:33ZShuai LiIs there a solution to easily delete all the data related to a partition?If we call the delete partition API, only the partition record is deleted from partition service. All the data related to this partition (e.g. file, storage records, user groups) remains in their locations.
Since partion is a "data parti...If we call the delete partition API, only the partition record is deleted from partition service. All the data related to this partition (e.g. file, storage records, user groups) remains in their locations.
Since partion is a "data partition", if I delete a data partition, I want to delete all the related data in this data partition. I probably need to call many other API to clean up the data. Not all services provide APIs to easily delete all data related to a partition.
Is there any consideration on this requirement?https://community.opengroup.org/osdu/platform/system/partition/-/issues/25Upgrade to Log4J 2.172021-12-21T00:29:17ZDavid Diederichd.diederich@opengroup.orgUpgrade to Log4J 2.17The Apache Foundation released another Log4j2 update, version 2.17, which address a denial of service vulnerability.
This issue tracks progress to upgrade this dependency for this project.The Apache Foundation released another Log4j2 update, version 2.17, which address a denial of service vulnerability.
This issue tracks progress to upgrade this dependency for this project.https://community.opengroup.org/osdu/platform/system/partition/-/issues/24Log4J Expedient Updates and Patches2021-12-15T17:15:30ZDavid Diederichd.diederich@opengroup.orgLog4J Expedient Updates and PatchesThis issue associates MRs that were applied to this project quickly to get a patched version ready as soon as possible. The intent is to provide a reference point for later, more thoughtful, analysis.This issue associates MRs that were applied to this project quickly to get a patched version ready as soon as possible. The intent is to provide a reference point for later, more thoughtful, analysis.https://community.opengroup.org/osdu/platform/system/partition/-/issues/23Apache log4j CVE-2021-442282021-12-14T15:56:43ZDmitrii GerashchenkoApache log4j CVE-2021-44228https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-44228
Zero-day vulnerability affects log4j and can lead to remote code execution. This is a critical issue and needs to be resolved as soon as possible.
---
Apache Log4j2 <=2.14...https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-44228
Zero-day vulnerability affects log4j and can lead to remote code execution. This is a critical issue and needs to be resolved as soon as possible.
---
Apache Log4j2 <=2.14.1 JNDI features used in configuration, log messages, and parameters do not protect against attacker controlled LDAP and other JNDI related endpoints. An attacker who can control log messages or log message parameters can execute arbitrary code loaded from LDAP servers when message lookup substitution is enabled. From log4j 2.15.0, this behavior has been disabled by default. In previous releases (>2.10) this behavior can be mitigated by setting system property "log4j2.formatMsgNoLookups" to “true” or it can be mitigated in prior releases (<2.10) by removing the JndiLookup class from the classpath (example: zip -q -d log4j-core-\*.jar org/apache/logging/log4j/core/lookup/JndiLookup.class).https://community.opengroup.org/osdu/platform/system/partition/-/issues/22API spec for partition service can not be loaded in Gitlab2021-11-25T13:16:02ZDmitrii GerashchenkoAPI spec for partition service can not be loaded in GitlabAPI spec for partition service can not be loaded in Gitlab: https://community.opengroup.org/osdu/platform/system/partition/-/blob/master/docs/api/partition_openapi.yaml
![image](/uploads/7d888ca0cfd788037c134f302f68d10b/image.png)
Path...API spec for partition service can not be loaded in Gitlab: https://community.opengroup.org/osdu/platform/system/partition/-/blob/master/docs/api/partition_openapi.yaml
![image](/uploads/7d888ca0cfd788037c134f302f68d10b/image.png)
Path "/actuator/health" is duplecated.Dmitrii GerashchenkoDmitrii Gerashchenkohttps://community.opengroup.org/osdu/platform/system/partition/-/issues/21Use HPA for kubernetes service2021-11-05T18:56:36ZRostislav Vatolinvatolinrp@gmail.comUse HPA for kubernetes serviceImplement practices described here: https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/Implement practices described here: https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/https://community.opengroup.org/osdu/platform/system/partition/-/issues/20Update the partitionListCache without rebuild on partition create, delete2022-11-24T11:47:11ZDmitrii GerashchenkoUpdate the partitionListCache without rebuild on partition create, deleteFor now, the `partitionListCache` is purged on `createPartition` or `deletePartition` invocation that leads to the unnecessary request to storage on the next invocation of `getAllPartitions` method.
To optimize this behavior the `partit...For now, the `partitionListCache` is purged on `createPartition` or `deletePartition` invocation that leads to the unnecessary request to storage on the next invocation of `getAllPartitions` method.
To optimize this behavior the `partitionListCache` could be updated on `createPartition` or `deletePartition` invocation without request to storage.https://community.opengroup.org/osdu/platform/system/partition/-/issues/19MS CloudTableClient has not timeouts2021-10-01T11:44:22ZDmitrii GerashchenkoMS CloudTableClient has not timeoutsMS TableStorage's client - CloudTableClient uses default timeout settings.
The client can try to connect to the MS server for up to 2 minutes: 3 retry attempts with 30 seconds delay between attempts.
The MaximumExecutionTime is null.
I...MS TableStorage's client - CloudTableClient uses default timeout settings.
The client can try to connect to the MS server for up to 2 minutes: 3 retry attempts with 30 seconds delay between attempts.
The MaximumExecutionTime is null.
I created a dummy server and tested the case when MS TableStorage responds with latency. There is no timeout for a response in the client so the client could be blocked infinitely.
The client doesn't throw errors on long TableStorage's latencies what could be the cause of 504 errors for API consumers.
Also, it means that we can't see any exceptions even if MS TableStorage responds with latencies.Dmitrii GerashchenkoDmitrii Gerashchenkohttps://community.opengroup.org/osdu/platform/system/partition/-/issues/18partition-core shouldn't contain SPI implementations2021-11-08T10:08:02ZDmitrii Gerashchenkopartition-core shouldn't contain SPI implementationsPartitionServiceImplCache is implemented in the core module: https://community.opengroup.org/osdu/platform/system/partition/-/blob/master/partition-core/src/main/java/org/opengroup/osdu/partition/service/CachedPartitionServiceImpl.java
...PartitionServiceImplCache is implemented in the core module: https://community.opengroup.org/osdu/platform/system/partition/-/blob/master/partition-core/src/main/java/org/opengroup/osdu/partition/service/CachedPartitionServiceImpl.java
Partition-core shouldn't contain SPI implementations. SPI should be implemented in CSPs.
Also, it makes some problems within CSP if it needs to use some special logic or even if it doesn't need cache at all: https://community.opengroup.org/osdu/platform/system/partition/-/blob/master/provider/partition-aws/src/main/java/org/opengroup/osdu/partition/provider/aws/service/PartitionServiceDummyListCacheImpl.java#L24
CachedPartitionServiceImpl doesn't contain any complicated logic so it could be combined with PartitionServiceImpl within providers modules or even be removed for ones that don't need it.Dmitrii GerashchenkoDmitrii Gerashchenkohttps://community.opengroup.org/osdu/platform/system/partition/-/issues/17Add memory limits2021-09-28T15:39:46ZRostislav Vatolinvatolinrp@gmail.comAdd memory limitsAdding memory limits:
AKS node autoscaler uses memory limits to add nodes to cluster when HPA(Horizontal Pod Autoscaler) needs more capacity. Values were experimentally determined. Given implementation allows managing the list of envs w...Adding memory limits:
AKS node autoscaler uses memory limits to add nodes to cluster when HPA(Horizontal Pod Autoscaler) needs more capacity. Values were experimentally determined. Given implementation allows managing the list of envs where limits are enabled.https://community.opengroup.org/osdu/platform/system/partition/-/issues/16Partition service's (azure-provider) latency is more than 300 seconds2021-10-01T11:44:27ZDmitrii GerashchenkoPartition service's (azure-provider) latency is more than 300 secondsThere are latencies (more than 300 seconds) on Partition API (azure-provider).
An inspection showed that there is 2 minutes timeout for Azure TableStorage which can be the cause of the latencies.
10 minutes latency reproduced locally w...There are latencies (more than 300 seconds) on Partition API (azure-provider).
An inspection showed that there is 2 minutes timeout for Azure TableStorage which can be the cause of the latencies.
10 minutes latency reproduced locally with the following conditions:
1. Endpoints GET /api/partition/v1/partitions or /api/partition/v1/partitions/{partitionId}
2. Not data in cache.
3. Azure Table storage is unavailable or responding too slow.
4. Many requests to API (more than 500).
Presumably, if a cache became outdated during high-load many simultaneous requests are send to TableStorage.
All requests which were sent before TableStorage response caching will create new requests to TableStorage and will be waiting for response up to 2 minutes. Finally, the API latency grows.
The solution is to use a cluster lock during the request to TableStorage. It's a copy of this solution from the Entitlements repository:
https://community.opengroup.org/osdu/platform/security-and-compliance/entitlements/-/blob/master/provider/entitlements-v2-azure/src/main/java/org/opengroup/osdu/entitlements/v2/azure/service/GroupCacheServiceAzure.java#L81
@Qualifier("cachedPartitionServiceImpl") was removed to make the bean "CachedPartitionServiceImpl" overridable.
CachedPartitionServiceImpl (defined in partition-core) was redefined with ProviderCachedPartitionServiceImpl (defined in partition-azure).
CachedPartitionService interface was introduced to resolve ambiguities for beans CachedPartitionService and PartitionServiceImpl. Both of them inherit IPartitionService. Now CachedPartitionService resolves ambiguities instead of @Qualifier("cachedPartitionServiceImpl").
New code was tested with the same conditions and the latency didn't grow.Dmitrii GerashchenkoDmitrii Gerashchenkohttps://community.opengroup.org/osdu/platform/system/partition/-/issues/15Upgrade Core GCP Dependency2022-02-11T21:56:28ZDavid Diederichd.diederich@opengroup.orgUpgrade Core GCP Dependencyhttps://community.opengroup.org/osdu/platform/system/partition/-/issues/14Upgrade Core IBM Dependency2022-02-11T21:56:46ZDavid Diederichd.diederich@opengroup.orgUpgrade Core IBM Dependencyhttps://community.opengroup.org/osdu/platform/system/partition/-/issues/13Upgrade Core Common Dependency2022-02-11T21:56:50ZDavid Diederichd.diederich@opengroup.orgUpgrade Core Common Dependencyhttps://community.opengroup.org/osdu/platform/system/partition/-/issues/11Upgrade dependencies to resolve security vulnerabilities2022-11-24T12:49:06ZRostislav Vatolinvatolinrp@gmail.comUpgrade dependencies to resolve security vulnerabilitiesPartition service has dependencies with critical bugs and vulnerabilities:
Updating:
guava, because it has security vulnerabilities: [CVE-2020-8908](https://nvd.nist.gov/vuln/detail/CVE-2020-8908)
bom for netty because of: [CVE-2021-2...Partition service has dependencies with critical bugs and vulnerabilities:
Updating:
guava, because it has security vulnerabilities: [CVE-2020-8908](https://nvd.nist.gov/vuln/detail/CVE-2020-8908)
bom for netty because of: [CVE-2021-21290](https://nvd.nist.gov/vuln/detail/CVE-2021-21290) and [CVE-2021-21409](https://nvd.nist.gov/vuln/detail/CVE-2021-21409) and [CVE-2021-21295](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-21295)
Upgrading version of spring: [WS-2016-7107](https://www.whitesourcesoftware.com/vulnerability-database/WS-2016-7107) and [WS-2020-0293](https://www.whitesourcesoftware.com/vulnerability-database/WS-2020-0293) and [CVE-2020-5421](https://nvd.nist.gov/vuln/detail/CVE-2020-5421)
Fixes were applied in common libraries.https://community.opengroup.org/osdu/platform/system/partition/-/issues/10Customized readiness check API2022-11-24T12:48:15ZMingyang ZhuCustomized readiness check APIWe'd like to use spring boot built-in actuator health endpoint for the partition service health check API, and implement a customized health indicator.
Partition service implements the cache layer, different cloud provider's implementat...We'd like to use spring boot built-in actuator health endpoint for the partition service health check API, and implement a customized health indicator.
Partition service implements the cache layer, different cloud provider's implementation implements the cache differently. It will be good to make sure the cache infrastructure and connection are ready before the pod serves the traffic.
To achieve this, the service will implement the custom health indicator, and get a dummy key from the cache instance. No matter it is memcache, redis, it expects no exception to claim the pod is ready to serve the traffic. This health check can be enabled or disabled by the configuration.Mingyang ZhuMingyang Zhuhttps://community.opengroup.org/osdu/platform/system/partition/-/issues/5Partition Service - New API to list all tenants2020-11-05T17:04:18ZDuvelis CaraoPartition Service - New API to list all tenants## Context & Scope
Partition service will provide a new API :
- List all partitions Id (GET)
## Use case
Legal service CRON (legalTag status change) and Schema service requires list of all partitions.
## Consequences
Client lib for P...## Context & Scope
Partition service will provide a new API :
- List all partitions Id (GET)
## Use case
Legal service CRON (legalTag status change) and Schema service requires list of all partitions.
## Consequences
Client lib for Partition service will be updated to support this new API.ethiraj krishnamanaiduDuvelis Caraoethiraj krishnamanaidu