OS Core Lib Azure issueshttps://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/issues2022-09-28T15:48:40Zhttps://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/issues/28Improve performance on CosmosStore pageable query method2022-09-28T15:48:40ZRafael FreireImprove performance on CosmosStore pageable query methodCurrently, CosmosStore has a queryItemsPage which interact with a BlockingIterable object twice per call. This object uses a Thread-sleep blocking way to iterate through page results causing a slow performance for heavy queries. Its impl...Currently, CosmosStore has a queryItemsPage which interact with a BlockingIterable object twice per call. This object uses a Thread-sleep blocking way to iterate through page results causing a slow performance for heavy queries. Its implementation can be improved by substituting the check then use style for a for-each approachRafael FreireRafael Freirehttps://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/issues/24Make system token generation more robust2022-08-23T15:11:30ZArsen GrigoryanMake system token generation more robust**Summary**: There was a problem in generating a credential to access partition service from cached secret because of rotation.
The Service principal is used to access Partition service for retrieving partition-specific credentials to E...**Summary**: There was a problem in generating a credential to access partition service from cached secret because of rotation.
The Service principal is used to access Partition service for retrieving partition-specific credentials to ECK.
Java service reads environment variable once at start-up. If service principal is changed (rotated) after service startup, service is unable to access Partition service. Change of service principal requires restart of service (restart of all pods) to force re-read of service principal value. This causes unnecessary down-time (returning errors or permission failures back to client).
**Proposed solution**: Service must be more robust to pick up new service principal value when old value fails. The token generation should attempt to retrieve the key from source if generation returns 401 after using the cached secret.https://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/issues/200.13.0 Getting NoSuchMethodError at com.azure.storage.blob.specialized.BlobAs...2022-08-30T11:52:13ZTsvetelina Ivanova0.13.0 Getting NoSuchMethodError at com.azure.storage.blob.specialized.BlobAsyncClientBase.lambda$downloadStreamWithResponse$22When using version 0.13.0 we get exception NoSuchMethodError at com.azure.storage.blob.specialized.BlobAsyncClientBase.lambda$downloadStreamWithResponse$22 when try to read files from blob storage.
In version 0.13.0 a new version of azu...When using version 0.13.0 we get exception NoSuchMethodError at com.azure.storage.blob.specialized.BlobAsyncClientBase.lambda$downloadStreamWithResponse$22 when try to read files from blob storage.
In version 0.13.0 a new version of azure-storage-blob is introduced 12.13.0. In class BlobAsyncClientBase on line 1069 a call to FluxUtil.createRetriableDownloadFlux() is made.
In class FluxUtil method createRetriableDownloadFlux() does not exists. (This class is in library azure-core).
**Stack Trace:**
java.lang.NoSuchMethodError: com.azure.core.util.FluxUtil.createRetriableDownloadFlux(Ljava/util/function/Supplier;Ljava/util/function/BiFunction;IJ)Lreactor/core/publisher/Flux;
at com.azure.storage.blob.specialized.BlobAsyncClientBase.lambda$downloadStreamWithResponse$22(BlobAsyncClientBase.java:1069)
at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:113)
at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1816)
at reactor.core.publisher.MonoFlatMap$FlatMapInner.onNext(MonoFlatMap.java:249)
at reactor.core.publisher.FluxSwitchIfEmpty$SwitchIfEmptySubscriber.onNext(FluxSwitchIfEmpty.java:74)
at reactor.core.publisher.Operators$ScalarSubscription.request(Operators.java:2398)
at reactor.core.publisher.Operators$MultiSubscriptionSubscriber.set(Operators.java:2194)
at reactor.core.publisher.Operators$MultiSubscriptionSubscriber.onSubscribe(Operators.java:2068)
at reactor.core.publisher.FluxFlatMap.trySubscribeScalarMap(FluxFlatMap.java:192)
at reactor.core.publisher.MonoFlatMap.subscribeOrReturn(MonoFlatMap.java:53)
at reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:57)
at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:157)
at reactor.core.publisher.FluxContextWrite$ContextWriteSubscriber.onNext(FluxContextWrite.java:107)
at reactor.core.publisher.FluxDoOnEach$DoOnEachSubscriber.onNext(FluxDoOnEach.java:173)
at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1816)
at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:151)
at reactor.core.publisher.FluxMap$MapSubscriber.onNext(FluxMap.java:120)
at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onNext(FluxOnErrorResume.java:79)
at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1816)
at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:151)
at reactor.core.publisher.FluxDelaySubscription$DelaySubscriptionMainSubscriber.onNext(FluxDelaySubscription.java:189)
at reactor.core.publisher.SerializedSubscriber.onNext(SerializedSubscriber.java:99)
at reactor.core.publisher.SerializedSubscriber.onNext(SerializedSubscriber.java:99)
at reactor.core.publisher.FluxTimeout$TimeoutMainSubscriber.onNext(FluxTimeout.java:180)
at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1816)
at reactor.core.publisher.MonoFlatMap$FlatMapInner.onNext(MonoFlatMap.java:249)
at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:127)
at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:127)
at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1816)
at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:151)
at reactor.core.publisher.SerializedSubscriber.onNext(SerializedSubscriber.java:99)
at reactor.core.publisher.FluxRetryWhen$RetryWhenMainSubscriber.onNext(FluxRetryWhen.java:174)
at reactor.core.publisher.Operators$MonoInnerProducerBase.complete(Operators.java:2664)
at reactor.core.publisher.MonoSingle$SingleSubscriber.onComplete(MonoSingle.java:180)
at reactor.core.publisher.Operators$ScalarSubscription.request(Operators.java:2400)
at reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain.onSubscribeInner(MonoFlatMapMany.java:150)
at reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain.onNext(MonoFlatMapMany.java:189)
at reactor.core.publisher.SerializedSubscriber.onNext(SerializedSubscriber.java:99)
at reactor.core.publisher.FluxRetryWhen$RetryWhenMainSubscriber.onNext(FluxRetryWhen.java:174)
at reactor.core.publisher.MonoCreate$DefaultMonoSink.success(MonoCreate.java:165)
at reactor.netty.http.client.HttpClientConnect$HttpIOHandlerObserver.onStateChange(HttpClientConnect.java:414)
at reactor.netty.ReactorNetty$CompositeConnectionObserver.onStateChange(ReactorNetty.java:671)
at reactor.netty.resources.DefaultPooledConnectionProvider$DisposableAcquire.onStateChange(DefaultPooledConnectionProvider.java:201)
at reactor.netty.resources.DefaultPooledConnectionProvider$PooledConnection.onStateChange(DefaultPooledConnectionProvider.java:457)
at reactor.netty.http.client.HttpClientOperations.onInboundNext(HttpClientOperations.java:637)
at reactor.netty.channel.ChannelOperationsHandler.channelRead(ChannelOperationsHandler.java:93)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436)
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324)
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:311)
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:432)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276)
at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1371)
at io.netty.handler.ssl.SslHandler.decodeNonJdkCompatible(SslHandler.java:1245)
at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1285)
at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:507)
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:446)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
at io.netty.channel.kqueue.AbstractKQueueStreamChannel$KQueueStreamUnsafe.readReady(AbstractKQueueStreamChannel.java:544)
at io.netty.channel.kqueue.AbstractKQueueChannel$AbstractKQueueUnsafe.readReady(AbstractKQueueChannel.java:381)
at io.netty.channel.kqueue.KQueueEventLoop.processReady(KQueueEventLoop.java:211)
at io.netty.channel.kqueue.KQueueEventLoop.run(KQueueEventLoop.java:289)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:748)M14 - Release 0.17https://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/issues/18Version Incompatibility with azure blob storage library in multiple services2022-07-11T19:53:59Zharshit aggarwalVersion Incompatibility with azure blob storage library in multiple servicesMultiple services (storage, legal known) are facing issues after upgrading to latest core lib version due to failures while connecting with blob store
Potential issue seems to be recent version upgrade in blob libraries
https://communi...Multiple services (storage, legal known) are facing issues after upgrading to latest core lib version due to failures while connecting with blob store
Potential issue seems to be recent version upgrade in blob libraries
https://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/merge_requests/175/diffs#442292b8a7efeabbe4cc176709b833b1792140ec
cc: @kibattulHarshit SaxenaHarshit Saxenahttps://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/issues/17Issue in Publisher Facade2022-01-21T09:59:41ZAbhishek Kumar (SLB)Issue in Publisher FacadeServices using Publisher Fascare from this MR.
They would fail with below error:
```
***************************
APPLICATION FAILED TO START
***************************
Description:
Field pubSubAttributesBuilder in org.opengroup.osdu....Services using Publisher Fascare from this MR.
They would fail with below error:
```
***************************
APPLICATION FAILED TO START
***************************
Description:
Field pubSubAttributesBuilder in org.opengroup.osdu.azure.publisherFacade.EventGridPublisher required a bean of type 'org.opengroup.osdu.azure.publisherFacade.models.PubSubAttributesBuilder' that could not be found.
The injection point has the following annotations:
- @org.springframework.beans.factory.annotation.Autowired(required=true)
Action:
Consider defining a bean of type 'org.opengroup.osdu.azure.publisherFacade.models.PubSubAttributesBuilder' in your configuration.
```
**Root cause:**
Autowiring beans which are not declared as Spring bean:<br>
`@Autowired
private PubSubAttributesBuilder pubSubAttributesBuilder;
`
<br>
`
@Lazy
@Builder
public class PubSubAttributesBuilder {
`
**Solution:**
Remove unused reference of `private PubSubAttributesBuilder pubSubAttributesBuilder` from `src/main/java/org/opengroup/osdu/azure/publisherFacade/EventGridPublisher.java` & `src/main/java/org/opengroup/osdu/azure/publisherFacade/ServiceBusPublisher.java`Nikhil Singh[MicroSoft]Nikhil Singh[MicroSoft]https://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/issues/8Fix security vulnerabilities in old dependencies2022-07-11T19:47:00ZRostislav Vatolinvatolinrp@gmail.comFix security vulnerabilities in old dependenciesFix security vulnerabilities in old dependencies:
1) removing netty-bom, because netty-bom is taken from os-core-common
2) removing jackson-bom, because Jackson-bom is taken from os-core-common
3) remove javax.inject, because it is take...Fix security vulnerabilities in old dependencies:
1) removing netty-bom, because netty-bom is taken from os-core-common
2) removing jackson-bom, because Jackson-bom is taken from os-core-common
3) remove javax.inject, because it is taken from os-core-common
4) replace azure-spring-boot-metrics-starter with micrometer-registry-azure-monitor to avoid bringing extra problem dependencies
5) add explicitly version of json-smart to avoid having old version with security vulnerabilities
6) remove the unused version of azure-core-http-netty
7) remove unused dependencies msal4j, azure-core, reactor-bom from dependencyManagement section
The library is proven in projects legal and partition:
https://community.opengroup.org/osdu/platform/security-and-compliance/legal/-/merge_requests/111
https://community.opengroup.org/osdu/platform/system/partition/-/merge_requests/51https://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/issues/7EG management SDK has to be moved to GA2022-07-11T19:41:13ZKomal MakkarEG management SDK has to be moved to GAEG management is using preview [SDK](https://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/blob/master/pom.xml#L55), which can be risky.
Move to the GA version of it.EG management is using preview [SDK](https://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/blob/master/pom.xml#L55), which can be risky.
Move to the GA version of it.harshit aggarwalharshit aggarwalhttps://community.opengroup.org/osdu/platform/system/lib/cloud/azure/os-core-lib-azure/-/issues/4Unit Tests are erroring out with "ERROR in EnvironmentCredential"2022-12-08T17:13:47ZKomal MakkarUnit Tests are erroring out with "ERROR in EnvironmentCredential" [logs.txt](/uploads/70da303b896cea4224d44ca0b6f8f87d/logs.txt) contains the failure report.
The same should be handled gracefully.
The tests are not failing though. [logs.txt](/uploads/70da303b896cea4224d44ca0b6f8f87d/logs.txt) contains the failure report.
The same should be handled gracefully.
The tests are not failing though.