Skip to content

Adding default ttl for all topics

Arturo Hernandez [EPAM] requested to merge ah/set-proper-ttl-bus into master

All Submissions:

  • [YES/NO] Explanation of change - YES
  • [YES/NO] I have updated the documentation accordingly. - NA
  • [YES/NO/NA] My code follows the code style of this project. - YES

Current Behavior or Linked Issues

Subscription topics are getting filled out, this change will create default rules to get rid of dead-letters that are not being needed anymore (Tested this already manually in glab).

If the subscription gets filled storage service will start throwing error like:

storage-fd496cf6b-4tdg8 storage 2022-10-06 19:53:11.758 ERROR storage-fd496cf6b-4tdg8 --- [nio-80-exec-185] o.o.o.a.p.ServiceBusPublisher            correlation-id=a0e5de38-7742-4b47-b510-2cd356047cff data-partition-id=opendes api-method=PUT operation-name={PUT [/records], consumes [application/json], produces [application/json]} user-id=8b2a56ba-edf5-47ce-94b6-42c336ec8172 app-id=678fadf8-e5a8-46cd-a75d-4d6cc95d9bc9:Retry limit Exceeded.Unable to publish message with message id a0e5de38-7742-4b47-b510-2cd356047cff-0
storage-fd496cf6b-4tdg8 storage 2022-10-06 19:53:11.755 ERROR storage-fd496cf6b-4tdg8 --- [nio-80-exec-190] o.o.o.a.l.Slf4JLogger                    correlation-id=a0e5de38-7742-4b47-b510-2cd356047cff data-partition-id=opendes api-method=PUT operation-name={PUT [/records], consumes [application/json], produces [application/json]} user-id=8b2a56ba-edf5-47ce-94b6-42c336ec8172 app-id=678fadf8-e5a8-46cd-a75d-4d6cc95d9bc9:storage.app The maximum entity size has been reached or exceeded for Topic: 'OSDU-MVP-DP1GLAB-KY7V-BUS:TOPIC:RECORDSTOPIC~31'. Size of entity in bytes:5368733776, Max entity size in bytes: 5368709120. For more information please see https://aka.ms/ServiceBusExceptions .  QuotaType: EntitySize Reference:e748e058-ca50-4a12-938e-5e2f5846070b, TrackingId:aebd0ca20000000e02246c68633f0fa3_G82_B45, SystemTracker:osdu-mvp-dp1glab-ky7v-bus:topic:recordstopic~31, Timestamp:2022-10-06T19:53:11, errorContext[NS: osdu-mvp-dp1glab-ky7v-bus.servicebus.windows.net, PATH: recordstopic, REFERENCE_ID: EDD3BB300BDEBF620BDEBFA7633F0B04_G82, LINK_CREDIT: 258] {correlation-id=a0e5de38-7742-4b47-b510-2cd356047cff, data-partition-id=opendes} org.opengroup.osdu.core.common.model.http.AppException: Failed to publish message in service bus|        at

Does this introduce a breaking change?

  • [NO] Introduces fix and autodeletion rules.

This introduces workaround/fix for dead letter automatic cleanup.

Other information

Ref: #244 Ref : https://community.opengroup.org/osdu/platform/system/storage/-/jobs/1439182

Edited by Arturo Hernandez [EPAM]

Merge request reports