Azure Gremlin driver throwing NoHostAvailableException

Azure Gremlin driver is throwing lot of org.apache.tinkerpop.gremlin.driver.exception.NoHostAvailableException exceptions after M23 deployment. In such cases service responds with 509 error code thus impacting service availability.

Based on discussions with Azure Support, Gremlin Java drivers with versions 3.5.* and 3.6.* have known compatibility issues with Cosmos DB and our recommended version is 3.4.13. Azure Cosmos DB for Gremlin support and compatibility with TinkerPop features | Microsoft Learn. Azure does not promises on support for the protocol in 3.5. Apache Tinkerpop made changes to the protocol handling on some Gremlin client drivers for versions > 3.4.13 that were incompatible with some Cosmos DB Gremlin API driver implementations. Additionally, certain driver features in newer versions are not supported by Gremlin API.