Skip to content

Pluggable PostgreSQL Driver

Rustam Lotsmanenko (EPAM) requested to merge fat-jar-dependency-refactoring into main

Description:

Link to ADR: https://gitlab.opengroup.org/osdu/pmc/community-implementation/-/issues/22
More details about the solution: https://community.opengroup.org/osdu/documentation/-/wikis/Pluggable-approach-for-Mappers-and-Drivers

Current:

Blockers for plugin approach:

  • All dependencies, even Driver specific like Postgres-JDBC, are located in the root pom, making them transitive for consumers even if Postgres-OSM Driver is not used.
  • Redundant dependencies are present in root pom.
  • Postgres OSM Driver compiled without its dependencies in JAR making it not possible to use it as a plugin.

Not blockers, but part of code cleanup:

  • @ConditionalOnProperties is present, but it is redundant, since Drivers won't be bundled all together, and there is no need to configure them that way.
  • Module and artifact ID naming are not aligned with convention,

These identifiers should be comprised of lowercase letters, digits, and hyphens only.: https://maven.apache.org/maven-conventions.html

Expected:

  • Added Maven shade plugin, which allows to build uber JAR with required dependencies, and shade them to avoid classpath conflicts.
  • Removed Spring @ConditionalOnProperties annotation. We do not need it anymore, Driver implementations won't be loaded to the application all at once.
  • Removed redundant dependencies. Driver shouldn't have redundant dependencies, it complicates uber JAR build.
  • Move dependencies to module pom files, to not force consumers to download redundant ones.
  • Module and artifact ID renamed according to convention.

Key components of pluggable configuration:

  1. Maven shade plugin, used to compile uber jar, with unique dependencies onboard:
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-shade-plugin</artifactId>
  1. Important parts of the maven shade plugin configuration:
  • We do not need to deploy the reduced by maven shade POM file into the registry, to keep the ability to use the original jar file as is.
  • The plugin jar will be deployed to the registry in addition to the original jar.
  • The plugin jar will be classified, making it possible to use :classifier in other maven tasks.
  • Signature files should be excluded from dependencies.
                        <configuration>
                            <!-- do not deploy reduced pom --> 
                            <createDependencyReducedPom>false</createDependencyReducedPom>
                            <!-- deploy plugin jar --> 
                            <shadedArtifactAttached>true</shadedArtifactAttached>
                            <!-- use the classifier to distinct plugin jar and original jar -->
                            <shadedClassifierName>plugin</shadedClassifierName> 
                            <!-- filter out signatures from signed jars, to prevent SecurityException -->
                            <filters>
                                <filter>
                                    <artifact>*:*</artifact>
                                    <excludes>
                                        <exclude>META-INF/*.SF</exclude>
                                        <exclude>META-INF/*.DSA</exclude>
                                        <exclude>META-INF/*.RSA</exclude>

  • Dependency relocation is necessary to prevent classpath conflict in case any consumer uses the same dependencies:

                            <relocations>
                                <relocation>
                                    <pattern>org.postgresql</pattern>
                                    <shadedPattern>shaded.org.postgresql</shadedPattern>
                                </relocation>

Driver usage from a service perspective:

Untitled_Diagram.drawio_9_

Driver artifact:

Untitled_Diagram.drawio_11_

Artifact in package registry:

Untitled_Diagram.drawio_12_

Changes include:

  • Refactor (a non-breaking change that improves code maintainability).
  • Bugfix (a non-breaking change that solves an issue).
  • New feature (a non-breaking change that adds functionality).
  • Breaking change (a change that is not backward-compatible and/or changes current functionality).

Changes in:

  • Community Implementation

Dev Checklist:

  • Added Unit Tests, wherever applicable.
  • Updated the Readme, if applicable.
  • Existing Tests pass
  • Verified functionality locally
  • Self Reviewed my code for formatting and complex business logic.

Other comments:

Two major requirements should be considered when the Driver is built as a Plugin:

  1. Driver JAR should contain all required dependencies a.k.a UBER JAR. Since Maven won't be help to download all transitive dependencies of Driver JAR, we should combine them at the compile step.
  2. Driver JAR should minimize them to only unique dependencies. For example, osdu-core-common.jar should not be part of it, but Postgres jdbc.jar should. This will help to avoid classpath conflicts.
Edited by Rustam Lotsmanenko (EPAM)

Merge request reports