Refactoring WorkflowEngine and WorkflowRunService

Description:

Main goals of refactoring ingestion core:

• Simplify class structure for better maintainability and future improvements.
• Introduce more interfaces with implementations to separate responsibilities.
• Change some method access modifiers for better extensibility.

For example, Airflow v3 support can now be implemented more easily.

The majority of changes were made in core module, with some dependent changes in GC module. Other modules remain unchanged.

Main changes:

• Introduced new interfaces:
  • IAirflowApiClient, to handle Airflow authorization and API calls.
  • IWorkflowRunExtension, a dedicated interface for extension methods.
• Reorganized Workflow Engine implementation classes by moving shared logic into an abstract class.
• Changed injection of IWorkflowEngineService beans from @Conditional annotations to a centralized provider class (WorkflowEngineServiceProvider).
• These changes allow to simplify the class structure in the GC module.

Detailed changes

• WorkflowRunServiceImpl: Introduced some protected methods and changed some private methods to protected for future code reuse.
• AirflowWorkflowEngineServiceImpl and AirflowV2WorkflowEngineServiceImpl: Refactored these classes by creating a new abstract class, AbstractAirflowWorkflowEngineService, for common code. All distinct code is now placed in AirflowWorkflowEngineServiceImpl and AirflowV2WorkflowEngineServiceImpl, which extend AbstractAirflowWorkflowEngineService.
• IWorkflowEngineService: Added a new default method, getVersion(), to the interface.
• CloudConnectedOuterServicesBuilder: Refactored dependencies to depend on the IWorkflowEngineService interface instead of a concrete class for retrieving the version
• WorkflowEngineServiceProvider: Created this new class to provide IWorkflowEngineService beans to the Spring context depending on settings (Airflow version).
• IAirflowApiClient: Created a new interface with a callAirflow() method to handle Airflow authorization and API calls. Existing functionality for authorizing Airflow and calling APIs in the workflow-core module has been moved from AirflowWorkflowEngineServiceImpl, AirflowV2WorkflowEngineServiceImpl, and AirflowV2WorkflowEngineExtension to BasicAuthAirflowApiClient, a new implementation of IAirflowApiClient.
• AirflowV2WorkflowEngineExtension: Simplified. It now only implements IWorkflowEngineExtension. (Previously, it extended AirflowV2WorkflowEngineServiceImpl and implemented IWorkflowEngineService interface).
• IWorkflowEngineService Beans injection: The classes AirflowWorkflowEngineServiceImpl, AirflowV2WorkflowEngineServiceImpl, and AirflowV2WorkflowEngineExtension are no longer marked as @Service classes. Instead, the WorkflowEngineServiceProvider class is now responsible for injecting IWorkflowEngineService beans.
• IWorkflowRunExtension: Created a new interface and implementation WorkflowRunExtensionImpl. Used in RunDetailsApi to call IWorkflowEngineExtension methods.
• Injection configuration: The injection configuration has been simplified. Now, a single condition in WorkflowEngineServiceProvider (depending on the osdu.airflow.version2 setting) is used, instead of using @ConditionalOnProperty annotations on AirflowWorkflowEngineServiceImpl, AirflowV2WorkflowEngineServiceImpl, and AirflowV2WorkflowEngineExtension.

Dependent changes in gc module:

• GcpComposerAirflowApiClient: Created new class for authorizing Airflow using ComposerIaapClient or ComposerV2Client (ComposerClient interface).
• Removed: GcpComposerAirflowExperimentalEngineServiceImpl and GcpComposerAirflowV2EngineServiceImpl classes.
• Removed: Condition classes: ComposerAirflowExperimentalCondition and ComposerAirflowV2Condition.

There are no changes in other provider modules.

There are no changes in logic.

Diagram of changes (before and after refactoring):

How to test:

Integration and acceptance tests were performed. All previously passing tests are still passing.

Changes include:

• Refactor (a non-breaking change that improves code maintainability).
• Bugfix (a non-breaking change that solves an issue).
• New feature (a non-breaking change that adds functionality).
• Breaking change (a change that is not backward-compatible and/or changes current functionality).

Changes in:

• Common code
• GCP
• Azure
• AWS
• IBM

Dev Checklist:

• Added Unit Tests, wherever applicable.
• Updated the Readme, if applicable.
• Existing Tests pass
• Verified functionality locally
• Self Reviewed my code for formatting and complex business logic.