Implement a configuration file instead of using environment variables
Problem description:
Currently used approach with passing configuration via environment variables, now it's about 15 variables. Since count of settings continue to grow, management of settings being quite difficult. This makes difficult to setup work environment, use as library for write tests, benchmarks, apps, etc. In general, getting environment variables in the library code isn't good approach, this is like hidden dependency. Consumers may want to use Open-ETP-Server as shared library (request #34 (closed)) for get faster communication with server, for this case definitely better to pass configuration via API instead of environment variables.
Suggested implementation:
Move reading configuration from shared libraries to application level, pass all settings via API at initialization step. Introduce "openETPServer_config.json" file, which will be one place for all settings. Mechanism with mapping values to environment variables will allow to keep compatibility with current approach.
Possible configuration files location:
- In the root folder
- "config" sub-folder
- In the "src/bin/openETPServer/"
Most likely there will be also required separate configs for 'Becnhmarks', 'osduStubServer', but they currently used only on local machines.
Additional features:
- All values in the default JSON must reference to environment variables by the syntax $ENV{variable_name}. This approach will help to keep compatibility with the current approach (pass configuration via environment variables).
- Configuration file "openETPServer_config.json" should be read by default, but can be overridden via environment variable like "OPEN_ETP_SERVER_ENV". For example, for local development, there should be "openETPServer_config.local.json" (OPEN_ETP_SERVER_ENV=local).
Preliminary sample of JSON
{
"DataPartitions":
{
"PartitionMode": "$ENV{RDMS_DATA_PARTITION_MODE}", // RDMS_DATA_PARTITION_MODE (single, multiple)
"ConnectivityMode": "$ENV{RDMS_DATA_CONNECTIVITY_MODE}", // RDMS_DATA_CONNECTIVITY_MODE (osdu, standalone)
"StandaloneConnectionStrings": // Optional list of DB connection strings for "standalone" connectivity mode
{
"single": "$ENV{POSTGRESQL_CONN_STRING}" // POSTGRESQL_CONN_STRING
}
}
"OSDU": // Not required for "standalone" mode
{
"CloudProvider": "$ENV{CLOUD_PROVIDER}", // CLOUD_PROVIDER
"DomainName": "$ENV{DOMAIN_NAME}", // DOMAIN_NAME
"KeyVaultUrl": "$ENV{KEYVAULT_URL}", // KEYVAULT_URL
"PartitionUrl": "$ENV{PARTITION_URL}", // PARTITION_URL
"EntitlementsUrl": "$ENV{ENTITLEMENTS_URL}", // ENTITLEMENTS_URL
"StorageUrl": "$ENV{STORAGE_URL}" // STORAGE_URL
},
"CloudProviders":
{
"Azure": {
"EnableMsi": "${AZURE_ENABLE_MSI}" // AZURE_ENABLE_MSI
},
"GC": {
// Currently there are no special settings
},
"AWS": {
// Not implemented
}
},
"Logging":
{
"LogLevel": "$ENV{LOG_MIN_SEVERITY}" // LOG_MIN_SEVERITY
}
}
Preliminary development plan (consists two stages):
- Remove all usages of environment variables from shared libraries code (except Azure host specific IDENTITY_ENDPOINT and IDENTITY_HEADER).
- Create DTO models with configuration.
- Read all necessary environment variables in the application level (src/bin/openETPServer), prepare configuration and pass to the server at the initialization step.
- Use passed configuration in the server code and remove all usages of environment variables.
- Fix tests/benchmarks.
- Implement reading configuration files from disk (in the app level "src/bin/openETPServer").
- Implement serialization of DTO models with configuration, support default values.
- Add support references to environment variables: $ENV{variable_name}
- Prepare default configuration file and for local development.
- Read configuration from disk instead of environment variables.
- Implement ability to override default config via OPEN_ETP_SERVER_ENV variable.
- Update CMake - add copy/install config files.
- Implement configs for Benchmarks and osduStubServer.
- Fix tests