Skip to content
Snippets Groups Projects
Commit 67073e98 authored by Oleksandr Kosse (EPAM)'s avatar Oleksandr Kosse (EPAM)
Browse files

Merge branch 'okosse' into 'master' 

[GONRG-10793] Remove search host configuration and related properties from deployment files

See merge request !866
parents 86c53628 e7331dcb
No related branches found
No related tags found
1 merge request!866[GONRG-10793] Remove search host configuration and related properties from deployment files
Pipeline #308517 failed
Stage: build
Stage: coverage
Stage: containerize
Stage: scan
Stage: deploy
Stage: integration
Stage: publish
Hide whitespace changes
Inline Side-by-side
Showing
with 16 additions and 29 deletions
devops/core-plus/deploy/templates/configmap.yaml
+ 0
− 1
View file @ 67073e98
......@@ -21,4 +21,3 @@ data:
SCHEMA_BASE_HOST: {{ .Values.data.schemaHost | quote }}
SECURITY_HTTPS_CERTIFICATE_TRUST: {{ .Values.data.securityHttpsCertificateTrust | quote }}
STORAGE_HOST: {{ .Values.data.storageHost | quote }}
SEARCH_BASE_HOST: {{ .Values.data.searchHost | quote }}
devops/core-plus/deploy/values.yaml
+ 0
− 1
View file @ 67073e98
......@@ -14,7 +14,6 @@ data:
schemaHost: "http://schema"
securityHttpsCertificateTrust: "true"
storageHost: "http://storage"
searchHost: "http://search"
# Deployment
requestsCpu: "35m"
requestsMemory: "640Mi"
......
devops/gc/deploy/README.md
+ 0
− 1
View file @ 67073e98
......@@ -45,7 +45,6 @@ First you need to set variables in **values.yaml** file using any code editor. S
| **data.schemaHost** | schema host | string | `http://schema` | yes |
| **data.securityHttpsCertificateTrust** | whether https is enabled | boolean | `true` | yes |
| **data.storageHost** | storage host | string | `http://storage` | yes |
| **data.searchHost** | search host | string | `http://search` | yes |
| **data.redisIndexerHost** | The host for redis instance. If empty (by default), helm installs an internal redis instance | string | - | yes |
| **data.redisIndexerPort** | The port for redis instance | digit | `6379` | yes |
......
devops/gc/deploy/templates/configmap.yaml
+ 0
− 2
View file @ 67073e98
......@@ -20,6 +20,4 @@ data:
REDIS_GROUP_PORT: {{ .Values.data.redisIndexerPort | quote }}
SCHEMA_BASE_HOST: {{ .Values.data.schemaHost | quote }}
SECURITY_HTTPS_CERTIFICATE_TRUST: {{ .Values.data.securityHttpsCertificateTrust | quote }}
SPRING_PROFILES_ACTIVE: "gcp"
STORAGE_HOST: {{ .Values.data.storageHost | quote }}
SEARCH_BASE_HOST: {{ .Values.data.searchHost | quote }}
devops/gc/deploy/values.yaml
+ 0
− 1
View file @ 67073e98
......@@ -17,7 +17,6 @@ data:
schemaHost: "http://schema"
securityHttpsCertificateTrust: "true"
storageHost: "http://storage"
searchHost: "http://search"
# Deployment
requestsCpu: "100m"
requestsMemory: "640Mi"
......
devops/gke-indexer-cicd.yml
+ 1
− 1
View file @ 67073e98
......@@ -103,7 +103,7 @@ steps:
- task: Maven@3
inputs:
mavenPomFile: '$(integrationTestGcpPom)'
options: '--settings $(mavenSettings) -Dsurefire.useFile=false -DVSTS_FEED_TOKEN=$(VSTS_FEED_TOKEN) -DDEFAULT_DATA_PARTITION_ID_TENANT1=$(DEFAULT_DATA_PARTITION_ID_TENANT1) -DDEFAULT_DATA_PARTITION_ID_TENANT2=$(DEFAULT_DATA_PARTITION_ID_TENANT2) -DELASTIC_HOST=$(ELASTIC_HOST) -DELASTIC_PASSWORD=$(ELASTIC_PASSWORD) -DELASTIC_USER_NAME=$(ELASTIC_USER_NAME) -DENTITLEMENTS_DOMAIN=$(ENTITLEMENTS_DOMAIN) -DINDEXER_HOST=$(INDEXER_HOST) -DLEGAL_TAG=$(LEGAL_TAG) -DOTHER_RELEVANT_DATA_COUNTRIES=$(OTHER_RELEVANT_DATA_COUNTRIES) -DSEARCH_HOST=$(SEARCH_HOST) -DSEARCH_INTEGRATION_TESTER=$(SEARCH_INTEGRATION_TESTER) -DSTORAGE_HOST=$(STORAGE_HOST) compile'
options: '--settings $(mavenSettings) -Dsurefire.useFile=false -DVSTS_FEED_TOKEN=$(VSTS_FEED_TOKEN) -DDEFAULT_DATA_PARTITION_ID_TENANT1=$(DEFAULT_DATA_PARTITION_ID_TENANT1) -DDEFAULT_DATA_PARTITION_ID_TENANT2=$(DEFAULT_DATA_PARTITION_ID_TENANT2) -DELASTIC_HOST=$(ELASTIC_HOST) -DELASTIC_PASSWORD=$(ELASTIC_PASSWORD) -DELASTIC_USER_NAME=$(ELASTIC_USER_NAME) -DENTITLEMENTS_DOMAIN=$(ENTITLEMENTS_DOMAIN) -DINDEXER_HOST=$(INDEXER_HOST) -DLEGAL_TAG=$(LEGAL_TAG) -DOTHER_RELEVANT_DATA_COUNTRIES=$(OTHER_RELEVANT_DATA_COUNTRIES) -DSEARCH_INTEGRATION_TESTER=$(SEARCH_INTEGRATION_TESTER) -DSTORAGE_HOST=$(STORAGE_HOST) compile'
publishJUnitResults: false
javaHomeOption: 'JDKVersion'
mavenVersionOption: 'Default'
......
indexer-core-plus/docs/baremetal/README.md
+ 3
− 5
View file @ 67073e98
......@@ -22,7 +22,6 @@ Must have:
| name | value | description | sensitive? | source |
|----------------------------------------------|--------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------|--------|
| `SPRING_PROFILES_ACTIVE` | ex `anthos` | Spring profile that activate default configuration for Google Cloud environment | false | - |
| `OPENID_PROVIDER_CLIENT_ID` | `*****` | Client id that represents this service and serves to request tokens, example `workload-identity-legal` | yes | - |
| `OPENID_PROVIDER_CLIENT_SECRET` | `*****` | This client secret that serves to request tokens | yes | - |
| `OPENID_PROVIDER_URL` | `https://keycloack.com/auth/realms/master` | URL of OpenID Connect provider, it will be used as `<OpenID URL> + /.well-known/openid-configuration` to auto configure endpoint for token request | no | - |
......@@ -67,11 +66,11 @@ This variable should be present in environment of service that need that variabl
Example:
```json
"elasticsearch.port": {
"sensitive": false, <- value not sensitive
"sensitive": false, <- value not sensitive
"value": "9243" <- will be used as is.
},
"elasticsearch.password": {
"sensitive": true, <- value is sensitive
"sensitive": true, <- value is sensitive
"value": "ELASTIC_SEARCH_PASSWORD_OSDU" <- service consumer should have env variable ELASTIC_SEARCH_PASSWORD_OSDU with elastic search password
}
```
......@@ -265,7 +264,6 @@ You will need to have the following environment variables defined.
| `LEGAL_TAG` | ex `opendes-demo-legaltag` | valid legal tag with a other relevant data countries from `DEFAULT_OTHER_RELEVANT_DATA_COUNTRIES` | no | - |
| `DEFAULT_DATA_PARTITION_ID_TENANT1` | ex `opendes` | HTTP Header 'Data-Partition-ID' | no | - |
| `DEFAULT_DATA_PARTITION_ID_TENANT2` | ex `opendes` | HTTP Header 'Data-Partition-ID' | no | - |
| `SEARCH_HOST` | ex `http://localhost:8080/api/search/v2/` | Endpoint of search service | no | - |
| `STORAGE_HOST` | ex `http://os-storage-dot-opendes.appspot.com/api/storage/v2/` | Storage API endpoint | no | output of infrastructure deployment |
| `SECURITY_HTTPS_CERTIFICATE_TRUST` | ex `false` | Elastic client connection uses TrustSelfSignedStrategy(), if it is 'true' | false | output of infrastructure deployment |
| `TEST_OPENID_PROVIDER_CLIENT_ID` | `********` | Client Id for `$INTEGRATION_TESTER` | yes | -- |
......@@ -275,7 +273,7 @@ You will need to have the following environment variables defined.
**Entitlements configuration for integration accounts**
| INTEGRATION_TESTER | NO_DATA_ACCESS_TESTER |
| INTEGRATION_TESTER | NO_DATA_ACCESS_TESTER |
|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------|
| users<br/>users.datalake.ops<br/>service.storage.creator<br/>service.entitlements.user<br/>service.search.user<br/>service.search.admin<br/>data.test1<br/>data.integration.test<br/>users@{tenant1}@{groupId}.com | |
......
indexer-core-plus/src/main/resources/application.properties
+ 0
− 1
View file @ 67073e98
......@@ -51,7 +51,6 @@ STORAGE_SCHEMA_HOST=${STORAGE_API}/schemas
SCHEMA_BASE_HOST=http://schema
SCHEMA_PATH=/api/schema-service/v1/schema
SCHEMA_HOST=${SCHEMA_BASE_HOST}${SCHEMA_PATH}
SEARCH_HOST=${SEARCH_BASE_HOST}/api/search/v2
# OQM config
records-changed-topic-name=records-changed
......
provider/indexer-gc/docs/gc/README.md
+ 6
− 8
View file @ 67073e98
......@@ -8,7 +8,6 @@ Must have:
| name | value | description | sensitive? | source |
|----------------------------------------------|---------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------|---------------------------------------------------|
| `SPRING_PROFILES_ACTIVE` | ex `gcp` | Spring profile that activate default configuration for Google Cloud environment | false | - |
| `<ELASTICSEARCH_USER_ENV_VARIABLE_NAME>` | ex `user` | Elasticsearch user, name of that variable not defined at the service level, the name will be received through partition service. Each tenant can have it's own ENV name value, and it must be present in ENV of Indexer service, see [Partition properties set](#properties-set-in-partition-service) | yes | - |
| `<ELASTICSEARCH_PASSWORD_ENV_VARIABLE_NAME>` | ex `password` | Elasticsearch password, name of that variable not defined at the service level, the name will be received through partition service. Each tenant can have it's own ENV name value, and it must be present in ENV of Indexer service, see [Partition properties set](#properties-set-in-partition-service) | false | - |
......@@ -72,15 +71,15 @@ It is possible, but not necessary to adjust consumer throughput via Partition se
```json
"max.sub.parallel.streams": {
"sensitive": false,
"value": 2
"sensitive": false,
"value": 2
},
"max.sub.thread.per.stream": {
"sensitive": false,
"sensitive": false,
"value": 5
},
"max.sub.max.outstanding.elements": {
"sensitive": true,
"sensitive": true,
"value": 100
}
```
......@@ -94,11 +93,11 @@ Example:
```json
"elasticsearch.port": {
"sensitive": false, <- value not sensitive
"sensitive": false, <- value not sensitive
"value": "9243" <- will be used as is.
},
"elasticsearch.password": {
"sensitive": true, <- value is sensitive
"sensitive": true, <- value is sensitive
"value": "ELASTIC_SEARCH_PASSWORD_OSDU" <- service consumer should have env variable ELASTIC_SEARCH_PASSWORD_OSDU with elastic search password
}
```
......@@ -188,7 +187,6 @@ You will need to have the following environment variables defined.
| `DEFAULT_DATA_PARTITION_ID_TENANT1` | ex `opendes` | HTTP Header 'Data-Partition-ID' | no | - |
| `DEFAULT_DATA_PARTITION_ID_TENANT2` | ex `opendes` | HTTP Header 'Data-Partition-ID' | no | - |
| `SEARCH_INTEGRATION_TESTER` | `ewogICJ0....` or `tmp/service-acc.json` | Service account for API calls. Note: this user must have entitlements configured already | yes | <https://console.cloud.google.com/iam-admin/serviceaccounts> |
| `SEARCH_HOST` | ex `http://localhost:8080/api/search/v2/` | Endpoint of search service | no | - |
| `STORAGE_HOST` | ex `http://os-storage-dot-opendes.appspot.com/api/storage/v2/` | Storage API endpoint | no | output of infrastructure deployment |
| `SECURITY_HTTPS_CERTIFICATE_TRUST` | ex `false` | Elastic client connection uses TrustSelfSignedStrategy(), if it is 'true' | false | output of infrastructure deployment |
| `CUCUMBER_OPTIONS` | `--tags '~@indexer-extended'` OR `--tags '~@* and @indexer-extended'` | By default `--tags '~@indexer-extended'` to disable experimental feature testing | no | -- |
......
provider/indexer-gc/src/main/resources/application-gcp.properties deleted 100644 → 0
+ 0
− 4
View file @ 86c53628
oqmDriver=pubsub
service.token.provider=GCP
partition-auth-enabled=true
dead-lettering-required=true
provider/indexer-gc/src/main/resources/application.properties
+ 6
− 4
View file @ 67073e98
......@@ -56,10 +56,6 @@ SCHEMA_BASE_HOST=http://schema
SCHEMA_PATH=/api/schema-service/v1/schema
SCHEMA_HOST=${SCHEMA_BASE_HOST}${SCHEMA_PATH}
SEARCH_BASE_HOST=http://search
SEARCH_PATH=/api/search/v2
SEARCH_HOST=${SEARCH_BASE_HOST}${SEARCH_PATH}
# OQM config
records-changed-topic-name=records-changed
schema-changed-topic-name=schema-changed
......@@ -81,3 +77,9 @@ management.endpoints.web.exposure.include=health
management.health.probes.enabled=true
elasticsearch-properties-prefix=elasticsearch.8
# GC config
oqmDriver=pubsub
service.token.provider=GCP
partition-auth-enabled=true
dead-lettering-required=true
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment