Schema issueshttps://community.opengroup.org/osdu/platform/system/schema-service/-/issues2024-01-08T13:43:28Zhttps://community.opengroup.org/osdu/platform/system/schema-service/-/issues/144Schema Bootstrap failure issue2024-01-08T13:43:28ZIsha KumariSchema Bootstrap failure issueThe system schema has experienced substantial growth, expanding from an initial range of 200-300 to approximately 900. Consequently, I've been facing failures in the schema during script execution. Notably, the script is designed to rei...The system schema has experienced substantial growth, expanding from an initial range of 200-300 to approximately 900. Consequently, I've been facing failures in the schema during script execution. Notably, the script is designed to reinitiate the schema upon encountering failures
Additionally, the Bootstrap process is showing failures randomly due to the schema variations. To address this, I have implemented error-catching mechanisms and introduced a retry strategy to mitigate the impact of these intermittent failures.M22 - Release 0.25Isha KumariIsha Kumarihttps://community.opengroup.org/osdu/platform/system/schema-service/-/issues/132[Azure] The request with typo or unknown requestParam must fail with 400 Badr...2023-11-28T10:02:22ZSachin Jaiswal[Azure] The request with typo or unknown requestParam must fail with 400 BadrequestCurrently getSchemaInfo endpoint accepts incorrect requestParam like limit-1, This ideally should be limit=1 therefore we should reject all such incoming requestParam.
Only below requestParam should be accepted other request with typo ...Currently getSchemaInfo endpoint accepts incorrect requestParam like limit-1, This ideally should be limit=1 therefore we should reject all such incoming requestParam.
Only below requestParam should be accepted other request with typo or unknown requestParam must fail with 400 Badrequest.
- authority
- source
- entity
- schema version major
- schema version minor
- schema version patch
- status
- scope
- latestVersion
- limit
- offsetM22 - Release 0.25Sachin JaiswalVidyaDharani LokamSachin Jaiswalhttps://community.opengroup.org/osdu/platform/system/schema-service/-/issues/139Add /liveness_check2023-11-21T19:08:35ZRiabokon Stanislav(EPAM)[GCP]Add /liveness_checkNeed to add the endpoint '/liveness_check' in order to verify the operational status of the Schema Service.Need to add the endpoint '/liveness_check' in order to verify the operational status of the Schema Service.M22 - Release 0.25Riabokon Stanislav(EPAM)[GCP]Riabokon Stanislav(EPAM)[GCP]https://community.opengroup.org/osdu/platform/system/schema-service/-/issues/49Job Failed #3846292023-08-07T09:56:06ZAbhishek Kumar (SLB)Job Failed #384629The changes were done to the Azure module only though IBM bootstrapping is failing.
Could you please take a look and fix the bootstrapping issue.
Job [#384629](https://community.opengroup.org/osdu/platform/system/schema-service/-/jobs/3...The changes were done to the Azure module only though IBM bootstrapping is failing.
Could you please take a look and fix the bootstrapping issue.
Job [#384629](https://community.opengroup.org/osdu/platform/system/schema-service/-/jobs/384629) failed for 5342e3fc0accfe6fbeec0cb13c5fad533b42abfc:Anuj GuptaAnuj Guptahttps://community.opengroup.org/osdu/platform/system/schema-service/-/issues/50Job Failed #3836472023-08-07T09:55:59ZAbhishek Kumar (SLB)Job Failed #383647The ITs are passing in the SLB environment.
@nsarda Could you please coordinate with Microsoft team to fix all the IT related issues.
Job [#383647](https://community.opengroup.org/osdu/platform/system/schema-service/-/jobs/383647) faile...The ITs are passing in the SLB environment.
@nsarda Could you please coordinate with Microsoft team to fix all the IT related issues.
Job [#383647](https://community.opengroup.org/osdu/platform/system/schema-service/-/jobs/383647) failed for aa6a8d39a4e11d3696132e76ae5373a0963841ec:Aman VermaNeha SardaAshay BansodAman Vermahttps://community.opengroup.org/osdu/platform/system/schema-service/-/issues/112Azure - shared partition should not have same name as created partition2022-12-20T11:39:58ZArturo Hernandez [EPAM]Azure - shared partition should not have same name as created partitionAs for now the schema service does uses "opendes" as shared partition, in the azure architecture this will save schema records in the shared cosmosdb, this will cause duplication of schemas and will not allow to update anymore schemas un...As for now the schema service does uses "opendes" as shared partition, in the azure architecture this will save schema records in the shared cosmosdb, this will cause duplication of schemas and will not allow to update anymore schemas until shared is properly defined in the properties:
```
shared.tenant.name=opendes
###
azure.system.sharedTenant=${shared.tenant.name}
```
If we use opendes, the records will be saved in the Shared cosmosdb, however, if we try to upload schemas, there are already opendes authority schemas and will not be able to update the schema.
I would suggest to use value `osdu` for shared tenant.name, as most of the shared schema authority does have osdu.M14 - Release 0.17Arturo Hernandez [EPAM]Igor Zimovets (EPAM)Arturo Hernandez [EPAM]https://community.opengroup.org/osdu/platform/system/schema-service/-/issues/104Bootstrap keep failing with a character issue2022-12-09T13:38:22ZAbhishek Kumar (SLB)Bootstrap keep failing with a character issueThe bootstrap python script keep failing with below error for random schemas:
Expecting value: line 1 column 1 (char 0)
![image](/uploads/0dfddc815f6edcccfacb3158699d6076/image.png)
https://community.opengroup.org/osdu/platform/system...The bootstrap python script keep failing with below error for random schemas:
Expecting value: line 1 column 1 (char 0)
![image](/uploads/0dfddc815f6edcccfacb3158699d6076/image.png)
https://community.opengroup.org/osdu/platform/system/schema-service/-/jobs/1167345prerana dubeyprerana dubeyhttps://community.opengroup.org/osdu/platform/system/schema-service/-/issues/43Schema Notification2022-08-23T11:19:20ZAbhishek Kumar (SLB)Schema Notification## Schema Notifications <a name="schema-notifications"></a>
Schema service triggers a notification event whenever there is a create or update operation performed. In other words, these notifications are triggered whenever POST or PUT en...## Schema Notifications <a name="schema-notifications"></a>
Schema service triggers a notification event whenever there is a create or update operation performed. In other words, these notifications are triggered whenever POST or PUT end-points are called.
Subscribers can register themselves via registration service to a newly created topic and start receiving notification on their push-end-point via notification-service.
The event message body will have the below structure:
* Create event
```
[
{
"kind": "osdu:wks:wellbore:2.0.0",
"op": "create"
}
]
```
* Update event
```
[
{
"kind": "osdu:wks:wellbore:1.0.0",
"op": "update"
}
]
```Abhishek Kumar (SLB)Abhishek Kumar (SLB)https://community.opengroup.org/osdu/platform/system/schema-service/-/issues/76Azure IT is taking approximately 1 hour to finish2022-05-11T07:58:44ZAbhishek Kumar (SLB)Azure IT is taking approximately 1 hour to finishhttps://community.opengroup.org/osdu/platform/system/schema-service/-/jobs/683575https://community.opengroup.org/osdu/platform/system/schema-service/-/jobs/683575Neha SardaNeha Sardahttps://community.opengroup.org/osdu/platform/system/schema-service/-/issues/58Upgrade Core Azure Dependency2022-02-11T21:59:01ZDavid Diederichd.diederich@opengroup.orgUpgrade Core Azure DependencyDavid Diederichd.diederich@opengroup.orgDavid Diederichd.diederich@opengroup.orghttps://community.opengroup.org/osdu/platform/system/schema-service/-/issues/48GET API fails if SupersededBy is present in SHARED partition2022-01-10T04:03:36ZAbhishek Kumar (SLB)GET API fails if SupersededBy is present in SHARED partitionGET API call fails if the call is made from the PRIVATE tenant and it includes schema from SHARED partition.
If there are schemas in a shared partition with SupersededBy reference, then it tries to fetch the schema from the private tena...GET API call fails if the call is made from the PRIVATE tenant and it includes schema from SHARED partition.
If there are schemas in a shared partition with SupersededBy reference, then it tries to fetch the schema from the private tenant.
GET call returns records from both SHARED and PRIVATE Partitions.Abhishek Kumar (SLB)Abhishek Kumar (SLB)https://community.opengroup.org/osdu/platform/system/schema-service/-/issues/66Job Failed #5608922021-12-14T12:31:12ZAbhishek Kumar (SLB)Job Failed #560892Please help us with to investigate this error.
Job [#560892](https://community.opengroup.org/osdu/platform/system/schema-service/-/jobs/560892) failed for 709d197fd10fce98c14576feb2789608fe451893:Please help us with to investigate this error.
Job [#560892](https://community.opengroup.org/osdu/platform/system/schema-service/-/jobs/560892) failed for 709d197fd10fce98c14576feb2789608fe451893:Aman VermaAman Vermahttps://community.opengroup.org/osdu/platform/system/schema-service/-/issues/73Job Failed #6031572021-09-23T08:14:36ZAbhishek Kumar (SLB)Job Failed #603157Azure deployment is failing for master.
Job [#603157](https://community.opengroup.org/osdu/platform/system/schema-service/-/jobs/603157) failed for eb1b48daf8056c00b11755723489a3a68e1f38f5:Azure deployment is failing for master.
Job [#603157](https://community.opengroup.org/osdu/platform/system/schema-service/-/jobs/603157) failed for eb1b48daf8056c00b11755723489a3a68e1f38f5:Aman VermaAman Vermahttps://community.opengroup.org/osdu/platform/system/schema-service/-/issues/60AADSecurityConfigWithIstioEnabled performs authentication with istio enabled2021-07-29T05:43:28ZAbhishek Kumar (SLB)AADSecurityConfigWithIstioEnabled performs authentication with istio enabledThere is no difference in behavior when `azure.istio.auth.enabled` is `true` or `false`.
Both, `AADSecurityConfig` and `AADSecurityConfigWithIstioEnabled.java` performs same function.
```
http
.csrf().disable()
....There is no difference in behavior when `azure.istio.auth.enabled` is `true` or `false`.
Both, `AADSecurityConfig` and `AADSecurityConfigWithIstioEnabled.java` performs same function.
```
http
.csrf().disable()
.sessionManagement().sessionCreationPolicy(SessionCreationPolicy.NEVER)
.and()
.authorizeRequests()
.antMatchers("/", "/index.html",
"/v2/api-docs",
"/configuration/ui",
"/swagger-resources/**",
"/configuration/security",
"/swagger",
"/swagger-ui.html",
"/webjars/**").permitAll()
.anyRequest().authenticated()
.and()
.addFilterBefore(appRoleAuthFilter, UsernamePasswordAuthenticationFilter.class);
```
Vs
```
http.httpBasic().disable()
.csrf().disable();
```M7 - Release 0.10Aman VermaAman Vermahttps://community.opengroup.org/osdu/platform/system/schema-service/-/issues/28azure_bootstrap is failing2020-12-04T04:03:33ZAbhishek Kumar (SLB)azure_bootstrap is failingschema-service azure_bootstrap is failing.
https://community.opengroup.org/osdu/platform/system/schema-service/-/jobs/111856
Please look into it.schema-service azure_bootstrap is failing.
https://community.opengroup.org/osdu/platform/system/schema-service/-/jobs/111856
Please look into it.Aman VermaAman Vermahttps://community.opengroup.org/osdu/platform/system/schema-service/-/issues/9[System/Core] Schema Service2020-11-09T20:06:02ZStephen Whitley (Invited Expert)[System/Core] Schema ServiceThe schema service is both an implementation of the schema standard defined below and a service to help create, manage and share schemas between different users. It aims to both raise awareness of the different models used by teams to he...The schema service is both an implementation of the schema standard defined below and a service to help create, manage and share schemas between different users. It aims to both raise awareness of the different models used by teams to help reuse both of the data sources but also of the schemas themselves. This will hopefully help reduce the friction obtaining data from multiple sources.
**Schema service goals**:
- To enable easier creation of new schemas
- To discover popular schemas in use today
- To make reuse of existing schemas easier to achieve
**Schema Definition**:
A schema is the model definition for a particular entity. At its core it defines the property names, their types and relationships to other models. It also includes certain qualities to expect about the actual entities and the data its hold.
An entity is a specific instance of a schema. A schema is assigned an entityType which categorizes the data the schema defines. For example 'well' is an example of an entityType.
An entityType is assigned an 'authority'. An authority is a namespace to help differentiate types defined by different groups e.g. it allows for both an slb:well and a chevron:well type to exist.
With schema definitions we aim to define the standards for data models used by a DDMS. This will reduce the friction with different systems data sharing needs helping with our vision to enable data discovery and sharing.
**Principles**:
- Low threshold for Schema Registration
: We should make it as easy as possible to create a new schema and register it, Whether this be through examples, tooling to help auto-generate a schema or validators to give early feedback when invalid markup is used. Also it must be easy to reuse existing schema definitions - DE-schemas and external schemas - using composition.
- Easy to consume
: JSON schema allows for JSON pointers to other schema parts and fragments, which need to be chased to obtain a full schema. A registered DE schema is self-contained, i.e. it contains only internal references "$ref": "#/definitions/../target". This is particularly important for schemas, which originally refer to external URLs. All internal "$ref" are resolvable. Consumers find all the answers inside the single schema document. The schema has a predictable structure.
- Robust
: All Schemas are read-only and versioned. The major version is controlled by the schema producer, the minor version is controlled by the schema service. When a schema is assigned version 1.0.0 or higher no breaking changes can be applied on new schema versions. Breaking changes will be allowed on any schema version with major version 0.X.X.
- Breaking changes are:
- removed properties
- changes to existing properties e.g.changed types, changed required properties
: This gives consumers confidence that they can use data without fear of breaks in the future. However schemas can be deprecated meaning that no new data will be assigned to it.
**Schema attributes**
Schema definitions are based on in the [JSON schema standard](https://json-schema.org/latest/json-schema-core.html) and [OpenAPI 3](https://swagger.io/docs/specification/data-models/). There are subtle differences in the keyword support, which are explained in conjunction with OAS3 [here](https://swagger.io/docs/specification/data-models/keywords/).
- [x] AWS
- [x] Azure
- [x] IBM
- [ ] GCPM1 - Release 0.1Hrvoje MarkovicFerris ArgyleDania Kodeih (Microsoft)Wladmir FrazaoChris ZhangMichael CleminsonHrvoje Markovic2020-10-30https://community.opengroup.org/osdu/platform/system/schema-service/-/issues/5[Schema Service] Azure SPI Implementation2020-09-02T20:47:47Zethiraj krishnamanaidu[Schema Service] Azure SPI Implementation2020-08-31