Skip to content
GitLab
Explore
Sign in
Register
Primary navigation
Search or go to…
Project
I
Indexer
Manage
Activity
Members
Labels
Plan
Issues
Issue boards
Milestones
Iterations
Wiki
Requirements
Code
Merge requests
Repository
Branches
Commits
Tags
Repository graph
Compare revisions
Snippets
Locked files
Build
Pipelines
Jobs
Pipeline schedules
Test cases
Artifacts
Deploy
Releases
Package registry
Container Registry
Model registry
Operate
Environments
Terraform modules
Monitor
Incidents
Service Desk
Analyze
Value stream analytics
Contributor analytics
CI/CD analytics
Repository analytics
Code review analytics
Issue analytics
Insights
Model experiments
Help
Help
Support
GitLab documentation
Compare GitLab plans
Community forum
Contribute to GitLab
Provide feedback
Terms and privacy
Keyboard shortcuts
?
Snippets
Groups
Projects
Show more breadcrumbs
OSDU
OSDU Data Platform
System
Indexer
Commits
d0b38151
Commit
d0b38151
authored
5 months ago
by
Zhibin Mai
Browse files
Options
Downloads
Patches
Plain Diff
Update document
parent
6c3f0d11
No related branches found
No related tags found
1 merge request
!834
Fix the bug in a case that more than one extended/virtual properties to match...
Pipeline
#290491
failed
5 months ago
Stage: review
Stage: build
Stage: coverage
Stage: containerize
Stage: scan
Stage: deploy
Stage: integration
Stage: publish
Changes
1
Pipelines
4
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
docs/docs/IndexAugmenter.md
+9
-1
9 additions, 1 deletion
docs/docs/IndexAugmenter.md
with
9 additions
and
1 deletion
docs/docs/IndexAugmenter.md
+
9
−
1
View file @
d0b38151
...
...
@@ -352,6 +352,14 @@ It is not permitted to
implemented in the OSDU Data Definition pipeline to ensure that this reserved name does not appear as property in
the
`data`
block.
*
When the policy
`ExtractAllMatches`
is applied, the
`Index Augmenter`
will aggregate the
`unique`
values from all matched sources
to an array. The aggregated array could be large, especially when the relationship is
`ParentToChildren`
. There is a limitation
on the size of the array that ElasticSearch processes. A record will fail to be indexed if its property with array type
has more than 10,000 items. In order to balance the performance, the array size of the extended property should not be too
large. From M25, if the array size of the extended property exceeds the pre-defined limit, the extended property will be removed
and a message will be logged in the index of the record. The pre-defined limit is 2,000 by default. It can be
overridden by the setting in the Application.properties, e.g.
`augmenter.extended_list_value.max_size=5000`
.
## Deployment
...
...
@@ -381,4 +389,4 @@ the records from the `OSDU search` results, any one of the following mistakes ca
```
*
If not all extended properties are missing, the
`Configurations[]`
of the missing extended properties could be invalid.
The
`Index Augmenter`
engine can do basic syntax check on each configuration and only ignore the invalid ones.
\ No newline at end of file
The
`Index Augmenter`
engine can do basic syntax check on each configuration and only ignore the invalid ones.
This diff is collapsed.
Click to expand it.
Preview
0%
Loading
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Save comment
Cancel
Please
register
or
sign in
to comment