Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SOLR-17669: Reduce Memory Consumption by 80-90% when using Dynamic fields (DocumentObjectBinder) #3179

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

ds-manzinger
Copy link
Contributor

https://issues.apache.org/jira/browse/SOLR-17669

Description

Flame Graph in our Production System showed a siginificant high memory consumption of Pattern.matcher

Solution

Wildcard can only be at beginning or end of Field Name, we replaced Pattern Regex with startsWith and endsWith

Tests

No new tests needed, old tests are working fine with new code

Checklist

Please review the following and check all that apply:

  • I have reviewed the guidelines for How to Contribute and my code conforms to the standards described there to the best of my ability.
  • I have created a Jira issue and added the issue ID to my pull request title.
  • I have given Solr maintainers access to contribute to my PR branch. (optional but recommended, not available for branches on forks living under an organisation)
  • I have developed this patch against the main branch.
  • I have run ./gradlew check.
  • I have added tests for my changes.
  • I have added documentation for the Reference Guide

@psalagnac
Copy link
Contributor

psalagnac commented Feb 13, 2025

The doc mentions it is recommended to use simple mapping, not strictly required.

Is this functional change on purpose?

It is recommended that you include basic dynamic field mappings (like that shown above) in your schema. The mappings can be very useful.

@dsmiley
Copy link
Contributor

dsmiley commented Feb 15, 2025

@psalagnac , this isn't about the Solr schema, this is SolrJ, using a nifty feature to map Solr result documents from QueryResponse.getBeans to a user's domain object that has annotations.

+1 to the change

@ds-manzinger
Copy link
Contributor Author

Yes, it is only in SolrJ. Our Company used that feature quite many years (I think since Solr 4).

If there is a case, where a * in the middle of the name is possible, we can add the Pattern.matcher as fallback.
What would you suggest?

@dsmiley
Copy link
Contributor

dsmiley commented Feb 16, 2025

Yes Pattern is fine (as it was).
I hope this is tested, but I suppose maybe it isn't if tests pass without the middle wildcard. Can you please add a test?

@github-actions github-actions bot added the tests label Feb 16, 2025
@ds-manzinger
Copy link
Contributor Author

@dsmiley Now i added the Pattern as it was before and 2 additional Asserts with Wildcard in the middle and Wildcard at the beginning

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants