Skip to content

SOLR-18328 Add support for standard deviation in rollup for streaming expressions - #4691

Merged
epugh merged 2 commits into
apache:mainfrom
KhushJain:SOLR-18328
Aug 10, 2026
Merged

SOLR-18328 Add support for standard deviation in rollup for streaming expressions#4691
epugh merged 2 commits into
apache:mainfrom
KhushJain:SOLR-18328

Conversation

@KhushJain

Copy link
Copy Markdown
Contributor

https://issues.apache.org/jira/browse/SOLR-18328

Description

Added support for the std(col) (standard deviation) metric in rollup() streaming expressions.
The std(col) metric was non-functional; the StdMetric class was a stub copy-pasted from MeanMetricupdate() was a no-op, getValue() returned null, and newInstance() even returned a MeanMetric instead of a StdMetric.

Solution

Implemented StdMetric to compute the sample standard deviation:

StdMetric.java:

  1. update() now accumulates count, sum, and sumSq from the column value (handling Double/Float/Integer/Long, skipping nulls/other types).
  2. getValue() computes the sample standard deviation sqrt((n*sumSq - sum^2) / (n*(n-1))), returning 0.0 when count <= 1, and honoring outputLong (rounds to a long when set).
  3. Fixed newInstance() to return a StdMetric (was incorrectly returning MeanMetric), and replaced the unused doubleSum/longSum fields with sum/sumSq.

Tests

  • StreamExpressionTest.testRollupStdMetric (new): validates std(a_i) and std(a_f) alongside count(*) in a rollup(... over="a_s" ...) expression across multiple groups.
  • StreamingTest.testRollupStream: added StdMetric("a_i") / StdMetric("a_f") to the metrics array and asserted expected values per group.

Checklist

Please review the following and check all that apply:

  • I have reviewed the guidelines for How to Contribute and my code conforms to the standards described there to the best of my ability.
  • I have created a Jira issue and added the issue ID to my pull request title.
  • I have given Solr maintainers access to contribute to my PR branch. (optional but recommended, not available for branches on forks living under an organisation)
  • I have developed this patch against the main branch.
  • I have run ./gradlew check.
  • I have added tests for my changes.
  • I have added documentation for the Reference Guide
  • I have added a changelog entry for my change

@github-actions github-actions Bot added documentation Improvements or additions to documentation client:solrj tests labels Aug 1, 2026
@epugh epugh self-assigned this Aug 2, 2026
@KhushJain

Copy link
Copy Markdown
Contributor Author

Hey @epugh, thanks for reviewing!!

One of the existing flaky test failed testSingleShardInFlightRequestsDuringShutDown

@epugh epugh left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a question about docs, but otherwise sems to follow the pattern of some other ones.. It would be good to answer the other review comments, and tjhen I think we can move forward...

@KhushJain

Copy link
Copy Markdown
Contributor Author

Thanks @epugh @VishnuPriyaChandraSekar for reviewing it!!

Can we please back port this to 9x as well?

@VishnuPriyaChandraSekar VishnuPriyaChandraSekar left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for addressing my feedback :)

@KhushJain

Copy link
Copy Markdown
Contributor Author

Thanks for reviewing!!
Could one of you merge and back port to 9x please @epugh @VishnuPriyaChandraSekar

@epugh epugh left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@epugh
epugh merged commit 5c28547 into apache:main Aug 10, 2026
4 of 5 checks passed
epugh pushed a commit that referenced this pull request Aug 10, 2026
@KhushJain

Copy link
Copy Markdown
Contributor Author

@epugh will this be part of the 9.11 release?
Any idea when we’ll release 9.11, since the release was previously stopped?

@epugh

epugh commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

argh... so, in my backporrt, I got confused and I thought backport burped on lciense, but instead it's failing on java code pattern...

@epugh

epugh commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

FYI: the 9x support is in #4723

@epugh

epugh commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

@epugh will this be part of the 9.11 release? Any idea when we’ll release 9.11, since the release was previously stopped?

There has been discussion about getting 9.11 out this month.

epugh pushed a commit that referenced this pull request Aug 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

client:solrj documentation Improvements or additions to documentation tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants