SOLR-18328 Add support for standard deviation in rollup for streaming expressions - #4691
Conversation
|
Hey @epugh, thanks for reviewing!! One of the existing flaky test failed |
epugh
left a comment
There was a problem hiding this comment.
a question about docs, but otherwise sems to follow the pattern of some other ones.. It would be good to answer the other review comments, and tjhen I think we can move forward...
|
Thanks @epugh @VishnuPriyaChandraSekar for reviewing it!! Can we please back port this to 9x as well? |
VishnuPriyaChandraSekar
left a comment
There was a problem hiding this comment.
Thanks for addressing my feedback :)
|
Thanks for reviewing!! |
|
@epugh will this be part of the 9.11 release? |
|
argh... so, in my backporrt, I got confused and I thought backport burped on lciense, but instead it's failing on java code pattern... |
|
FYI: the 9x support is in #4723 |
There has been discussion about getting 9.11 out this month. |
https://issues.apache.org/jira/browse/SOLR-18328
Description
Added support for the
std(col)(standard deviation) metric inrollup()streaming expressions.The
std(col)metric was non-functional; theStdMetricclass was a stub copy-pasted fromMeanMetric—update()was a no-op,getValue()returnednull, andnewInstance()even returned aMeanMetricinstead of aStdMetric.Solution
Implemented
StdMetricto compute the sample standard deviation:StdMetric.java:update()now accumulatescount,sum, andsumSqfrom the column value (handlingDouble/Float/Integer/Long, skipping nulls/other types).getValue()computes the sample standard deviationsqrt((n*sumSq - sum^2) / (n*(n-1))), returning0.0whencount <= 1, and honoringoutputLong(rounds to alongwhen set).newInstance()to return aStdMetric(was incorrectly returningMeanMetric), and replaced the unuseddoubleSum/longSumfields withsum/sumSq.Tests
StreamExpressionTest.testRollupStdMetric(new): validatesstd(a_i)andstd(a_f)alongsidecount(*)in arollup(... over="a_s" ...)expression across multiple groups.StreamingTest.testRollupStream: addedStdMetric("a_i")/StdMetric("a_f")to the metrics array and asserted expected values per group.Checklist
Please review the following and check all that apply:
mainbranch../gradlew check.