feat: Implement IntervalAwareBalancerStrategy to balance segment count per interval - #19944
Draft
rbankar7 wants to merge 3 commits into
Draft
feat: Implement IntervalAwareBalancerStrategy to balance segment count per interval#19944rbankar7 wants to merge 3 commits into
rbankar7 wants to merge 3 commits into
Conversation
…datanode per interval
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
The default
roundRobinsegment distribution creates a skew in how many segments each historical must scan to answer scan/export endpoint calls that target a narrow, recent time range. Segments covering the same interval can land unevenly across historicals, so some historicals do disproportionately more work for such queries.This PR adds a new
intervalAwarebalancer strategy that spreads segments covering the same time interval as evenly as possible across the historicals in a tier.Added
IntervalAwareBalancerStrategyFor a segment being placed, moved, or dropped, the "cost" of a server is the number of segments already projected on that server for the segment's interval. The least-loaded-for-this-interval server is preferred.
SegmentCountsPerInterval(viaServerHolder.getProjectedSegmentCounts()), so each placement is O(numServers) with an O(1) lookup per server, rather than the O(segmentsPerServer) pairwise computation ofcost. This keeps the coordinator duty-cycle time low.cost, it does not model query-time-decay across the whole retention window; it only equalises the per-interval segment count, which is what matters for workloads that query a narrow, recent time range.ServerHolder.isDecommissioning()), so a draining server is fully evacuated regardless of interval balance. Without this, a single-replica segment on a draining node (source count = 1) could never satisfy thebestCount + 1 < sourceCountguard and would be stranded, stalling decommissioning. The caller (StrategicSegmentAssigner#moveSegment) already excludes a decommissioning source from the destination list, so any chosen destination strictly makes progress towards draining the server, and decommissioning targets are additionally guarded bycanLoadSegmentreturningfalse.Added the
perDatasourceoptionThe count can be scoped two ways via the
perDatasourceflag on the factory:perDatasource=true(default): counts only segments of the same datasource for the interval. Each datasource is balanced independently — optimal when a query targets a single datasource.perDatasource=false: counts segments of all datasources for the interval — optimal when a query unions multiple datasources covering the same time range.Registered the
intervalAwarestrategyAdded the
intervalAwaresubtype toBalancerStrategyFactoryso it can be selected via the coordinator dynamic config:{ "strategy": "intervalAware" } { "strategy": "intervalAware", "perDatasource": false }This strategy is only consulted for initial placement when round-robin assignment is disabled (
useRoundRobinSegmentAssignment=false, which also requiressmartSegmentLoading=false). It is always used for balancing moves.Release note
Added a new
intervalAwarebalancer strategy that distributes the segments of an interval evenly across the historicals in a tier, reducing per-historical scan skew for workloads that query a narrow, recent time range. It can be enabled by setting the balancerstrategytointervalAwarein the coordinator dynamic config, with an optionalperDatasourceflag (defaulttrue) to control whether the per-interval count is scoped to a single datasource or across all datasources.Key changed/added classes in this PR
IntervalAwareBalancerStrategyIntervalAwareBalancerStrategyFactoryBalancerStrategyFactoryThis PR has: