Package org.apache.pinot.broker.routing.instanceselector

Class MultiStageReplicaGroupSelector

java.lang.Object

org.apache.pinot.broker.routing.instanceselector.MultiStageReplicaGroupSelector

All Implemented Interfaces:

InstanceSelector

public class MultiStageReplicaGroupSelector
extends Object

Instance selector for multi-stage queries which can ensure that Colocated Tables always leverage Colocated Join whenever possible. To achieve this, this instance-selector uses InstancePartitions (IP) to determine replica-groups, as opposed to IdealState used by other instance-selectors. Moreover, this also uses the requestId generated by Pinot broker to determine the replica-group picked for each table involved in the query, as opposed to using a member variable. There may be scenarios where an instance in the chosen replica-group is down. In that case, this strategy will try to pick another replica-group. For realtime tables, this strategy uses only CONSUMING partitions. This is feature is in Beta.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.pinot.broker.routing.instanceselector.InstanceSelector

InstanceSelector.SelectionResult

Field Summary

Fields

Modifier and Type	Field	Description
protected AdaptiveServerSelector	_adaptiveServerSelector

Constructor Summary

Constructors

Constructor	Description
MultiStageReplicaGroupSelector(String tableNameWithType, org.apache.helix.store.zk.ZkHelixPropertyStore<org.apache.helix.zookeeper.datamodel.ZNRecord> propertyStore, org.apache.pinot.common.metrics.BrokerMetrics brokerMetrics, AdaptiveServerSelector adaptiveServerSelector)

Method Summary

Modifier and Type	Method	Description
protected org.apache.pinot.common.assignment.InstancePartitions	getInstancePartitions()
void	init(Set<String> enabledInstances, org.apache.helix.model.IdealState idealState, org.apache.helix.model.ExternalView externalView, Set<String> onlineSegments)	Initializes the instance selector with the enabled instances, ideal state, external view and online segments (segments with ONLINE/CONSUMING instances in the ideal state and pre-selected by the SegmentPreSelector).
void	onAssignmentChange(org.apache.helix.model.IdealState idealState, org.apache.helix.model.ExternalView externalView, Set<String> onlineSegments)	Processes the segment assignment (ideal state or external view) change based on the given online segments (segments with ONLINE/CONSUMING instances in the ideal state and pre-selected by the SegmentPreSelector).
void	onInstancesChange(Set<String> enabledInstances, List<String> changedInstances)	Processes the instances change.
InstanceSelector.SelectionResult	select(org.apache.pinot.common.request.BrokerRequest brokerRequest, List<String> segments, long requestId)	Selects the server instances for the given segments queried by the given broker request, returns a map from segment to selected server instance hosting the segment and a set of unavailable segments (no enabled instance or all enabled instances are in ERROR state).

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

_adaptiveServerSelector

protected final AdaptiveServerSelector _adaptiveServerSelector

Constructor Detail

MultiStageReplicaGroupSelector

public MultiStageReplicaGroupSelector(String tableNameWithType,
                                      org.apache.helix.store.zk.ZkHelixPropertyStore<org.apache.helix.zookeeper.datamodel.ZNRecord> propertyStore,
                                      org.apache.pinot.common.metrics.BrokerMetrics brokerMetrics,
                                      @Nullable
                                      AdaptiveServerSelector adaptiveServerSelector)

Method Detail

init

public void init(Set<String> enabledInstances,
                 org.apache.helix.model.IdealState idealState,
                 org.apache.helix.model.ExternalView externalView,
                 Set<String> onlineSegments)

Description copied from interface: InstanceSelector

Initializes the instance selector with the enabled instances, ideal state, external view and online segments (segments with ONLINE/CONSUMING instances in the ideal state and pre-selected by the SegmentPreSelector). Should be called only once before calling other methods.

Specified by:

init in interface InstanceSelector

onInstancesChange

public void onInstancesChange(Set<String> enabledInstances,
                              List<String> changedInstances)

Processes the instances change. Changed instances are pre-computed based on the current and previous enabled instances only once on the caller side and passed to all the instance selectors.

Updates the cached enabled instances and re-calculates segmentToEnabledInstancesMap and unavailableSegments based on the cached states.

Specified by:

onInstancesChange in interface InstanceSelector

onAssignmentChange

public void onAssignmentChange(org.apache.helix.model.IdealState idealState,
                               org.apache.helix.model.ExternalView externalView,
                               Set<String> onlineSegments)

Processes the segment assignment (ideal state or external view) change based on the given online segments (segments with ONLINE/CONSUMING instances in the ideal state and pre-selected by the SegmentPreSelector).

Updates the cached maps (segmentToOnlineInstancesMap, segmentToOfflineInstancesMap and instanceToSegmentsMap) and re-calculates segmentToEnabledInstancesMap and unavailableSegments based on the cached states.

Specified by:

onAssignmentChange in interface InstanceSelector

getInstancePartitions

protected org.apache.pinot.common.assignment.InstancePartitions getInstancePartitions()

select

public InstanceSelector.SelectionResult select(org.apache.pinot.common.request.BrokerRequest brokerRequest,
                                               List<String> segments,
                                               long requestId)

Description copied from interface: InstanceSelector

Selects the server instances for the given segments queried by the given broker request, returns a map from segment to selected server instance hosting the segment and a set of unavailable segments (no enabled instance or all enabled instances are in ERROR state).

Specified by:

select in interface InstanceSelector

Parameters:

brokerRequest - BrokerRequest for the query

segments - segments for which instance needs to be selected

requestId - requestId generated by the Broker for a query

Returns:

instance of SelectionResult which describes the instance to pick for a given segment