Package org.apache.pinot.broker.routing.segmentselector

Class RealtimeSegmentSelector

  • All Implemented Interfaces:
    SegmentSelector
    public class RealtimeSegmentSelector
extends Object
implements SegmentSelector
    Segment selector for real-time table which handles the following scenarios:
    • When HLC and LLC segments coexist (during LLC migration), select only HLC segments or LLC segments
    • For HLC segments, only select segments in one group
    • For LLC segments, only select the first CONSUMING segment for each partition to avoid duplicate data because in certain unlikely degenerate scenarios, we can consume overlapping data until segments are flushed (at which point the overlapping data is discarded during the reconciliation process with the controller).

    • Method Summary

      Modifier and Type Method Description
      void init​(org.apache.helix.model.IdealState idealState, org.apache.helix.model.ExternalView externalView, Set<String> onlineSegments)
      Initializes the segment selector with the ideal state, external view and online segments (segments with ONLINE/CONSUMING instances in the ideal state and pre-selected by the SegmentPreSelector).
      void onAssignmentChange​(org.apache.helix.model.IdealState idealState, org.apache.helix.model.ExternalView externalView, Set<String> onlineSegments)
      Processes the segment assignment (ideal state or external view) change based on the given online segments (segments with ONLINE/CONSUMING instances in the ideal state and pre-selected by the SegmentPreSelector).
      Set<String> select​(org.apache.pinot.common.request.BrokerRequest brokerRequest)
      Selects the segments queried by the given broker request.

    • Constructor Detail

      • RealtimeSegmentSelector

        public RealtimeSegmentSelector()

    • Method Detail

      • init

        public void init​(org.apache.helix.model.IdealState idealState,
                 org.apache.helix.model.ExternalView externalView,
                 Set<String> onlineSegments)
        Description copied from interface: SegmentSelector
        Initializes the segment selector with the ideal state, external view and online segments (segments with ONLINE/CONSUMING instances in the ideal state and pre-selected by the SegmentPreSelector). Should be called only once before calling other methods.
        Specified by:
        init in interface SegmentSelector

      • onAssignmentChange

        public void onAssignmentChange​(org.apache.helix.model.IdealState idealState,
                               org.apache.helix.model.ExternalView externalView,
                               Set<String> onlineSegments)
        Description copied from interface: SegmentSelector
        Processes the segment assignment (ideal state or external view) change based on the given online segments (segments with ONLINE/CONSUMING instances in the ideal state and pre-selected by the SegmentPreSelector).
        Specified by:
        onAssignmentChange in interface SegmentSelector

      • select

        public Set<String> select​(org.apache.pinot.common.request.BrokerRequest brokerRequest)
        Description copied from interface: SegmentSelector
        Selects the segments queried by the given broker request. The segments selected should cover the whole dataset (table) without overlap.
        Specified by:
        select in interface SegmentSelector