org.apache.flink.runtime.resourcemanager.slotmanager

Class FineGrainedSlotManager

java.lang.Object

org.apache.flink.runtime.resourcemanager.slotmanager.FineGrainedSlotManager

All Implemented Interfaces:

AutoCloseable, SlotManager

public class FineGrainedSlotManager
extends Object
implements SlotManager

Implementation of SlotManager supporting fine-grained resource management.

Constructor Summary

Constructors

Constructor and Description

FineGrainedSlotManager(org.apache.flink.util.concurrent.ScheduledExecutor scheduledExecutor, SlotManagerConfiguration slotManagerConfiguration, SlotManagerMetricGroup slotManagerMetricGroup, ResourceTracker resourceTracker, org.apache.flink.runtime.resourcemanager.slotmanager.TaskManagerTracker taskManagerTracker, SlotStatusSyncer slotStatusSyncer, ResourceAllocationStrategy resourceAllocationStrategy, org.apache.flink.api.common.time.Time requirementCheckDelay)

Method Summary

Modifier and Type	Method and Description
void	clearResourceRequirements(org.apache.flink.api.common.JobID jobId) Notifies the slot manager that the resource requirements for the given job should be cleared.
void	close() Closes the slot manager.
void	freeSlot(SlotID slotId, AllocationID allocationId) Free the given slot from the given allocation.
Collection<SlotInfo>	getAllocatedSlotsOf(InstanceID instanceID)
ResourceProfile	getFreeResource()
ResourceProfile	getFreeResourceOf(InstanceID instanceID)
int	getNumberFreeSlots()
int	getNumberFreeSlotsOf(InstanceID instanceId)
int	getNumberRegisteredSlots()
int	getNumberRegisteredSlotsOf(InstanceID instanceId)
ResourceProfile	getRegisteredResource()
ResourceProfile	getRegisteredResourceOf(InstanceID instanceID)
Map<WorkerResourceSpec,Integer>	getRequiredResources() Get number of workers SlotManager requested from ResourceActions that are not yet fulfilled.
long	getTaskManagerIdleSince(InstanceID instanceId)
void	processResourceRequirements(ResourceRequirements resourceRequirements) Notifies the slot manager about the resource requirements of a job.
boolean	registerTaskManager(TaskExecutorConnection taskExecutorConnection, SlotReport initialSlotReport, ResourceProfile totalResourceProfile, ResourceProfile defaultSlotResourceProfile) Registers a new task manager at the slot manager.
boolean	reportSlotStatus(InstanceID instanceId, SlotReport slotReport) Reports the current slot allocations for a task manager identified by the given instance id.
void	setFailUnfulfillableRequest(boolean failUnfulfillableRequest)
void	start(ResourceManagerId newResourceManagerId, Executor newMainThreadExecutor, ResourceActions newResourceActions) Starts the slot manager with the given leader id and resource manager actions.
void	suspend() Suspends the component.
boolean	unregisterTaskManager(InstanceID instanceId, Exception cause) Unregisters the task manager identified by the given instance id and its associated slots from the slot manager.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

FineGrainedSlotManager

public FineGrainedSlotManager(org.apache.flink.util.concurrent.ScheduledExecutor scheduledExecutor,
                              SlotManagerConfiguration slotManagerConfiguration,
                              SlotManagerMetricGroup slotManagerMetricGroup,
                              ResourceTracker resourceTracker,
                              org.apache.flink.runtime.resourcemanager.slotmanager.TaskManagerTracker taskManagerTracker,
                              SlotStatusSyncer slotStatusSyncer,
                              ResourceAllocationStrategy resourceAllocationStrategy,
                              org.apache.flink.api.common.time.Time requirementCheckDelay)

Method Detail

setFailUnfulfillableRequest

public void setFailUnfulfillableRequest(boolean failUnfulfillableRequest)

Specified by:

setFailUnfulfillableRequest in interface SlotManager

start

public void start(ResourceManagerId newResourceManagerId,
                  Executor newMainThreadExecutor,
                  ResourceActions newResourceActions)

Starts the slot manager with the given leader id and resource manager actions.

Specified by:

start in interface SlotManager

Parameters:

newResourceManagerId - to use for communication with the task managers

newMainThreadExecutor - to use to run code in the ResourceManager's main thread

newResourceActions - to use for resource (de-)allocations

suspend

public void suspend()

Suspends the component. This clears the internal state of the slot manager.

Specified by:

suspend in interface SlotManager

close

public void close()
           throws Exception

Closes the slot manager.

Specified by:

close in interface AutoCloseable

Throws:

Exception - if the close operation fails

clearResourceRequirements

public void clearResourceRequirements(org.apache.flink.api.common.JobID jobId)

Description copied from interface: SlotManager

Notifies the slot manager that the resource requirements for the given job should be cleared. The slot manager may assume that no further updates to the resource requirements will occur.

Specified by:

clearResourceRequirements in interface SlotManager

Parameters:

jobId - job for which to clear the requirements

processResourceRequirements

public void processResourceRequirements(ResourceRequirements resourceRequirements)

Description copied from interface: SlotManager

Notifies the slot manager about the resource requirements of a job.

Specified by:

processResourceRequirements in interface SlotManager

Parameters:

resourceRequirements - resource requirements of a job

registerTaskManager

public boolean registerTaskManager(TaskExecutorConnection taskExecutorConnection,
                                   SlotReport initialSlotReport,
                                   ResourceProfile totalResourceProfile,
                                   ResourceProfile defaultSlotResourceProfile)

Registers a new task manager at the slot manager. This will make the task managers slots known and, thus, available for allocation.

Specified by:

registerTaskManager in interface SlotManager

Parameters:

taskExecutorConnection - for the new task manager

initialSlotReport - for the new task manager

totalResourceProfile - of the new task manager

defaultSlotResourceProfile - of the new task manager

Returns:

True if the task manager has not been registered before and is registered successfully; otherwise false

unregisterTaskManager

public boolean unregisterTaskManager(InstanceID instanceId,
                                     Exception cause)

Description copied from interface: SlotManager

Unregisters the task manager identified by the given instance id and its associated slots from the slot manager.

Specified by:

unregisterTaskManager in interface SlotManager

Parameters:

instanceId - identifying the task manager to unregister

cause - for unregistering the TaskManager

Returns:

True if there existed a registered task manager with the given instance id

reportSlotStatus

public boolean reportSlotStatus(InstanceID instanceId,
                                SlotReport slotReport)

Reports the current slot allocations for a task manager identified by the given instance id.

Specified by:

reportSlotStatus in interface SlotManager

Parameters:

instanceId - identifying the task manager for which to report the slot status

slotReport - containing the status for all of its slots

Returns:

true if the slot status has been updated successfully, otherwise false

freeSlot

public void freeSlot(SlotID slotId,
                     AllocationID allocationId)

Free the given slot from the given allocation. If the slot is still allocated by the given allocation id, then the slot will be freed.

Specified by:

freeSlot in interface SlotManager

Parameters:

slotId - identifying the slot to free, will be ignored

allocationId - with which the slot is presumably allocated

getNumberRegisteredSlots

public int getNumberRegisteredSlots()

Specified by:

getNumberRegisteredSlots in interface SlotManager

getNumberRegisteredSlotsOf

public int getNumberRegisteredSlotsOf(InstanceID instanceId)

Specified by:

getNumberRegisteredSlotsOf in interface SlotManager

getNumberFreeSlots

public int getNumberFreeSlots()

Specified by:

getNumberFreeSlots in interface SlotManager

getNumberFreeSlotsOf

public int getNumberFreeSlotsOf(InstanceID instanceId)

Specified by:

getNumberFreeSlotsOf in interface SlotManager

getRequiredResources

public Map<WorkerResourceSpec,Integer> getRequiredResources()

Description copied from interface: SlotManager

Get number of workers SlotManager requested from ResourceActions that are not yet fulfilled.

Specified by:

getRequiredResources in interface SlotManager

Returns:

a map whose key set is all the unique resource specs of the pending workers, and the corresponding value is number of pending workers of that resource spec.

getRegisteredResource

public ResourceProfile getRegisteredResource()

Specified by:

getRegisteredResource in interface SlotManager

getRegisteredResourceOf

public ResourceProfile getRegisteredResourceOf(InstanceID instanceID)

Specified by:

getRegisteredResourceOf in interface SlotManager

getFreeResource

public ResourceProfile getFreeResource()

Specified by:

getFreeResource in interface SlotManager

getFreeResourceOf

public ResourceProfile getFreeResourceOf(InstanceID instanceID)

Specified by:

getFreeResourceOf in interface SlotManager

getAllocatedSlotsOf

public Collection<SlotInfo> getAllocatedSlotsOf(InstanceID instanceID)

Specified by:

getAllocatedSlotsOf in interface SlotManager

getTaskManagerIdleSince

@VisibleForTesting
public long getTaskManagerIdleSince(InstanceID instanceId)