faust.assignor.copartitioned_assignor

Copartitioned Assignor.

class faust.assignor.copartitioned_assignor.CopartitionedAssignor(topics: Iterable[str], cluster_asgn: MutableMapping[str, faust.assignor.client_assignment.CopartitionedAssignment], num_partitions: int, replicas: int, capacity: int = None) → None[source]

Copartitioned Assignor.

All copartitioned topics must have the same number of partitions

The assignment is sticky which uses the following heuristics:

  • Maintain existing assignments as long as within capacity for each client

  • Assign actives to standbys when possible (within capacity)

  • Assign in order to fill capacity of the clients

We optimize for not over utilizing resources instead of under-utilizing resources. This results in a balanced assignment when capacity is the default value which is ceil(num partitions / num clients)

Notes

Currently we raise an exception if number of clients is not enough for the desired replication.

get_assignment() → MutableMapping[str, faust.assignor.client_assignment.CopartitionedAssignment][source]
Return type

MutableMapping[str, CopartitionedAssignment]