polaris.run.allocation.usable_cores

polaris.run.allocation.usable_cores(cores_per_node: int, logger) → Tuple[int, ...]

Which cores a launch may use on a node of this allocation, by number.

The configuration says how many; it does not say which, and the two differ where a site keeps cores back for itself. Aurora keeps core 0, and PALS refuses a launch bound to a core the job was not given, so a scheduler numbering cores from zero hands out one that cannot be used.

What the kernel allows this process is what the job may use on this node, and nodes in an allocation are alike, which is the assumption cores_per_node already makes. Those ids are hardware threads on every machine Polaris runs on, two to a core, and Polaris places one rank or thread per physical core and leaves the sibling idle – so the ids are reduced to one per core by what the kernel says shares a core, and the first cores_per_node of those are taken. A Perlmutter CPU node allows 256 ids and is configured with 128 cores; an Aurora node allows 204 – ids 1-51, 53-103 and their siblings, with 0, 52 and theirs held back – and is configured with 102.

The count of cores the job may use is held against the configured one and a disagreement is reported, since it means the machine’s configuration and the machine differ. A reading that leaves fewer than half the configured cores is not believed at all: the process may have been started bound to a corner of the node, and numbering the whole allocation from that corner would starve every step. The configured range is used instead, and said so.

Parameters:

• cores_per_node (int) – The configured number of cores on a node

• logger (logging.Logger) – Where a distrusted or disagreeing reading is reported

Returns:

core_ids (tuple of int) – The usable cores, one id per physical core, in order