polaris.ModelStep

class polaris.ModelStep(component, name, subdir=None, indir=None, ntasks=None, min_tasks=None, openmp_threads=None, gpus=None, min_gpus=None, gpus_per_task=0, min_gpus_per_task=0, memory=None, min_memory=None, cached=False, namelist=None, streams=None, yaml=None, update_io_tasks=True, update_pio=None, make_graph=False, mesh_filename=None, partition_graph=True, graph_filename='graph.info', make_yaml=False)

Variables:

• namelist (str) – The name of the namelist file

• streams (str) – The name of the streams file

• yaml (str) – The name of the yaml file

• config_models (list of str) – If config options are available for multiple models, a list of valid models from which config options should be taken. For example, for MPAS-Ocean this would be ['ocean', 'mpas-ocean'] and for Omega it is ['ocean', 'omega'], since both models share the generic ocean config options.

• update_io_tasks (bool) – Whether to modify model config options controlling IO tasks so the number of IO tasks and the stride between them are consistent with the number of nodes and cores (one IO task per node).

• make_graph (bool) – Whether to make a graph file from the given MPAS mesh file. If True, mesh_filename must be given.

• mesh_filename (str) – The name of an MPAS mesh file to use to make the graph file

• partition_graph (bool) – Whether to partition the domain for the requested number of cores. If so, the partitioning executable is taken from the partition option of the [executables] config section.

• graph_filename (str) – The name of the graph file to partition

• model_config_data (list) – a list used internally to keep track of updates to the default model config options from calls to polaris.ModelStep.add_yaml_file() polaris.ModelStep.add_namelist_file() and polaris.ModelStep.add_model_config_options()

• streams_data (list) – a list used internally to keep track of updates to the default streams from calls to polaris.ModelStep.add_streams_file()

• make_yaml (bool) – Whether to create a yaml file with model config options and streams instead of MPAS namelist and streams files

• streams_section (str) – The name of the streams section in yaml files

__init__(component, name, subdir=None, indir=None, ntasks=None, min_tasks=None, openmp_threads=None, gpus=None, min_gpus=None, gpus_per_task=0, min_gpus_per_task=0, memory=None, min_memory=None, cached=False, namelist=None, streams=None, yaml=None, update_io_tasks=True, update_pio=None, make_graph=False, mesh_filename=None, partition_graph=True, graph_filename='graph.info', make_yaml=False)

Make a step for running the model

Parameters:

• component (polaris.Component) – the component that the step belongs to

• name (str) – The name of the step

• subdir (str, optional) –

  the subdirectory for the step. If neither this nor indir

  are provided, the directory is the name

• indir (str, optional) – the directory the step is in, to which name will be appended

• ntasks (int, optional) – the target number of tasks the step would ideally use. If too few cores are available on the system to accommodate the number of tasks and the number of cores per task, the step will run on fewer tasks as long as as this is not below min_tasks

• min_tasks (int, optional) – the number of tasks the step requires. If the system has too few cores to accommodate the number of tasks and cores per task, the step will fail

• openmp_threads (int, optional) – the number of OpenMP threads to use

• gpus (int, optional) – the number of GPUs to use, as a total for the step rather than a count per task

• min_gpus (int, optional) – the minimum number of GPUs required, again as a total

• gpus_per_task (int, optional) –

  the number of GPUs per task to use

  Deprecated since version 1.1.0: Use gpus instead

• min_gpus_per_task (int, optional) –

  the minimum number of GPUs per task required

  Deprecated since version 1.1.0: Use min_gpus instead

• memory (int, optional) – the amount of memory in MB the step would ideally be given

• min_memory (int, optional) – the amount of memory in MB the step needs in order to run at all

• cached (bool, optional) – Whether to get all of the outputs for the step from the database of cached outputs for this component

• namelist (str, optional) – The name of the namelist file, default is namelist.<component>

• streams (str, optional) – The name of the streams file, default is streams.<component>

• update_io_tasks (bool, optional) – Whether to modify model config options controlling IO tasks so the number of IO tasks and the stride between them are consistent with the number of nodes and cores (one IO task per node).

• update_pio (bool, optional) – Deprecated alias for update_io_tasks.

• make_graph (bool, optional) – Whether to make a graph file from the given MPAS mesh file. If True, mesh_filename must be given.

• mesh_filename (str, optional) – The name of an MPAS mesh file to use to make the graph file

• partition_graph (bool, optional) – Whether to partition the domain for the requested number of cores. If so, the partitioning executable is taken from the partition option of the [executables] config section.

• graph_filename (str, optional) – The name of the graph file to partition

• make_yaml (bool, optional) – Whether to create a yaml file with model config options and streams instead of MPAS namelist and streams files

Methods

__init__(component, name[, subdir, indir, ...])	Make a step for running the model
add_dependency(step[, name])	Add step as a dependency of this step (i.e. this step can't run until the dependency has finished).
add_input_file([filename, target, database, ...])	Add an input file to the step (but not necessarily to the MPAS model).
add_model_config_options(options[, config_model])	Add the replacement model config options to be parsed when generating a namelist or yaml file if and when the step gets set up.
add_namelist_file(package, namelist)	Add a file with updates to namelist options to the step to be parsed when generating a complete namelist file if and when the step gets set up.
add_output_file(filename[, validate_vars, ...])	Add the output file to the step
add_property_check(filename, check_properties)	Add a single conservation comparison for an output file
add_streams_file(package, streams[, ...])	Add a streams file to the step to be parsed when generating a complete streams file if and when the step gets set up.
add_yaml_file(package, yaml[, ...])	Add a file with updates to yaml config options to the step to be parsed when generating a complete yaml file if and when the step gets set up.
check_properties()	Check conservation properties of the output files of this step.
constrain_resources(available_resources)	Constrain cpus_per_task and ntasks based on the number of cores available to this step
dynamic_model_config(at_setup)	Add model config options, namelist, streams and yaml files using config options or template replacements that need to be set both during step setup and at runtime
map_yaml_configs(configs, config_model)	A mapping between model config options from different models.
map_yaml_options(options, config_model)	A mapping between model config options from different models.
map_yaml_streams(streams, config_model)	A mapping between model streams from different models.
map_yaml_to_namelist(options)	A mapping from yaml model config options to namelist options.
partition([graph_file])	Partition the domain for the requested number of tasks
process_inputs_and_outputs()	Process the model as an input, then call the parent class' version
run()	Run the step.
runtime_setup()	Update IO task model config options, make graph file, and partition graph file (if any of these are requested)
set_model_resources([ntasks, min_tasks, ...])	Update the resources for the step.
set_resources([cpus_per_task, ...])	Update the resources for the subtask.
set_shared_config(config[, link])	Replace the step's config parser with the shared config parser
setup()	Setup the command-line arguments
update_io_tasks_config([config_model])	Modify model config options so the number of IO tasks and the stride between them are consistent with how MPI tasks are distributed across nodes (one IO task per node).
update_namelist_pio([config_model])	Deprecated alias for update_io_tasks_config().
validate_baselines()	Compare variables between output files in this step and in the same step from a baseline run if one was provided.
work_path(*filenames)	Get the absolute path to a file or directory in the step's work directory

Attributes

cores	the number of cores this step needs, in total
gpus	the number of GPUs this step needs, in total
may_span_nodes	whether this step's cores and GPUs may come from several nodes
min_cores	the number of cores this step needs in order to run at all
min_gpus	the number of GPUs this step requires, in total