Job Options

JobOptions

JobOptions are used to store parameters for jobs. They contain all the info necessary for on-the-fly GUI generation.

class pipeliner.job_options.BooleanJobOption(*, label: str, default_value: bool, help_text: str = '', in_continue: bool = False, only_in_continue: bool = False, deactivate_if: JobOptionCondition | None = None, jobop_group: str = 'Main')

Bases: JobOption

Define a job option as a boolean

Parameters:

label (str) – A verbose label for the parameter. This is what appears in a run.job file
default_value (bool) – The default value
help_text (str) – Text that will be displayed in the GUI if help is clicked
in_continue (bool) – If this parameter can be modified in a job that is continued

get_boolean() → bool

Get a boolean value

Returns:: The value
Return type:: bool

joboption_type = 'BOOLEAN'

validate() → List[JobOptionValidationResult]

Basic validation of the input parameters

Returns:

The validation: error of one exists

Return type:

JobOptionValidationResult

class pipeliner.job_options.DirPathJobOption(*, label: str, default_value: str = '', help_text: str = '', must_be_in_project: bool = False, must_exist: bool = True, in_continue: bool = False, only_in_continue: bool = False, is_required: bool = False, required_if: JobOptionCondition | None = None, deactivate_if: JobOptionCondition | None = None, validation_regex: str | None = None, regex_error_message: str | None = None, jobop_group: str = 'Main')

Bases: StringJobOption

A JobOption for directory paths

Parameters:

label (str) – A verbose label for the parameter.
default_value (None) –
help_text (str) – Text that will be displayed in the GUI if help is clicked.
in_continue (bool) – If this parameter can be modified in a job that is continued
must_be_in_project (bool) – Does the directory have to be in the project
must_exist (bool) – Does this directory have to exist when the job option is validated? Usually true, unless it will be created by a previous command in the job

check_dir() → List[JobOptionValidationResult]

Check that the specified dir exists.

Returns:: An error if the dir does not exist, or a warning if the dir does not exist and must_exist is False
Return type:: List[JobOptionValidationResult]

joboption_type = 'DIRPATH'

validate() → List[JobOptionValidationResult]: Basic validation of the input parameters

class pipeliner.job_options.ExternalFileJobOption(*, label: str, pattern: str = '', default_value: str = '', help_text: str = '', in_continue: bool = False, only_in_continue: bool = False, is_required: bool = False, required_if: JobOptionCondition | None = None, deactivate_if: JobOptionCondition | None = None, validation_regex: str | None = None, regex_error_message: str | None = None, suggestion: str = '', jobop_group: str = 'Main')

Bases: JobOption

Define a job option as a file name

GUI will open a file browser for the user to find an input

Parameters:

label (str) – A verbose label for the parameter. This is what appears in a run.job file
default_value (str) – will usually be set from the string and pattern.
pattern (str) – Info about the search string for file types. It should be in the format <description> (<example>) e.g.: “MRC files (*.mrc)”
help_text (str) – Text that will be displayed in the GUI if help is clicked
in_continue (bool) – If this parameter can be modified in a job that is continued
suggestion (str) – A hint that will appear in the data entry field if no no default value is set, it will not be used. IE: ‘Enter file path here’ If you want the suggestion used as the default then just put it in default_value

check_file() → List[JobOptionValidationResult]

Check that specified input files exist, also checks in path

Returns:: An error message if the file does not exist
Return type:: str

get_basename_mapping() → Dict[str, str]

Get a mapping of names from input files to unique basenames.

The purpose of this method is to get unique basenames (i.e. the file name alone without any directory part) for all input files, by adding a prefix if necessary to make sure files that would otherwise have identical basenames are all assigned a unique basename to use for output files.

Returns:

A dict with the full original input file names as keys and the unique basenames as values, for example:

{
  "Import/job001/myfile.txt": "job001_myfile.txt",
  "Import/job002/myfile.txt": "job002_myfile.txt",
}

joboption_type = 'FILENAME'

validate() → List[JobOptionValidationResult]

Basic validation of the input parameters

Returns:

The: validation error of one exists

Return type:

List[JobOptionValidationResult]

class pipeliner.job_options.FloatJobOption(*, label: str, default_value: str | float | None, suggested_min: float | None = None, suggested_max: float | None = None, step_value: float = 1.0, help_text: str = '', in_continue: bool = False, only_in_continue: bool = False, is_required: bool = False, required_if: JobOptionCondition | None = None, deactivate_if: JobOptionCondition | None = None, hard_min: float | None = None, hard_max: float | None = None, jobop_group: str = 'Main')

Bases: JobOption

Define a job option as a slider for inputting numbers as floats

The min value and max value are for display only, they do not limit the actual values that can be input

Parameters:

label (str) – A verbose label for the parameter. This is what appears in a run.job file
default_value (float) – The default value for the parameter
suggested_min (float) – The suggested minimum value for the slider
suggested_max (float) – The suggested maximum value for the slider
step_value (float) – The step value for the slider
help_text (str) – Text that will be displayed in the GUI if help is clicked
in_continue (bool) – If this parameter can be modified in a job that is continued
hard_min (float) – An error will be raised if the value is < this value
hard_max (float) – An error will be raised if the value is > this value

get_number() → float

Get the value of a JobOption as a number

Returns:

The value

Return type:

float

Raises:

ValueError – If the value cannot be converted to a float
ValueError – If the value is required and missing

joboption_type = 'FLOAT'

validate() → List[JobOptionValidationResult]

Basic validation check on the job option

Returns:

The validation: error of one exists

Return type:

JobOptionValidationResult

class pipeliner.job_options.InputNodeJobOption(*, label: str, node_type: str, default_value: str = '', pattern: str = '', help_text: str = '', in_continue: bool = False, only_in_continue: bool = False, is_required: bool = False, required_if: JobOptionCondition | None = None, deactivate_if: JobOptionCondition | None = None, validation_regex: str | None = None, regex_error_message: str | None = None, suggestion: str = '', jobop_group: str = 'Main', node_kwds: List[str] | None = None)

Bases: JobOption

Define a job option as a file name, create an input node for it

GUI will open a file browser for the user to find an input

Parameters:

label (str) – A verbose label for the parameter. This is what appears in a run.job file
node_type (str) – Top level node type for this node
default_value (str) – will be set by the specific class method
pattern (str) – Info about the search string for file types. It should be in the format <description> (<example>) IE: “MRC files (*.mrc)”
help_text (str) – Text that will be displayed in the GUI if help is clicked
in_continue (bool) – If this parameter can be modified in a job that is continued
suggestion (str) – A hint that will appear in the data entry field if no no default value is set, it will not be used. IE: ‘Enter file path here’ If you want the suggestion used as the default then just put it in default_value

check_file() → List[JobOptionValidationResult]

Check that specified input files exist

Returns:: An error message if the file does not exist
Return type:: str

get_basename_mapping() → Dict[str, str]

Get a mapping of names from input files to unique basenames.

The purpose of this method is to get unique basenames (i.e. the file name alone without any directory part) for all input files, by adding a prefix if necessary to make sure files that would otherwise have identical basenames are all assigned a unique basename to use for output files.

Returns:

A dict with the full original input file names as keys and the unique basenames as values, for example:

{
  "Import/job001/myfile.txt": "job001_myfile.txt",
  "Import/job002/myfile.txt": "job002_myfile.txt",
}

get_input_nodes() → List[Node]

Get the input node(s) related to this job option.

Most job option types do not create input nodes and so they should rely on this default implementation that returns an empty list.

Options that do create nodes should return a list of the corresponding Node objects.

joboption_type = 'INPUTNODE'

validate() → List[JobOptionValidationResult]

Basic validation of the input parameters

Returns:: The validation error of one exists
Return type:: List[JobOptionValidationResult]

class pipeliner.job_options.IntJobOption(*, label: str, default_value: int | None, suggested_min: int | None = None, suggested_max: int | None = None, step_value: int = 1, help_text: str = '', in_continue: bool = False, only_in_continue: bool = False, is_required: bool = False, required_if: JobOptionCondition | None = None, deactivate_if: JobOptionCondition | None = None, hard_min: int | None = None, hard_max: int | None = None, jobop_group: str = 'Main')

Bases: JobOption

Define a job option as a slider for inputting numbers as integers

The slider min value and max value are for display only, they do not limit the actual values that can be input

Parameters:

label (str) – A verbose label for the parameter. This is what appears in a run.job file
default_value (int) – The default value for the parameter
suggested_min (int) – The suggested minimum value for the slider
suggested_max (int) – The suggested maximum value for the slider
step_value (int) – The step value for the slider
help_text (str) – Text that will be displayed in the GUI if help is clicked
in_continue (bool) – If this parameter can be modified in a job that is continued
hard_min (int) – An error will be raised if the value is < this value
hard_max (int) – An error will be raised if the value is > this value

Raises:

ValueError – If the suggested_min, suggested_max, default, hard_min , hard_max, or step values are not integers

get_number() → int

Get the value of a JobOption as a number

Returns:

The value

Return type:

int

Raises:

ValueError – If the value cannot be converted to a float
ValueError – If the value is required and missing

joboption_type = 'INT'

validate() → List[JobOptionValidationResult]

Basic validation check on the job option

Returns:

The validation: error of one exists

Return type:

JobOptionValidationResult

class pipeliner.job_options.JobOption(*, label: str, default_value: Any = '', help_text: str = '', in_continue: bool = False, is_required: bool = False, required_if: JobOptionCondition | None = None, deactivate_if: JobOptionCondition | None = None, only_in_continue: bool = False, jobop_group: str = 'Main')

Bases: object

A JobOption stores a parameter, its value, and info about the GUI for that param

This is a general class with several more specialised class subclasses.

check_dir() → List[JobOptionValidationResult]

Check that the value is an existing directory

Returns:: A JobOptionValidationResult for any errors
Return type:: list

check_file() → List[JobOptionValidationResult]

Check that specified input files exist

Returns:: A JobOptionValidationResult for any errors
Return type:: list

check_required_not_empty() → None: Check if the job option is required but has no value

get_basename_mapping() → Dict[str, str]

Get a mapping of names from input files to unique basenames.

The purpose of this method is to get unique basenames (i.e. the file name alone without any directory part) for all input files, by adding a prefix if necessary to make sure files that would otherwise have identical basenames are all assigned a unique basename to use for output files.

Returns:

A dict with the full original input file names as keys and the unique basenames as values, for example:

{
  "Import/job001/myfile.txt": "job001_myfile.txt",
  "Import/job002/myfile.txt": "job002_myfile.txt",
}

get_boolean() → bool

get_input_nodes() → List[Node]

Get the input node(s) related to this job option.

Most job option types do not create input nodes and so they should rely on this default implementation that returns an empty list.

Options that do create nodes should return a list of the corresponding Node objects.

get_list() → List[str]

get_number() → int | float

get_string() → str

Get the value of a JobOption as a string

Returns:: The value
Return type:: str
Raises:: ValueError – If the value is required and missing

joboption_type = 'ERROR: no job option type set'

set_string(set_to: str) → None

Set the value of a JobOption to a string

Parameters:: set_to (str) – The string to set the value to

validate() → List[JobOptionValidationResult]

Basic validation of input parameters

will be set by individual JobOption subtypes

class pipeliner.job_options.JobOptionCondition(conditions: List[Tuple[str, str, str | bool | int | float]], operation: Literal['ANY', 'ALL'] = 'ANY')

Bases: object

Definines conditions for JobOption based on other JobOptions

Used to determine if a JobOption should be required or deactivated

Has the condition been met?

Parameters:

jobops (dict) – The JobOptions dict for the job

Returns:

(bool: Has the condition been met?, List[conditions that passed],: List[conditions that failed]

Return type:

Tuple

class pipeliner.job_options.JobOptionValidationResult(result_type: Literal['warning', 'error', 'info'], raised_by: List[JobOption], message: str)

Bases: object

A class for handling validation of joboptions

type

One of ‘warning’, ‘error’ or ‘info’

Type:: str

raised_by

List of JobOption objects that raised the warning/error. Generally this will only be one JobOption, unless the warning/error was raised from comparing two or more JobOptions

Type:: list

message

Description of the error/warning

Type:: str

class pipeliner.job_options.MultiExternalFileJobOption(*, label: str, pattern: str = '', default_value: str = '', help_text: str = '', in_continue: bool = False, only_in_continue: bool = False, is_required: bool = False, required_if: JobOptionCondition | None = None, deactivate_if: JobOptionCondition | None = None, validation_regex: str | None = None, regex_error_message: str | None = None, suggestion: str = '', jobop_group: str = 'Main', allow_upload: bool = False)

Bases: JobOption

Define a job option as multiple file names

GUI will open a file browser for the user to find an input for each one. The value for this should be a string of values separated by ::: which will be converted into a list by: [x.strip() for x in the_str.split(“:::”)]

Parameters:

label (str) – A verbose label for the parameter. This is what appears in a run.job file
default_value (str) – will usually be set from the string and pattern
pattern (str) – Info about the search string for file types. It should be in the format <description> (<example>) e.g.: “MRC files (*.mrc)”
help_text (str) – Text that will be displayed in the GUI if help is clicked
in_continue (bool) – If this parameter can be modified in a job that is continued
suggestion (str) – A hint that will appear in the data entry field if no no default value is set, it will not be used. IE: ‘Enter file path here’ If you want the suggestion used as the default then just put it in default_value
allow_upload (bool) – Should a file upload be allowed for this input? If yes a copy of the file will end up in the project. This should be turned off for external programs and files that are expected to be very large like ML models.

check_file() → List[JobOptionValidationResult]

Check that specified input files exist

For each input file that does not exist, a JobOptionValidationResult of type “error” is returned.

Returns:: class:~pipeliner.job_options.JobOptionValidationResult]: if any of the files do not exist
Return type:: List[

get_basename_mapping() → Dict[str, str]

Get a mapping of names from input files to unique basenames.

The purpose of this method is to get unique basenames (i.e. the file name alone without any directory part) for all input files, by adding a prefix if necessary to make sure files that would otherwise have identical basenames are all assigned a unique basename to use for output files.

Returns:

A dict with the full original input file names as keys and the unique basenames as values, for example:

{
  "Import/job001/myfile.txt": "job001_myfile.txt",
  "Import/job002/myfile.txt": "job002_myfile.txt",
}

get_list() → List[str]

Return the items in self.value as a list of strings

Returns:

The items in the JobOptions comma separated string: as a list, with any white space at the beginning and/or end removed

Return type:

List[str]

joboption_type = 'MULTIFILENAME'

validate() → List[JobOptionValidationResult]

Basic validation of the input parameters

Returns:: The validation errors of any exist
Return type:: List[JobOptionValidationResult]

class pipeliner.job_options.MultiInputNodeJobOption(*, label: str, node_type: str, default_value: str = '', pattern: str = '', help_text: str = '', in_continue: bool = False, only_in_continue: bool = False, is_required: bool = False, required_if: JobOptionCondition | None = None, deactivate_if: JobOptionCondition | None = None, validation_regex: str | None = None, regex_error_message: str | None = None, suggestion: str = '', jobop_group: str = 'Main', node_kwds: List[str] | None = None)

Bases: JobOption

Define a job option as a file name, create an input node for it

GUI will open a file browser for the user to find an input

Parameters:

label (str) – A verbose label for the parameter. This is what appears in a run.job file
node_type (str) – Toplevel node type for this node.
node_kwds – Optional[List[Optional[List[str]]]]: keywords for each toplevel type in node_types. The index of the toplevel type is used to match the kwds to toplevel type.
default_value (str) – will be set by the specific class method
pattern (str) – Info about the search string for file types. It should be in the format <description> (<example>), e.g. “MRC files (*.mrc)”
help_text (str) – Text that will be displayed in the GUI if help is clicked
in_continue (bool) – If this parameter can be modified in a job that is continued
suggestion (str) – A hint that will appear in the data entry field if no no default value is set, it will not be used. IE: ‘Enter file path here’ If you want the suggestion used as the default then just put it in default_value

check_file() → List[JobOptionValidationResult]

Check that specified input files exist

For each input file that does not exist, a JobOptionValidationResult of type “error” is returned.

Returns:: class:~pipeliner.job_options.JobOptionValidationResult]: if any of the files do not exist
Return type:: List[

get_basename_mapping() → Dict[str, str]

Get a mapping of names from input files to unique basenames.

The purpose of this method is to get unique basenames (i.e. the file name alone without any directory part) for all input files, by adding a prefix if necessary to make sure files that would otherwise have identical basenames are all assigned a unique basename to use for output files.

Returns:

A dict with the full original input file names as keys and the unique basenames as values, for example:

{
  "Import/job001/myfile.txt": "job001_myfile.txt",
  "Import/job002/myfile.txt": "job002_myfile.txt",
}

get_input_nodes() → List[Node]

Get the input node(s) related to this job option.

Most job option types do not create input nodes and so they should rely on this default implementation that returns an empty list.

Options that do create nodes should return a list of the corresponding Node objects.

get_list()

joboption_type = 'MULTIINPUTNODE'

validate() → List[JobOptionValidationResult]

Basic validation of the input parameters

Returns:: The validation results (if any)
Return type:: List[JobOptionValidationResult]

class pipeliner.job_options.MultiStringJobOption(*, label: str, default_value: str = '', help_text: str = '', in_continue: bool = False, only_in_continue: bool = False, is_required: bool = False, required_if: JobOptionCondition | None = None, deactivate_if: JobOptionCondition | None = None, validation_regex: str | None = None, regex_error_message: str | None = None, jobop_group: str = 'Main')

Bases: JobOption

Create a job option object for a multiple strings. For things like keyword lists

The value for this should be a string containing values separated by ::: which will be converted into a list by: [x.strip() for x in the_str.split(“:::”)]

The only reason there can’t just be a self.get_list() function in a regular StringJobOption is the GUI must handle them differently.

For regex validation all values in the list of inputs must match the regex

Parameters:

label (str) – A verbose label for the parameter.
default_value (None) –
help_text (str) – Text that will be displayed in the GUI if help is clicked.
in_continue (bool) – If this parameter can be modified in a job that is continued

get_list() → List[str]

Return the items in self.value as a list of strings

Returns:

The items in the JobOptions separated string: as a list, with any white space at the beginning and/or end removed

Return type:

List[str]

joboption_type = 'MULTISTRING'

validate() → List[JobOptionValidationResult]: Basic validation of the input parameters

class pipeliner.job_options.MultipleChoiceJobOption(*, label: str, choices: List[str], default_value_index: int, help_text: str = '', in_continue: bool = False, only_in_continue: bool = False, is_required: bool = True, required_if: JobOptionCondition | None = None, deactivate_if: JobOptionCondition | None = None, jobop_group: str = 'Main')

Bases: JobOption

Define a job option as a pull down menu

Radio is a misnomer, the GUI will display a pull down menu with the options

Parameters:

label (str) – A verbose label for the parameter. This is what appears in a run.job file
choices (list) – A list str options for the menu
default_value_index (int) – Index of the initial option for the radio; also used as the default value
help_text (str) – Text that will be displayed in the GUI if help is clicked
in_continue (bool) – If this parameter can be modified in a job that is continued

Raises:

ValueError – If the index of the initial option is not in the radio menu

joboption_type = 'MULTIPLECHOICE'

validate() → List[JobOptionValidationResult]

Basic validation of the input parameters

Returns:: The validation error of one exists
Return type:: List[JobOptionValidationResult]

class pipeliner.job_options.SearchStringJobOption(*, label: str, default_value: str = '', help_text: str = '', must_be_in_project: bool = False, in_continue: bool = False, only_in_continue: bool = False, is_required: bool = False, required_if: JobOptionCondition | None = None, deactivate_if: JobOptionCondition | None = None, validation_regex: str | None = None, regex_error_message: str | None = None, jobop_group: str = 'Main')

Bases: StringJobOption

A JobOption for search strings

Parameters:

label (str) – A verbose label for the parameter.
default_value (None) –
help_text (str) – Text that will be displayed in the GUI if help is clicked.
in_continue (bool) – If this parameter can be modified in a job that is continued

check_file() → List[JobOptionValidationResult]

Check that at least one input file exists

Returns:: Error if no files found
Return type:: List[JobOptionValidationResult]

get_list()

joboption_type = 'SEARCHSTRING'

validate() → List[JobOptionValidationResult]: Basic validation of the input parameters

class pipeliner.job_options.StringJobOption(*, label: str, default_value: str = '', help_text: str = '', in_continue: bool = False, only_in_continue: bool = False, is_required: bool = False, required_if: JobOptionCondition | None = None, deactivate_if: JobOptionCondition | None = None, validation_regex: str | None = None, regex_error_message: str | None = None, jobop_group: str = 'Main')

Bases: JobOption

Create a job option object for a string parameter.

Parameters:

label (str) – A verbose label for the parameter.
default_value (None) –
help_text (str) – Text that will be displayed in the GUI if help is clicked.
in_continue (bool) – If this parameter can be modified in a job that is continued

joboption_type = 'STRING'

validate() → List[JobOptionValidationResult]: Basic validation of the input parameters

pipeliner.job_options.files_exts(name: str = 'File', exts: List[str] | None = None, exact: bool = False) → str

Produce a description of files and their extensions

In a format compatible with PyQt5 QFileDialog

Parameters:

name (str) – The type of file IE: ‘Micrograph movies’
exts (list) – The acceptable extensions or file search strings, e.g. ‘.txt’ to find *.txt or ‘_half1.star’ for *_half1.star
exact (bool) – Match the file name(s) exactly, do not prepend a *

Returns:

Formatted for PyQt5 QFileDialog: ‘name (*ext1 *ext2 *ext3)’

Return type:

str