General Utilities

These utilites are used by the pipeliner for basic tasks such as nice looking on-screen display, checking file names, and getting directory and file names

pipeliner.utils.check_for_illegal_symbols(check_string: str, string_name: str = 'input', exclude: str = '')

Check a text string doesn’t have any of the disallowed symbols.

Illegal symbols are !*?()^/#<>&%{}$.”’ and @.

Parameters

check_string (str) – The string to be checked
string_name (str) – The name of the string being checked; for more informative error messages
exclude (str) – Any symbols that are normally in the illegal symbols list but should be allowed.

Raises

ValueError – If illegal symbols are present in the test string

pipeliner.utils.clean_jobname(jobname: str) → str

Makes sure job names are in the correct format

Job names must have a trailing slash, cannot begin with a slash, and have no illegal characters

Parameters: jobname (str) – The jon name to be checked
Returns: The job name, with corrections in necessary
Return type: str

pipeliner.utils.date_time_tag(compact: bool = False) → str

Get a current date and time tag

If can return a compact version or one that is easier to read

Parameters

compact (bool) – Should the returned tag be in the comapct form

Returns

The datetime tag

compact format is: YYYYMMDDHHMMSS

verbose form is: YYYY-MM-DD HH:MM:SS

Return type

str

pipeliner.utils.decompose_pipeline_filename(fn_in: str) → Tuple[str, Optional[int], str]

Breaks a job name into usable pieces

Returns everything before the job number, the job number as an int and everything after the job number setup for up to 20 dirs deep. The 20 directory limit is from the relion code but no really necessary anymore

Parameters

fn_in (str) – The job or file name to be broken down in the format: <jobtype>/jobxxx/<filename>

Returns

The decomposed file name: (str, int, str)

[0] Everything before ‘job’ in the file name

[1] The job number

[2] Everything after the job number

Return type

tuple

Raises

ValueError – If the input file name is more than 20 directories deep

pipeliner.utils.find_common_string(input_strings: List[str]) → str

Find the common part of a list of strings starting from the beginning

Parameters: input_strings (list) – List of strings to compare
Returns: The common portion of the strings
Return type: str
Raises: ValueError – If input_list is shorter than 2

pipeliner.utils.fix_newlines(file_path: str)

Replace LF+CR new lines in files with LF, because RELION doesn’t like them

Parameters: file_path (str) – The file to fix

pipeliner.utils.get_pipeliner_root() → pathlib.Path

Get the directory of the main pipeliner module

Returns: The path of the pipeliner
Return type: path

pipeliner.utils.make_pretty_header(title: str)

Make nice looking headers for on-screen display

Parameters

title (str) – The text to put in the header

Returns

A nice looking header

-=-=-=-=-=-=-=-=-=
It looks like this
-=-=-=-=-=-=-=-=-=

Return type

str

pipeliner.utils.print_nice_columns(list_in: List[str], err_msg: str = 'ERROR: No items in input list')

Takes a list of items and makes three columns for nicer on-screen display

Parameters

list_in (str) – The list to display in columns
err_msg (str) – The message to display if the list is empty

pipeliner.utils.quotate_command_list(commands: List[list]) → List[list]

Adds quotation marks to commands arguments that need them

If a command is to be run in terminal some args need to be quotated. Quotation marks are not needed if the command list is run with subprocess.run but they are if the command is run as a string in a qsub script or in the terminal

Any arg that contains a space or the set of characters !*?()^#<>&%{}$@ will be quotated

Parameters

commands (list) – The commands are a list of lists. Each item in the main list is a single command, which itself is a list of the individual arguments

Returns

A correctly quotated command list

The list is in the same list of lists format

Return type

list

pipeliner.utils.smart_strip_quotes(in_string: str) → str

Strip the quotes from a string in a intellegant manner

Remove leading and ending ‘ and ” but don’t remove them internally

Parameters: in_string (str) – The input string
Returns: the string with leading and ending quotes removed
Return type: str

pipeliner.utils.touch(filename: str)

Create an empty file

Parameters: filename (str) – The name for the file to create

pipeliner.utils.truncate_number(number: float, maxlength: int) → str

Return a number with no more than x decimal places but no trailing 0s

This is used to format numbers in the exact same way that RELION does it. IE: with maxlength 3; 1.2000 = 1.2, 1.0 = 1, 1.23 = 1.23. RELION commands are happy to accept numbers with any number of decimal places or trailing 0s. This function is just to maintain continuity between RELION and pipeliner commands

Parameters

number (float) – The number to be truncated
maxlength (int) – The maximum number of decimal places

pipeliner.utils.wrap_text(text_string: str)

Produces <= 55 character wide wrapped text for on-screen display

Parameters: text_string (str) – The text to be displayed