General Utilities

These utilites are used by the pipeliner for basic tasks such as nice looking on-screen display, checking file names, and getting directory and file names

pipeliner.utils.check_for_illegal_symbols(check_string: str, string_name: str = 'input', exclude: str = '')

Check a text string doesn’t have any of the disallowed symbols.

Illegal symbols are !*?()^/#<>&%{}$.”’ and @.

Parameters
  • check_string (str) – The string to be checked

  • string_name (str) – The name of the string being checked; for more informative error messages

  • exclude (str) – Any symbols that are normally in the illegal symbols list but should be allowed.

Raises

ValueError – If illegal symbols are present in the test string

pipeliner.utils.clean_jobname(jobname: str) str

Makes sure job names are in the correct format

Job names must have a trailing slash, cannot begin with a slash, and have no illegal characters

Parameters

jobname (str) – The jon name to be checked

Returns

The job name, with corrections in necessary

Return type

str

pipeliner.utils.date_time_tag(compact: bool = False) str

Get a current date and time tag

If can return a compact version or one that is easier to read

Parameters

compact (bool) – Should the returned tag be in the comapct form

Returns

The datetime tag

compact format is: YYYYMMDDHHMMSS

verbose form is: YYYY-MM-DD HH:MM:SS

Return type

str

pipeliner.utils.decompose_pipeline_filename(fn_in: str) Tuple[str, Optional[int], str]

Breaks a job name into usable pieces

Returns everything before the job number, the job number as an int and everything after the job number setup for up to 20 dirs deep. The 20 directory limit is from the relion code but no really necessary anymore

Parameters

fn_in (str) – The job or file name to be broken down in the format: <jobtype>/jobxxx/<filename>

Returns

The decomposed file name: (str, int, str)

[0] Everything before ‘job’ in the file name

[1] The job number

[2] Everything after the job number

Return type

tuple

Raises

ValueError – If the input file name is more than 20 directories deep

pipeliner.utils.find_common_string(input_strings: List[str]) str

Find the common part of a list of strings starting from the beginning

Parameters

input_strings (list) – List of strings to compare

Returns

The common portion of the strings

Return type

str

Raises

ValueError – If input_list is shorter than 2

pipeliner.utils.fix_newlines(file_path: str)

Replace LF+CR new lines in files with LF, because RELION doesn’t like them

Parameters

file_path (str) – The file to fix

pipeliner.utils.get_pipeliner_root() pathlib.Path

Get the directory of the main pipeliner module

Returns

The path of the pipeliner

Return type

path

pipeliner.utils.make_pretty_header(title: str)

Make nice looking headers for on-screen display

Parameters

title (str) – The text to put in the header

Returns

A nice looking header

-=-=-=-=-=-=-=-=-=
It looks like this
-=-=-=-=-=-=-=-=-=

Return type

str

pipeliner.utils.print_nice_columns(list_in: List[str], err_msg: str = 'ERROR: No items in input list')

Takes a list of items and makes three columns for nicer on-screen display

Parameters
  • list_in (str) – The list to display in columns

  • err_msg (str) – The message to display if the list is empty

pipeliner.utils.quotate_command_list(commands: List[list]) List[list]

Adds quotation marks to commands arguments that need them

If a command is to be run in terminal some args need to be quotated. Quotation marks are not needed if the command list is run with subprocess.run but they are if the command is run as a string in a qsub script or in the terminal

Any arg that contains a space or the set of characters !*?()^#<>&%{}$@ will be quotated

Parameters

commands (list) – The commands are a list of lists. Each item in the main list is a single command, which itself is a list of the individual arguments

Returns

A correctly quotated command list

The list is in the same list of lists format

Return type

list

pipeliner.utils.smart_strip_quotes(in_string: str) str

Strip the quotes from a string in a intellegant manner

Remove leading and ending ‘ and ” but don’t remove them internally

Parameters

in_string (str) – The input string

Returns

the string with leading and ending quotes removed

Return type

str

pipeliner.utils.touch(filename: str)

Create an empty file

Parameters

filename (str) – The name for the file to create

pipeliner.utils.truncate_number(number: float, maxlength: int) str

Return a number with no more than x decimal places but no trailing 0s

This is used to format numbers in the exact same way that RELION does it. IE: with maxlength 3; 1.2000 = 1.2, 1.0 = 1, 1.23 = 1.23. RELION commands are happy to accept numbers with any number of decimal places or trailing 0s. This function is just to maintain continuity between RELION and pipeliner commands

Parameters
  • number (float) – The number to be truncated

  • maxlength (int) – The maximum number of decimal places

pipeliner.utils.wrap_text(text_string: str)

Produces <= 55 character wide wrapped text for on-screen display

Parameters

text_string (str) – The text to be displayed