Star File Utilities

Star file reading utilities

Tools for reading and modifying star files

class pipeliner.jobstar_reader.BodyFile(fn_in)

Bases: object

A star file that lists the bodies in a multibody refinement

data

A gemmi cif object containing the data from

Type: gemmi.cif.Cif

bodies

The block that contains info about the bodies

Type: gemmi.cif.Block

count_bodies()

Count the number of bodies in the BodyFile

Returns: The count
Return type: int

class pipeliner.jobstar_reader.ExportStar(fn_in)

Bases: object

Class for a star file that lists exported jobs

This functionality is from RELION 3.1 and will probably be deprecated

data

A gemmi cif object containing the data from the star file

Type: gemmi.cif.Cif

jobs

The block that contains info about the exported jobs

Type: gemmi.cif.Block

class pipeliner.jobstar_reader.JobStar(fn_in)

Bases: object

A class for star files that define a pipeliner job parameters

data

A gemmi cif object containing the data from the job.star file

Type: gemmi.cif.Cif

pipeline_info

The cif data block for the job section of the file

Type: gemmi.cif.Block

options

The cif data block for the ‘options’ section of the file

Type: gemmi.cif.Block

count_blocks()

Count the number of blocks in the file

Returns: The number of blocks
Return type: int

count_jobopt()

Count the number of items in the job options section of the file

Returns: The number of items
Return type: int

count_pipeline()

Count the number of items in the pipeline section of the file

Returns: The number of items
Return type: int

get_all_options(against=None)

Get a dict of all the parameters in the file

Parameters: against (iterable) – Only return joboptions that are present in this list, if None returns all the job options
Returns: The parameters dict in the format {param_name: value}
Return type: dict

get_block(block_name)

Get a specific block from the file

Parameters: block_name (str) – The name of the block to retrieve
Returns: The desired block
Return type: gemmi.cif.Block

get_continue_status()

get_jobtype()

class pipeliner.jobstar_reader.OutputNodeStar(fn_in)

Bases: object

A starfile that lists output nodes for exported jobs

This functionality is from RELION 3.1 and will probably be deprecated

data

A gemmi cif object containing the data from the star file

Type: gemmi.cif.Cif

jobs

The block that contains info about the exported nodes

Type: gemmi.cif.Block

get_output_nodes()

Get a list of output node names and types from an OutputNodeStar

This functionality is from RELION 3.1 and will probably be deprecated

Returns: [[node 1 name, node 1 type], …, [node n name, node n type]]
Return type: list

class pipeliner.jobstar_reader.RelionStarFile(fn_in)

Bases: object

A general class for starfiles that contain data, such as particles files

The input file is checked for reserved word errors when initialising a RelionStarFile object

data

A gemmi cif object containing the data from the star file

Type: gemmi.cif.Cif

count_block(blockname=None)

Count the number of items in a block that only contains a single loop

This is the format in most relion data star files

Parameters: blockname (str) – The name of the block to count
Returns: The count
Return type: int

get_block(blockname=None)

Get a block from the star file

Parameters: blockname (str) – The name of the block to get. Use None if the file has a single unnamed block
Returns: The desired block
Return type: (gemmi.cif.Block)

class pipeliner.jobstar_reader.StarfileCheck(fn_in=None, cifdoc=None, is_pipeline=False)

Bases: object

A class for checking the validity of star files and converting them

Checks for reserved word errors which can be common in star files written by some versions of RELION. If the starfile was written by RELION 3.x converts it to the pipeliner format

version

Info from the star file’s #version line what program wrote it and what version number it is

Type: str

fn_in

The path to the star file.

Type: str

cifdoc

The cif doc containing the data for the file

Type: gemmi.cif.Cif

has_been_corrected

If the file has been corrected

Type: bool

has_been_converted

If the file has been converted from RELION 3.1 to RELION 4.0/Pipeliner format

Type: bool

is_pipeline

If the file is a pipeline star file

Type: bool

check_pipeline_version()

See if a pipeline is the RELION 3.x or the new RELION 4.0/pipeliner style

Returns: The pipeline version relion3, relion4, or undefined
Return type: str
Raises: ValueError – If the pipeline has features of both RELION 3.x and RELION 4.0/Pipeliner versions

check_reserved_words()

Make sure the starfile doesn’t contain any illegally used reserved words

Only operates on a file only as trying to read in a cif doc with reserved word errors raises a parse error. Overwrites the original file if it is corrected. The old file is saved as <filename>.old

convert_relion3_1_pipeline(display_warning=True)

Convert a pipeline from RELION 3.x to RELION 4.0/Pipeliner format

Backs up the original file as <filename>.old

Parameters: display_warning (bool) – Should a warning be displayed on screen?

pipeliner.jobstar_reader.bool_jobop(jobop)

Checks if a value should be interpreted as True

Returns: True if the value is true, otherwise False
Return type: bool

pipeliner.jobstar_reader.compare_starfiles(starfile1, starfile2)

See if two starfiles contain the same information

Direct comparison can be difficult because the starfile columns or blocks can be in different orders

Parameters

starfile1 (str) – Name of the first file
starfile2 (str) – Name of the second file

Returns

(bool, bool) [0] True if the starfile structures are identical (Names of blocks and columns) [1] True if the data in the two files are identical

Return type

tuple

pipeliner.jobstar_reader.convert_coordinates_job(fn_in)

Convert RELION 3.1 style coordinates files to RELION 4 style

RELION 3.x has an empty file called coords_suffix_xxx.star RELION 4.0/pipeliner has a starfile with two columns for micrograph name and coords file. If errors are encountered with the conversion it sets the node type for that file as UNSUPPORTED_deprecated_format

Parameters: fn_in (str) – Path to the RELION 3.x style file
Returns: The new file name
Return type: str

pipeliner.jobstar_reader.convert_oldstyle_names(fn_in)

Converts model and particle nodes from RELION 3.x to RELION 4 names

The model files is RELION 3.x have been merged in to the _optimiser files in RELION 4. The particle files have moved from a system where the name and location of the file define the particles to the particles being explicitly defined in the file itself.

Parameters: fn_in (str) – The path to the file to convert
Returns: The name of that file’s node in RELION 4
Return type: str

pipeliner.jobstar_reader.convert_pipeline_node(name, oldtype)

Convert node names from a RELION 3.1 to the RELION 4.0 formats

Every job type has a specific method for converting its own node types

Parameters

name (str) – The node name
oldtype (str) – The original node type

Returns

job name or error and if it was successful: (str, bool)

Return type

tuple

pipeliner.jobstar_reader.convert_proctype(jobname, jobops=None)

Convert a process from the RELION to pipeliner style

The pipeliner has much more specific job types than RELION 3.x/4.0 so conversion is necessary

Parameters

jobname (str) – The name of the job
jobops (dict) – Dict of joboptions with {dict_name or label: value}

Returns

job name or error and if it was successful: (str, bool)

Return type

tuple

pipeliner.jobstar_reader.convert_relion20_datafile(starfile, dtype, outname=None, boxsize=None, og_name='convertedOpticsGroup1', og_num=1)

Convert a relion 2.x format starfile to relion 3.x/4.x format

Adds on an optics groups block and renames the main data block with the relion 3/x/4.x format. Fixes te descrepency between how Relion 2.x and Relion 3.x/4.x define pixel size. Adds missing fields to the data block.

Parameters

starfile (str) – The name of the file to process
outname (str) – The name of the output file, ‘converted_<starfile>.star’ by default.
dtype (str) – The type of file, must be in (movies, micrographs, ctf_micrographs, particles)
boxsize (int) – The box size of particles in pix. This is required for particles
og_name (str) – The name for the optics group that will be created for the data in the file
og_num (int) – Optics group number for the optics group that will be created for the data in the file

pipeliner.jobstar_reader.count_movs_mics_parts(node)

Count the number of objects in a movies, mics or particles starfile

Parameters: node (Node) – The Node object for the star file
Returns: Count of object if the file contained movies, micrographs, or particles returns None if the file is not found or it does not contain the correct data type
Return type: int

pipeliner.jobstar_reader.get_job_type(job_options_dict: dict) → Tuple[str, str]

Get the job type from a job.star file

Includes back compatibility with the old naming convention in RELION 3.1 files (_rlnJobType vs _rlnJobTypeLabel)

Parameters

job_options_dict (dict) – A dict of the parameters from a job.star file in the format {param_name: param_value}

Returns

(job_type (str), job_type_label (str))

The job type is the pipeliner format string job type. The job_type_label is the label used in the star file to designate the job type, which is different between RELION 3.x and RELION 4.0/Pipeliner

Return type

tuple

pipeliner.jobstar_reader.get_joboption(dict_name, label, jobops)

Get a job option value from muliple dictionary types

Return a job option where the jobops dict might contain the dictionary entries (from jobstar file) or the labels (from runjob file)

Parameters

dict_name (str) – The joboption key IE: from a job.star file
label (str) – The joboption label IE: from a run.job file
jobops (dict) – The dict of joboptions with {dict_name or label: value}

Returns

The job option value

Return type

str

pipeliner.jobstar_reader.modify_jobstar(fn_template: str, params_to_change: dict, out_fn: str) → str

Edits the a template job.star file and saves a copy

Parameters

fn_template (str) – Path to the template star file to be modified
params_to_change (dict) – The Parameters to change in the template in the format {param_name: new_value}
out_fn (str) – Name for the output file. If the same as the template files it will be overwritten

Returns

The name of the output file

Return type

str

pipeliner.jobstar_reader.star_loop_as_list(starfile, block, columns=None)

Returns a set of columns from a starfile loop as a list

Parameters

starfile (str) – The file to read from
block (str) – The name of the block to get the data from
columns (list) – Names of the columns to get, if None then all columns are returned

Returns

(list, list) [0] The names of the columns in order, [c1, c2, c3]. [1] The rows of the column(s) as a list of lists [[r1c1, r1c2, r1c3],[r1c1, r1c2, r1c3],[r1c1, r1c2, r1c3]]

Return type

tuple

Raises

ValueError – If the specified block is not found
ValueError – If the specified block does not contain a loop
ValueError – If any of the specified columns are not found

pipeliner.jobstar_reader.star_pairs_as_dict(starfile, block)

Returns paired values from a starfile as a dict

Parameters

starfile (str) – The file to read from
block (str) – The name of the block to get the data from

Returns

{parameter: value}

Return type

dict

Raises

ValueError – If the specified block is not found
ValueError – If the specified block is a loop and not a pair-value

Starfile writing utilities

Writes star files in the same style as RELION

pipeliner.star_writer.write(doc, filename: str, commentline: str = 'Relion version 4.0 / CCP-EM_pipeliner vers 0.0.1')

Write a Gemmi CIF document to a RELION-style STAR file.

Parameters

doc (gemmi.cif.Cif) – The data to write out
filename (str) – The name of the file to write the data to
commentline (str) – The comment line to put in the written star file

pipeliner.star_writer.write_jobstar(in_dict: dict, out_fn: str)

Write a job.star file from a dictionary of options

Parameters

in_dict (dict) – Dict of job option keys and values
out_fn (str) – Name of the file to write to

pipeliner.star_writer.write_to_stream(doc, out_stream: IO[str], commentline: str = 'Relion version 4.0 / CCP-EM_pipeliner vers 0.0.1')

Write a Gemmi CIF document to an output stream using RELION’s output style.

Parameters

doc (gemmi.cif.Cif) – The data to write out
out_stream (str) – The name of the file to write the data to
commentline (str) – The comment line to put in the written star file