Star File Utilities

Star file reading utilities

Tools for reading and modifying star files

class pipeliner.jobstar_reader.BodyFile(fn_in)

Bases: object

A star file that lists the bodies in a multibody refinement

data

A gemmi cif object containing the data from

Type

gemmi.cif.Cif

bodies

The block that contains info about the bodies

Type

gemmi.cif.Block

count_bodies()

Count the number of bodies in the BodyFile

Returns

The count

Return type

int

class pipeliner.jobstar_reader.ExportStar(fn_in)

Bases: object

Class for a star file that lists exported jobs

This functionality is from RELION 3.1 and will probably be deprecated

data

A gemmi cif object containing the data from the star file

Type

gemmi.cif.Cif

jobs

The block that contains info about the exported jobs

Type

gemmi.cif.Block

class pipeliner.jobstar_reader.JobStar(fn_in)

Bases: object

A class for star files that define a pipeliner job parameters

data

A gemmi cif object containing the data from the job.star file

Type

gemmi.cif.Cif

pipeline_info

The cif data block for the job section of the file

Type

gemmi.cif.Block

options

The cif data block for the ‘options’ section of the file

Type

gemmi.cif.Block

count_blocks()

Count the number of blocks in the file

Returns

The number of blocks

Return type

int

count_jobopt()

Count the number of items in the job options section of the file

Returns

The number of items

Return type

int

count_pipeline()

Count the number of items in the pipeline section of the file

Returns

The number of items

Return type

int

get_all_options(against=None)

Get a dict of all the parameters in the file

Parameters

against (iterable) – Only return joboptions that are present in this list, if None returns all the job options

Returns

The parameters dict in the format {param_name: value}

Return type

dict

get_block(block_name)

Get a specific block from the file

Parameters

block_name (str) – The name of the block to retrieve

Returns

The desired block

Return type

gemmi.cif.Block

get_continue_status()
get_jobtype()
class pipeliner.jobstar_reader.OutputNodeStar(fn_in)

Bases: object

A starfile that lists output nodes for exported jobs

This functionality is from RELION 3.1 and will probably be deprecated

data

A gemmi cif object containing the data from the star file

Type

gemmi.cif.Cif

jobs

The block that contains info about the exported nodes

Type

gemmi.cif.Block

get_output_nodes()

Get a list of output node names and types from an OutputNodeStar

This functionality is from RELION 3.1 and will probably be deprecated

Returns

[[node 1 name, node 1 type], …, [node n name, node n type]]

Return type

list

class pipeliner.jobstar_reader.RelionStarFile(fn_in)

Bases: object

A general class for starfiles that contain data, such as particles files

The input file is checked for reserved word errors when initialising a RelionStarFile object

data

A gemmi cif object containing the data from the star file

Type

gemmi.cif.Cif

count_block(blockname=None)

Count the number of items in a block that only contains a single loop

This is the format in most relion data star files

Parameters

blockname (str) – The name of the block to count

Returns

The count

Return type

int

get_block(blockname=None)

Get a block from the star file

Parameters

blockname (str) – The name of the block to get. Use None if the file has a single unnamed block

Returns

The desired block

Return type

(gemmi.cif.Block)

class pipeliner.jobstar_reader.StarfileCheck(fn_in=None, cifdoc=None, is_pipeline=False)

Bases: object

A class for checking the validity of star files and converting them

Checks for reserved word errors which can be common in star files written by some versions of RELION. If the starfile was written by RELION 3.x converts it to the pipeliner format

version

Info from the star file’s #version line what program wrote it and what version number it is

Type

str

fn_in

The path to the star file.

Type

str

cifdoc

The cif doc containing the data for the file

Type

gemmi.cif.Cif

has_been_corrected

If the file has been corrected

Type

bool

has_been_converted

If the file has been converted from RELION 3.1 to RELION 4.0/Pipeliner format

Type

bool

is_pipeline

If the file is a pipeline star file

Type

bool

check_pipeline_version()

See if a pipeline is the RELION 3.x or the new RELION 4.0/pipeliner style

Returns

The pipeline version relion3, relion4, or undefined

Return type

str

Raises

ValueError – If the pipeline has features of both RELION 3.x and RELION 4.0/Pipeliner versions

check_reserved_words()

Make sure the starfile doesn’t contain any illegally used reserved words

Only operates on a file only as trying to read in a cif doc with reserved word errors raises a parse error. Overwrites the original file if it is corrected. The old file is saved as <filename>.old

convert_relion3_1_pipeline(display_warning=True)

Convert a pipeline from RELION 3.x to RELION 4.0/Pipeliner format

Backs up the original file as <filename>.old

Parameters

display_warning (bool) – Should a warning be displayed on screen?

pipeliner.jobstar_reader.bool_jobop(jobop)

Checks if a value should be interpreted as True

Returns

True if the value is true, otherwise False

Return type

bool

pipeliner.jobstar_reader.compare_starfiles(starfile1, starfile2)

See if two starfiles contain the same information

Direct comparison can be difficult because the starfile columns or blocks can be in different orders

Parameters
  • starfile1 (str) – Name of the first file

  • starfile2 (str) – Name of the second file

Returns

(bool, bool) [0] True if the starfile structures are identical (Names of blocks and columns) [1] True if the data in the two files are identical

Return type

tuple

pipeliner.jobstar_reader.convert_coordinates_job(fn_in)

Convert RELION 3.1 style coordinates files to RELION 4 style

RELION 3.x has an empty file called coords_suffix_xxx.star RELION 4.0/pipeliner has a starfile with two columns for micrograph name and coords file. If errors are encountered with the conversion it sets the node type for that file as UNSUPPORTED_deprecated_format

Parameters

fn_in (str) – Path to the RELION 3.x style file

Returns

The new file name

Return type

str

pipeliner.jobstar_reader.convert_oldstyle_names(fn_in)

Converts model and particle nodes from RELION 3.x to RELION 4 names

The model files is RELION 3.x have been merged in to the _optimiser files in RELION 4. The particle files have moved from a system where the name and location of the file define the particles to the particles being explicitly defined in the file itself.

Parameters

fn_in (str) – The path to the file to convert

Returns

The name of that file’s node in RELION 4

Return type

str

pipeliner.jobstar_reader.convert_pipeline_node(name, oldtype)

Convert node names from a RELION 3.1 to the RELION 4.0 formats

Every job type has a specific method for converting its own node types

Parameters
  • name (str) – The node name

  • oldtype (str) – The original node type

Returns

job name or error and if it was successful: (str, bool)

Return type

tuple

pipeliner.jobstar_reader.convert_proctype(jobname, jobops=None)

Convert a process from the RELION to pipeliner style

The pipeliner has much more specific job types than RELION 3.x/4.0 so conversion is necessary

Parameters
  • jobname (str) – The name of the job

  • jobops (dict) – Dict of joboptions with {dict_name or label: value}

Returns

job name or error and if it was successful: (str, bool)

Return type

tuple

pipeliner.jobstar_reader.convert_relion20_datafile(starfile, dtype, outname=None, boxsize=None, og_name='convertedOpticsGroup1', og_num=1)

Convert a relion 2.x format starfile to relion 3.x/4.x format

Adds on an optics groups block and renames the main data block with the relion 3/x/4.x format. Fixes te descrepency between how Relion 2.x and Relion 3.x/4.x define pixel size. Adds missing fields to the data block.

Parameters
  • starfile (str) – The name of the file to process

  • outname (str) – The name of the output file, ‘converted_<starfile>.star’ by default.

  • dtype (str) – The type of file, must be in (movies, micrographs, ctf_micrographs, particles)

  • boxsize (int) – The box size of particles in pix. This is required for particles

  • og_name (str) – The name for the optics group that will be created for the data in the file

  • og_num (int) – Optics group number for the optics group that will be created for the data in the file

pipeliner.jobstar_reader.count_movs_mics_parts(node)

Count the number of objects in a movies, mics or particles starfile

Parameters

node (Node) – The Node object for the star file

Returns

Count of object if the file contained movies, micrographs, or particles returns None if the file is not found or it does not contain the correct data type

Return type

int

pipeliner.jobstar_reader.get_job_type(job_options_dict: dict) Tuple[str, str]

Get the job type from a job.star file

Includes back compatibility with the old naming convention in RELION 3.1 files (_rlnJobType vs _rlnJobTypeLabel)

Parameters

job_options_dict (dict) – A dict of the parameters from a job.star file in the format {param_name: param_value}

Returns

(job_type (str), job_type_label (str))

The job type is the pipeliner format string job type. The job_type_label is the label used in the star file to designate the job type, which is different between RELION 3.x and RELION 4.0/Pipeliner

Return type

tuple

pipeliner.jobstar_reader.get_joboption(dict_name, label, jobops)

Get a job option value from muliple dictionary types

Return a job option where the jobops dict might contain the dictionary entries (from jobstar file) or the labels (from runjob file)

Parameters
  • dict_name (str) – The joboption key IE: from a job.star file

  • label (str) – The joboption label IE: from a run.job file

  • jobops (dict) – The dict of joboptions with {dict_name or label: value}

Returns

The job option value

Return type

str

pipeliner.jobstar_reader.modify_jobstar(fn_template: str, params_to_change: dict, out_fn: str) str

Edits the a template job.star file and saves a copy

Parameters
  • fn_template (str) – Path to the template star file to be modified

  • params_to_change (dict) – The Parameters to change in the template in the format {param_name: new_value}

  • out_fn (str) – Name for the output file. If the same as the template files it will be overwritten

Returns

The name of the output file

Return type

str

pipeliner.jobstar_reader.star_loop_as_list(starfile, block, columns=None)

Returns a set of columns from a starfile loop as a list

Parameters
  • starfile (str) – The file to read from

  • block (str) – The name of the block to get the data from

  • columns (list) – Names of the columns to get, if None then all columns are returned

Returns

(list, list) [0] The names of the columns in order, [c1, c2, c3]. [1] The rows of the column(s) as a list of lists [[r1c1, r1c2, r1c3],[r1c1, r1c2, r1c3],[r1c1, r1c2, r1c3]]

Return type

tuple

Raises
  • ValueError – If the specified block is not found

  • ValueError – If the specified block does not contain a loop

  • ValueError – If any of the specified columns are not found

pipeliner.jobstar_reader.star_pairs_as_dict(starfile, block)

Returns paired values from a starfile as a dict

Parameters
  • starfile (str) – The file to read from

  • block (str) – The name of the block to get the data from

Returns

{parameter: value}

Return type

dict

Raises
  • ValueError – If the specified block is not found

  • ValueError – If the specified block is a loop and not a pair-value

Starfile writing utilities

Writes star files in the same style as RELION

pipeliner.star_writer.write(doc, filename: str, commentline: str = 'Relion version 4.0 / CCP-EM_pipeliner vers 0.0.1')

Write a Gemmi CIF document to a RELION-style STAR file.

Parameters
  • doc (gemmi.cif.Cif) – The data to write out

  • filename (str) – The name of the file to write the data to

  • commentline (str) – The comment line to put in the written star file

pipeliner.star_writer.write_jobstar(in_dict: dict, out_fn: str)

Write a job.star file from a dictionary of options

Parameters
  • in_dict (dict) – Dict of job option keys and values

  • out_fn (str) – Name of the file to write to

pipeliner.star_writer.write_to_stream(doc, out_stream: IO[str], commentline: str = 'Relion version 4.0 / CCP-EM_pipeliner vers 0.0.1')

Write a Gemmi CIF document to an output stream using RELION’s output style.

Parameters
  • doc (gemmi.cif.Cif) – The data to write out

  • out_stream (str) – The name of the file to write the data to

  • commentline (str) – The comment line to put in the written star file