Contents

Classification tutorial (EMPIAR-10304)

Classification tutorial (EMPIAR-10304)

This tutorial shows how to convert raw tilt-series from EMPIAR-10304 (E. coli. ribosomes) into a ~4.9A resolution structure and resolve 8 different conformations.

Total running time required to complete this tutorial: 32 hrs.

We first use the command line to download and decompress a tbz file containing: 1) a script to download the raw tilt-series from EMPIAR, 2) corresponding metadata with tilt angles and acquisition order, and 3) an initial model:

# cd to a location in the shared file system and run:

wget https://nextpyp.app/files/data/nextpyp_class_tutorial.tbz
tar xfz nextpyp_class_tutorial.tbz
source download_10304.sh

Note

Downloading the raw data from EMPIAR can take several minutes.

Open your browser and navigate to the url of your nextPYP instance (e.g., https://nextpyp.myorganization.org).

Step 1: Create a new project

Data processing runs are organized into projects. We will create a new project for this tutorial

  • The first time you login into nextPYP, you should see an empty Dashboard:

    Create new project

  • Click on Create new project, give the project a name, and select Create

  • Select the new project from the Dashboard and click Open

  • The newly created project will be empty and a Jobs panel will appear on the right

Step 2: Import raw tilt-series

Import the raw tilt-series downloaded above ( <1 min)

  • Go to Import Data and select Tomography (from Raw Data)

    Import dialog

  • A form to enter parameters will appear:

    File browser

  • Go to the Raw data tab:

    • Set the Location of the raw data by clicking on the icon and browsing to the directory where the you downloaded the raw movie frames

    • Type *.mrc in the filter box (lower right) and click on the icon to verify your selection. 12 matches should be displayed

    • Click Choose File Pattern to save your selection

    • Click on the Microscope parameters tab

    • Set Pixel size (A) to 2.1

    • Set Acceleration voltage (kV) to 300

    • Set Tilt-axis angle (degrees) to 90.0

  • Click Save and the new block will appear on the project page

  • The block is in the modified state (indicated by the sign) and is ready to be executed

  • Clicking the button Run will show another dialog where you can select which blocks to run:

  • Since there is only one block available, simply click on Start Run for 1 block. This will launch a process that reads one tilt image, applies the gain reference (if applicable) and displays the resulting image inside the block

  • Click inside the block to see a larger version of the image

Step 3: Pre-processing

Movie frame alignment, CTF estimation and particle picking ( 4 min)

  • Click on Tilt-series (output of the Tomography (from Raw Data) block) and select Pre-processing

  • Go to the Frame alignment tab:

    • Check Single-file tilt-series

    • Click on the CTF determination tab

    • Set Max resolution to 5.0

    • Click on the Tomogram reconstruction tab

    • Check Erase fiducials

    • Set Binning factor for reconstruction to 12

    • Set Thickness of reconstruction (unbinned voxels) to 3072

    • Uncheck Resize squares to closest multiple of 512

    • Click on the Resources tab

    • Set Threads per task to 42

    • Set Memory per task to 100

    • Set other runtime parameters as needed (see Computing resources)

  • Click Save, Run, and Start Run for 1 block. Follow the status of the run in the Jobs panel

Step 4: Particle detection

Particle detection from virion surfaces ( 2 min)

  • In the Pre-processing block, click on the menu icon and select the Edit option.

  • Go to the Particle detection tab:

    • Set Detection radius (A) to 80

    • Set Detection method to auto

    • Set Threshold for contamination detection to 2.0

    • Set Minimum contamination size (voxels) to 60

    • Set Minimum distance between particles to 2

    • Check Local refinement

    • Set Z-axis detection range (binned voxels) to 40

    • Set Particle detection threshold to 2

  • Click Save, Run, and Start Run for 1 block

  • Navigate to the Reconstruction group inside the Tilt-series tab to inspect the picked coordinates

Step 5: Reference-based refinement

Reference-based particle alignment ( 26 hr)

  • Click on Particles (output of the Pre-processing block) and select Particle refinement

  • Go to the Sample tab:

    • Set Molecular weight (kDa) to 2000

    • Set Particle radius (A) to 150

    • Click on the Extraction tab

    • Set Box size (pixels) to 64

    • Set Image binning to 4

    • Click on the Refinement tab

    • Specify the location of the Initial model by clicking on the icon , navigating to the folder where you downloaded the data for the tutorial, and selecting the file EMPIAR-10304_init_ref.mrc

    • Click in Alignments from sub-volume averaging to select the initial parameters text file tomo-preprocessing-*_original_volumes.txt from Pre-processing

    • Set Max resolution (A) to 22.0

    • Check Use signed correlation

    • Check Skip refinement

    • Click on the Constrained refinement tab

    • Set First exposure for refinement to 15

    • Set Last exposure for refinement to 25

    • Set Optimizer - Exhaustive search points to 5000000

    • Check Refine particle alignments

    • Set Phi range (degrees), Psi range (degrees) and Theta range (degrees) to 180

    • Set Translation range (voxels) to 50

    • Check Invert CTF handedness

    • Click on the Reconstruction tab

    • Set Max tilt-angle to 50

    • Set Min tilt-angle to -50

    • Click on the Resources tab

    • Set Threads per task to the maximum allowable by your system

    • Set Memory per task to at least 4x the number of Threads per task

    • Set Walltime per task to 72:00:00

  • Save your changes, click Run and Start Run for 1 block

  • One round of refinement and reconstruction will be executed. Click inside the block to see the results

Step 6. Filter particles

Identify duplicates and particles with low alignment scores ( 3 min)

  • Click on Particles (output of the Particle refinement block) and select Filter particles

  • Go to the Particle filtering tab:

    • Set Score threshold to 15.0

    • Set Min distance between particles (A) to 20

    • Specify the location of Input parameter file by clicking on the icon and selecting the file tomo-coarse-refinement-*_r01_02.par.bz2

    • Set Lowest tilt-angle to -7.0

    • Set Highest tilt-angle to 7.0

    • Set Min number of projections per particle to 1

    • Check Generate reconstruction after filtering

    • Click on the Refinement tab

    • Specify the location of the Initial model by clicking on the icon and selecting the file tomo-coarse-refinement-*_r01_02.mrc

  • Click Save, Run, and Start Run for 1 block. You can see how many particles were left after filtering by looking at the job logs.

Step 7 (optional): Permanently remove bad particles

Permanently remove bad particles to improve processing efficiency downstream ( 1 min)

  • Edit the settings of the existing Filter particles block

  • Go to the Particle refinement tab:

    • Check Permanently remove particles

    • Uncheck Generate reconstruction after filtering

  • Click Save, Run, and Start Run for 1 block to launch the job

Step 8. Fully constrained refinement

Tilt-geometry parameters and particle poses are refined in this step ( 10 min)

  • Click on Particles (output of the Particle filtering block) and select Particle refinement

    • Set Box size (pixels) to 256

    • Set Image binning to 1

    • Click on the Refinement tab

    • Specify the location of the Initial model by clicking on the icon and selecting the file tomo-fine-refinement-*_r01_02.mrc

    • Select the location of the Initial parameter file by clicking on the icon and selecting the file tomo-fine-refinement-*_r01_02.par.bz2 (select the file tomo-fine-refinement-*_r01_02_clean.par.bz2 if bad particles were permanently removed in the previous step)

    • Set Max resolution (A) to 18:14

    • Check Use signed correlation

    • Set Last iteration to 3

    • Check Skip refinement

    • Click on the Constrained refinement tab

    • Set First exposure for refinement to 15

    • Set Last exposure for refinement to 25

    • Set Optimizer - Exhaustive search points to 0

    • Set Optimizer - Max step length to 100

    • Check Refine tilt-geometry

    • Check Refine particle alignments

    • Set Phi range, Psi range and Theta range to 30.0

    • Set Translation range (voxels) to 30.0

    • Click on the Exposure weighting tab

    • Check Dose weighting

    • Check Global weights

    • Set Frame weight fraction to 4

  • Click Save, Run, and Start Run for 1 block to execute three rounds of refinement and reconstruction

  • Click inside the Particle refinement block to inspect the results

Step 9: Create shape mask

Use most recent reconstruction to create a shape mask ( <1 min)

  • Click on Particles (output of Particle refinement block) and select Masking

  • Go to the Masking tab:

    • Select the Input map by click on the icon and selecting the file tomo-coarse-refinement-*_r01_03.mrc

    • Set Threshold for binarization to 0.4

    • Check Use normalized threshold

    • Set Width of cosine edge (pixels) to 8

  • Click Save, Run, and Start Run for 1 block to run the job

  • Click on the menu icon of the Masking block, select the Show Filesystem Location option, and Copy the location of the block in the filesystem (we will use this in the next step))

  • Click inside the Masking block to inspect the results of masking

Step 10. Region-based local refinement

Constraints of the tilt-geometry are applied over local regions ( 25 min)

  • Edit the settings of the existing Particle refinement block and go to the Refinement tab:

  • Go to the Refinement tab:

    • Set Max resolution (A) to 18:14:12:10:8:6:5

    • Set Last iteration to 8

    • Specify the location of the Shape mask produced in Step 10 by clicking on the icon , navigating to the location of the Masking block by copying the path we saved above, and selecting the file frealign/maps/mask.mrc

    • Click on the Constrained refinement tab

    • Set First exposure for refinement to 18

    • Set Last exposure for refinement to 22

    • Set Number of regions to 8,8,2

    • Set Tilt-angle range (degrees) and Tilt-axis range (degrees) to 5.0

    • Set Phi range, Psi range and Theta range to 5.0

    • Set Translation range (voxels) to 20.0

  • Click Save, Run, and Start Run for 1 block to run the job

  • Click inside the Particle refinement block to inspect the results

Step 11: Particle-based CTF refinement

Per-particle CTF refinement using most recent reconstruction ( 2 hr)

  • Edit the settings of the existing Particle refinement block and go to the Refinement tab:

  • Go to the Refinement tab:

    • Set Max resolution (A) to 18:14:12:10:8:6:5:4.5

    • Set Last iteration to 9

    • Click on the Constrained refinement tab

    • Set First exposure for refinement to 15

    • Set Last exposure for refinement to 25

    • Uncheck Refine tilt-geometry

    • Uncheck Refine particle alignments

    • Check Refine CTF per-particle

    • Set Defocus 1 range (A) and Defocus 2 range (A) to 2000.0

  • Click Save, Run, and Start Run for 1 block

  • Click inside the Particle refinement block to inspect the results

Step 12: Region-based refinement after CTF refinement

Constraints of the tilt-geometry are applied over local regions ( 20 min)

  • Edit the settings of the existing Particle refinement block and go to the Refinement tab:

    • Set Max resolution (A) to 18:14:12:10:8:6:5:4.5:6:5:4.5

    • Set Last iteration to 12

    • Click on the Constrained refinement tab

    • Set First exposure for refinement to 18

    • Set Last exposure for refinement to 22

    • Set Number of regions to 16,16,4

    • Set Optimizer - Max step length to 20.0

    • Check Refine tilt-geometry

    • Set Translation range (pixels) to 20.0

    • Check Refine particle alignments

    • Uncheck Refine CTF per-particle

    • Click on the Exposure weighting tab

    • Set Frame weight fraction to 2

  • Click Save, Run, and Start Run for 1 block to run the job

  • Click inside the Particle refinement block to inspect the results

Step 13: 3D classification

Constrained classification ( 3 hr)

  • Click on Particles (output of the Particle refinement block) and select Particle refinement to create a new block

  • Go to the Refinement tab:

    • Specify the location of the Initial model by clicking on the icon and selecting the file tomo-coarse-refinement-*_r01_12.mrc

    • Select the location of the Initial parameter file by clicking on the icon and selecting the file tomo-coarse-refinement-*_r01_12.par.bz2

    • Set Max resolution (A) to 8

    • Set Last iteration to 20

    • Click on the Constrained refinement tab

    • Uncheck Refine tilt-geometry

    • Uncheck Refine particle alignments

    • Click on the Classification tab

    • Set Number of classes to 8

    • Click on the Exposure weighting tab

    • Specify the location of the External weights by clicking on the icon and selecting the file frealign/weights/global_weights.txt from the file location of the previous block

  • Click Save, Run, and Start Run for 1 block

  • Click inside the Particle refinement block to inspect the results

Tip

Click on the round blue markers (top right of the page) to inspect different classes or go to the Class view or Classes Movie tabs to show all classes simultaneously

Note

Running times were measured running all tilt-series in parallel on nodes with 124 vCPUs, 720GB RAM, and 3TB of local SSDs

See also

previous

Tomography tutorial (EMPIAR-10164)

next

Single-particle session