Single-particle tutorial¶

This tutorial shows how to process single-particle raw movies from T20S proteasome (EMPIAR-10025) into a high-resolution 3D structure.

We first download and decompress a tbz file containing a subset of 20 movies, the gain reference, and an initial model:

# cd to a location in the shared file system and run:

wget https://nextpyp.app/files/data/nextpyp_spr_tutorial.tbz
tar xvfz nextpyp_spr_tutorial.tbz

Step 1: Create a new project¶

The first step is to create a new folder where all the data will be stored.

# create new project

mkdir T20S
cd T20S

Step 2: Pre-processing¶

Data pre-processing consists of doing movie frame alignment, CTF estimation and particle picking:

# launch pre-processing

pyp -data_mode spr                                  \
    -scope_voltage 300                              \
    -scope_pixel 0.66                               \
    -data_path="PATH_TO/spr_tutorial/14*.tif"       \
    -gain_reference="PATH_TO/spr_tutorial/Gain.mrc" \
    -gain_flipv                                     \
    -detect_rad 80                                  \
    -detect_method all                              \
    -detect_dist 40                                 \
    -slurm_tasks 7                                  \
    -slurm_memory 70                                \
    -slurm_merge_tasks 7                            \
    -slurm_merge_memory 70

The parameters for all pre-processing operations can be controlled using option groups of the form:

--movie_*: options for movie frame alignment
--ctf_*: options for estimation of the CTF
--detect_*: options for particle picking

Some examples of options for data pre-processing are:

--movie_iters 5: frame alignment iterations
--ctf_max_res 4: max resolution used for CTF fitting
--detect_method auto: particle picking method

The results of data pre-processing are saved in the pkl/ and webp/ folders under the project directory.

Tip

Use pyp --help to get a complete list of options. The parameter list is very long and is organized into groups to facilitate navigation. For example, all parameters to control gain correction are under -gain_*.

Step 4: Filter bad particles¶

This step removes bad particles based on assigned particle scores during refinement. We first need to create a new T20S_clean folder:

# make new project folder
cd ..
mkdir T20S_clean
cd T20S_clean

# filter bad particles

pcl -data_parent=`pwd`/../T20S"                                      \
    -clean_spr_auto                                                  \
    -clean_dist 20                                                   \
    -clean_parfile=`pwd`/../T20S/frealign/maps/T20S_r01_04.par.bz2   \
    -clean_check_reconstruction                                      \
    -no-clean_discard                                                \
    -refine_model=`pwd`/../T20S/frealign/maps/T20S_r01_04.mrc

Tip

Check the results in the frealign/maps folder to confirm that the filtering operation was successful.

Step 5: Permanently remove bad particles¶

Remove bad particles from metadata (this step cannot be undone):

pcl -clean_discard                      \
    -no-clean_check_reconstruction

Step 6: Particle refinement¶

The next step is to do local alignments using a lower level of binning (using only clean particles). We first need to rename frealign/maps to frealign/maps_clean:

# save coarse refinement results

mv frealign/maps frealign/maps_clean

# launch fine refinement

csp -extract_box 256                                                            \
    -extract_bin 2                                                              \
    -refine_mode 1                                                              \
    -reconstruct_cutoff="1"                                                     \
    -refine_iter 2                                                              \
    -refine_rhref "6:4:3"                                                       \
    -refine_maxiter 6                                                           \
    -refine_fboost                                                              \
    -no-refine_skip                                                             \
    -refine_parfile=`pwd`/frealign/maps_clean/T20S_clean_r01_02_clean.par.bz2   \
    -refine_model=`pwd`/frealign/maps_clean/T20S_clean_r01_02.mrc

Note

Every time pyp commands are executed, the parameters are saved in a .pyp_config.toml file in the project directory. This means that parameter values are “remembered” and you only need to specify the ones that change between consecutive runs. For example, if you executed the csp command above and you want to run an additional refinement iteration, you can just run: csp -refine_maxiter 7.

Step 7: Create shape mask¶

This step will create a shape mask using the most recent reconstruction:

pmk -mask_model=`pwd`/frealign/maps/T20S_clean_r01_06.mrc  \
    -mask_threshold 0.3

Step 9: Particle-based CTF refinement¶

This step refines the CTF per-particle using an 8x8 grid:

csp -refine_maxiter 9       \
    -csp_refine_ctf         \
    -csp_Grid_spr "8,8"

Step 11: Dose weighting¶

This step performs per-frame dose-weighting to increase the contribution of high-quality frames:

# launch dose-weighting reconstruction

csp -extract_fmt frealign_local     \
    -dose_weighting_enable          \
    -dose_weighting_fraction 4      \
    -dose_weighting_transition 0.75 \
    -refine_iter 4                  \
    -refine_maxiter 4               \
    -no-csp_frame_refinement

Step 12: Particle refinement after frame alignment¶

This step does additional 3D refinement using the drift-corrected particles and the dose-weighted reconstruction:

# launch frame refinement

csp -refine_iter 5                  \
    -refine_maxiter 5               \
    -no-refine_skip

Step 13: Map sharpening¶

The final step does masking, sharpening, and produces FSC resolution plots:

psp -sharpen_input_map=`pwd`/frealign/frame/*_r01_half1.mrc  \
    -sharpen_automask_threshold 0.5                          \
    -sharpen_adhoc_bfac -50

Note

Output maps and FSC plots will be saved in the frealign/maps folder.

Single-particle tutorial

Contents