Overview of UI¶
Dashboard and projects¶
The Dashboard is the top-level page of
nextPYP
Projects are logical units within the dashboard and are used to organize data processing runs
Blocks¶
Each Block represents a unit of processing (data import, pre-processing, refinement, masking, etc.). They provide control over the corresponding processing parameters and an interface to visualize the results. Projects consist of sequences of connected blocks.
Tip
Blocks can be moved around and organized within the canvas
Multiple blocks can be selected and moved as a set
Block connectivity¶
Each block in nextPYP has inputs and outputs of specific types. Two blocks can be connected only if their input/outputs types are compatible. The program does not allow for incompatible block connections.
Types¶
Single-particle projects and tomography projects have similar block types. A list of block types and corresponding inputs and outputs is given below:
Import
Description: Import movie frames or micrographs in MRC, DM4, TIF, or EER format.
Input: None
Output: Movies
Pre-processing
Description: Movie frame alignment, CTF estimation, and particle picking.
Input: Movies
Output: Particles
Particle refinement
Description: Particle alignment, classification and per-particle CTF refinement.
Input: Particles
Output: Particles
Particle filtering
Description: Removal of bad particles from downstream analysis.
Input: Particles
Output: Particles
Movie refinement
Description: Per-particle movie frame alignment and dose weighting.
Input: Particles or Frames
Output: Frames
Create mask
Description: Creation of shape masks.
Input: Particles or Frames
Output: None
Post-processing
Description: Masking, map sharpening, and Fourier Shell Correlation (FSC) plots.
Input: Particles or Frames
Output: None
Data import
Decription: Import raw tilt-series data (with or without frames) in MRC, DM4, TIF, or EER format.
Input: None
Output: Tilt-series
Pre-processing (legacy)
Description: Frame and tilt-series alignment, tomogram reconstruction, CTF estimation, and particle picking.
Input: Tilt-series
Output: Particles
Pre-processing
Description: Frame and tilt-series alignment, tomogram reconstruction, and CTF estimation.
Input: Tilt-series
Output: Tomograms
Denoising (train)
Description: Train a neural network for tomogram denoising.
Input: Tomograms
Output: Tomograms
Denoising (eval)
Description: Evaluate a neural network for tomogram denoising.
Input: Tomograms
Output: Tomograms
Segmentation (open surfaces)
Description: 3D segmentation using pre-trained model.
Input: Tomograms
Output: None
Particle-Picking
Description: Import, manual, size-based, virions, or template-search particle picking.
Input: Tomograms
Output: Particles
MiLoPYP (train)
Description: Train neural network for pattern mining.
Input: Tomograms
Output: MiLoPYP Model
MiLoPYP (eval)
Description: Evaluate neural network model for pattern mining.
Input: MiLoPYP Model
Output: MiLoPYP Particles
Particle-Picking (train)
Description: Train neural network for particle picking.
Input: Particles, MiLoPYP Particles
Output: Particles Model
Particle-Picking (eval)
Description: Evaluate neural network for particle picking.
Input: Tomograms
Output: Particles
Particle refinement
Description: Constrained particle alignment and classification, region-based refinement, and per-particle CTF refinement
Input: Particles
Output: Particles
Particle filtering
Description: Removal of bad particles from downstream analysis.
Input: Particles
Output: Particles
tomoDRGN (train-vae)
Description: Train tomoDRGN model to learn structural heterogeneity.
Input: Tomograms
Output: DRGN Model
tomoDRGN (analyze)
Description: Analyze structural heterogeneity from tomoDRGN model.
Input: DRGN Model
Output: DRGN Particles
tomoDRGN (analyze-volumes)
Description: Volume space heterogeneity analysis from tomoDRGN model.
Input: DRGN Model
Output: DRGN Particles
tomoDRGN (filter-star)
Description: Select particle subsets from tomoDRGN clusters.
Input: DRGN Particles
Output: Particles
Movie refinement
Description: Per-particle tilt movie refinement and reconstruction, data-driven dose-weighting.
Input: Particles or Frames
Output: Frames
Create mask
Description: Creation of shape mask.
Input: Particles or Frames
Output: None
Post-processing
Description: Masking, map sharpening and Fourier Shell Correlation (FSC) plots.
Input: Particles or Frames
Output: None
Operations¶
Users can access block-level operations using the menu icon located at the top-right corner of each block. The following operations are supported:
Rename block.
Create a new block with the same input connection and parameter settings as the current block
Open a dialog to Edit/Read block parameters. Click Save, Reset or (to discard your changes)
Reveal the location of the latest set of logs in the Jobs panel
Reveal location of files in the filesystem for the block
Reset state to allow re-running the block
Delete all files associated with the block
Delete block. This operation cannot be undone. If a block has connections downstream, all connected blocks will be deleted (user will be prompted to confirm this operation)
Status¶
Blocks can be in one of four states (indicated by icons displayed on the top bar of each block):
Block is up-to-date (no graphical indication)
Newly created (block was created and is ready to be executed)
Modified (block parameters were modified and the block needs to be updated)
Running (the block is currently running)
Parameters¶
Block parameters are specified using dialog forms. These are shown every time a new block is created or copied, or when clicking the icon and selecting the Edit option.
Jobs panel¶
The Jobs panel is used to monitor the status of all jobs launched by nextPYP
Jobs can be in one of four states:
Scheduled
Running
Completed
Canceled
Failed
Jobs are arranged hierarchically according to their dependencies, and the number of jobs in each state is continuously updated
To facilitate navigation, jobs are grouped chronologically into Today, This Week and Older
The arrows and are used to expand or collapse groups of jobs
Most jobs in nextPYP have three phases: Launch, Split and Merge. See Compute resources for more details.
Tip
A summary of currently running jobs from all projects can be found at the bottom of the Dashboard page
Running jobs can be cancelled by clicking on the icon in the Jobs panel
Job logs can be accessed by clicking the icon next to the job name in the Jobs panel
Log windows can be docked/undocked by clicking the icon
Navigation¶
Use the breadcrumb menu at the top of the page to quickly navigate to the Dashboard or the current Project page:
