Process movies

As of version 1.2, RELION incorporates functionality to process movies from fast digital cameras, such as direct-electron detection devices (DDDs). The following has been tested on several data sets from our FEI Falcon camera. Other detectors have not been tested yet. Please share your experiences with us if you have.

Get organised

Organize all your movies and the average micrographs (i.e. single-image files that are the average of all frames of your individual movies) inside one or more directories that are inside your RELION project directory (from where you would launch the GUI). We like to call these directories "Micrographs/" if all micrographs are in one directory, or "Micrographs_15jan13/" and "Micrographs_23jan13/" if they are in different directories (e.g. because they were collected on different dates. But, you can organise yourself as you like.

Name the average micrographs and movies in the following way:

001.mrc
001_movie.mrcs
24.01.13.15.04.mrc
24.01.13.15.04_movie.mrcs
myverybest.mrc
myverybest_movie.mrcs
...

Note that all average micrographs are in MRC format with an .mrc extension, and all movies are MRC stacks with a .mrcs extension. It is important that movies have the same rootname (e.g. 001, or 24.01.13.15.04, or whatever you like) as their corresponding average micrographs, plus a movie-identifier (in this case "movie") that follows the rootname after an additional "_" character.

What to do first

First, use the average micrographs to extract "normal" particle stacks, and follow the Recommended_procedures for 3D refinement.

Preprocessing movies

Then, extract (box out) the individual movie-frames of all particles from the original micrograph movies. To do this, go back to the Preprocessing run-type in the RELION GUI, and:

  * Keep everything on the I/O tab as it was when you extracted the "normal" particles   (also do NOT change the rootname)
  * Set "Run Ctffind3?" to No
  * Set "Extract particles from micrographs?" to Yes
  * Set "Use movies instead of micrographs?" to Yes
  * Set "Rootname of movie files" to the movie-identifier mentioned above (in the example above "movie")
  * Set "Number of frames to average?" to any desired number from 1 to half the number of frames in your movies
  * Keep everything on the operate tab as it was when you extracted the "normal" particles   
  * Let it run... !

(By this time you will probably already be aware of disc quota, full discs, etc....) For each movie, a separate stack with all movie frames of all particles in that micrograph will be stored along the extracted "normal" particle stacks in the Particles directory. All metadata regarding these particle-movies will be stored in a large STAR file in your project directory that is called particles_movie.star (where "particles" is the Particle rootname given on the I/O tab; and "movie" is your movie-identifier).

3D refinement with movies

Finally, go to the 3D auto-refine run-type in the GUI and load the settings of the run that you want to perform movie-processing on. Note that this previous 3D auto-refine run was done with (most likely a classified subset of) the average-micrographs particles. As long as the naming conventions outlines above are maintained, then the program will figure out itself which movie particles correspond to which average-particle in the 3D auto-refine. In the 3D auto-refine GUI:

  * Select "continue old run" from the pull-down menu on the top.
  * Possibly change the "Output rootname" (also see below)
  * Select the last optimiser.star file from the run at the "Continue from here" option
  * Under the Optimisation tab, set "Realign movie frames?" to Yes.
  * Set "Input movie frames" to the particles_movie.star file generated in the section above.
  * Select suitable parameters for "Running avg window" and the standard deviations of the priors on the rotations and translations. 
  * If you plan to use these movies for subsequent particle polishing (see below), then you can skip the rotational searches
  * Keep everything on the CTF and Sampling tabs unchanged
  * Let it run... !

Regarding the parameter choices on the Movie tab: the defaults worked well for our ribosome data, but this will depend strongly on your total dose, how much your particles move etc. One can run this procedure multiple times (if you change the "Output rootname" on the I/O tab, previous results will not be overwritten). You can then select optimal parameter settings based on the gold-standard FSC curve.

Particle polishing

For relatively small particles (e.g. sub-MegaDalton), following beam-induced movements by aligning particles in running averages of a few movie frames may become very noisy. To make this procedure more robust, a so-called particle-polishing procedure was implemented in RELION-1.3. It improves the processing in three ways:

1. Linear movement tracks are fitted through the (possible very noisy) movement tracks as determined by the original movie-refinement (as outlined above). To further increase robustness to noisy tracks, multiple neighbouring particles on the same micrograph are considered simultaneously in this fitting. This reduces noise, since neighbouring particles are often observed to move in similar directions. To still be able to model complicated movement patterns, where particles in different areas of the micrograph move in different directions, the user defines a standard deviation of the particle-distance, which is used to calculate a Gaussian weight on the least-squares fit of these multiple particles. Thereby, particles far away from each other contribute little to each others fitted tracks, whereas particles within a 1-2 standard deviations still contribute significantly to each others fitted tracks.
2. A resolution and dose-dependent weighting scheme is devised to model unresolved beam-induced movements and radiation damage. Radiation damage affects the high-resolution components at much lower electron dose than the low-resolution components. Consequently, whereas the high-resolution signal in the later frames may be completely gone, these frames may still contribute positively to the low-resolution signal (and thereby the orientability of the particles). The resolution-dependent decay of each movie frame is modelled by a B-factor, which is estimated from the gold-standard FSC between individual movie-frame reconstructions. These FSCs are converted into relative Guinier plots, which are the logarithm of the amplitudes of the individual-frame reconstructions divided by the amplitudes of the average reconstruction from all frames versus the square of the resolution. Linear fits through these Guinier plots (which may often be performed for resolution higher than 20 Angstroms) then yield a slop (the B-factor) and an intercept (a resolution-independent scale-factor) which are used to device the resolution-dependent weghting scheme. Often, the B-factors are relatively large during the first few electrons/squared Angstroms dose due to unresolved, large initial beam-induced movements, and then also become relatively large again in the later frames when radiation damage sets in. A similar behaviour is also often seen for the intercepts, although sometimes for those the initial frames are as good as the middle ones.
3. All movie-frames for each particle are then summed together, taking the fitted movement tracks (of beam-induced translations) as well as the resolution-dependent weights into account. The result is a set of shiny or polished particles, which have increased signal-to-noise ratios compared to the particles that were extracted from the average micrographs of the original movies. Re-classification of these polished particles (in 2D or in 3D) may often work better than with the original, averaged particles, such that the final classified data set may be further improved.

In practice, the procedure is performed on the output _data.star file from the 3D-movie refinement above (e.g. Refine3D/run1_ct22_data.star). Note that when performing the original 3D movie-refinement, the rotational searches may be skipped, since in the polishing procedure beam-induced rotations will be ignored in the summation of the aligned movie frames anyway. If one skips the rotational searches, the 3D movie-refinement will be much quicker, and by default also the maximization (i.e. reconstruction) step is skipped, which further accelerates the procedure. In that case, only the _data.star file is output at the end of the 3D movie-refinement; and it is this file thats to be input in the particle polishing job-type window.

The following parameters are filled in on the GUI:

• input STAR file: the output _data.star from the original 3D movie-refinement run. (PS: the one WITHOUT _it??? in its name!)
• output rootname: Note that for each different output rootname, a copy of the particles stacks with this name will be created in the Particles/ directory. To save disc space, be careful not to copy your data too many times.
• running average window: just re-provide the same value you gave for the 3D movie-processing refinement
• stddev on particle distance: small values (e.g. 100 pixels) allow for complicated beam-induced movement patterns to be modelled accurately, although averaging over more particles with larger values (e.g. 300 pixels) may increase robustness to noise in the movement tracks from the original 3D movie-refinement.
• High-res limit per-frame maps: by default the individual-frame reconstructions will be calculated until Nyquist. This make considerable amounts of CPU time and RAM, whereas in many cases single-frame reconstructions do not reach Nyquist anyway. To decrease computational costs, one may provide a high-resolution limit (e.g. 6 Angstroms) to which the single-frame reconstructions are calculated.
• Low-res limit B-factor estimation: The per-frame B-factors are calculated by fitting straight lines through so-called relative Guinier plots (the logarithm of the amplitudes of the individual-frame reconstruction divided by the amplitudes of the average reconstruction from all movie frames). The way these relative Guinier plots are calculated often results in useful linear regions to resolutions as low as 20 Angstroms. Manual inspection of these curves (which are stored in frame0??_guinier.star) is often useful to confirm that fitting a straight line through the specified range in the relative Guinier plot is indeed useful.

Describe All parameters Mention continuation of old runs, possibility to edit the bfactors.star file etc

Post-processing

Whether you only perform the original movie-processing refinement, or you also perform particle polishing, remember to always get the most out of your final reconstruction by calculating mask-corrected FSCs and sharpening your map, as implemented in the postprocessing job-type, and as explained on the Analyse results page.