Deshaker (Latest version: 2.5)

Parameter

Description

Source pixel aspect

Tells the filter what pixel aspect ratio the source clip has. It's very important that you set this correctly, or the rotation (for example) won't be made correctly. You may enter any value if you can't find the preset you're looking for.

Video type

Selects between progressive video and interlaced video (for input and output). If the video is interlaced you must also determine if the video has the first or last field on the first line. In general, DV uses lower field first and all other formats (including HDV) use upper field first. You can easily check that your setting is correct by looking at the motion vectors for the two fields when there's motion in the clip. If the vectors point in opposite directions most of the time the setting is incorrect.

For Deshaker, a rolling shutter means that the camcorder shake parameters (panning, rotation and zoom) aren't necessarily constant over the entire frame, but can change slightly from one line to the next. If you enable this setting, Deshaker will therefore allow these parameters to change linearly in the vertical direction during pass 1 (and you will see a range of values under the output video). This makes extracting these parameters more reliable and it also makes it possible for pass 2 to remove the rolling shutter distortions if you keep the setting enabled during that pass too. Although the actual camcorder shake isn't linear during the capture of a frame, it's usually a good enough approximation to make the distortions disappear almost completely. But I still strongly recommend using the internal camcorder stabilizer too.

If you have one of those compact CMOS cameras with bad (or no) internal stabilization, and the clip has very fast shakes (for example if the camera is mounted on a bike or similar), you probably won't be able to get a very good result from Deshaker, since the distortions will be far from linear in this case. Get a CCD camera capable of high shutter speeds for these kind of videos.

Different camcorder models may have different rolling shutter speeds (how fast the shutter rolls downwards). And a certain model could possibly even have different speeds under different conditions. To be able to handle all speeds properly, you can enter an amount from 0 to 100%. You can try which amount setting looks best, or you can measure it more precisely yourself using the following procedure:

Put the camcorder on a tripod (or something flat) and point it towards some vertical line (or bar) that covers the whole height of the frame. Now, using a fast shutter speed (to make the line sharp), record some video where you pan left and right (direction doesn't matter) at a pretty fast constant speed, as well as when it's completely still. Then, pick a still frame and a panning frame (where the vertical line is slanted). If you recorded interlaced video, the panning frame will contain two fields (and the line will appear twice), which is what you need. If you recorded progressive video, you'll need two consecutive frames instead. Then, find the horizontal position (x-pos) of the vertical line at 5 places, namely at St = top of still frame, Sb = bottom of still frame, Ft = top of first panning field/frame, Fb = bottom of first panning field/frame, Nt = top of next panning field/frame. Now, the rolling shutter amount should be calculated as: 100 * (Fb - Ft + St - Sb) / (Nt - Ft).

The default rolling shutter amount is 88%, which was measured from the Sony HDR-HC1(E) camcorder. (Deshaker versions before 2.4 had 88% fixed.)
If you measure the amount for another camcorder, please let me know so I can publish the values for other users. Here are the ones I've got so far:

• Canon EOS 500D: 71%
• Canon EOS 5D Mark II: Probably somewhere between 66% - 72%
• Canon HV20, at 1440x1080 25p: 38%
• Casio Exilim EX-F1 Pro, at 1280x720 30fps: 22%
• Casio Exilim EX-F1 Pro, at 1920x1080 60fps: 72%
• HD Hero, at 1280x720 60fps: 82%
• Kodak Zi8: Somewhere between 80% - 100%
• Oregon Scientific ATC5K: 100%
• Panasonic HDC-SD300 EG-K: 73%
• Pentax K7, at 1536x1024 30fps: 75%
• Sanyo Xacti HD1010, at 1280x720 60fps: 48%
• Sanyo Xacti HD1010, at 1920x1080 30fps: 28%
• Sony HDR-FX7: 86.4% (+/- 2.0)
• Sony HDR-HC1: 88%
• Sony HDR-HC1E: 88%
• Sony HDR-V1U: 86.4% (+/- 2.0)
• Sony HVR-A1E: 82%

Note that only rolling shutter distortions that are caused by moving/shaking the camcorder can be removed. Fast moving objects within the frame will sadly still look as strange as before.

You cannot use the deep analysis algorithm with rolling shutter video.

Log file

The name of the log file to write in pass 1 and read in pass 2.

Append to file

If checked, it appends to the log file instead of overwriting it. This is useful if you want to process certain frames again.

Video output

What kind of output to generate during pass 1. If you select Motion vectors you will see the previous image with arrows showing how to move the blocks to match the current source image. Only white arrows are used to calculate panning, rotation and zoom factor. For interlaced video both of the fields are shown, the first field above the last.

Block size

The width and height of the blocks used during the image matching.

Scale

The maximum scale to do the image matching on. Full is best but very slow.

Use pixels

How many pixels in the blocks to actually use when matching. All is best but very slow.

Color mode

The image matching can be made in color or grayscale. Grayscale is a little faster but the matching is worse. This setting has nothing to do with the color of the final output of pass 2.

Initial search range

In the most reduced scale this percentage gives the maximum shift while matching. Lower = slightly faster.

Differential search range

When going from a reduced scale to twice the size, the filter already has approximate shifts. The differential search range is how many pixels up, down, left and right from these approximate shifts to look when matching. Lower values make processing a lot faster but when objects move quickly over the frames, or if you have very fast/big rotations and/or zooms, matching can fail for some blocks.

Detect rotation/zoom

If Deshaker has problems finding a stable background to match on, and you know there is no notable rotation/zoom in the problematic frames, you can turn off detection of rotation and/or zoom (individually) for these frames (or the entire clip). This will make Deshaker assume there is no rotation/zoom and therefore not accept groups of motion vectors that indicate rotation/zoom. And this should make it easier for Deshaker to find the background. But as I said, there mustn't be any rotation/zoom in the analyzed frames for this to work. In fact, there might be problems even when the camera is moving forward quickly, because it looks rather similar to zooming.

Discard motion of blocks that have match value less than X

If the best match value found while matching a certain block is below X this block is discarded. The blocks in higher scales that depend on this block are also discarded. These discarded blocks show no motion vector in the output video of pass 1.

Discard motion of blocks that have 2nd best match larger than best minus X

If the second best match value found while matching a certain block is close to the best match value, this means that the best shift found isn't very reliable. It could just as well be this second shift that's the correct one. For example a blue sky has good match values for all shifts. Setting this parameter correctly discards those unreliable blocks. The blocks in higher scales that depend on these blocks are also discarded. These discarded blocks show no motion vector in the output video of pass 1.

Discard motion of blocks that move more than X pixels in wrong direction.

When calculating the best values for panning, rotation and zoom based on all motion vectors, the filter soon discovers that some blocks have motion that don't quite fit in with the "main motion". If a block has a motion vector that is more than X pixels off this main motion it is discarded in order to get more precise values. The motion vectors for these blocks are drawn in red in the output video of pass 1 and are not used in any way. Since the panning, rotation and zoom are all done in only 2 dimensions, it's possible to get a lot of these blocks that don't seem to fit in with the main motion when the effect of perspective gets big. This happens when the camera is set to wideangle and not just rotating (along any axis), but actually moving. Here you should use high X values (or maybe limit the matching area to the most distant parts). When zoomed in a lot or when the camera is stationary, the perspective effect is low and you should use low X values (even 1) to discard motion of objects moving even very slightly.

Discard motion of blocks that move more than X pixels (absolute motion)

Simply discards all blocks that have a motion vector that is longer than X pixels. Might be useful in some cases.

Remember discarded areas to next frame

When enabled, this feature makes Deshaker try to ignore approximately the same areas from one frame to the next. Deshaker will then become a lot more successful in ignoring moving objects. As long as they enter the scene rather slowly (by not covering too much of the background), Deshaker will usually be able to ignore those objects even if they eventually grow to cover most of the frame.

You cannot use the deep analysis algorithm with rolling shutter video.

Skip frame if less than X percent of all blocks are ok.

If less than X percent of all blocks are ok (i.e. the rest of the blocks either have no acceptable match or have been discarded), skip this frame (set panning and rotation to 0 and zoom scale to 1)

Ignore pixels outside area

Only area is used during matching. If you check Let area follow motion, the area will move along with the motion vectors between different frames. You can use it if you want to stabilize on a moving object instead of the background. Just don't expect this to work perfectly :)

Ignore pixels inside area

area is not used during matching.

Same destination properties as source

If checked, the destination video will get the same size and pixel aspect ratio as the source and the two settings below will be ignored.

Destination pixel aspect

Tells the filter to generate video with a certain pixel aspect ratio. If you plan to watch the clip on a computer monitor you'll probably want to use square pixels. You may enter any value if you can't find the preset you're looking for.

Destination video size

The size of the output video you want from pass 2.

The difference between true interlaced output and this interlaced progressive output is that the video becomes twice as high (in order to fit two frames into one) and more importantly that the lower field (or rather "frame" in this case) isn't calculated with a one line offset as in interlaced video.

Resampling

The algorithm to use when calculating pixel colors from the source image. Note that none of these are very good when the destination size is a lot lower than the source. Instead, add a separate 2:1 reduction filter (or something) after the Deshaker to reduce scale a lot.

Edge compensation

A side-effect from stabilizing video is black or strange looking borders caused by moving the image around to compensate the shaking. To reduce or eliminate this effect you can zoom into the image. The following options do this in different ways:

• None: No zoom is added.

• Adaptive zoom: Determines, for each frame, how much zoom is needed to avoid borders. This additional zoom is then smoothed together with the stabilizing zoom changes using the zoom smoothness parameter, so the borders still become visible sometimes.

• Adaptive zoom only: Same as adaptive zoom, but the zoom smoothness parameter is only applied to this adaptive zoom and not for stabilization. So, with this setting no smoothing will be made to the zooming made in the clip.

• Fixed zoom: Deshaker will add a constant zoom to completely eliminate any borders. This zoom may be different for each scene of the clip.

• Adaptive+fixed: First, adaptive zoom will be used. Then, a smaller fixed zoom will be added on top of this to completely eliminate any borders. This will usually result in less zoom than fixed zoom.

• Adaptive+fixed only: First, adaptive zoom only will be used. Then, a smaller fixed zoom will be added on top of this to completely eliminate any borders.

If you're using any edge compensation (i.e. anything but None), your can limit the amount of zoom needed to avoid the borders by using lower values for Motion smoothness and especially Max. correction limits.

Previous and future frames to fill in borders

This is a kind of additional edge compensation. When the current frame doesn't contain any image data in the "real world area" we want in our output, it can search in past and future frames and use that image data instead. As long as that area is part of the object (or background) that was stabilized on in pass 1, this usually works perfectly. But when other objects move in these areas, it can look a bit strange. It's usually better than black anyway, so I strongly recommend using this feature. Just make sure you crop any borders from the source clip before Deshaker. Otherwise this won't work very well at all.

If the current frame is number 50, it searches frames in the following order to find the image data it needs: 50, 49, 51, 48, 52, 47, 53, 46, 54 etc. So for each pixel the closest frame (chronologically) is always used. You can set how many frames to store internally for this feature. More is always better but keep in mind that each frame needs up to 8 MB of memory (for full HD) and it can take a long time to search through all frames for image data.

This option will also allow you to make the transitions between a frame and the frame(s) beneath it smoother. The transition width value you supply is how far in from the edge of a frame the transition will start. The transition will be applied also to the current frame, so you'll actually lose some information in the edges of the main frame by using this option. But I'd say it's worth it. If you don't like it, just set the transition width to 0.

Enabling 'Soft borders' improves the appearance of the border quite a lot, especially when there are objects moving there, but it is also very slow when there are lots of border pixels.

Extra zoom factor

An additional zoom factor to apply to the video. This can for example be used together with fixed zoom edge compensation to zoom out just enough for the borders to become slightly visible but still remain hidden in the overscan area of the TV. Then you will see more of the video while still not seeing any borders on the TV. Or you can use it to zoom in slightly to get less borders. This setting won't cause another resampling, so using a little extra zoom to finetune your results is perfectly safe, quality wise. (Btw, all output pixels are always resampled directly from the original source pixels exactly once no matter what settings you use. So, no double resampling ever. This includes areas taken from previous or future frames.)

Motion smoothness

These values determine how smooth the motion will be. You can set a parameter to 0 to turn off the smoothing completely. This can be useful if you don't want to stabilize zoom for example. But you must use a rather high zoom smoothness if you use adaptive zoom as edge compensation, or the video will zoom in and out very fast to avoid the borders. The motion smoothness calculation is not time-based but rather based on frame count, so you'll probably want to use larger values for video with high number of frames per second. And in this case, the fps for interlaced video will count as twice as high, since each of its frames contains two "sub frames" (fields). If you enter -1 as smoothness you will get "infinite smoothness", meaning the camera will appear to be stationary at the position of the first frame that was processed in pass 1. This feature usually doesn't work perfectly, though.

Max. correction limits

To keep the corrections (that cause the black borders) small during fast pans (for example), you can limit the maximum corrections. Especially when using fixed zoom as edge compensation these settings keep the zoom from being too large. What these settings really do is automatically lower the motion smoothness values in certain parts of the video when it becomes necessary.