Reports of the demise of this Web site are greatly exaggerated! We at sheldonbrown.com thank Harris Cyclery for its support over the years. Harris Cyclery has closed, but we keep going. Keep visiting the site for new and updated articles, and news about possible new affilations.


Using Deshaker Image Stabilization for Bicycle Video
find us on FB
John Allen photo
by John "Shake, Rattle and Roll" Allen
Spoke Divider

Deshaker, a plug-in for VirtualDub

You really only want to perform some video processing functions once -- for example, inverting upside-down video; image stabilization; resizing. The freeware Windows video processing application VirtualDub, (at http://virtualdub.com/) is lean and fast. By preprocessing in VirtualDub and saving the resulting file, you avoid having your main video editing suite perform the same processing again and again as it renders video.

There is another article on this site about VirtualDub. Please read it in connection with this article.

VirtualDub performs most functions using filters. One VirtualDub filter, Deshaker, is the best image stabilization software I have encountered. VirtualDub is supplied with a number of filters by default, but Deshaker is an add-on. You get it from the Deshaker Web page (at http://www.guthspot.se/video/deshaker.htm), and install it in the VirtualDub folder on your computer. Like VirtualDub, Deshaker is freeware, but the Web site requests a donation. I paid.

VirtualDub and Deshaker are available in 32-bit and 64-bit versions. The 64-bit version runs significantly faster, and so I recommend it, if your computer supports it. Some filters are available only in 32-bits, so you may want to keep a 32-bit version of VirtualDub as well.

Filters are the top item in VirtualDub's Video menu. When you click on that item, the filter dialog box will open. Click on "Add" to see the list of available filters. Then click on a filter in the list to add it. Filter information will appear in the filter dialog box. You may more than one filter at a time, though certain combinations are not compatible. Think of the filter list as a sort of pipeline, working in order from top to bottom of the list. So, for example, you may invert the video image, and adjust color in one pass through VirtualDub.

Adding a filter in VirtualDub

Adding a plugin in VirtualDub

Spoke Divider

Spoke Divider

Deshaker and Bicycle Video

There is another article on this site about image stabilization in general. While excellent documentation is available on the Deshaker site, here are some specifics relating to bicycle video.

Bicycle video is going to be shaky, whether you shoot it with a helmet camera or with a camera mounted on the bicycle. There will be small, rapid shakes and also wide sweeps from a turn of the head, or of the bicycle.

Shoot at the highest resolution your camera, editing suite and tolerance for processing time can support, to avoid much loss of sharpness. I typically shoot at 1080 x 1920. There is not much noticeable loss of quality with the slight enlargement of the image which I use in Deshaker, displaying at the same resolution. Display at a lower resolution will avoid any loss of sharpness -- at that resolution. Shoot progressive video. Most action cameras do. Interlaced video doesn't process as well and should preferably be deinterlaced before running Deshaker -- see our article about deinterlacing. .

Deshaker runs two passes through the video file, analyzing it on the first pass and adjusting it on the second. In this way, Deshaker can look ahead in time, preventing it from confusing panning with shaking. Single-pass image stabilization can run to the end of its range, resulting in jerky motion during wide pans.

Deshaker lets you restrict the part of the image which it senses, and that can be useful. More about that later.

The screen shot below is of the Deshaker interface, with my preferred settings for 1920 x 1080 HD video from a forward-facing helmet camera with a fisheye lens. A few settings (pixel aspect ratio, scan type, rolling shutter setting) are specific to the camera. You may refer to the screen shot while reading the instructions below.

deshaker

Deshaker's default motion-smoothness values of 1000 produce floating-on-a cloud-smooth pans. For bicycle video, I recommend lower values for horizontal and vertical pan, around 100. These settings will eliminate the rapid shake that is usual on a bicycle, while decentering the image less and causing minimal geometric distortion in images shot with a fisheye lens. I set motion smoothness for zooming to zero, to avoid "ballooning" of the image.

On the first pass, I select "Run Video Analysis Pass" in VirtualDub's File menu, and "Uncompressed RGB/YCbCr" in the Video menu. On this pass, Deshaker will save a log file, and VirtualDub will not save a video file. Still, processing will run much faster without a compressor -- more than twice as fast as with the XVid MPEG-4 compressor. Evidently, VirtualDub sends the video to the compressor for processing even if it isn't saving a file.

For each video that you stabilize, save a log file under a different name, so you can re-use it with different stabilization setting. The video file must be the same geometrically and frame-by-frame to work with the same log file.

Processing on the second pass can be slow. In 1920 x 1080 HD resolution, on my computer with an Intel i7-3770 processor and using the XVid MPEG-4 codec, it's about 7.5 frames per second, so it takes about 4 times as long as the clip ran. Still, I prefer to have the entire stabilized video file to work with, and so I let my computer run Deshaker when I'm working on something else.

Uncompressed video files are so huge (5 GB per minute!) that you will quickly run out of disk space. These files won't play at normal speed either, even from a fast hard drive. So, on the second pass, select "Save as AVI", and a compressor in the file menu. The free XVid MPEG-4 compressor is my favorite. If the original file is in any format other than AVI, you can save the stabilized version using the same filename and the different filetype. Keeping the original file, log file and processed file in the same folder (or the processed file in a parallel folder on a different hard drive, for less drive wear) makes them easy to find.

(There is a workaround to get VirtualDub to save a file as an MP4. This results in a smaller file which also processes faster in some video editors. There is a link to an article on how to do that at the bottom of this page. Or you may reprocess the .AVI file)

At the output of Deshaker, an enlarged or rotated image will be cropped unless you select an image size in the second pass large enough to hold all of it. VirtualDub can run multiple filters, but making additional changes to the video image only after the second pass will save processing time: any geometric change before Deshaker must be done on both passes, so the input on the second pass will match the log file.

VirtualDub's rotate filter works OK to turn the image upside down or on its side, but introduces jaggies when rotating by another amount, so it's better to use another application for that. It usually makes more sense, all in all, to run other processes after Deshaker. The exception would be if you are correcting for geometric distortions such as fisheye distortion which affect the operation of Deshaker itself.

Eliminating Borders

As the image shifts position to remove shake, black border areas will appear unless you use one of the several tricks that Deshaker offers.

One is adaptive zoom, which enlarges the image to push the border areas off the screen. If adaptive zoom is turned on in Deshaker, the image will "balloon" in size whenever there is a big shake. If the camera is facing forward or rearward, this can make it look as if you are suddenly speeding up or slowing down on your bicycle -- even riding backwards.

Another option is to use earlier and later frames to fill in the border areas. This doesn't work perfectly with a moving camera. Images of objects closer to the camera change size faster, and so motion forward or backward results in poor matches where the other images fill in. Still, the fill works well enough that it's worth doing even if the results aren't perfect. The transitions between the current and other images are less distracting if you set Deshaker to create soft boarders. If the camera is looking sideways, you can get a good match if the margins of the image are all at the same distance (for example, a wall).

Filling in borders using neighboring frames will work poorly if you run the geometric correction before Deshaker. With a rotated image, for example, the fill will be only at the corners, where the image reaches the edges of the frame.

On the second pass, setting motion smoothness for zooming to zero eliminates the "ballooning" effect. Setting 100 for horizontal, vertical and rotational motion smoothness (or 200 at 50 or 60 frames per second) will eliminate annoying rapid shake and produce a result more like your perceived head motion as you are riding -- while minimizing decentering of the image.

You might also disable stabilization against rotation in the second pass if there is little rotational shake, reducing the size of border areas. But rotational shake can be annoying.

If the camera doesn't change position or zoom, you can use the stabilization options more freely. Stabilized images can then look as smooth as if the camera is on a tripod. Visible errors with fill frames are then less likely.

If you have decided from the start to use only a fixed, zoomed-in area of your image, you can have Deshaker crop it for you. An extra zoom factor of about 1.1 will keep border areas and fill out of the picture almost all the time with the motion-smoothness settings I recommend. As Deshaker itself resizes the image, there will be only one re-encoding. In case black borders remain, you may remove them later by zooming the image in your editing suite. It's time-consuming, but you can make better artistic decisions than Deshaker can, recentering the image if necessary.

I recommend that you have Deshaker look backward and forward the full 30 frames to fill in the border areas. When these areas fill in seamlessly, you have a larger image area to work with.

Tricks

It is possible to play some interesting tricks with Deshaker's Motion Smoothness setting in Pass 2. Setting Motion Smoothness for panning, zoom or rotation to -1 in pass 2 results in "infinite" smoothness: the position or rotation remains the same as in the first frame. (There can be a very slow drift, which you may have to correct in post-processing.) One use for this is to stabilize the image if the camera is on a tripod but the tripod is disturbed. I had a (non-bicycling) video where the camera was unintentionally zoomed in, then back out. All of the motion, fortunately, was in the zoomed-in area. Saving this segment of video as a file and setting Zoom to -1 kept the scale of the image constant. Fill using neighboring frames avoided black borders for the most part. Cropping the processed image and overlaying it on a still from the unzoomed section produced a result with only the slightest visible artifacting! You can extract a file, or set In and Out points in VirtualDub, to save time by stabilizing only the short segment of a video with the disturbance.

Another trick is to create 3D video from 2d video shot from a camera in motion. For this, you want zero vertical panning and very smooth horizontal panning. Video shot out the side window of an aircraft, for example, can produce hyperstereo of the landscape below, so it looks like a table-top model.

Spoke Divider

Spoke Divider

Rolling Shutter

Many cameras have a "rolling shutter", which scans the image from top to bottom, rather than capturing it all at once. The image will then appear stretched vertically if the camera is panning upwards quickly, squished if the camera is panning downward and skewed if it is panning sideways.. If Deshaker knows the amount of rolling shutter for your camera, it can compensate for this. It cannot, however, compensate for very rapid shake which occurs during the scan of each video frame. If this shake is vertical, objects at different heights in the image may become compressed and expanded -- the dreaded "Jell-o" effect. Very rapid horizontal shake can make vertical lines appear wavy.

The Deshaker Web site lists rolling-shutter values for a number of cameras. I have contributed measurements for my cameras:

The Deshaker site describes a special test procedure to measure the rolling shutter amount of a camera, but you can calculate it without having to do a special test, using a video clip with a rapid pan across a building or other object that has horizontal and vertical lines at right angles. This is common enough in helmet-camera videos, when the cyclist turns the head. If the camera has a rolling shutter, the object will appear skewed. Stop the image on the screen, so you can advance it a frame at a time, just as with the test described on the Deshaker Web site. Lay a straightedge -- the edge of a sheet of paper will do -- along a vertical edge which appears skewed in the image. Extend the line upward and downward to measure the horizontal positions at the top and bottom of the screen.

Checking for rolling shutter without having to do a special test shoot.
The camera was panning and so the buildings are skewed.
Testing for rolling shutter

The still-frame values in the rolling shutter formula serve only to make a correction if the vertical line in the test shot was not precisely vertical -- so, draw a right angle across a horizontal line in the image you are measuring from, and carry it to the top and bottom of the screen to simulate the locations of the top and bottom of a still frame. A sheet of paper conveniently has square corners and can be used to derive the vertical line from the horizontal. The horizontal and vertical lines should cross near the center of the screen, to avoid geometric errors caused by the lens itself, and especially if it is a fisheye lens.

Rolling-shutter correction is linear from top to bottom of the frame, and so a slight jitter may remain near the top and bottom if there was rapid shake. I prefer to mount my forward-facing camera on my helmet rather than on the handlebar because motions are smoother than those of the bicycle.

Lenses and Geometric Distortion

Deshaker can produce astonishingly good results with a conventional lens -- one which images straight lines as straight lines. Telephoto and "normal" lenses are of this type, as are some wide-angle lenses. These lenses project an accurate geometric representation of the subject onto the flat sensor in the camera. This kind of lens cannot look behind itself, or extremely far out to the sides.

A fisheye image is easy to recognize: straight lines curve like parentheses near edges of the image. A fisheye lens can cover a wider angle, up to 180 degrees and sometimes even more. Because a fisheye lens looks in very different directions at the same time and compresses the image at the edges, shake will be different there.

There is a lot of image area out at the edges, and Deshaker takes an average when calculating how to reposition the image to stabilize it. Including the edges in the sensed area when using a fisheye lens will leave the center of the image shaky -- though the center is usually the main area of interest. With a fisheye lens, I recommend tracking an area only about half as high and wide as the image when running Deshaker's pass 1, the analysis pass. (For example,  if you are shooting in 1080 x 1920 HD, you might exclude everything outside 270 pixels from the top and bottom, and 480 pixels from the right and left. If the horizon is high in your picture as it often is in a bicycle video, place the sensed rectangle closer to the top.) Some visible shake will still occur in your final result, but it will mostly be near the edges of the image. You may want to use a smaller sensing area if you are going to crop the image significantly. Processing in pass 1 will be faster with a smaller analyzed area.

In a fisheye image with rapid panning, image stabilization will result in visible changes in image geometry as compressed and curved parts come closer to the center. The image can become "wavy" when panning direction reverses. This issue is nearly eliminated with the low motion-smoothness settings I recommend. Another way to minimize distortion is to crop the image after stabilizing it, but the best way of all is the old-fashioned way, to avoid shake as much as possible in the first place!

You may also want to reduce the analyzed area in pass 1 if a camera has a non-fisheye wide-angle lens (that is, if straight lines remain straight at the edges of the image). With these lenses, objects appear larger near the edges of the image, and image geometry changes noticeably as the camera pans, or if it shakes wildly. With a normal or telephoto lens, though, it's better to sense the entire image, giving Deshaker more to work with.

Deshaker offers the option to have the analyzed area follow motion in the image. This can be useful if objects are moving through the image and making it hard for Deshaker to track the stationary background. However, if the camera pans, this setting can move the analyzed area completely out of the image. If you are going to use this setting, monitor the display in Deshaker's Pass 1 to make sure that you haven't defeated stabilization in this way.

Deshaker numbers the frames in the log file and so you have the option to run more than one log file on the same video, in case you need to use different settings in tricky places, or make manual adjustments. This can be time-consuming, but if you are a perfectionist, there you go.

Some action cameras have their own internal image stabilization. This can work quite well, but using it along with post-processing in Deshaker produces the best results of all. I describe how to do this, with examples, in my review of the Sony HDR-AS100V action camera.

A 360-degree camera such as the Garmin VIRB 360 needs to use its own image stabilization and/or the image needs to be cropped before running Deshaker. Deshaker cannot stabilize the entire 360-degree image, as the camera is looking in every direction at once.

Resynchronizing audio

In my experience, Deshaker running in VirtualDub sometimes inserts several blank frames at the start of the video, and shifts the audio track by a few frames. To address this issue, I import the complete, unedited file which VirtuaDub has produced into my editing suite, Pinnacle Studio Ultimate, detach the audio, placing it in a separate track, and then shift the start of that track to where it needs to be. Any good editing suite should be able to do this. Often, placing the start of the audio at the first non-blank frame of video will realign them. I use the two tracks together from that point onward. It is also possible to resynchronize the audio and video using a cue such as a hand clap, but unless you have intentionally created a cue, you may have to go hunting for one. We have an article covering that topic.

The image below shows the start of a one-minute-long clip as displayed in the timeline of Pinnacle Studio Ultimate 20. The display will be similar in any multi-track video editing suite.

end of Deshaker clip

The image above shows the end of the clips. The processed clip is missing the last several frames. The audio extends farther than shows with the original clip, but in fact, this audio was recorded!

The moral of the story is, if you are recording a clip on which you will use Deshaker, record a second or two extra at the start and end. And if you didn't do that, append extra frames before processing, so none of your original clip is lost.

The clip here was shot using a Samsung Galaxy S4 Mini Android smartphone. Oddly, VirtualDub running Deshaker identified the frame rate as 31.48 frames per second. I got jerky motion when I inserted the processed clip into Pinnacle Studio. AVS4YOU Video Converter correctly identified the clip as running at 30 frames per second (well, actually, 29.999, but close enough) and by first copying the clip using that software, I got a good result. I still had to shift the video and audio, though.

Spoke Divider

Spoke Divider

Links:

The Garmin VIRB 360 camera

The Sony AS100V helmet camera

the Mobius M800 action camera/dashcam

The Contour HD1080 helmet camera

The GoPro Helmet Hero HD helmet camera

The GoPro Session action camera

Synchronizing multi-camera shoots

Image stabilization for bicycle video

VirtualDub video processor

Image stabilization plugin for VirtualDub

Deinterlacing in VirtualDub

Saving to MP4 in VirtualDub

Using VirtualDub to improve video from VHS tape

Pinnacle and Avid editing software

Five Ways to Create a Picture in Picture in Pinnacle Studio Ultimate

Pinnacle overwrites voiceovers...

Techmoan Web site -- reviews of action cams

Articles by Sheldon Brown and Others


Reports of the demise of this Web site are greatly exaggerated! We at sheldonbrown.com thank Harris Cyclery for its support over the years. Harris Cyclery has closed, but we keep going. Keep visiting the site for new and updated articles, and news about possible new affilations.

Copyright © 2012 John Allen

Harris Cyclery Home Page

If you would like to make a link or bookmark to this page, the URL is:
https://www.sheldonbrown.com/deshaker.html
Last Updated: John Allen June 9, 1012 22:00 EDT