VideoScript Professional Application Note:
Convolution & VideoScript

 

Version 1.1 4 April 2000.

Copyright © Tim Molteno 1999-2003. All rights reserved.

VideoScript versions 1.8.3 and later include convolution operations. VideoScript Professional also features a set of plug-in functions that facilitate advanced image processing.

Note: Currently convolution operates on images only. Version 2.0 will include convolution of frames.

Table of Contents

  1. Filter.Convolute
  2. Filter.GaussianBlur
  3. Filter.MedianFilter
  4. Filter.Sharpen
  5. Edge Detection (Soebel)
  6. Putting it all together - Optimizing movie compression using VideoScript.

Filter.Convolute

Perform a convolution of the source parameter with the kernel, this convolution uses cyclic co-ordinates. A convolution kernel is just an image, for historical reasons, convolutios kernels are usually specified using integer elements. This is not necessary.

Filter.Convolute(
    image source,
    image kernel)
returns image

If the kernel is large, this operation can be very slow.

Example: We will use the Filter.KernelLoG() function to generate a Laplacian of Gaussian Filter.Kernel This kernel is commonly used to detect edges in images. Typically edge detection increases the noise level in images, therefore combining a Laplacian edge detector with a Gaussian noise reduction (smoothing) can yield optimal results.

Here is the source code showing how the above image was generated.

//
// Convolutions in VideoScript
//
// In this example, we generate a Laplacian of Gaussian (LoG)
// convolution kernel which has a width of 13 pixels and a
// standard deviation of 2.6 pixels.
//
// We then normalize the kernel so that its sum is 1. This stops
// the convolution from scaling the overall brightness of the Filter.
//
set i to (Fractal.Mandelbrot() as image);
//
// Get ourselves a normalized Filter.Kernel..
//
set k to Filter.KernelLoG(13, gray -> true, sigma -> 2.6);
set k to k / Filter.KernelSum(k);
//
// Have a look at the Filter.Kernel
//
set window "kernel" to ((k - min(k)) / max(k)) with size to {200,200};
set c to Filter.Convolute(i,k);
//
// View the result.
//
set window "convolution" to c;

Filter.GaussianBlur

A Gaussian blur, is a convolution of an image with a Gaussian Filter.Kernel The radius parameter specifies the standard deviation of the Gaussian. The larger the radius, the more blurred the Filter.

Filter.GaussianBlur(
    image source,
    number radius)
returns image

A gaussian blur is an effective method for reducing some types of noise in images. If you are compressing a movie which has frames with smoothly varying colors, then using this function on each frame can improve compression significantly.

 

Filter.Median

A median filter is a nonlinear filtering technique, useful for removing shot noise from an Filter. Shot noise is a sprinkling of random pixels - often more dramatic in images taken at low light.

Filter.Median(
    image source,
    image kernel)
returns image

Example:

We create a test image, and add five thousand random colored pixels to it. This 'noisy' image is then median-filtered using Median Cross filters of 3, 5 and 7 pixels in diameter. The results show how effective median filters are at noise removal.

set i to (Fractal.Mandelbrot() as image);
set window "original" to i;

//
// Add some noise.
//
repeat 5000 times
    set i[Math.Random() * 319, Math.Random() * 239]
        to {Math.Random(), Math.Random(), Math.Random()} as color;

set window "noisy" to i;
set window "MedianCross(3)" to Filter.Median(i,Filter.KernelMedianCross(3));
set window "MedianCross(5)" to Filter.Median(i,Filter.KernelMedianCross(5));

 
Original Image (with noise)

 
MedianCross(3)

 
MedianCross(5)

 
MedianCross(7)

 

Filter.Sharpen

If we choose our kernel carefully, a convolution can accentuate the edges in an Filter.

Filter.Sharpen(
    image source,
    number level -> 10)
returns image

The 'sharpen' function accepts two parameters, the first is the image to sharpen, the second is a level parameter which should be larger than 4.0. This parameter determines how much to accentuate the edges. The following figure shows the effect of Filter.Sharpen() on our test Filter.

Example: Sharpening a movie.

Here is an example showing how to use VideoScript to sharpen an entire movie. Perhaps the movie was slightly blurred when recorded. Well the following VideoScript shows how to help this situation.

set in to file "input.mov";
set out to movie;

repeat length of in times increment i
begin
    set i to in[i] as image;
    set i to Filter.Sharpen(i,5);
    append (i as frame) to out;
end

set file "sharp.mov" to out;

 

Edge Detection (Soebel)

Soebel edge detection involves two convolutions, one with a kernel which detects edges in the horizontal direction and another which detects edges in the vertical direction. Each convolution produces and Filter. If we add the squares of these images and take the square-root, then we end up with an image consisting only of the edges.

In color images, there are several difficulties, since there may be an edge in the red channel, but not in the blue channel. In the example below one can see this effect.

Edge detection is often combined with thresholding - this lets one make yes/no decisions on the existence of edges in an Filter.

Source Code:

//
//	Soebel Edge detection using VideoScript.
//
//	We use the Filter.Convolute function.
//
//	We are using the optimized Soebel kernels [Jahne pp232].
//	These minimize the error in the size and direction of the
//	edges.
//

set i to (Fractal.Mandelbrot() as image);

//
//	This shows how to specify convolution kernels
//
set soebelY to {{3,	10,	3},
                {0,	0,	0},
                {-3,	-10,	-3}} / 32;
set soebelX to {{3,	0,	-3},
                {10,	0,	-10},
                {3,	0,	-3}} / 32;

set x to Filter.Convolute(i,soebelX);
set y to Filter.Convolute(i,soebelY);
set window "edges" to sqrt(pow(y,2) + pow(x,2));

The results of Optimized Soebel edge detection on our sample Filter.

 

Example: Optimizing Movie Compression

In this example, we show how to take a movie which is recorded in very poor quality, low contrast - slightly out of focus as well. We will perform three stages of image filtering on the original movie, and measure the improvements in compression which they allow.

  1. The first filtering stage will be to sharpen the image (reduce the effect of the lack of focus)
  2. The second filtering stage will be to enhance the contrast using Filter.Sharpen
  3. The third stage will be to do a gaussian blur to remove some of the extra noise (improving the compression again)

 

Other Ideas for movie processing

Cross-linked with InterXlink (PERL) 0.8.1 @ 2005-07-18 13:48:56
glaucoma implant