vil : Core image library

A set of classes to represent and manipulate images

Introduction

Broadly there are two sorts of image one is interested in - images in memory (all parts of which can be accessed directly) and external images (eg in files) which can only be accessed indirectly. In most cases images in files are loaded into memory in one go, where they can then be manipulated. However, for some very large images this is not possible or desirable. In this case it is useful to be able to load in a sub-section of the image, manipulate it, and possible write it out again. vil supports both memory images and external images.

The core class is vil_image_view<T> which gives a view of an image in memory. The only way to get at the pixel data in an image is through a vil_image_view<T>.

Large images (in files) can be accessed through the vil_image_resource class, but one can only get at the data by asking for a view of it, which loads the requested section into memory and creates a suitable vil_image_view<T>.

Images

The vil_image_view<T> represents a view of a multi-plane image of given type. A pointer is stored to the top-left pixel in the first plane of the image (top_left_ptr()) and integers indicating how to get to neighbours in the i (istep()), j (jstep()) and plane (planestep()) directions.

The advantages of this approach are

• It is actually faster than pointer indirection on most architectures
• It allows one to access non-contiguous data as if it were a single plane
• One can use it to wrap up other image classes transparently.
• It is simple to extend to 3D images, and avoids huge arrays of pointers

Note that the same colour image can either be viewed as a one plane image with RGB pixels (using vil_image_view<vil_rgb<vxl_byte> >) or as a 3 plane byte image (using vil_image_view<vxl_byte> with nplanes() == 3).

vil_image_view<vil_rgb<vxl_byte> > rgb_image;
rgb_image = vil_load("my_image.jpg");

// Create a 3 plane view of rgb
vil_image_view<vxl_byte > three_plane_image =  rgb_image;
// or explicitly
three_plane_image =  vil_view_as_planes(rgb_image);
// or with less typing
three_plane_image = vil_load("my_image.jpg");

// Now create rgb view of 3 plane image (note: will set view to empty if not possible)
vil_image_view<vil_rgb<vxl_byte> > rgb_image2 = three_plane_image;
// or explicitly
rgb_image2 = vil_view_as_rgb(three_plane_image);

// If you want a greyscale image no matter whether the underlying image is rgb or grey
vil_image_view<vxl_byte> image2 = vil_convert_to_grey_using_average<vxl_byte>(vil_load("my_image.jpg"),vxl_byte());

There are some conversions that will try a fast view conversion only.

• vil_view_as_planes(view): Create a multi-plane view from a multi-component view.
• vil_view_as_rgb(view): Create an rgb pixel view from a multi-plane view.
• vil_view_as_complex(view): Create a complex pixel view from a multi-plane view.

There are some conversions that will process each pixel to do a conversion.

• vil_convert_to_grey_using_average(view, new_pixel_type): Create a greyscale view from any image.
• vil_convert_cast(source, dest): Convert, for example, a byte image to float image

Copying

Since the vil_image_view<T> is a ‘view’ of the actual image data, copying one only copies the ‘view’, not the image data itself - you get two views looking at the same chunk of memory. Some cunning smart pointer stuff is used to ensure that the actual data remains as long as a valid view is looking at it. (Note that this may not always be the case, since the view can be of a chunk of memory that the view does not have direct control of, such as a video buffer). This view copying will work between different types of view if it is possible to reconfigure the view very cheaply. If you wish to copy the image data itself, then use the vil_copy_deep(src_im) function. This copies the raw data into a newly created space, and returns a new view of it. Alternatively, use the method dest_im.deep_copy(src_im), or the function vil_copy_reformat(src_im, dest_im).

Example of loading, copying then processing:

vil_image_view<vxl_byte> image;
image = vil_load("test_image.jpg");
vil_image_view<vxl_byte> image2 = vil_copy_deep(image);
my_invert_image(image2);
vil_save(image2,"output_image.jpg");

Example of creating an image in memory

unsigned ni=256;
unsigned nj=256;
unsigned nplanes=3;
vil_image_view<vxl_byte> image(ni,nj,nplanes);
for (unsigned p=0;p<nplanes;++p)
  for (unsigned j=0;j<nj;++j)
    for (unsigned i=0;i<ni;++i)
      image(i,j,p) = vxl_byte(i+j+p);

Example of creating an image in memory, using pointer arithmetic

  unsigned ni=256;
  unsigned nj=256;
  unsigned nplanes=3;
  vil_image_view<vxl_byte> image(ni,nj,nplanes);
  vxl_byte* plane = image.top_left_ptr();
  for (unsigned p=0;p<nplanes;++p,plane += image.planestep())
  {
    vxl_byte* row = plane;
    for (unsigned j=0;j<nj;++j,row += image.jstep())
    {
      vxl_byte* pixel = row;
      for (unsigned i=0;i<ni;++i,pixel+=image.istep())
        *pixel = vxl_byte(i+10*j+100*p);
    }
  }

Resizing

When one resizes (using set_size) a vil_image_view<T> the view disconnects from the data (which may then be deleted if no other views are connected), allocates a new chunk of memory for the new image and sets the view to look at it.

Note that if the set_size does not change the image size, then nothing is done and the view remains unchanged.

Manipulating Views

There are a variety of ways one can view the same data, allowing one to appear to change the data simply by changing ones view of it.

For instance one can obtain a transposed view of an image simply by swapping ni-nj and istep-jstep. This is particularly useful when implementing decomposable filters, as one need only write it for one direction, then can apply it to the original image and its transpose.

vil_image_view<vxl_byte> src_image = vil_load("image_file.jpg");
vil_image_view<vxl_byte> tmp_image1, tmp_image2;
vil_exp_filter_i(src_image, tmp_image1, 2.0);  // Apply filter to rows of src_image
vil_exp_filter_i(vil_transpose(tmp_image1), tmp_image2, 2.0);  // Apply filter to cols of tmp_image1
vil_image_view<vxl_byte> filtered_image = vil_transpose(tmp_image2);
// Note that
// vil_exp_filter_i(vil_transpose(tmp_image1),vil_transpose(filtered_image),2.0)
// would only work correctly if filtered_image was already the correct size.
// If it isn't, then the view created by vil_transpose(filtered_image) would
// be resized, and disconnected from filtered_image itself, which would remain
// unchanged.

View manipulations include:

• vil_transpose : Return transposed view
• vil_flip_lr : View which reflects along i (ie i -> ni-1-i left/right)
• vil_flip_ud : View which reflects along j (ie j -> nj-1-j up/down)
• vil_view_as_planes: View rgb<T> data as planes of T
• vil_view_as_rgb : View plane data as rgb<T> pixels
• vil_crop : View a window into an image
• vil_decimate : View a trivially subsampled image
• vil_plane : View a single plane of an image

Operations on Views

Useful simple operations on views include

• vil_print_all(os,view) : Prints out view in a grid
• vil_fill(view,value) : Fills view with value
• vil_fill_row(view,j,value) : Fills row j with value
• vil_fill_col(view,i,value) : Fills col i with value
• vil_fill_mask(view,mask,value) : Fills region defined by boolean mask image
• vil_fill_disk(view,ci,cj,r,val): Fills disk in image with value
• vil_clamp : Clamp image pixels to given range
• vil_histogram_equalise(image) : Apply histogram equalisation

vil_image_view_base_sptr

vil_image_view_base_sptr is used internally by vil, and is not intended for users of vil. Anywhere you see a function returning one of these, (e.g. vil_load() ) you can assign it directly to a vil_image_view<T>. If you want a pixel-type independent image container, see vil_memory_image below.

Image resources

The vil_image_resource class is an abstract base class for image data, views of which can be obtained (or changed) with the get_view()/put_view() functions. You cannot directly construct one of these vil_image_resource classes. Instead you use a helper function, which will return a smart pointer - expect to see lots of vil_image_resource_sptr objects. The types of derived classes, and means of creating them are

• Representing an image in a file: e.g. vil_pnm_image, vil_jpeg_image. These are created using vil_load_image_resource(), and vil_new_image_resource()
• vil_memory_image: Representing an image in memory (essentially a wrapper around a view.) This is created using vil_new_image_resource() or vil_new_image_resource_of_view().
• Representing a filtered version of an image in a file (without loading in memory): e.g. vil_crop_image_resource and vil_decimate_image_resource. These are created using the equivalent functions: vil_crop(), vil_decimate(), etc.
• Representing the outcome of an image processing algorithm (see next section) e.g. vil_convolve_1d_resource. These are created using the equivalent functions vil_convolve_1d().

vil_image_resource objects in most cases only deal with scalar-type pixels. Of course once you get a view of real image data in memory, you can easily convert a multi-planar view into a multi-component view.

interfaces

There are two reasons to still use vil1 (currently core/vil1)

• vil1 has one or two image processing functions and image loaders that have yet to be ported to vil
• You may have lots of code that uses vil1, and haven't yet converted to vil.

There are several functions to convert between the various types of vil1 and vil images.

• vil_vil1_to_image_resource(vil1_image): creates a vil_image_resource.
• vil_vil1_to_image_view(vil1_memory_image_of<T>): creates a vil_image_view<T>.
• vil_vil1_from_image_view(vil_image_view<T>): creates a vil1_memory_image_of<T>.

Reporting

Currently most error reporting either happens by calling vcl_abort, or by returning a null image. The null images reporting in particular will transition to exception-based reporting. You can retain the old behaiour by setting VXL_LEGACY_ERROR_REPORTING with CMake.

Mathematics on images

Functions to evaluate image properties:

• vil_math_value_range - Get range of values in view
• vil_math_sum(sum,im,p) - Sum of elements in plane of view
• vil_math_mean - Mean of elements
• vil_math_mean_and_variance - Mean and variance over view
• vil_math_rms(image,rms_im) - Compute root mean of squares for each (multiplane) pixel
• vil_math_rss(image,rss_im) - Compute root sum of squares for each (multiplane) pixel
• vil_math_sum_sqr(image,ss_im) - Compute sum of squares for each (multiplane) pixel
• vil_math_mean_over_planes(im,mean_im)

Functions to modify images:

• vil_math_scale_values(im,s) : im(x,y,p) = s*im(x,y,p)
• vil_math_scale_and_offset_values(im,s,o) : im(x,y,p) = s*im(x,y,p)+o
• vil_math_image_sum(imA,imB,im_sum) : im_sum = imA+imB
• vil_math_image_difference(imA,imB,im_diff) : im_diff = imA-imB
• vil_math_image_abs_difference(imA,imB,im_diff) : im_diff = |imA-imB|
• vil_math_image_product(imA,imB,imP) : imP(x,y,p) = imA(x,y,p)*imB(x,y,p)
• vil_math_image_ratio(imA,imB,imP) : imP(x,y,p) = imA(x,y,p)/imB(x,y,p)
• vil_math_add_image_fraction(imA,fa,imB,fb) : imA = fa*imA + fb*imB (Useful for moving averages!)
• vil_math_sqrt(im) : im(x,y,p) = sqrt(im(x,y,p)) or zero if -ive

Other related maths functions

• vil_math_integral_image(imA,im_sum) : Images integrating values
• vil_math_integral_sqr_image(imA,im_sum,im_sum_sq) : Images integrating values and square of values

Interpolating images

Bilinear interpolation (of single planes of scalar types) can be obtained using

• vil_bilin_interp : Interpolate, using assert to check if in valid region
• vil_bilin_interp_raw : Interpolate, assuming point in valid region (seg fault otherwise)
• vil_bilin_interp_safe : Interpolate, returning zero if outside image
• vil_bilin_interp_safe_extend : Interpolate, returning nearest edge value if outside

Other methods of sampling with bilinear interpolation include:

• vil_sample_profile_bilin : Sample values along a line
• vil_sample_grid_bilin : Sample values over a grid, storing in a vector
• vil_resample_bilin : Sample values over a grid, storing in a image

Bicubic interpolation: For each of the bilinear interpolation functions there is a corresponding bicubic interpolation function. Just use "bicub" instead of "bilin" in the name.

---

vil_algo : Computational Imaging Library

A core level 2 library containing some simple algorithms for manipulating images. Note that the argument order is typically of the form vil_f(src_im,dest_im,params). Examples include

• vil_convolve_1d - Convolve with 1D filter - all manner of edge effects catered for
• vil_convolve_2d - Convolve with 2D filter (no edge effects catered for yet!)
• vil_correlate_1d - Similar to vil_convolve_1d but with reversing the kernel
• vil_correlate_2d - Similar to vil_convolve_2d but with reversing the kernel
• vil_gauss_filter_gen_ntap - Generate an n-tap FIR filter from a Gaussian function
• vil_gauss_filter_5tap - Apply 1d Gaussian filter using 5 element kernel
• vil_gauss_filter_1d - Apply 1d Gaussian filter of arbitrary SD and kernel width
• vil_gauss_filter_2d - Apply 2d Gaussian filter of arbitrary SD and kernel width
• vil_exp_filter_i - Apply exponential filter along i axis (fast recursive method)
• vil_exp_filter_j - Apply exponential filter along j axis (fast recursive method)
• vil_exp_filter_2d - Apply exponential filter along i and j (fast recursive method)
• vil_exp_grad_filter_i - Apply exponential gradient filter along i axis (fast recursive method)
• vil_exp_grad_filter_j - Apply exponential gradient filter along j axis (fast recursive method)
• vil_sobel_3x3 - to generate X/Y gradient images using 3x3 Sobel operators
• vil_gauss_reduce - smooth and sub-sample to produce half sized image
• vil_median - Compute median over area described by vil_structuring_element
• vil_fft_2d_fwd,bwd - Perform in-place 2D FFT on an image of a complex type
• vil_suppress_non_max_3x3 - Zero all pixels which are not local maxima
• vil_corners - Harris cornerness operator
• vil_find_peaks - Locate local peaks in an image
• vil_orientations - Compute angle and edge magnitude at each pixel
• vil_distance_transform - Apply distance transform to images
• vil_histogram_equalise - Replace every pixel by its intensity rank
• vil_suppress_non_max_edges - Compute gradient magnitude and zero any non-maximal values

See also the vipl library for a more generic image processing interface and more image processing algorithms.

Morphological functions (using vil_structuring_element)

• vil_binary_erode
• vil_binary_dilate
• vil_binary_opening
• vil_binary_closing
• vil_greyscale_erode
• vil_greyscale_dilate
• vil_greyscale_opening
• vil_greyscale_closing
• vil_median

Other related maths functions

• vil_histogram(image,hist,min,max,nbins) : Compute histogram of image values
• vil_histogram_byte(image,histo) : Compute histogram from byte image

Region finding

The boundaries of thresholded regions can be found using

• vil_find_4con_boundary_above_threshold
• vil_find_4con_boundary_below_threshold (Given a point inside a closed, 4-connected region defined by a threshold, these functions find the 4-connected boundary pixels of the region)
• vil_blob : Functions to find all connected regions in an image

General utilities

• vil_tile_images : Generate single image by tiling together a set of images

Parameter Conventions

For your ease of use, all vil functions and methods adhere to the following parameter conventions:

• Index types - All pixel index and image size types are unsigned.
• Specifying a pixel - func(i,j) assumes plane 0 or func(i,j,p)
• Specifying a window - func(i0,ni,j0,nj) window includes all planes.

Examples

There are plenty of working examples in the examples subdirectory - these can be treated as a mini-tutorial.

Developers

Ian Scott is responsible for co-ordinating significant changes to vil. http://sourceforge.net/sendmessage.php?touser=261110