Dr John Szymanski
Senior Lecturer
Media Technology Research Group
Department of Electronics, University of York
Heslington,
York, YO10 5DD, UK

Room: P/B007
Tel: +1904 432354
Fax: +1904 432335
Email: jes1@ohm.york.ac.uk

 

Multi-point imaging of architectural and heritage structures

        In association with Skycell Ltd. (www.skycell.ltd.uk) work is under way to develop airborne imaging systems suitable for the controlled and repeatable imaging of architectural and heritage structures at close range. Using a digital camera carried by a remotely controlled airship, it is possible to acquire large quantities of digital imagery from an assortment of views and ranges. But there are a range of major research issues involved in the consolidation of the massive amounts of data involved (10Gb per day of survey work) into a single, unified high-resolution 'seamless' archive.
        Even though the images may at first seem to be 'direct' rather than 'indirect', the data recorded on disk in fact represents a geometrical projection of the region of interest (dependent on the camera position and orientation) and distorted by lens and camera-specific factors.

The basic concept of being able to construct and interact with a massive composite high-resolution seamless image of a heritage structure.
The composite here of the South Transept of York Minster is about 8,500x6,200 pixels in size.
(Click image for zoomed version)

Background:

        Photogrammetry is an established technique for the accurate recording of buildings and structures, but has limitations. In general, a photograph offers an excellent way of recording variations in contrast and colour, but is rather worse at providing 3D and textural information – problems with parallax, perspective and lighting effects can be introduced depending on the position of the camera and the shape/structure of the region of interest.
        Laser ranging systems are a newer development which are still relatively expensive and harder to use, but provide a superior way of obtaining quantitative range and texture information.
        Geomatics involves the use of these methods in combination – the aim being to combine the two different data sets to construct an interpretation which is true to the original, in the sense that it not only contains high-resolution textural and colour information, but also places this information in 3D space.
        The natural next step is to use multiple sources of information (such as many different images and ranges taken from different angles) to construct a single master archive.
        Most previous work in this area has concentrated on fairly low resolution VR applications where the camera and ranging system may be some distance from the surfaces of interest. The aim of the York work is more ambitious - to move towards the creation of extremely high resolution digital archives from close-range images.

Approach:

The York work concentrates on four related aspects:
1) Data acquisition – to obtain high-resolution data, the imaging system is flown on a remotely controlled airship, allowing extremely high quality and close-range imaging.
2) Laser scanning – laser scanning equipment can be used to project grid lines onto a region. These assist in step 3 below.
3) Data processing and data fusion – the images first need to be corrected for their range, angle of view, lens distortions and exposure effects. They can then be used to construct a new massive integrated data set.
4) Content engineering – the ability to explore (change view, pan, scan, zoom, and measure) the resultant massive (Gb scale) data set with ease.

The Skycell aerial imaging vehicle carrying a digital camera in front of the Jesse Window in Wells Cathedral
(Click image for larger version)

Data Acquisition:

        The vehicle, developed by Skycell Ltd. at York, is soft and moves slowly - it cannot cause damage to surfaces or structures. The equipment cradle, camera, control unit, video downlink and power systems are hung below the main body of the balloon. The imaging platform is stable, controllable and is entirely suitable for flight indoors or outdoors in low winds. Larger systems are required for worse conditions, or a series of tethers can be used to provide stability.
        In short,

  • The vehicle can be flown and controlled by radio to desired imaging points.
  • A real-time video downlink allows instant analysis and a camera preview.
  • The camera can be controlled by a radio uplink.
  • Individual images or groups of images can be obtained from any point
  • High-resolution close-range views can be taken that would otherwise require expensive scaffolding or pneumatic lifts.

A composite (scaled down) of part of the Jesse window in Wells Cathedral. The full size version is 2,600x10,000 pixels in size - seventy-five times larger.

Data Fusion:

        If multiple images have been obtained, representing overlapping views of the same area, then it is possible to construct a vast high-quality composite data set from these separate images. Construction of such high-quality seamless images is difficult and processor intensive – matching points in overlapping images can be used to determine unknown parameters such as the camera angle and position, but additional information from ranging or scanning systems would ease the process greatly.
        The research at York is aimed at attacking two key issues associated with this process:

  • The massive nonlinear optimization problem associated with estimating the unknown camera positions and orientations (the EOP - Exterior Orientation Parameters) associated with each image, and the unknown internal distortions due to the camera optics and configuration (the IOP - Interior Orientation Parameters) for each image. The York work is investigating the introduction a priori information and the use of both hard and soft constraints to increase the effectiveness of the nonlinear parameter estimation algorithm.
  • Automation of the registration process between overlapping pairs or sets of images - this is most often carried out by using pairs of user-specified control points chosen from regions of overlap between images. This process is not only extremely slow and labour-intensive, but is also sub-optimal in the sense that only very limited amounts of the image data are ever actually exploited. Instead a multi-resolution wavelet based approach is being investigated which will allow automatic alignment of image sections with residual mismatches being used to provide information about appropriate geometrical corrections.

Delivery:

        Delivery of the archive to a wide (non-expert) user base is also a crucial part of the York work. The aim is to exploit the use of ECW (Enhanced Compressed Wavelet) techniques to allow both the exploration and navigation (panning, zooming, etc.) of multi-gigabyte archives and the straightforward analysis of the image content (overlays, comparisons, links to metadata and other relevant information, etc.).
        Click here to see a video (5Mb - Windows Media Video) illustrating the extent to which it is possible to zoom in and navigate the composite image (video copyright the University of York and Skycell Ltd).
        The ECW format means that access to a large composite archive is quite feasible even for non-technical users using a standard web browser over a normal internet link. Click here for an example of a 52 megapixel composite image of the South transept of York Minster in compressed ECW format (1.3Mb - you will need the free "ER Viewer" software, available from the ER Mapper web site at www.ermapper.com, where free ECW plugins for Photoshop and PaintShopPro are also available).

Using the ECW image format, the user can navigate the image at varying levels of zoom in real time.



Department of Electronics Home Page

University of York Home Page

John Szymanski's Research Home Page