Department of Electronics

The particular emphasis of my audio research at York is source signal separation - the idea that is it possible to 'demix' a single monophonic musical signal into multiple tracks, with each track corresponding to a different source instrument or voice in the original signal.

This work is fundamentally different to the various basic (phase, lag and reverberation) 'tricks' often employed to obtain a so-called 'pseudo stereo' output. Even Jack Somers of RCA, one of the creators of the pseudo stereo approach, felt that with such techniques

"...you can spread the sound around the room, but there is no way to get the feeling, as in true stereo, of the proper positioning of the individual instruments..." Time Magazine (1961)

But our work over the last decade has proven that it *is* possible to use to recognise key characteristics of different instruments within a mixture and to extract these instruments, creating a new multi-track "master" that can be further processed using standard studio techniques.

Applications include remastering of mono and stereo originals to full surround sound, improved studio tools in terms of processes such as remixing, audio compression/expansion, etc., as well as completely novel creative effects, extraction of sung vocals, from music, enhanced content-based classification of music for archiving and internet delivery, improved onset/event detection, restoration of damaged audio, and forensic audio.

NOTE:

Interest in signal separation techniques is growing. In recent months, two commercial firms have announced products and services intended to deliver some degree of extraction or separation for practical purposes.

Celemony's Melodyne editor "is the first product with DNA Direct Note Access, our groundbreaking technology, that allows for the first time to edit individual chord tones in audio recordings".

Audionamix (aka Mist Technologies) market their UnMixingStation which is a "fully dedicated professional audio service solution, the first and only facility able to unmix any kind of mono or stereo source".

Unfortunately, we are unable to provide advice and information about the quality or suitability of these products for particular applications.

My image processing research concentrates on the problem of assembling a massive seamless 3D 'mosaic' image from a large set of overlapping originals.

There is growing interest in such work, and some very clear examples of its potential are shown by this 13 gigapixel panoramic view of Harlem, produced from a set of 2045 original images, or this even bigger 17 gigapixel image of Yosemite, both produced by the Autopano software package.

Although producing such high-quality multi-row mosaics is a computational feat, it was rendered much easier by the fact that these images were acquired from static cameras - the Autopano site states that

"Autopano imposes only one constraint in terms of your photos – the camera must rotate around its nodal point, the basic principle of panoramic photography."

Hence, other than colour balancing, the key problem in these examples was obtaining the (already approximately known if an automatic motorised data acquisitions system is used) angles of tilt and rotation of each picture.

The work at York is intended to be much more widely applicable, handling the most challenging general case where not only the angles, but also the varying positions of the camera are completely unknown for every recorded frame. In other words, the aim is to be able to process a large set of images acquired from arbitrary positions, angles and ranges, from any camera, without requiring the photographer to measure and record any additional information about the image set and without the need of specialised or automatic collection systems.