Virtual Eye breaks new ground in the Louis Vuitton Pacific Series

by Richard Gladwell, Sail-World.com on 26 Jan 2009
One night in the winter of 1991 a group of five people gathered in a meeting room in Television New Zealand to talk about ways of improving the television coverage of the upcoming 1992 America’s Cup.

Paul France, then TVNZ’s General Manager of Production was frustrated. A keen sailor, he had been the Producer of Television NZ’s America’s Cup coverage in Fremantle, and had really been unable to tell the story other than with conventional video.

His team couldn’t show the nuances of yacht racing, the rules and tactics. More importantly France was totally reliant on a cameraman and boat to be exactly in the right position at the right time to catch that crucial incident which could determine the outcome of a race, or indeed the fate of the America’s Cup itself.

A system had been tried in Fremantle consisting of the yachts being 'shot' using accurate survey equipment located on hills around the race course, and the output used to produce a GPS track of the yachts. While it looked promising, the system didn’t deliver.

Ian Taylor, of then fledgling Animation Research Ltd (at that time a co-operative venture with Otago University), and his team had done some work in making animated yachts move across a screen, but lacked the ability to make them react to yacht movement on the water.

A week later, and visit to the Animation Research Ltd facilities in the bowels of an old brick television studio, the ideas started to come together.

Tom Schnackenberg, who headed up North Sails in Auckland, had developed a tactical matchracing game which could move two yachts in response to signals generated by pushing keys on a PC keyboard to turn the yachts left or right.

With a little adaption the engine of Schnakenberg’s game was driving the images generated by the ARL team, and the yachts could be moved at will.

Several thousand kilometres away in Silicon Valley, the computer generated graphics industry was in its infancy.

Connecting with Silicon Valley

Jim Clark's company, Silicon Graphics had developed the first computer generated images which were cut into a movie – Terminator 2. (Jurassic Park which featured substantial amounts of computer generated images was just a couple of years away.)

A few more conversations later, and Ian Taylor decided to make a bold move - putting one of his bright young team, Paul Sharp, into Silicon Graphic’s campus for three months. There Sharp would learn how to generate the graphics being used in the movie industry – and apply these to the America’s Cup. A few months later ARL had a basic working product at the 1992 Louis Vuitton Cup in San Diego – acceptable graphics, suitable for broadcast, that could be viewed from any chosen position – like being in a helicopter.

The only outstanding issue was how to generate the data from the America’s Cup yachts – to drive the live, or real-time, graphics and produce the broadcast video.

Up stepped Alan Trimble, then a Software Developers’ Advocate at Silicon Graphics Inc. Trimble was also a keen sailor, and his job was helping developers bend or break the rules at Silicon Graphics to ensure that they got the right assistance with their projects and applications.

It was Trimble who took on board the task of developing the first on board 'black box' which had a GPS unit, gyroscopic compass and transmitter – all located in the bowels of the competing yachts.

Sharp was joined by a second ARL developer, Stu Smith, and together the two worked some incredible hours over the 1992 Louis Vuitton Cup and America’s Cup to tweak and tune the breakthrough application in sports broadcasting.

Other large international networks were aware of the ARL project, but never believed it would work. But it did, and although others tried imitations they were never quite the same.

Back where it all started

Eighteen years later, the same meeting room at Television New Zealand still overlooks the Viaduct Basin.

America’s Cups have come and gone from Auckland. But ARL technology has become an integral part of any yachting broadcast, not just at America’s Cup level but at a myriad of other events and sports.

At the Louis Vuitton Pacific Series, ARL will be making another giant step in the television coverage of sport – the entire broadcast of the two and a half week regatta will be done using virtual cameras only. Almost.

There will be one land based video camera on North head at the entrance to the Waitemata Harbour, but aside from that all footage will be computer generated using data supplied from the competing boats – and screened live, in real-time.

Computers of various types, functions and locations will be interfaced to provide all aspects of the coverage.

In another irony, the Director of the original TVNZ broadcast in San Diego, Denis Harvey, who was the one who had to splice the first yacht racing graphics into the video broadcast back to New Zealand in 1992 is also involved in the Louis Vuitton Pacific Series television production, again as Director. Except this time Harvey has the opposite problem to that which faced him in 1992. In 2009, Harvey will be producing the first live outside broadcast of a sporting event, that doesn't use on course cameras, and with just one camera based on North Head, Harvey and his team will be have to decide how to fit just the single video camera into the multi camera animation feed.

The biggest breakthrough, however, is one viewers won't see.

Crucial to the generation of any animated sailing graphics is the onboard 'black box'. In 1992 this was crafted by Alan Trimble's engineering team and usually rebuilt each night because of water damage and part failure.

Until now the 'black boxes' have all been commissioned by the event organisers and the data feed supplied to broadcasters. As well as the GPS position, heading and speed of the yachts, pitch and yaw are often transmitted enabling the software developers to generate realistic onscreen graphics. However the reliability and quality of this feed has always been an issue, forcing developers to run the graphics system in a small delay of a few to over 30 seconds depending on the quality of the data feed from the competing yachts.

To resolve this issue ARL have commissioned their own 'black box' or rather a 'blue box' which will be in use for the Louis Vuitton Pacific series. This small waterproof box branded as 'µTrack'. It is installed on each yacht and tracks every maneuver they make. 'µTrack' was designed from the ground up by ex ARL Systems Manager, Brent Russell and Taylor says it is a revolution.

'In 2007 the tracker we used came in a box the size of the average yachtie's beer cooler. The tracker that Brent has designed is not only more reliable but you could fit 20 of them in the box we used to use. Testing over the past couple of days has delivered the most consistent data we have ever seen,' he adds.

The primary driver behind the broadcast system being used in the Louis Vuitton Pacific Series is one of cost. No more is there the usual outside broadcast costs of gyroscopic cameras, camera operators, camera boats, crew and all the issues associated with getting a signal ashore.

All the signals come ashore via regular mobile communication links, and are fed into the ARL graphics software to produce the product known as Virtual Eye.

'Our next step, which is quite a small one really', says Taylor 'is to turn this all into a turnkey solution for sailing worldwide, so that regatta organisers anywhere can just install the onboard box into the competing yachts, we hook up our computers and produce a broadcast quality feed.'

'We can do everything from data collection, to visualization a