The Games Industry has always had an inferiority complex about film, and no wonder;

 

Film has over a century of history, and various workflows and conventions that everyone agrees upon. It has also had a dependable technology- cogs shunting first nitrate, then celluloid, and now polymer plastic through a light source. A film maker from the early part of the last century would still find the principles and workings of today’s cameras and projectors very familiar.

 

The Games industry on the other hand, thrives on technological leaps. A game developer from the 70’s wouldn’t recognise the code, workflow or technology of today’s console game. An envious Games industry has long coveted Film’s ability to engage an audience by its illusive portrayal of reality. It seems fascinated by how photorealistic characters might add a new dimension to the playability and immersive qualities of games.

 

Indeed, one can see the development of games as a journey towards photorealism- a long historic line of consoles, not only getting faster in terms of interactivity, but also more convincing in terms of higher pixel resolutions and resultant spectacle.

 

But in order to achieve this dream of creating convincing looking characters, Games need more computational firepower. Even today’s consoles cannot calculate and shift enough pixels across the screen to describe a convincing human form in all its detail, something film cameras have effortlessly done for decades, with exceptional quality.

 

That could all be about to change, thanks to advances in the humble silicon chip.

 

If you know a little about computers you may have heard of AMD. Indeed, your computer may be driven by the AMD Athlon chip. AMD (short for the now quaint sounding Advanced Micro Devices) designs and produces microprocessor and graphics technologies for the computer.

 

If AMD are to be believed we are finally going to see the convergence of film and games, as the last barrier- that of film quality on the console or computer- finally comes tumbling down.

 

The salubrious Marquee Club is usually the place to see tipsy supermodels and the beautiful party people of New York but for one day in August the nerds took over. It was here that AMD unveiled the new world of possibilities opened up by the world’s first teraflop chip, and as we’ll see, much more besides. It might seem odd to announce the production of a new silicon chip in a nightclub. What’s a teraflop chip and why is this tiny piece of silicon such a big deal? The technical definition is that it’s capable of a trillion floating point operations per second, but AMD makes an effort to bring the importance of this dry statistic to life for the rest of us. It claims this computational power is greater than ALL the different console models ever produced in the history of gaming (including the PS2, PS3, Xbox360 and Wii together), and all condensed into an unassuming chip the size of a thumbnail.

 

Not prone to going half measures, AMD have placed TWO of these chips on a graphics card, the Radeon ATI HD 4870 X2. That’s the big deal. To provide a little visual entertainment and to illustrate the point, AMD had painstakingly assembled almost every games console from the last 30 years, creating what must be a nerd’s paradise in the club. But perhaps even more surprising are the brains behind this silicon brawn, in the form of AMD’s corporate vision of the future, which could have real repercussions for animators, gamers and film makers alike. AMD weren’t so much launching a new product as a new medium, dubbed portentously ‘Cinema 2.0’

 

“Cinema 2.0 is really the holy grail of gaming” explains Neal Robison, AMD’s Director of Games Developer Relations “being able to create games that have the visual fidelity of today’s feature films” In AMD’s future, you won’t just play movies, you'll play in them. Charlie Boswell, Director of Digital Media and Entertainment, explains that “the cinematic and the interactive are racing to get to one another. Cinema 2.0… is going to accelerate that at an amazing speed”. So much for the hyperbole, but AMD seem to be able to back this up with some pretty influential and clever friends.

 

Take the real star of the show, Jules Urbach, whose company Jules World has been developing new tools that can take advantage of the new Radeon 4870 Processor. Jules made his first game Hell Cab for Time Warner Interactive at the tender age of 18, but since then he’s gravitated towards Hollywood, working on Transformers and Spiderman 3. His mission is to bring real-time film quality to the world of CGI, animation and games, and his rapid-fire sales patter could be discounted as so much evangelising, were it not interspersed with startling visual demonstrations. As an opener he showed a 30 second film of a New York street erupting into chaos as a chase ensues with a character and her robot assailant. It looked like a CGI sequence from any blockbuster film, with the characters composited into the shot footage. The big shock was that this scene was 100% CGI, and being generated in real time by AMD Radeon chips. To emphasise this, the camera viewpoint was later swivelled round via a mouse to reveal this wasn’t a film clip, but a live 3D virtual set being rendered there and then in real time. That’s a double whammy- firstly it was CGI, and secondly a scene of Hollywood film quality being rendered in 3D space at 60 frames per second in front of everyone’s eyes. Now, compare this with the fact that it’s not uncommon for a single frame of CGI for a feature film to take 40 minutes or even hours to render. The implication dawned on the audience - film resolution in real time means that games could finally be created that were indistinguishable from reality.

 

This is the promise of Cinema 2.0. -the immediacy of games combined with the up till now unattainable quality of film. Jules Urbach is no stranger to this idea. He’s used his talent in CGI to develop “OTOY” 3D software that allows relatively weak computers to display incredibly detailed graphics.

 

High tech author and industry insider George Gilder claims “As calculated by its Hollywood users, such as James Cameron (Titanic), Jules accelerates the rendering process by an unbelievable factor… it's said to reduce the time to complete a video frame from five days at Pixar and Industrial Light & Magic to a real-time, 30 frames a second at Jules World. This technology cuts the cost of animating a film by between 40 and 70 percent”.

 

But Urbach is not satisfied with photorealistic streets and robots. The photorealistic human is in his sights. “Even when you throw a lot of money at this problem, a character that is completely photo-real is very difficult to do…. skin is a very difficult surface to model, light enters and scatters and hits blood, bone and tissue….but our work in Cinema 2.0 wouldn’t be complete if we couldn’t account for that.”

 

This is where Lightstage 6 comes in. Lightstage 6 is a domed walk-in structure that is a real-time, full body 3D scanner. This 30 feet tall structure was developed by Paul Debevec, associate director of graphics research at USC's Institute for Creative Technologies (USC ICT). It’s composed of thousands of LED lights and three high speed cameras capturing thousands of frames per second. It captures not only the picture of the subject but also 3D geometry, diffuse, specular and normal maps. These cleverly allow the lighting of the face to be changed instantly in CGI afterwards. Urbach is now talking about a new model that has a resolution of 32,000 points an inch. That’s very high resolution indeed.

 

Another technology that may help herald the birth of Cinema 2.0 comes from Manchester, now being acclaimed by Hollywood; Image Metrics’ markerless performance driven facial animation system. We’re familiar with the motion capture process, but faces remain problematic. In Beowulf, for instance, up to a hundred tiny markers were used, plotting movements in the face’s fifty or so muscles. Image Metrics takes a different route. Powerful software provides a way of tracking the important features of one person’s face from a video source, allowing it to control or ‘drive’ a realistic CGI face. It’s the real-time potential of this computer vision software, coupled with Lightstage’s body scanning that excites Urbach. It also represents a new workflow that changes the role of animation in CGI. If you scan your actor in via Lightstage, and then drive its features by Image Metrics software, where does the animator fit in? Cinema 2.0 seems intent on automating chunks of production that were once the realm of the 3D modeler, animator and technical artist. Speed and immediacy are the watchwords of Cinema 2.0, and animation certainly isn’t speedy.

 

But capturing and rendering huge amounts of data in real time is only one aspect of Cinema 2.0, and not just for gamers. The much vaunted immediacy and greater detail means film directors don’t have to wait before seeing the idea in their head translated onto the screen. Robert Rodriguez (Spy Kids, Sin City) has become a convert. “Artists just want not to be stopped….to be able to move at the speed of thought…now technology has caught up”. Rodriguez has always embraced new technology, being an early adopter of the all-digital film workflow.

 

Cinema 2.0 also means high definition assets from a film can go directly into a game. No more costly down-converting of polygons or even rebuilding of assets to fit into a game. The creation of such CGI assets has always been expensive, and to create different versions for a film and a game is time-consuming. “Cinema 2.0 helps you realise your vision for the film and the videogame at the same time, making it almost the same experience” enthuses Rodriguez.

 

It’s not hard to see why AMD are pinning their hopes on the concept of Cinema 2.0; they’ve been losing out in the chip wars to companies like Nvidia and Intel. The introduction of the Radeon 4870 X2 graphics card soon led to retaliation by Nvidia in the form of slashing prices on its competing graphics cards, a move which Wall Street didn't like very much. Nvidias’s stock dipped, showing how sensitive such developments are to the IT industry.

 

Nvidia’s own strategy seems more vague, based around visual computing in all its aspects, including engineering, design and simulation. At their NVision conference shindig at San Jose in August, NVIDIA co-founder Jen-Hsun Huang regally told those gathered "Visual computing is transforming the videogame industry; transforming the film industry…Let the era of visual computing begin." Suddenly the chip-makers are wooing the film-makers.

 

In fact, Cinema 2.0 suits AMD’s design philosophy; creating ‘scalable’ solutions where many chips can be linked together to achieve ever higher peaks in performance. This means rather than designing separate processors for different uses and budgets, you just string together more chips to suit your customers needs. By contrast, Nvidia concentrates on single very high performance chips.

 

So AMD’s Cinema 2.0 concept is a vehicle to sell the same chips to both the casual and hardcore gamers, as well as to Hollywood and the myriad of VFX houses and visual artists out there. They’re fuelling interest in a nascent media, claiming turf in a corner of the chip wars that hasn’t reached critical mass yet. By promoting Cinema 2.0, they bring that time nearer. Nigel Dessau, AMD’s Chief Marketing Officer comments “its technology available for the same price fundamentally as a games console is today….this is not a niche solution, it’s really a mass market solution”

 

But as all big technology shifts in moving image production bring changes in the roles and workflows of those involved, it will be interesting to see how animation adapts, especially if several of its unique selling points are to be automated. However, other recent time-saving developments such as marker based Motion Capture have not led to the death of animation, but rather invigorated the market, allowing animators to move into more specialist areas than before. As an example, Gollum as a character is not successful because it was motion captured, but for what sympathetic and skilled animators added to those motion capture files to make him work so well.

 

Anyway, Cinema 2.0 could be about to kill off one of the animation industry’s biggest bugbears- that of rendering time. Now, that would be a ‘sacrifice’ every CG studio would be willing to make! So Cinema 2.0, if it takes off like AMD and Jules Urbach imagine, may be just another opportunity for animation to reach even more eyeballs than ever.

 

As an afterthought, in a discussion about the Olympics on BBC2’s Newsnight (21st August), artist Grayson Perry, reacting to comments about Beijing’s spectacular opening, joked that London’s cheaper endeavour should be “created in CG by some games company in Northumberland and piped down to London”. Be careful for what you wish for, Grayson. Cinema 2.0 may make this easier than you think, and allow you to compete virtually too!

 

 

KEY MOVERS AND SHAKERS IN CINEMA 2.0

 

IMAGE METRICS AND THE EMILY PROJECT

 

Image Metrics began the Emily project in March 2008 as a way of testing how close they could get to replicating a real person in CGI. Actress Emily O’Brien sat in front of a standard video camera and answered innocuous questions on tape. Later a 3D facial scan of Emily was used with Paul Debevec’s Lightstage to develop a CG character, and a rig was created using the video as a reference. The rig provided a way to enable the software to control the CG face. Image Metrics software analyzed the video to gather all of the details that drive expression, from eye and lip movement movement, to skin deformations and wrinkles. These facial parameters were then applied to the CG model. In this case, it was a model of Emily herself, but it could just as easily been another face.

 

Once the capture and rigging processes were finalized, the 90-second animation took just one week to complete- a long way from the real-time solution envisaged in Cinema 2.0, but attractive enough to be used extensively in games such as over 80 characters for Rockstar’s Grand Theft Auto 4 and four characters for Devil May Cry 4.

 

"Our goal was to create a completely convincing, animated computer-generated face," explained David Barton, Image Metrics' producer. " it was a challenging goal, but one we were confident we could accomplish by pairing our technology with the facial scanning system developed by Paul Debevec and his team at ICT…to achieve a new level of believability in facial animation." But is it convincing? Emily O’Brien, the actress, when presented with the CGI version of herself exclaimed “It’s me…if you showed this to me and didn’t tell me, I really wouldn’t be able to tell”.

 

 

PAUL DEBEVEC AND LIGHTSTAGE 6

 

Paul Debevec is the associate director of graphics research at USC's acclaimed Institute for Creative Technologies (USC ICT)

 

Lightstage 6 was Debevecs first full body capture system, and prosaically came about when USC gave Debevec extra space to expand his cramped workshop. Debevec’s research is revolutionising CGI and Visual Effects with prolific work on 3D Scanning, High Dynamic Range images, post production relighting and 360 degree displays. The idea behind Lightstage 6 is that a human figure is rotated over 45 seconds on a turntable at the centre of a dome constructed of lights and cameras, and is bathed in a variety of lighting configurations at very high speed (the 990Hz frequency is imperceptible to the subject). This creates what is called a dataset of the subject under all lighting conditions, from a multitude of positions at the same point in time.

 

Currently that’s 33 different pictures of the subject at any one 30th of a second, all with slightly different lighting. Once recorded on disk, you can skip between these positions (and morph to viewpoints that weren’t even filmed!), giving the final impression you are orbiting around the subject, as if it was in 3D space.

 

Because you have images of the figure under any lighting condition you can later make the figure fit into any environment. The Lightstage even creates a matte, so you can ‘key’ the figure anywhere. Even more cleverly, Debevec found a way to virtually relight the subject in post-production, by collecting lighting characteristics from other locations. Extra processes can be included to create a wireframe and other 3D CGI data from the human figure.

 

JULES URBACH

 

Jules Urbach is founder and CEO of Los Angeles-based companies OTOY and Lightstage. He has been working with microprocessor manufacturer AMD since 2006 on various projects. As an obsessive gamer Jules reportedly turned down a place at Harvard to make video games. He programmed his first popular game Hell Cab (released by Time Warner Interactive), at the age of 18. In 1999 he co-founded Groove Alliance, a leading developer of online videogames. Groove Alliance was one of the first game companies to create 3-D products exclusively for online use, creating titles for Nickelodeon, Disney, Shockwave and Electronic Arts, among others. Their ‘3D Groove’ gaming platform enabled narrowband broadcasting of unrivaled 3D videogames directly to any Windows or Macintosh Web browser, no mean feat at the time.

 

This interest in distributing high 3D quality down small pipes led a decade later to the creation of OTOY which is now developing technology for delivering real-time “cinematic quality” 3D rendering through web browsers. According to Urbach, Cinema 2.0 is the next leap in the evolution of film, like sound, color, cinemascope, 70mm, THX, stereoscopic 3D, and IMAX. “This changes everything” he says.