Tuesday, May 24, 2011

Acting Unnatural – Technological Changes to an Old Art Form

greek-tragedy-and-comedy-masksI work in theatre. Yes, that’s r-e not e-r. I often spend time inside of theaters, but when I’m talking about the craft/industry as a whole I use the r-e spelling.

I’m primarily a director, though I’ve done my share of collegiate acting. Even in my young career, I’ve worked with a lot of actors. I like them (for the most part). Whenever I’m pondering my career choice or plotting out my approach to a given play, I hold one truth as my organizing principle: theatre needs actors.

That sounds a little too intuitive to warrant mantra-like repetition, so let me explain. Theatre does not exist without the actor. A beautifully painted, cleverly designed, artfully constructed set is a failure if it does not support the actor, his character, and his conflict with the other players onstage. The relationship between the audience and the performer is integral to a play’s success.

In other media, the actor’s role is often defined by the technology used. It often feels like film brings the audience remarkably close to the actors, but there’s no denying that there’s a lens, a machine, a cinematographer, an editor, and a director between us and the performer. Then there’s animation – be it for movies, TV, or videogames – which cherry picks the tools of the actor most salient to the project, often valuing the voice above all others.

Advances in technology are changing the way storytellers use actors, as well as how audiences appreciate them. There’s a parallel to be made with the advent of gas and electric lighting, the invention of motion pictures, etc. But the changes here are so drastic they bear investigation.

In high school, I remember spending afternoons trying to stump SmarterChild, an AIM chatbot that used natural language processing to emulate human chat behavior. Annie Dorsen’s play Hello Hi There, which ran at Performance Space 122 in New York City this past winter, is an improvised conversation between two chatbots, each loaded with Shakespeare, Bible passages, and material from a 1971 debate between Noam Chomsky and Michel Foucault. Thanks to the chatbots’ algorithms, the show is different every night. The topic of their 45-minute debate: human nature.


Remember a minute ago when I said “Theatre does not exist without the actor”? Well, apparently the National Endowment for the Arts agrees with me, as they refused Hello Hi There a grant because it lacks actors. I bring this up not out of arrogance but out of curiosity. I am fascinated by Hello Hi There, and I want it to succeed. To hear that it failed to receive NEA funding confuses me.

It confuses Dorsen, too. “That’s a strange definition of theater,” she told the New York Times ArtsBeat blog. “No one knows how to define theater. To decide that it’s only theater if there are actors seems strange.”

It is strange, so maybe we should expand our definition of actor to include these chatbots. After all, Dorsen used Chomsky’s assertion that the ability to create new thoughts with language equates to humanity as a jumping off point. Her computers generated new material every evening. Does that make them human? If so, are they then not capable of being actors? Perhaps Hello Hi There has actors after all.

If a digital entity can play the role of human, can actors fully play their roles digitally? Rockstar’s new videogame L.A. Noire (a procedural cop drama set in 1947 Los Angeles) pushes the boundaries of realism in gaming with its revolutionary facial capturing technology. Developer Team Bondi used a new technology called MotionScan (created by Depth Analysis) to digitally map the actors’ faces to character models with remarkably fidelity.

The results are astonishing. Protagonist Cole Phelps is instantly recognizable as actor Aaron Staton from Mad Men. MotionScan’s 32 3D cameras recording Staton at 60 frames per second gave the animators incredibly detailed footage with which to work. Team Bondi wisely rolled this technology into L.A. Noire’s detective gameplay. Lie detection in interrogation scenarios relies on picking up certain facial cues, and MotionScan tracks them in spades.

Team Bondi and Depth Analysis explain the tech behind L.A. Noire (via IGN)

Depth Analysis hasn’t gotten us out of the Uncanny Valley just yet. The high production values aside, there are still discrepancies between the MotionScan animation and the more conventional “ping-pong ball” motion capture in L.A. Noire. As you can see in the above video, actors cannot move while in the Depth Analysis camera chamber. Their body capture is done separately. Occasionally, the seams between these two performances are visible. It could also just be that MotionScan’s level of detail puts it ancestors to shame.

L.A. Noire’s daring grasp for true photorealism steers gaming away from traditional animation and more toward filmmaking. All of the characters in L.A. Noire resemble their real-life counterparts (if you play it, keep an eye out for other Mad Men actors – they won’t be hard to find). Actors aren’t merely lending their voices and maybe their faces for textures. The player interacts with their image the way moviegoers watch them onscreen.

johnny depp on the rango set

Game and filmmakers are also getting ever more creative with the interplay between voice-over and animation (a good idea considering that MotionScan doesn’t look cheap). For Rango, director Gore Verbinski had his Johnny Depp-led cast act out scenes on a soundstage (see above). Audio from these sessions became the voice-over track, and the Industrial Light & Magic used the video as an animation reference. A similar process was used for the game Uncharted 2 (minus the Wild West sets), allowing Nolan North to believably play an action hero ten years his junior.

Programs are invading the world of actors. Actors are invading the world of programs. What about the audience?

Common sense says we benefit either way, so long as the product is good, but there are some grumblings about classification re: legacy and marketing. This applies more to movies than it does to videogames, as we’re well past the FMV horrors of Night Trap and it doesn’t look like we’re circling back anytime soon.

The official trailer for Steven Spielberg’s The Adventures of Tintin and already the Oscar debates have begun. Spielberg has viewed the project from the start as an animated feature, and he enlisted Peter Jackson and Weta Digital to provide the motion capture for the film. This, of course, set off the animation purists who believe performance capture isn’t true animation.

andy serkisFlashback to 2006 when Happy Feet won the Academy Award for Best Animated Feature. People were incensed at the mo-capped dance numbers (which remind me a bit of old rotoscoped Cab Calloway numbers), so much so that 2007’s winner, Pixar’s Ratatouille, included this disclaimer in its credits: “Our Quality Assurance Guarantee: 100% Genuine Animation! No motion capture or any other performance shortcuts were used in the production of this film.”

Just as the NEA snubbed Hello Hi There for excluding actors, so are animators snubbing motion capture for including them. It’s about control: How is the story being told and who has the final say in that decision? There’s artistic license and value to be found on both extremes. In the middle is the actor.

Technology affects the performer’s craft. No one’s putting Billy West on a stage; his best asset is his malleable voice. Some stage actors appear to devour scenery in front of the camera, while many film stars come off flat and bland in front of a live audience. Perhaps in the future, tech-savvy actors will carve out their own niche.

Or we’ll just watch the robots act.