Robert Downey Jr relaxes during an augmented reality segment from Esquire's December 2009 issue.

The virtual 3D technology known as augmented reality has found its way onto chip bags, Lego boxes and rapper Eminem's website. Now, it's found its way between actor Robert Downey Jr's legs. The smirking Sherlock Holmes star appears on the cover of Esquire's December issue, perched atop a black-and-white AR code next to a headline teasing the deepest integration of print media and AR to date: "WTF? A living, breathing, moving, talking magazine?"

The augmented reality issue of Esquire is the latest in a series of unconventional features the 76-year-old men's magazine has produced in an attempt to reenergize creative thinking around what is considered ‘old' and ‘new' media, and offer readers and advertisers something different in the process.

"I challenged my staff three years ago to come up with ways to expand the boundaries of what people generally think magazines are capable of," says Esquire's editor-in-chief, David Granger. "I get really sick of magazines being lumped into this category of ‘old media' and I just really want to try to prove that we can do things that are fun and exciting and novel."

Last October, Esquire animated the cover of its 75th anniversary edition and a Ford ad with blinking E Ink electronic paper technology. A square flap in the cover of the February issue revealed quotes from inside the issue alongside a Discovery Channel ad and the May issue's perforated cover allowed readers to mix and match the faces of George Clooney, Justin Timberlake and Barack Obama.

The augmented reality issue, available on newsstands Monday, is more ambitious than previous experiments. The magazine contains six black and white AR markers that launch interactive video content when readers hold them up to a webcam. The features include the cover photo of Robert Downey Jr, a celebrity fashion spread starring actor Jeremy Renner, a "Funny Joke From a Beautiful Woman" and a Lexus ad.

Created with New York-based digital shop The Barbarian Group and animation and design company Psyop, the AR features added six weeks to the issue's production schedule and a six-figure cost to the budget.

The idea for the augmented reality issue arose during discussions between Granger and The Barbarian Group co-founder and CEO Benjamin Palmer, who featured in Esquire's Best and Brightest issue a year ago.

Initially, the two pondered embedding a USB stick into the magazine to give readers bonus digital content but prohibitive costs killed the idea. "We wanted to do something that enhanced the magazine-reading experience but didn't take you away from the magazine," says Palmer. "We didn't want to do something that could essentially be done on the Internet."

Though augmented reality has appeared on magazine covers before - including Popular Science and Boards' sister publication Strategy - Palmer calls the Esquire project the deepest mash up between print and digital to date. The Barbarian Group's creative team and Psyop's directors sat in on editorial meetings to brainstorm potential AR features with the editors, writers and designers.

Unlike previous, browser-based AR magazine features, Esquire readers must download an 80MB custom app containing the video content. Though the AR marker reader software is available in the public domain, The Barbarian Group designed a real-time graphics engine based on programming code written in-house to power more robust 3D-animated experiences.

There is potential to design interactive high-definition AR content, but there's a trade off between quality and bandwidth. In other words, if readers are able to control the interactive experience by tilting the magazine, more data is required

"It's almost like you're making a video game engine," says Palmer. "We could technically do this with beautiful, luscious, high-definition video, but you'd have to get somebody to download two-to-three gigabytes of high definition."

Jeremy Renner bundles up with Burberry in an interactive AR fashion spread.

Gillian Jacobs stars in the augmented "Funny Joke From A Beautiful Woman" feature.

For the animation team in Psyop's New York office, the primary challenge was balancing the high-quality production values the company is known for in the commercial sphere with AR's programming requirements.

As a result, the animation style is very illustrative, full of broad shapes and bright colors like a newspaper comic strip. The most stripped-down segment is the Robert Downey Jr cover because the user has complete control and thus it sucked up the most data.

When a reader holds the AR marker on the front of the magazine to a webcam, the cover lines explode into a suspended cloud of type as Downey leaps toward the screen and proceeds to explain the concept. Tilt the magazine forward, for example, and he steps off the page with a megaphone in hand to announce the "home invasion" portion.

The fashion spread shows actor Jeremy Renner (The Hurt Locker) swapping sweaters, coats and winter scarves as readers turn the magazine to control the weather. The programming for this segment was less intensive and so Psyop had more leeway to animate richer visuals.

"We imagined a circular runway stage where he would showcase his fashion wares as well as interact with the environment," says Psyop director Marie Hyon. "We wanted to come up with something that felt whimsical playful but also fashion-y."

Of all the segments, the "Funny Joke From A Beautiful Woman" starring actress Gillian Jacobs has the fullest look because it is the least interactive of the animated features. The twist is the video is time-coded so Jacobs tells a different, dirtier joke if viewers activate the AR after midnight.

"We went full-on illustrative and wanted to play around with having a handheld feeling of camera move," says Hyon. "Even though there's animation happening around her, the joke visuals feel dynamic and alive in her space."

One reason most of the augmented reality segments star celebrities is because celebrities, unlike many other pretty faces to grace the pages of consumer magazines, can act. Both Hyon and Psyop director Marco Spier flew to Los Angeles to turn each of the photo shoots into visual effects productions.

For the cover shoot, the magazine had scheduled four-hours to shoot Robert Downey Jr and tacked on an extra hour for the AR segment. Psyop and Esquire's writers had developed a basic script and storyboards but hoped the actor would ad lib and rewrite once on set to better showcase his manic charisma.

"It adds a little bit of pressure and a little bit of tension to the whole shoot," says Granger. "When Robert got to the shoot, he wasn't completely clear what augmented reality was and so he had to wrap his mind around it.

"It was key we had a talented person who was willing to play along and get enthusiastically involved with it," he continues. "Otherwise it could've been a disaster."

So can Esquire's readers expect more augmented reality features in the future? Having Lexus as an exclusive augmented reality advertiser helped to amortize costs, but in order to make AR a regular staple, Granger says he must find ways to keep the budget down.

"I think if we're able to do it in future, it'll be because we'll have multiple advertisers in the issue who are using the technology," he says. "We have some advertiser interest for our March issue but we're trying to decide whether we can pull it off. We'd have to start today to do an issue that comes out in early February."

A more realistic target for the next augmented Esquire is the May 2010 issue, he says.

www.esquire.com/ar

www.barbariangroup.com

www.psyop.tv