AR: A 40-Year-Old Technology is Driving a New Generation of Apps
初出：Web Site Expert #37（2011年7月23日発売）
Augmented Reality has a lot in common with 3D movies. Both were invented decades ago, both are enjoying a renaissance, and both, as we will see, can be used in ways that transform the experience--or simply as attention-attracting gimmicks. Invented in 1968 by computer graphics pioneer Ivan Sutherland, AR was once the sole domain of academic researchers, requiring serious programming skills and expensive equipment like head-mounted displays. Now, the learning curve and hardware costs have precipitously dropped--putting augmented reality in the mainstream of commercial application development.
Mark Billinghurst, director of the Human Interface Technology Laboratory New Zealand, has helped propel this revolution. He is best known for his collaboration with Hirokazuo Kato on ARToolKit, a library that solves one of the key problems of augmented reality by enabling an AR application to determine where users are looking in the real world.
“I always knew there would come a point where that barrier to entry would be lowered--and it has,” Billinghurst told me, speaking from Christchurch. Evolving display hardware has made a dramatic difference. The smartphone, Billinghurst noted, has more computational power than the high-end graphics computer that once sat on a few researchers’ desks. Smartphones are being joined by the camera-equipped iPad2, gesture-driven gaming consoles, and webcam-equipped PCs and laptops, both of which remain the platform of choice for AR-augmented e-commerce.
Flash makes a difference
If hardware advances have driven down AR costs, so have development tools--most notably those that take advantage of Flash. “Around ten years ago, you started seeing computer vision-based augmented reality appearing on the PC, but mostly as native, standalone applications developed with C++,” Billinghurst said. “Then around three years ago, the Flash platform developed to the point where you could start using it for computer vision--and people started porting applications over to it.” For end users, Flash meant an AR experience without having to download additional software. On the development side, Flash opened the door to the hundreds of thousands of software engineers familiar with the platform who could potentially create AR applications.
These days, Billinghurst said, you can create AR applications in Flash ActionScript code, with calls to a Flash AR tracking library, such as the GPL-licensed FLARToolKit--an ActionScript 3 version of the ARToolKit--which enables an application to overlay content atop a tracking marker. That content is produced by a Flash compatible 3D graphics library, such as Papervision3D.
“On mobile, you might download one of the free AR browsers, such as the junaio browser from metaio or the Layar Reality Browser from Layar--both cross-platform applications supporting Android, iOS, and other mobile operating systems. From there, the developer creates server-based code, typically written in PHP or another scripting language, that specifies the points of interest and the appropriate overlays.”
A second option is to build a native application for the Android or iPhone platform. “In this case you’
Billinghurst says that the next breakthrough for AR tools may put the technology in the hands of non-programmers--enabling authors, artists, and other creative people to invent their own AR experiences. “For example, we developed a piece of software called BuildAR that lets ordinary people, including children, assemble simple AR scenes using a simple drag and drop mouse interface on their PC.”
The AR landscape
Will over-exposure turn that ploy into a gimmick? Here, 3D movies are instructive. While 3D is here to stay, the best examples, like Avatar and Werner Herzog’
Those experiences fall into four different categories: e-commerce, games, information, and professional. The last of these is worth mentioning up front--because it hints at where the technology is headed. These are state-of-the art applications designed for specialists--surgeons and engineers, among them. “You are starting to see AR providing professional training to, for example, show the names of the component parts of an engine, or to help with a surgical procedure,” said Billinghurst. “These are still largely custom software running on PCs, because they must provide a higher quality AR experience than you can get on a mobile device or through a browser. But I think these sorts of applications will spread to lower end devices over the next five to 10 years.”
AR development with an SDK
To get a better idea of how AR development works, I spoke with Ben Blachnitzky, director of research and development for metaio GmbH, which, along with Layar, makes one of the most popular augmented reality browsers, as well as related SDKs and other development tools. An early step in the development process is for the software to be able to identify the object to be tracked. Objects with strongly demarcated colors and lines are good candidates, while more subtle, gradated images, are not. “One of the first things the SDK tells you, on a percentage basis, is how trackable your object is,” he said. The developer also provides the actual dimensions of the object, thereby enabling the software to accurately track it. This data, including the image and measurements, is stored in an XML file.
Once the software “knows” the object it is tracking, it generates a matrix of coordinate values that show the center of the object with respect to a virtual camera, as well as its rotation. That data can then be used to manipulate the rendered, virtual image, thereby syncing or “gluing it” to the real-time image. What you see on the display is a blending of the two.
Blachnitzky said that while you don’
One of metaio’
The next obvious step would be to use a 3D image instead, and Blachnitzky says we are not far off. “A 3D image might appear to attach to the body, turn when you turn, and might even wrinkle appropriately as you raise an arm. We’
AR Games: Ogmento’
s Paranormal Activity
Based in New York and Los Angeles, the game company Ogmento has hung its hat on AR. The company is best known for Paranormal Activity, an iPhone game some reviewers have said is better than the movie franchise it was based on.
“Unlike most games where you sit on the couch and play in front of the screen, this game takes place in the real world,” said Ogmento CEO and co-founder Ori Inbar. “The back-story is that demons and ghosts are taking over the world and your job is to create sanctuaries that will protect you favorite neighborhood spots.” Players cast spells by drawing a shape on a paper that is then tracked by the software when the phone is in camera mode. The game also uses Google Maps, with an overlay showing areas of demonic activity in red, and sanctuaries in blue. “A third part of the game are missions that are incorporated in real-life locations. The game automatically creates those for cemeteries or churches, grocery stores and hospitals around you, and that leads you to the places as far as the story line. It may say there’
Ogmento has experimented with more site-specific versions of this idea. At the Game Developers Conference in San Francisco, the company placed missions at the Moscone Center venue, Alcatraz Island, and a nearby hotel. “In New York, we put missions in some famous haunted places, like Washington Square,” Inbar said. “I learned from developing the game that 20,000 people were buried there around 200 years ago.”
Part of the inspiration for these types of games came from Inbar’
Initial development took six months, and the team was ultimately surprised how much they managed to cram in: geolocation, mapping, and of course augmented reality and 3D graphics, which were created with the Unity 3 engine. “For the AR part of the game, we came up with the idea of extracting the content from the real world. You sketch a shape on a paper and the graphics represent what you just drew, becoming part of the game’
Sidebar - Virtual tags and text transformation
While much of the attention for AR centers on 3D graphics, another, quieter form is in superimposing labels on real world items. Amateur astronomers were early beneficiaries through applications like Star Walk (Vito Technology), which provides virtual labels for stars, planets, constellations and meteor showers. Other mobile AR apps can do similar tricks here on earth by correlating location information with the image captured on a video camera.
A cousin of these apps can be glimpsed in Google Goggles, which can use a mobile device’
Sidebar - Putting AR to work on websites: Zugara’
s Matt Szymczyk
Los Angeles based website firm Zugara has taken the AR plunge. Founded in 2001 as an interactive Web developer, the firm got its start doing interactive marketing for the Sony Playstation, and continued with work for Lexus, Toyota, Casio and other American arms of Japanese companies. But after experimenting with AR in 2009, the firm repositioned itself as an AR software developer. The move put Zugara’
Here are excerpts from our conversation:
- How should website designers look at AR: as a promising technology, a gimmick, or perhaps both?
Any online technology can be a gimmick. It’
s the strategy behind it and how you are deploying it that matters. I’ ve been meeting with companies for two years, and am seeing a shift. Now they are asking smarter questions, like how AR can help with product development. People are starting to see the longer trajectory of AR, not just what’ s happening now. But it’ s definitely not just marketing. I think marketing has worn it out already. That’ s why we are focusing on e-commerce. Augmented reality can really help in areas where you want to go online and see how something works.
- What are some examples?
s a Converse mobile app to see what a shoe would look like on your feet. IKEA has an app that simulates how furniture would look in your home. Total Immersion worked with Olympus to let you interact with a 3D representation of a camera. The application serves both as an e-commerce vehicle and as a training manual. Samsung lets you see how an LCD screen would look on your wall.
The point of these AR applications is to help e-commerce retailers increase the sales conversion rate and decrease returns, which are the two biggest pain points. In a physical store, the sales conversion rate is something like 20 to 40 percent, while online, it is about 2-3 percent. We see AR as a way to help bridge that gap. When you are in a store, you can hold an item up to yourself to see how it looks. AR makes it possible to do something like that online, because you can step away and see the apparel item overlaid on you. These kinds of things represent a whole new “funnel” in the sales cycle for purchasing online.
- You are not a big proponent of mobile AR. Why not?
Nobody is going to dispute that mobile is the future of AR, but the hype around mobile AR is ten times worse than the hype around AR in general. There are too many expectations on what mobile AR is going to do. There are issues surrounding processing power, battery life, and the user interface. There are also multiple development platforms to account for ― iOS, Android, Windows and Blackberry. The iPad, technologically, still doesn’
t meet the minimum specifications needed to run the average AR application on a desktop. Besides, are people really going to walk around using their phone as a viewfinder? Probably not.
- Is the iPad2 in the same boat as mobile?
Google just conducted a study on how people actually use their tablets. The number one thing was gaming. From the gaming side, there might be a lot of practical applications with AR using the front forward facing cameras. But as you go down the list, you see shopping is less than 50 percent: people aren’
t going to shop on an iPad. With any hardware device, you need to first understand how consumers will use it.
- What should website developers know about incorporating AR?
Follow the KISS principle―keep it simple stupid. If you are doing AR for a quick engagement, you want to have the fewest barriers possible. Augmented reality represents a revolution for how humans will interact with computers, but most people aren’
t ready for it yet. This has brought the focus back to usability and interface design for AR, both of which have been overlooked at the expense of technological advances. We learned a lot of those lessons over last three years. Older people are not all familiar with their web camera. Younger people, around the median age of 28, quickly grasp gestural control, where older people tend to take longer to adapt to it.
- Does it suggest that gestures may be the future user interface for augmented reality?
s all going to be gestures. You won’ t be sitting in front of the computer rotating a piece of paper and image.