If you’re going to be in New York between November 19th and August 12th, download the Beyond Planet Earth Augmented Reality app developed by Patched Reality onto your iPhone, iPod Touch, or iPad 2, and stop by a certain museum that specializes in natural history (Map). The exhibit itself is stunning – Gizmodo calls it a “Can’t miss”. The curators and museum staff have really outdone themselves.

Beyond Planet Earth AR on iTunes
YouTube video preview

If you can’t make it to the museum, you can still have fun at home with the web version.

In all, there are 11 augmentations that you can collect. Taking photos is encouraged – so go ahead and take a picture of yourself with the Curiosity Mars rover or a lunar space elevator, and share it with your friends on Facebook or Twitter. If you stop by, I hope you enjoy the show and learn a lot in the process!

Chunk Generosity Diorama revealed with Moo Vision

Ben & Jerry’s “Moo Vision” has now been available for two weeks (via iTunes), and its been great to see the reaction.  Here are some quotes, movie poster style:

“a must try for ice cream fans” - Augmented Planet

“Ben & Jerry’s iPhone App Gets Even More Psychedelic with Augmented Reality” – Mashable

“Ben & Jerry’s is now the first major brand to sport natural feature tracking on the iPhone” – ReadWriteWeb

I was thrilled to be involved in developing the first 3rd-party iPhone app to use matio’s Unifeye Mobile SDK, and working with Circ.us, Edelman, and metaio was fantastic.  But even better is knowing that we’ve only scratched the surface of what’s possible, and I can’t wait to see what else comes along and to get involved in making it happen.

The project had some interesting challenges.  The Unifeye SDK performs natural feature tracking (NFT), which means that it is able to track 2D images that aren’t limited to black & white blocky markers.  In the ideal case when using NFT, you’ll have control over the target image to be recognized and tracked, so that you can optimize the image for track-ability by the vision system.  For this app, the target images were Ben & Jerry’s pint lids, so we very well couldn’t ask them to reprint them!  Also, pint lids are circular and AR libraries generally tend to deal with rectangular images. Therefore, the tracking configuration went through several iterations to get a result that tracked well under various lighting conditions and camera angles. Major kudos to Stefan Misslinger at metaio and his engineering team in Germany for their fantastic support and tweaks to the SDK.

Here’s a video of the app in action (with editing by the multi-talented Paul Jannicola):

DVD Case iPad Stand with USB cord attached.

I’ve been on the look out for an iPad stand that I can throw in my laptop bag.  I’ve come across various DIY options, but all of them seemed either way too much work or not portable.  The .69 business card holder was the frontrunner, but I don’t have an Office Depot nearby, and also didn’t like the fact that it wasn’t something that would fold flat. Inspired by this Gizmodo post, I came up with a solution that you can create in a couple minutes.  All you need is a standard single DVD case, and a couple of binder clips, and you’ve got yourself a foldable, portable, flat, lightweight, adjustable, iPad case for $0:

1. Empty the case
2. Fold it back to the angle you prefer
3. Fold the outside clear plastic around the side where the DVD should go, and use the binder clips to secure it.
4. Insert your iPad into the DVD case interior clips

It folds in half for a quick stash in your bag, but is pretty rough around the edges.  For a cleaner fold:

1. 

   DVD Case iPad Stand folded up.

   Remove the binder clips

2. Re-clip them to the side of the DVD case opposite where the DVD would normally go
3. Close the case

It works in both portrait and landscape orientations, and even accommodates the data cable if you bend the clips forward a bit to give it clearance.

DVD Case iPad Stand in landscape mode.

It might not take win any beauty contests or take the edge off the $30/month you’re shelling out for the 3G data plan, but just might give you a warm fuzzy feeling that you did the green thing by finding a use for your old stuff.

I’d love to hear if anyone tries this out.  Have any improvement suggestions?

DVD Case iPad Stand from the back.

DVD Case iPad Stand at a steep angle.

DVD Case iPad Stand at a shallow angle.

Today I’m announcing the first public availability of an application framework I’ve been developing that complements FLARToolkit and FLARManager, and makes it possible to rapidly produce Flash augmented reality applications.

This past year I spent many many hours with FLARToolkit and  Eric Socolofsky’s excellent FLARManager.  FLARManager makes the overall FLARToolkit configuration much simpler, especially if you want to mirror the camera or use multiple markers, and has some nice additional features like marker smoothing.

However, after two or three projects I found myself spending a lot of time writing very similar 3D asset loading and manipulation code, which FLARManager wasn’t really created to address.  This is where the framework I’m announcing today comes in.  A simple “show an animated 3D model on a marker” application can be written in about 5 minutes with about 10 lines of mxml code.  Of course, much more complex applications are possible, such as the first application to use it, which lets you explore a 17th century “Cabinet of Curiosities”  at The Getty Museum web site: http://bit.ly/GettyAugsburgAR.

The key features of the framework are:

• Clean separation between tracking libraries and rendering engines, allowing for easy mix and match
• Configure 3D scenes with a simple XML file and specify scale, position, and rotation
• Group 3D models and assign them to different AR markers
• Configure Collada animation clips (especially useful for Maya-exported models)
• Easily specify marker detection/removal behavior - fade out, fade in out, stay visible, or disappear immediately.
• Pluggable and extensible scene renderers – currenlty Papervision3D, Away3D, and Away3DLite.
• Integration with FLARManager and flare*NFT (commercial only)
• Sound manager for configuring and playing sounds
• Extend all XML configuration items with application-specific data

The framework is dual licensed in the same manner as FLARToolkit – it is licensed under GPL for non-commercial projects, or projects where you are willing to share your source code.  For use in projects where you do not want to share your source code, a commercial license is available. Send commercial inquiries to: .

The framework doesn’t have an official name yet, but I wanted to be able to share it with fellow ARE2010 attendees.  You can check out the framework here:

svn co http://projects.patchedreality.com/FLARFramework/releases/v0.1 PatchedRealityFLARFramework

Or browse the source at:

http://projects.patchedreality.com/FLARFramework/releases/v0.1

There is a simple example application in the “examples/Simple” directory.  More information and documentation to come.

If you’re going to be at the ARE2010conference, look me up and I’d be happy to give you a demonstration. I hope you’ll download it and give it a try, and enjoy spending more of your time on application/game logic than boilerplate code.  And if you do, please share your experiences.

Okay.  Here is something I didn’t predict would come out in 2010 – a carbon fiber quadricopter AR input device and gaming platform for the iPhone.  Why wait for Apple to open up their camera API, when you can slap one onto a self-stabilizing 4-propeller geeks’ dream toy?  The clever folks at Parrot and int13 have developed the “AR.Drone”, a remote-controlled copter with two on-board cameras (one for flight stabilization, and one for web cam broadcast and AR gaming).  The AR software is capable of detecting other drones in flight or markers on the ground, making it possible to play both solo games and multi-player games against freinds’ AR.Drones.  Parrot has also opened up their SDK to developers to create their own games for the AR.Drone.   Developer prototype kits can be had for $1200, and start shipping in February.

IMHO, this is the most exciting development in AR games to come along to-date, and I can’t wait to get my hands on one (or two).  It combines nearly every one of my childhood and adult toy loves – remote control, robotics, flying, mobile gaming, and augmented reality - into a single package.  Kudos to Parrot and int13 for a truly innovative product.

If the rest of 2010 has more surprises in store of the likes of this, it is truly shaping up to be an amazing year for augmented reality.  What do you think?  Am I overstating the awesomeness of this new development, or do you share my breathless enthusiasm?

Rouli of Games Alfresco and Augmented Times has started a wonderful thread on Augmented Reality in 2010.  I figured I’d jump into the conversation with my take on what will emerge in 2010.  My general outlook for next year is that more focus will be paid to utility and entertainment uses of AR, and less focus will be paid to AR as marketing gimmick.

Augmented Reality for home shopping will explode

While AR browsers like Layar and Wikitude will continue to focus their attention on discovering information that is in the world at large, another class of AR applications will emerge that helps people see what could be in the comfort of their own home.  We’ll see a lot more applications released by manufacturers that sell products that go in people’s homes.  These applications will be more sophisticated than the recent IKEA campaign in Germany, as they will make use of the actual smartphone video stream to make sense of the user’s environment, and also allow people to purchase the products they’ve previewed right within the app.

(Image via MobiADNews)

Products that people will be able to “try before they buy” will run the gamut from furniture, artwork, electronics, window treatments, clothing, and maybe even paint colors.  This type of application will be to 2010 what the “hold a marker up to your webcam to see a marketing message” was in 2009.  And there will likely be both good and bad executions of the basic concept.

See-through eyewear will still be out of reach of the average consumer

As much as I’m chomping at the bit to see this happen, I think that any decent implementation of a see-through display will still be at the >$1000 price point.  One possible scenario that would change this is if one of the major game console players partners with a display manufacturer to release an AR-specific console that includes the eyewear, and they bank on selling millions of units.  The Nintendo Wii controller is a recent example of AR tech gone mainstream.  Steven Feiner remarked to me that he would have gladly paid hundreds of dollars for a device with the motion tracking abilities of the Wii controller before it came out.  However, I think this is not likely to happen in 2010, but perhaps 2011?  I would love nothing more than to be proven wrong on this one.

Games, games, and more games

I agree with Lester of Augmented Planet that “marker-based games will be the next wonder”, and Noah Zerkin that we’ll see “multi-user, multi-device, multi-perspective games”.  There will also be natural feature tracking games thrown in the mix that don’t require markers.  Apple will inevitably open up their camera API to developers, and the pent up demand will result in hundreds of AR games for the iPhone alone.

Augmented reality will raise consumer consciousness

There will be a number of companies that release applications in 2010 that help a consumer make more informed choices about the products they purchase day-to-day.  A user will be able to point their phone at a product and see information not readily available on the packaging.  Information such as manufacturing practices of the company that made the product, the distance the product had to travel, nutritional value, use of organics, etc.  Tish Shute speculates about “Green Tech AR apps” in her interview with Ori Inbar.  I’m going to optimistically say that 2010 will see the first real applications emerge in this area, although perhaps the data sets will not be as rich as they could be.  See Giuseppe Costanza’s excellent “Food Tracer” demo on a Symbian device:

Smartphones will ship with two video cameras

One camera pointing at the user, and one away, will make possible crude applications of augmented reality video conferencing.  At the very least, a smartphones with two cameras will allow both “magic mirror” and “through the looking glass” applications to be targeted to the same device.  There are already handsets on the market with two cameras, but generally one camera is used for snapshots and the other dedicated to video conferencing.  The first phones that allow simultaneous access to both cameras will most likely arrive on the Android platform.  This is really mere speculation on my part, but I hope to see a handset like this in 2010.

That’s just a handful of things I think we’ll be seeing in 2010, but I’m positive there will be more surprises next year than there were this year, and I’m fully looking forward to seeing what happens.  I especially hope that Noah Zerkin’s expectation that we will see “a step towards a more precise civilian positioning system” comes to pass, although I’m a bit more skeptical.  Until that happens, AR applications and games that rely solely on GPS + compass tracking will continue to disappoint.  What do you think?  Check out the running list of predictions and commentary at Augmented Times and simulcast at Games Alfresco, and join in the conversation.

This was a fun project to put together. It’s somewhat unique in the increasingly crowded field of FLARToolkit-based marketing applications out there, as it makes use of the pan, tilt, and roll of the marker as inputs to a game (good old SOH-CAH-TOA came in handy for calculating that part).  For this project, I decided to give the FLARManager a try, mostly for its pretty decent smoothing algorithm.  The game wouldn’t be much fun to play if the playing field was jumping around sporatically.

The game takes a bit of concentration to play, since the game piece always levitates upward (like Criss Angel, get it?), instead of what one’s sense of gravity would expect.  But once you get the hang of it, some of the trickier levels towards the end are a lot of fun to play (my favorite is level four).

I programmed the gameplay, 3D rendering, and Facebook/high score integration, and Matt Dominianni provided the level design and programmed many of the UI screens.  The application was produced by Circ.us for the A&E Television Network.

You can play the game at the official Criss Angel site here: Five Lives of Criss Angel Augmented Reality Experience.

This is one of the best uses of FLARToolkit that I have seen to-date. I’m glad to see people making use of AR to solve actual problems, and not solely for the wow factor. You can’t get much more un-wow than a USPS priority mail box, but a simple app like this could help make everyday decisions like how to ship that recent eBay sale all that much easier.

I see a lot of promise for AR in the home, helping people answer questions about how well real objects will work in their personal space before schlepping the actual atoms.

Try the app out here.

Via Games Alfresco, via Living in an Augmented Reality.

I just finished developing a new augmented reality experience for SAP, to promote the SAP EcoHub during the Sapphire 09 conference in Orlando, FL this week.  The project was a collaboration between myself, Ori Inbar of Games Alfresco, and Seventh, Inc.  I programmed it, Ori produced it, and Seventh, Inc created the 3D artwork.

The project was completed on a pretty tight schedule, so there isn’t anything super ground-breaking about the experience.  It does, however, include some interesting interactions as far as these flash-based promotional AR apps go – the experience responds to the rotation of the marker, and also allows you to click on individual 3D cubes that take you to the corresponding EcoHub solution page.

The code is based upon the FLARToolkit and Papervision3D, and should be available under GPL license from SAP.

Check it out here, and let me know what you think.

On my current FLARToolkit project (release soon!), I needed the ability to trigger more than one animation for a Collada file.  No problem, I figured, I’ll just use the convenient DAE.play(clip:String) method.  Unfortunately, I discovered that the implementation of that method didn’t make use of the “optional” clip name:

/**
* Plays the animation.
 *
 * @param clip Optional clip name.
 */
public function play(clip:String=null):void
{
    _currentFrame = 0;
    _currentTime = getTimer();
    _isPlaying = (_isAnimated && _channels && _channels.length);
}

Luckily, the original author, Tim Knip, had coded enough of the foundations that adding in support for multiple clips was pretty straightforward. If you need to play multiple animations for your Collada models, and can’t wait for the fix to make it into Papervision3D, you can download a replacement: DAE.as.

The bug report is Issue 191, if you want to follow progress on its inclusion in the Papervision3D trunk.