We have a Wii running 24 hours a day in our lounge at R/GA San Francisco. It’s only 20 feet away from me, yet I hardly have any time to play with it. However, my lack of playtime doesn’t stop me from realizing the huge impact that devices like this, as well as other gesture-based and multi-touch devices (iPhone, iPad/tablets), are having on the future of user interfaces. As well, we’re starting to work with Oblong, who are at the forefront of this movement – I’m super excited about that!

Will the Mouse go away from my desktop? Probably not too soon – but eventually, I’d expect to use it less and less and less. The ‘ol mouse was so useful to us for so long, but it has limits (not to mention health risks); it adds a physical barrier between the user and the experience. The iPad helped us see how we could come yet one step closer to the world of touch in our day-to-day computing. Now, we are seeing how this can be evolved to include larger, fully gesture-based interfaces that will someday change the way we interact with basic computer tools.

“There’s no reason why dealing with a spreadsheet or sorting e-mail couldn’t be as wonderful as tai chi,” said computer and social scientist Katherine Isbister at NYU’s Polytechnic Institute in Brooklyn. “Games are the perfect ecosystems for evolving fun, and hopefully we might be able take those lessons elsewhere.”

And Microsoft’s new Kinect:

Interactive Puppet Prototype with Xbox Kinect from Theo Watson on Vimeo.

More on to get started with Kinect and the OpenFrameworks can be found here.

And Oblong, who are truly creating the future of computer UI:

I’m hoping the performance will improve as people learn to optimize for this new device. Then again, wasn’t that the point of HTML5? It’s supposed to be open and run everywhere (HTML5 is supported).

OK, now we’re talking: the future of interfaces, currently on display at SIGGRAPH. Some Augmented-Virtual-Reality and the like. Although it’s not a true hologram, the touchable holography is pretty sweet, and I’m sure it will soon get attention for creative media/marketing applications; interacting with games and/or sports for example. I was excited to see the Augmented Reality for Ordinary Toys. One of my minor beefs with other implementations of AR, was the prep required to make an object recognizable. This solution seems to show any object getting some sort of identity for the system to respond to.I think I like the scratchable input example the best. I tend to like things that start with raw, dynamic data, and then offer the ability to visualize (or realize) this data in any form; plus, I like doing stuff with sound! Just think about all the cool uses for this. As someone else commented, it would be neat to see this done with multiple input devices (instead of 1) so that they could measure the gestures in 2 dimensions. 

Finally, posting some valuable info capture from the MAX show. Thanks to Crazy J!

The Adobe Image Foundation (AIF) Toolkit
The Adobe Image Foundation (AIF) Toolkit preview release includes a high-performance graphics programming language that

Adobe is developing for image processing, codenamed Hydra, and an application to create, compile and preview Hydra filters and effects. The toolkit contains a specification for the Hydra language, several sample filters, and sample images provided by AIF team members. The AIF technology delivers a common image and video processing infrastructure which provides automatic runtime optimization on heterogeneous hardware. It currently ships in After Effects CS3 and will be used in other Adobe products in the future. The next release of Flash Player, codenamed Astro, will leverage Hydra to enable developers to create custom filters, effects and blend modes. http://labs.adobe.com/wiki/index.php/AIF_Toolkit

Flash Player 9 “Moviestar”
This is the latest beta release of the Flash Player (not sure when they plan on releasing it publicly). Some of you may already be aware of the new features Adobe introduced in this player release, but for those of you who aren’t here are a few cool features. Support for H.264 video and HE-AAC audio codecs. This means support for up to 1080P HD video as well as the ability to take advantage of existing tools and services that leverage this industry standard format. They demoed this technology during the keynote with 720P full screen video and it looked amazing.

Multi-core support for vector rendering. At MAX the Flash Player team explained that there is currently support for up to 4 processors (cores) and that this may increase in the future. They also explained that this ONLY affects rendering. It will NOT increase the performance of ActionScript. The reason for this is because Flash is a single threaded platform and script execution cannot be divided among several processors. The rendering performance improvement results from dividing the final render frame into pieces and sending each piece to a separate processor for concurrent rendering.

Full screen mode with hardware scaling. The new full screen mode allows you to choose a rectangular area of the stage to magnify full screen. This mode will also take advantage of the video card for high performance scaling. This is the reason the full screen video during the keynote looked so good and performed so well.

Flash Player cache for common platform components, such as the Flex framework. This is a very interesting feature but for security reasons it currently only affects Adobe signed components. They hope to extend the feature to custom components when they can determine a safe way to do so. It’s basically a cache, similar to a browser cache, but it caches flash components in a domain independent manner. What this means is that if your site uses a particular component, such as the Flex framework, it will be cached by the Flash Player. When users revisit your site, or visit any other site that happens to use the same component, the component will be retrieved from the cache instead of downloaded again. This can drastically reduce download times for complex applications that leverage frameworks such as Flex. http://labs.adobe.com/technologies/flashplayer9/

Performance Tips for AS3
There was a very interesting session at MAX about performance in AS3. It compared the performance of various ActionScript statements that carry out the same task but are organized or structured in slightly different ways that have a significant effect on performance. The most important lesson from this session was TYPE YOUR VARIABLES! The performance difference between typed and untyped variables was about 10 fold! For those of you who haven’t gotten into this habit with AS2 now is the time start. It was also interesting to see that operations between two different types (such as int and uint) are much slower than if the types were the same. In the following example, the first for loop is more than 13 times faster than the second, and more than 23 times faster than the third:

var count:int = 10000000;
// comparing int to int (i < count)
for (var i:int = 0; i < count; i++);
// comparing uint to int (i < count), 13 times slower
for (var i:uint = 0; i < count; i++);
// comparing untyped to int and incrementing untyped, 23 times slower
for (var i = 0; i < count; i++);

There were some other performance tips that are a little more obvious and also have a benefit in AS2 as well as AS3. For example, assigning array length to a variable before using it in a for loop (some of you may already have a habit of doing this). In the following example the first for loop is almost 20 times faster than the second:

var a:Array = new Array(10000000);

// assign length to a variable

var len:int = a.length;

for (var i:int = 0; i < len; i++);

// use length directly, 20 times slower

for (var i:int = 0; i < a.length; i++);

The most surprising demonstration was the try-catch statement. In the demo, a complicated conditional statement that accomplished the same task as a try-catch statement was about 100 times faster! In my own experiments, a try-catch statement under normal conditions (i.e. no error is thrown) is a very small performance hit and is almost the same as there being no error handling code whatsoever. However, when an error is thrown, it’s about 300 times slower! The bottom line is it’s a good idea to use try-catch statements to handle unrecoverable scenarios and notify the programmer that they are doing something wrong (such as an index out of bounds exception). But it’s a bad idea to use them in place of simple conditional statements to handle valid scenarios that may occur under normal program execution.

Adobe MAX was last week in Chicago – I followed that with a trip immediately up to Microsoft, and back to San Francisco; so I’m lagging on getting my MAX post up. I went with Crazy J and we met up with Sam (now at Yahoo), Kaare and Yi from AKQA New York, Dave from AKQA DC, Nick Velloff and the Sauceman. We were totally surprised when Kevin Lynch showcased our Halo 3 Believe work during his keynote. It was especially cool for Crazy J to see his work up on the giant screen considering the fact that he masterminded the main video interaction and apparently stumped Kevin himself. Kevin, if you’re reading this, we did not use streaming FLV video; it’s not video at all. The rest of MAX was awesome; lots of great, inspirational stuff and I will be posting some cool things I learned over the next few days.

Has anyone seen this “Augmented Reality” stuff? I saw an in-person demo of it last Friday. It is “Real-time interactive-3D-video” that connects up with Print, Web and soon Mobile. It has the ability to track movement, including facial expressions, and can recognize physical objects (people, print) and interact with 3D objects, characters or environments all in real time. In the past it was targeted to outdoor installations and broadcast, (if you watch sports on TV, you’ve likely seen a version of their technology at work: Overlaying 3D objects on top of the action), but it also hooks up with desktop, the web and soon mobile.

This is a link to a crappy and limited video presentation- but you really need to see it presented in person to get it:

http://www.youtube.com/watch?v=g8Eycccww6k

Company site, more examples:

http://www.t-immersion.com/video_gallery/main.asp?idf=a0#

Because it’s all networked, we can track the user journey, hook it up to a database as well as integrate it with other digital touch points or mash-ups via web services. Maybe a user’s physical interactions can be captured and tracked to a back-end that mirrors, enhances or mashes the experience up with a web service (maps, weather, YouTube?) We could tap into an asset database to consistently keep the experience fresh and updated with current graphics, etc. It supports 3D models that we work in (i.e. Objects output from Maya).

We did it! Final Phase of Nike Copa America launched! This was a challenging project that took a solid three months to complete. Phase I started before the Copa tournament. We created a solution that would allow users to predict the outcome of each stage of the tournament by building their own bracket. Their data was stored until the final game. At this time (Sunday, July 15) we totaled the scores and ranking, determined the winner and created a search function on the site to allow anyone who entered the contest a way to see how they did. Big thanks to my great team: “Tony” Terry, Summer, Tim, Dwight, Mark, Stuart and Paul!

http://nikefootball.nike.com/nikefootball/siteshell/index.jsp#,la,0;copa,,