Archive for the 'code' Category

Openframeworks + Kinect still working

Thursday, July 28th, 2016

Last night I was looking at depthkit and 8i while researching options for video-based 3D capture, and I felt inspired to rescue my old kinect from the bottom of a drawer. I got a fresh copy of openframeworks and ten minutes later I had a running build of the openframeworks kinect example in my computer. It was literally like time-traveling to 2011. I remember capturing a point cloud of Amy pregnant back before Maya was born. A couple of months later the kinect found oblivion in the bottom of a drawer and I stopped using openframeworks until now.



Synchrony 2016

Saturday, January 9th, 2016

I am attending a demo party called Synchrony NYC. It is hosted at a place near Union Square in Manhattan called Babycastles and organized by my old friend @nickmofo, who invited me to give a talk about virtual reality.

Synchrony promotional image by Raquel Meyers.

A-FRAME, a markup language for browser-based VR

Wednesday, December 16th, 2015


I have been fooling around with ThreeJS and virtual reality boilerplates for desktop and mobile browsers using Oculus and Cardboard for a while, but this just takes things to a whole new level.

A-frame is described by its creators as

an open source framework for easily creating WebVR experiences with HTML. It is designed and maintained by MozVR (Mozilla’s virtual reality team research team). A-Frame wraps WebGL in HTML custom elements, enabling web developers to create 3D VR scenes that leverage WebGL’s power, without having to learn its complex low-level API. Because WebGL is ubiquitous in modern browsers on desktop and mobile, A-Frame experiences work across desktop, iPhone (Android support coming soon), and Oculus Rift headsets.

It is not the first time we see something like this —remember VRML and more recently GLAM— but this is the first time I sense a strong design and content oriented vision behind a toolset of this kind. It has been clearly built taking into consideration the full spectrum of creative people that currently fuel the web as well as the mobile space, and this I hope will help it stick around. To see what I mean just launch from the broswer in your iPhone if you have one (sorry androids), browse through the examples, and hit that cardboard icon.

Screen Shot 2015-12-20 at 11.04.51 AM

Finally, I just stole a drawing from an article by @ngokevin where he explains what’s so special about A-frame and the entity-component-system design pattern at its core.


Casey Reas Linear Perspective

Sunday, September 6th, 2015

Casey Reas just opened a show at the Charlie James Gallery in Chinatown LA last night. It is interesting to see how his generative work has recently shifted from the purely algorithmic —using rules and numbers as a base to create form from scratch— to a deconstructive commentary on media that utilizes content units —like digital photographs and video streams— as a source of [not quite] raw data that generates his quasi abstract forms over an extended period of time. One of his pieces, the one I photographed for this article, retrieves the main photograph from the cover page of the New York Times every day, and uses it as is as a topological stripe that stretches across the digital frame over and over again, weaving a familiar, yet unrecognizable tapestry across the big television screen that Casey chose as his canvas. Well done.



IML400 Spring 2015

Sunday, February 15th, 2015

The time has come to teach IML-400 at USC again, and this time around things are a little different. It is the first time I get a batch of students that had to take a prerequisite class, IML300, before they could join my class. This means I can jump ahead and make some assumptions about my students’ general knowledge that will hopefully help us move faster into the fun stuff and really take advantage of the browser as an interactive programming playground.

In addition to this, the class got split in two smaller groups of around twelve students, and I am only teaching one of these groups, while my colleague Raphael Arar is teaching the other one. When talking with Raphael about previous iterations and the future of the class, we decided to design a new Syllabus together based on my previous one, but taking into consideration Raphael’s teaching interests, the more advanced nature of this class, and aspects of the web that are a lot more mature today than they were during my previous iteration of IML400 a year ago. Specifically, I wanted students to put aside the page-based nature of the web we have today, and think about the things they can do using Web Audio and WebGL in emerging contexts like mobile WebVR for example.

I see my class not as design class, but as a creative innovation one. When thinking about new media, user interface, user interaction and user experience design are important things to understand, explore and develop as skills, but we are at a point where some design paradigms —like the page/scroll nature of the web today— have reached a degree of maturity that leaves very little room for the pure, unbiased creative experimentation that will eventually drive the emergence of fresh new media. There is so much more to the web that is coming to us.

Having a partner in crime on this teaching adventure has been the best thing ever happened to me and to this class. We are only a few weeks into the semester and Raphael and I have established a relationship where we exchange impressions about how the class is going every week, and iterate upon our teaching approach together. It’s really great to have someone to talk to at this level 😀





Friday, December 12th, 2014

Some time in the nineteen fifties a serious attempt was made to bring stereoscopic photography to the masses. Stereoscopic photography faces similar adoption challenges to 3D movies and Virtual Reality because up to this point there is no easy way to experience any of them without attaching a contraption to your face. An interesting note is that while 3D movies and Virtual Reality are fairly recent, stereoscopic photography has been around since the eighteen fifties, and even commercial viewers were mass produced back then.

In my quest to learn how to make my own Virtual Reality work, I got interested in the display of stereoscopic photography in VR. Inspired by the idea that properly placed in VR space, a stereoscopic photo can feel a lot like a VR sculpture of VR hologram, a moment frozen in time with a large potential to make the viewer feel “there with it” as opposed to just looking at a flat projection of it in a two dimensional picture.

While I was looking for cheap and easy stereoscopic camera systems I came across the historical mid twentieth century consumer cameras, and it was easier for me —or at least more reliable/fun/interesting— to get my hands on a couple of film cameras and start taking pictures than to find a digital solution. Eventually I came across a smartphone solution in Kickstarter that worked fairly well, but not before I was already shooting stereoscopic 35mm film all over the place. Here is an inventory of my stereoscopic gadgetry:

  • Revere Stereo 33 35mm Camera Released around 1953. In perfect condition. Works great.
  • Stereo Realist by the David White Company, available from 1947 to 1971. In perfect condition. Works great.
  • Poppy 3D for iPhone via Kickstarter. This works great too. Except it’s a little painful to manage the files. Overall easier than shooting film, developing it, scanning it and putting it together for digital viewing, but it could be easier on the viewing/playback aspect of the product.

Once I had my pictures, the next step was to build a program that could let me look at them through a VR headset. Since I have no time or interest to learn how to use a game engine, I was left with only one option: the web browser and WebGl/ThreeJS. I already knew there are experimental builds of Firefox and Chrome that are compatible with the Oculus Rift DK2 headset, and I also wanted something that could run on a mobile browser for Google Cardboard viewing. I knew where to get sample boilerplates from the VR Chrome team and Mozilla, so all I had left to do was to find a way to feed two textures onto the same piece of geometry while rendering for each eye. Luckily I found a code example that did just that, and it didn’t take me long to adjust it to my needs. You can find the code here (Stereovision).

A stereo card of a woman using a stereoscope circa 1901. Via Wikipedia

My Stereo Realist after a photo session

Stereovision for Cardboard on my iPhone. Featured photo taken with the Stereo Revere

Stereovision screenshot. Left and right images are rendered for the corresponding eye

PaperJS vs TwoJS

Thursday, August 14th, 2014

Inspired by a morning discussion with Tom White aka dribnet about javascript and 2D vector graphics, I came home tonight determined to find out how far could I get in ten minutes using PaperJS on the one hand, and TwoJS on the other hand, with the sole constraint that somewhere along the way I had to make use of SVG.

For those who don’t know, PaperJS is the web-based heir of Scriptographer, a Javascript plugin for Adobe Illustrator that lived up until CS5. As far as I know, PaperJS is far more powerful than it’s predecesor, and has been designed keeping in mind the interactive, object-oriented nature of the contemporary web browser.

Similarly, TwoJS is a relatively new web-based 2D drawing library inspired by the WebGL javascript library Threejs.

After ten minutes of PaperJS I produced a simple particle raster script that uses an image to draw a collection of SVG shapes over a grid, and I added some color manipulation and particle scattering functionality to produce images like the ones featured below, using pixel data from grayscale photographs of Brigitte Bardot and William Burroughs.

TwoJS took me in a completely different direction. All I did was write a script that pulled SVG code from the loaded page source, and created animations from it. Here are two examples: I<3VR and rolling W.