Programmable Reality Workshop at CHI2009.

Programming Reality Workshop, that I co-organized with Roel Vertegaal, Marcelo Coelho and Sajid Sadi, was a highlight of CHI 2009 conference for me bringing together an interesting group of people.


The first day of the workshop was a day of presentations. Some of the presentation files can be found here and the papers are here. Below is a basic summary of the presentations and discussions.

One general theme of discussions was designing interfaces for arbitrary shaped and/or deformable computing devices. For example, Hrvoje Benko from Microsoft Research was talking about his work on Microsoft Sphere and designing interaction techniques that could take advantage of unique properties of spherical displays. Roel Vertaagal was discussing various interface design approaches of foldable and bendable devices. I did a couple of projects in relation to this theme, in particular Gummi and D20 projects.

Microsoft SphereMicrosoft Sphere by Hrvoje Benko

Another theme was  dynamic architecture and re-configurable environments. Omar Khan from the Center for Virtual Architecture at University of Buffalo showed some of his re-configurable architectural environments built using composite elastometers that he and his group designed and implemented. Mette Ramsgaard Thomsen from CITA at Royal Danish Academy of Fine Arts presented projects on dynamically re-configurable living spaces with a strong emphasis on exploring the use of textiles and smart materials. Jinsil Seo from Simon Frazer University presented a number of “immersive” physical installations that she created.

Reflexive architectureReflexive Architecture by Omar Khan

Several talks was discussing various issues in computationally-controlled materials. In particular Tim Merrit presented his work on biological computing media, such as using genetically modified E. Coli bacteria to create glowing ambient displays. Eric Schweikardt talked about projects in Cornell Computational Synthesis Laboratory on personal digital fabrication, which I think is relevant to this topic. Marcelo Coelho and Sajid Sadi from MIT Media Lab talked about various techniques of integrating computational control and materials, such as designing paper and textile based digital devices and actuating them with SMAs. Anna Vallgarda from IT University in Copenhagen presented concept of “computational composites” and presented her Planks project. Keywon Chung discussed how properties of objects and materials can be considered when we enhancing them with computation.

sproutSprout by Marcelo Coelho.

Programmable matter was also briefly discussed. I feel that  “programmable matter” is a somewhat misleading term:  it currently refers to trying to make objects out of the tiny blocks that re-arrange themselves in space, e.g. see “Claytronics” project in CMU. In this sense we do not really “program”  matter but design a sort of  modular robotics systems. Eric Schweikardt from Cornell University and Michael Philetus Weller from Computational Design Lab at CMU talked about their project on modular robotics and  self-reconfigurable devices.

Cube latticeModular self-reconfigurable robot by Michael Philetus Weller.

Self-actuated interaction devices were presented by Olivier Bau from INRIA, France and Hyujung Kim from KAIST Korea. They showed two different approaches to building surfaces that can dynamically change their physical surfaces. Olivier uses a novel electromagnetic actuators that is woven into textile and Hyujung was using solenoids and stretchable fabrics. I also did a somewhat similar project using shape-memory alloys few years ago. There was also one more presentation from CMU’s Computational Design Lab by Greg Saul who showed his actuated paper devices.

Imprimer"Bubble Wrap" by Olivier Bau.

Tools for developing physical interfaces have been presented by Reto Wettach from University of Potsdam, Germany who tried to convince us to use his What-You-See-Is-What -You-Get Fritzing software environment for rapid prototyping of hardware devices.

fritzigFritzing environment screenshot.

Finally, the conceptual challenges of programmable reality have been discussed by me and Erin Solovey from Tufts University who discussed possible interaction frameworks for programmable reality.

Overall, a number of common and often overlapping themes emerged during discussions and presentations.

1. How can we combine computation and physical materials? The vision here is that basic construction blocks that we will use to make things should already have computation built into them so that architect or designer does not have to think about it.

2. How property of new materials, such as flexibility or thier shape can be used to create new interfaces and applications? For example, if devices become foldable or stretchable, how that can lead to new interaction paradigms? Its important to take advantage of the unique properties of these materials when designing interfaces.

3. How kinetic motion can be included into interaction with machines? The motion can be used for haptic feedback or to display information.

4. How can we make objects that can dynamically change their shape? This is often called “programmable matter”. These devices can be on scale from hand-held device to a room or a building. We should also consider implications of programmable matter on our living environments.

5. How can we program all these future devices and materials? And what “programming” actually means? Are we programming computers or we programming these materials and devices directly?

The second day was a prototyping day that was spent at MIT Media Lab.  The goal was to brainstorm and develop few prototypes of reconfigurable interfaces using available facilities. There were few interesting ideas quickly prototyped, such as modular device consisting of several inflatable cells requiring several people blowing into tubes to make it work. Unfortunately, due to circumstances beyond my control I could not participate in the making actual things, so I was mostly observing the results and taking pictures.

workshop-prototype1workshop-prototype2Ready to blow!


The after-workshop activities were in the “Miracle Of Science”, a common hang-out for MIT students. Burgers there were very good.


Respond to Programmable Reality Workshop at CHI2009.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s