Pianair

About Pianair

I had a lot of fun putting together this air piano – that is, a piano that you can play like an air guitar.

The inspiration behind it was simply to create fun and interaction: it’s fun to make noise, and it’s great when people can get involved in something they wouldn’t otherwise. However, it also developed into something more than that.

It’s also interesting to see what the Pianair – and you – can produce together. After all, without the range of the piano or the tactile feedback from the keys, it’s hard to be a Beethoven (and the maestro would have found it even more difficult, because there’s nothing to feel your way across to play).

Therefore if you’re not playing to sound good, why are you playing? And what does this say about our impetus to play in the first place? Can “music” simply be the sounds of our happiness as we enjoy our play?

Pianair can be download as patches for Max MSP here: pianair, or as a collective build over at mega.co.nz.

Artistic Statement

Pianair is an interactive project that foregrounds the humble enjoyment of noise-making. The piece reconstructs the piano – a revered instrument of great aural depth – as a low-resolution (both visually and tonally) game. It is playable not through the exacting movements of its renaissance ancestor, but through wide sweeps: gentle key strokes have been replaced by vertical air pokes, captured by webcam.

This low-resolution reinterpretation of the piano aesthetically reflects one of its biggest thematic questions. Compared with the cultural and musical depth of a grand piano, why is the Pianair so enjoyable? The pleasure derived from making  sounds with the Pianair toys with the distinction between music and noise, between recital and play (particularly by knocking balls around the screen as a result of playing notes).

Most importantly, it asks participants: how vital is the definition between noise and music? In interacting with Pianair, a sense of wonder is instilled that typically overrides any immediate desires to hear a pleasant melody. For a time, we revert to children as noise and play are championed.

There is also another side to Pianair – as an automaton. Pianair’s visual detection system will operate on multiple stimulus – not simply the human body. Pianair can therefore be pointed to detect and respond to anything passing its camera.

This means it can essentially “play” passing traffic, TV shows or even household pets. The great human effort needed to play and compose for piano is removed as the system sonifies the world around it. As an automaton, like many sonification efforts, the Pianair objectively reinterprets its subjective surroundings.

Directions for Use

  • Try to use Pianair in daylight. Use in low-light situations can cause noise to occur in the video, as the computer webcam attempts to increase sensitivity by increasing its ISO, and therefore floods the lens with noise which effects note detection.
  • Experiment with moving closer and further away from the camera to make your fingers bigger and smaller
  • Try it with friends!

Building a Pianair

Creating the Pianair predominantly involved working on three separate parts of Max MSP: Open GL rendering, physics and general Jitter manipulation, including the webcam.

I first used Jitter to offset the webcam input, setting it to record just one horizontal line. When a certain colour was visible on this line (in this case, it was  programmed to be a redish hue,  finger) a “1” would be returned. This formed the basis of the keys: each key would have to cross this line to be activated, and would also be apportioned a limited horizontal width.

the air piano was too fiddly to be a  replacement for a Steinway

The keys themselves were overlaid on to the webcam feed, allowing the user to see where their hands were in relation to the keys. This made it much easier to use, and created a more direct link with the piano aesthetic.

However, it soon became apparent that the air piano was too fiddly to be a  replacement for a Steinway (or even a cheap replica). The webcam fidelity was not good enough to capture fingers at a distance great enough to fit all the keys on the keyboard.

Beyond this technical drawback, playing notes using the “above trigger’ mechanism of the Pianair involves a more vertical action, slightly different from the piano’s leaver-push method and  awkward for traditional pianists.

This fatal issue made me reassess the function of the Pianair – everyone who had tried the flawed beta versions had become totally engaged with the project, even without the precision of a real piano. Why? Thinking around this question created  was what lead to the final version of the Pianair, as discussion in the artistic statement.

I felt that to create the artistic Pianair I desired, I needed to make a few more changes – to lo-fi the visuals, add obvious “gamification” and to reduce the octaves to a more manageable number. The “gamification” was achieved by turning the Pianair into a sort-of arcade experience, with players being able to juggle balls through hitting notes.

Pianair - an air piano in Max MSP by Jack Ratcliffe

This was developed using the basics of the Jitter physics library, which I would love to investigate in more depth. I also only touched the surface of OpenGL manipulation, and while anything too complicated it would undoubtably feel disjointed from the retro, clunky aesthetic I was creating, in future I would like to explore Max’s OpenGL in more detail.

In the end, the Pianair reminded me of the above piano fountain. It started its life as a tool to create music, but in the end it became something completely different, but equally engaging. And in creating that sense of fun – and the noise alongside it – I feel quite satisfied.

Recognition

I’d like to thank the guys at Cycling 74, not least for Max MSP, but for their excellent tutorials (shoutout to the Physics Introduction) and help files (particularly  OpenGL).

Also Cycling 74 forum users Frid, whose suggestion for overlays helped me bond the OpenGL with the webcam.

Finally, thanks to my tutor Chris Kiefer – this wouldn’t have been at all possible without you!