Tech Talk: Seeing Sounds and the ‘Art Game’


Singing Ribbons is a new art installation for the iPhone generation.


By Nathan Blades

Singing Ribbons is a new art installation by Matthew Maxwell for the iPhone generation. A series of paintings consisting of bold stripes of colour can be scanned by a special mobile app that converts each stripe into notes sung by a soprano.

Really, it’s an awesome concept. On a basic level, it’s a great form of interactive art. The works come alive with what you bring to the gallery, and the results leave with you. We are clearly far to prone to forgetfulness to simply remember the exhibit.

From a technological perspective, it takes the idea behind QR codes (Those ‘square barcodes’ that most smartphones these days can read), and explores it in a refreshing way. It’s no surprise that Mr Maxwell works in software, but has an education in fine art.

Beyond that, I think it’s an interesting demonstration of Synaesthesia, though that may not be intentional.

What’s that? Synaesthesia is a mental condition that switches up the wiring in your brain when recalling emotions and senses. Normally when we see, for example, the colour green, our brains go through a subconscious process of acknowledging what colour it is, using the ‘correct’ part of the brain to do so.

For a ‘synaesthete’, they may instead start smelling fried bacon or hear the sound of a tuba, as their brain tries to use an area that’s meant to process a different sense.

In practice, the most common form of synaesthesia are people feeling that letters and numbers have inherent colours. What’s more, only 1 in 2000 are likely to be synaesthetes. But that’s not as romantic or exciting to think about.

Drug use (I won’t say which ones, though you could probably guess) is often known to induce synaesthesia, often a kind where sounds trigger colours, or touch triggers sounds. Unsurprisingly, this is a lot closer to artistic representations.

The bars that make up each piece in Singing Ribbons reflect the decorations on army generals. What would be a meaningless pattern of colours to a layperson has a set of understandable rules and meanings to someone in the know. Just like how a synaesthete could feel another meaning to a set of apparently arbitrary letters.

Singing Ribbons isn’t the only example of interactive art, or in using technology as a medium. Across the history of video games, there’s a host of titles that take the art appreciation experience to your living room.

Once described as ‘art games’, they cause many debates over if games are inherently art, or if only this genre of subversive and abstract titles qualify. Many have taken to describing them as ‘un-games’, though that’s not always appropriate either.

Rez is the classic example. Released way back in 2002 for the Sega Dreamcast and Playstation 2 (and since re-released on Xbox Live), it is directed by Tetsuya Mizuguchi.

Across much of Mizuguchi’s work, he has a personal fascination with sound design in games. He designed Rez as a representation of Synaesthesia, and in reference to the Bauhaus art of Wassily Kandinsky.

Even though it’s a video game in a very traditional sense, enjoying it need not be about getting high scores. There’s a mode that lets you just enjoy the game without fear of failure.

More recently, the present Indie scene has had a revival of games that are much closer to the idea of the ‘un-game’. Dear Esther went from a free modification of an existing game in 2008 to a complete, priced title in February 2012. It lets you explore an abandoned, overcast island as a narrator explains the back story of the events that happened there in a vaguely obtuse and non-chronological fashion.

At the time, its total refusal to conform to standard perceptions of a ‘game’ caused a large stir among games journalists and even now I feel it’s a far cry from a successful experiment, but it was a well-publicised step into a bigger interest in interactive artworks.

As such, Proteus works out to be a more palatable take on that kind of Art Game. Using intentionally low resolution graphics, hazy neon gradients, and a dynamic interactive soundtrack (filled with wavy extended synths and hushed clarinets), players can wander through the landscape as day turns to night and clouds roll overhead.

As developing video games becomes easier, and the underground movement for producing counter-culture and intellectual games builds steam, the discussion of what constitutes ‘art’ or a ‘game’ will rage on.

Meanwhile, Pippin Barr’s Art Game, a game about producing art by playing a game within the game is the best take on the genre I’ve seen yet. Maybe the Tate Modern should have an Art Game exhibition this year.

Singing Ribbons will be available for viewings from 14-17 April at the Coningsby Gallery on Tottenham Street. For more information, check the gallery website.

Photo courtesy of OldsXCool via YouTube, with thanks.

Follow us @SW_Londoner

Related Articles