It comes from the Way Out West Hackathon 2013 and involves the Pixelfolders party hack team creating a pan flute-based game…
They’ve written in detail about their work.
For example, the flute controller involved Python running on a RaspberryPi, listening to the GPIO pins and forwarding “blow” events to the desktop via TCP.
In order to capture when the player blows into each pipe, there are four Arduino sound sensors per flute, positioned at the very tip of each bamboo pipe and wired to the GPIO pins on the RaspberryPi. These are read back using the Python package RPi.GPIO.
On the communications side of things, they write:
Alongside reading the sound sensor inputs, we run a local TCP/IP server that acted as a controller hub. This mean we can serve multiple game clients the flute events at once (mostly for debugging). The server listens for events on the Raspberry PI GPIO ports, and forwards them over TCP/IP to all connected game clients that are listening. It can also be run locally together with the game to simulate a flute if the flute is being blown by someone else. The clients can also send commands back to the server to tell the flute when to turn on and off the non-existent lights (which was one of the planned, but canned, features).
For the game client, they say, almost everything was written in Lua (a new one to me) “running in luajit with FFI bindings for libraries”, with some C++ support code, and libspotify for playing Spotify songs in the background using RtAudio…
Game rendering is very simple: first the four pipes are rendered into the framebuffer using a mix of lambert shading and pure nonsense. The pipes themselves never move, instead, the applied textures are sampled with a time-based offset to indicate movement. The coloured ‘blips’ area, where a player need to activate each pipe, is a small section of the pipe-geometry that is rendered on top of the individual pipes. To achieve the wobbly effect, the base-color texture is sampled along the horizontal direction with a good old sinus function, modulated with a time-since-pressed value sent to the GPU as a uniform.
It’s all very impressive.
For the flute itself, they sourced bamboo from a local flower shop in Gothenburg!
See the flute game in action. Note: the flute pipes are glued shut so no real flute sounds are generated…