Sonicode is a performance piece producing audio-visual work derived from barcodes. The barcode is input using a wireless scanner and informs a rhythmic pattern made of audio samples that I have collected from items around my flat. There are four primary tracks into which a barcode can be entered to create the rhythm of the work, and a further four tracks which allow binary mixing of the input signals to produce further sound output.
Background
I’ve had a fascination with barcodes since childhood. I remember the scene in Disney’s The Computer Wore Tennis Shoes (1995) where Dexter Riley (Kurt Russell) has been transformed into having a human/computer hybrid brain as a result of an electrical storm and is suddenly capable of amazing acts of memory and computation.
He picks up a jerky stick from a news-stand and is able to “read” the barcode to realise the cost is 89¢. Even as a child I was sceptical that a barcode was readable in this way as I (correctly) thought there was a direct mapping to the decimal number printed below the barcode and that it only contains a reference number from which the price could be looked up.
Later, I created my first barcode for a zine I worked on with friends back in 2001. The code is derived from the ISSN, and it made it easier for us to get the zine accepted for sale at larger shops .
My interest was recently reawakened when reading Code - The Hidden Language of Computer Hardware and Software (Petzold, Charles), which begins by explaining the history of encoding systems such as morse code and braille, then later the UPC barcode standard. I finally had a deep understanding of how the barcode worked!
A barcode represents an interesting interface between humans and code. Invented in the 1950s, it wasn’t until the 1980s that it started to become very popular as the costs associated with the technology decreased. Visually they are easy to recognise in form, but almost impossible to “read” or decode by eye – unless assisted by Hollywood magic.
Barcodes appear as a series of bars of differing thickness, but are actually composed of bars of uniform thickness where the bars representing the 1s of the binary string are printed. Each decimal number uses 7 bits, which at first seems unnecessarily large, but actually there are 3 bit patterns to represent the same decimal value. The patterns are divided into two halves (either side of the central control pattern) and the left has even parity and the right odd parity. This allows the scanner to determine if the code is upside down and reverse it if required.
Concept
When I’ve asked friends what they see when looking at a barcode, their answer always involves a series of thick and thin lines. However to the scanner the lines are of uniform width and a thick line is simply three adjacent lines.
I’m interested in exploring the idea of treating adjacent lines as a single thick line for rhythmic interpretation. I also want to look at using barcodes for direct waveform creation.
I intend to focus on the binary pattern of the barcode, its decimal representation being only of secondary interest. I’m also interested in the dynamics of how barcodes might combine. There are three control codes which mark the start, middle and end of the barcode and so these form a natural synchronisation point.
If time allows, then I hope to look at visualising the barcode and how visuals could be used to contextualize the audio generated.
I’ve also been interested in the relations between the barcode, the UPC number and the product to which it attached. The numeric association seems essentially meaningless, a result of the visual expression of the algorithm and the numeric id which was chosen, however there are some subtle patterns which indicate the genre of product and the country of origin. It is perhaps possible that a well tuned ear might be able to identify aspects of the code from the audio output. It could be interesting to tie the products attached to the barcode into the piece but I think this would have to be left for future exploration.
Inspiration
I’ve been greatly inspired by the work of Ryoji Ikeda. Last year I was fortunate to see one of his performances at the Barbican. The show was in two parts, the first performed live and unamplified using human clapping and cymbals. The second part was a recorded performance from his datamatics series. Both these works play very heavily on the overlapping of independent rhythms and the temporal and physical distances that get highlighted with the binaural interplay of the sounds arriving in our ears.
I was also very inspired by the work of Ei Wada who recently released an amazing video performance of creating live music using barcode scanners. This was around the time I was reading Code and figured into my idea of using barcodes for this project. In his video, he is actually mostly using fiducials and it seems the reflected light pattern from the scanner is feeding the audio.
Technical implementation
I picked up a basic understanding of how barcodes work from reading Code - The Hidden Language of Computer Hardware and Software (Petzold, Charles). I chose to focus on the UPC-A barcode standard as it had a good encoding reference and I created a very basic binary pattern for a single 7-bit value (see left).
This worked, but it wasn’t going to be practical to encode the entire algorithm using toggles and gates. I tried a few things before settling on using coll to output the binary list for each digit’s encoding.
This worked well and I used it to get a first sense of translating a barcode to audio.
My barcode scanner works like a keyboard which types the barcode very rapidly. I found that by using zl.group I could store the 13 digits of a barcode and push them as a single list.
I discovered that using a multislider I could display the barcode within Max. This was exciting but I realised that my encoding wasn’t correct since the displayed barcode didn’t match the original one I had scanned. I needed to switch to use EAN-13 as this is the standard which most products use for generating barcodes. However it was more complex as it has differing encoding patterns for the first half of the code depending on the first digit. This took up most of the coding time I spent on this project and resulted in the following four abstractions.
- barcode-digits-to-bin-list.maxpat
Takes a single digit and the encoding sequence, and returns the binary encoding as a list of 7 bits. - upc-to-bin84.maxpat
Takes the full 13 digit EAN, splits it and then using barcode-digits-to-bin-list turns it into a 84 bit list (this is the most complex part). - bin84-to-barcode.maxpat
Takes the 84 bits from the upc-to-bin84 abstraction and adds control codes and padding to make 101 bits. - upc-to-barcode.maxpat
A convenience wrapper to make it easier to group together the full encoding and reset.
I was very pleased when my encoding was fully working and could be verified by using a scanning app on my phone.
Now that I had a working binary list I wanted to be able to sonify more than one barcode at once. I had the idea that they should behave like tracks into which I could enter a barcode and the sound would be produced on each beat. I initially did this using a metro, but found that the barcodes would go out of sync, especially when adding a new barcode to a track.
Due to these frustrations, I started to look at using MSP (signals). I knew what I wanted to produce; a sound which was only active during the 1s in my binary list. I tried using a complex set of line~ objects, and then tried using techno~. I had some progress but it wasn’t sounding how I wanted. I abandoned this approach as I came up with a better way of using metro, however I think it is possible and would like to get it working as it presents interesting possibilities.
The new approach was working well and I had my four tracks. I now needed more samples. Due to the Covid-19 lockdown I was spending almost all my time in my flat and so I decided to try producing my own. I recorded these using my Røde VideoMic, and Olympus Dictaphone–probably not ideal equipment but it was what I had available. I made 30 samples in total and gave them minimal processing in Audacity. I like that my samples have an “amateur” quality to them and I can identify the item which produced the sound.
I moved on to visualising the barcode. I have experimented with shaders previously and we looked at them in class so I initially tried that. I got some very basic black and white pulses, which I think could look good projected at a live performance but I wanted to try for a smooth continuous stream of bars. I next tried using mgraphics and was able to do basic things like moving a rectangle in a fairly easy way but it was hard to build it up dynamically, and I was going to need around fifty rectangles. I ended using up using JSUI as this allowed me to use mgraphics with JavaScript, a language I’m already familiar with. This worked well and I settled on producing an accurate rendering on the barcodes with a line to indicate the current progress through the track. I felt this was useful as it visually connects the audio output and more importantly shows that a single “thick” bar is actually composed of consecutive 1s so multiple beats will be played.
Ideally I would continue to develop more visualisations from the binary representation and also a way to switch between them. I would have liked to make it so barcode could use a jit.window which didn’t seem possible with JSUI.
Performance notes
My performance begins with me scanning a barcode which appears within the Max patcher. I click the toggle and it begins to play each bar as a clap sound. Once the barcode has finished I add a second barcode to the next track and we can hear both sounds playing in an almost syncopated style across the stereo channels. I add one more barcode on each of the next loops until all four tracks have a barcode.
I now add a mix track on each loop. The barcodes are played verbatim, but the mixes allow expr objects to remix the patterns. A barcode has no inherent meaning, its representation is designed to achieve accurate scanning. The encoding is not two’s complement like many binary encodings, and so there is a large latent space between all the valid barcodes. This allows me to dive into, and consider ideas such as the logical union of two barcodes.
The piece grows in complexity and intensity where it is hard to pick out the rhythm of each track but the shared gaps of the control codes now form dominant points along the segments.
What the audience will experience if it were publicly presented
I envisage that I would be standing behind a table and would have a variety of products with barcodes that I could scan to produce the audio. I would play through several single barcodes, to show the slight differences between different product’s rhythms. It would also be fun if I could take an item from the audience, perhaps a beer can, and scan it–in order to show that it isn’t “fake” and that the audio is truly derived from the barcode.
Appendix
The source code is available at
GitHub