Art and Digital Technology:
Performance Project by New Media Artist Balam Soto

"The Body Sound Suit" captures human gesture and translates it into data . . . beautiful data . . .

 

The Body Sound Suit is a bio-gesture capture and trigger system; it is inspired by the arduino drum project.   

The suit captures human gestures/body movements and translates it into data; this captured data then triggers sound and graphics.  The body of the dancer becomes a generator of sound and visual data; the dancer creates his/her own audio and visual environment simply by moving the body – by dancing. 

This challenges the unilinear relationship between the dancer, the music and the visual environment, synchronizing control of all aesthetics into the body of the dancer and expanding the accepted relationship between the dancer and the music into a very intimate realtionship between dancer, music, technology and graphics. 

 

Building the Suit and Problem-Solving (relieving my headaches) 

I first had to decide which Xbee to use for this project.  This decision involved hours of research and numerous forum visits over several days before I decided to use the most popular Xbee module.  Upon finally making this decision, I rushed to the closest browser and ordered my very first Xbee.

I soon realized I that I would need more than one Xbee as well as some kind of adaptor to connect it to the compter and the arduino and a battery to power the sender adaptor.  I returned to my online research – quadruple checking to make sure I ordered everything I needed this time. 

I finally had all of my supplies, so now, I had to figure out how to set up the Xbees . . . back online. Fortunately there is a lot of information online which I utilized, as well as my own trials-and-errors, in order to teach myself how to best connect the system.

Performance artist Heidi Kirchofer interacting with the body sound suit

 

 

 

 

 

 

 

 

 

Balam holding an Xbee used in the interactive body sound suit project

 

 

Success didn’t happen overnight; several days of frustration ensued as I lost data and dropped connections between sender/receiver.  The Xbee adaptor “lilypad Xbee”, which I was using as a sender, was poorly documented so I had to find the answer through experimentation.
I began my experimentation by making a mental list of where the problem or problems may lie. My instinct told me the problem lay in the sender, the Xbee adaptor that was attached to the suit.  
I assumed the Xbees themselves were not at fault because I was able to easily program them when I had them connected to a computer, so I focused my testing on the adaptors.


First test on the list - I lowered the voltage that my battery was providing in the sender by incorporating a transistor into the system.  The transistor worked, as far as lowering the voltage; as a result I was losing about 50% less data.  I now had a somewhat functioning, wireless mess sitting in front of me.  On to the next test . . .


Second on the list - check the sender and receiver adaptors and the OS platforms in order to debug the communication between the Xbees.


I used both a pc and a mac for debugging in order to test the OS platform as a potential cause of the problem.  For the mac side I used the all powerful http://tomgerhardt.com/Cornflake/; for windows I used X-CTU software.  These programs read serial ports so would tell we if information was being dropped from the sender or receiver.  These programs confirmed that the sender adaptor was dropping information, solution - find a new adaptor.  


I returned online, this time to purchase a different model.  I ordered an extra one JUST IN CASE, which I ended up using after I accidently melted down the pins on the first one.  The second, un-melted, adaptor worked perfect – I had my wireless system up and running.

 

 Art and technology projects require lots of wiring and creativity.

This is the receiver adaptor I used:
 
Art and Technology project Body Sound Suit used this receiver adapter

I needed to replace the sender adaptor so chose this one:

Replacement sender for my art and technology interactive project
To power the sender, I decided to use this lithium battery:
 
Lithium battery used to power the body sound suit, an interactive art and technology project
 
Depending on the number of sensors used, this battery can last up to four hours on a single charge; optimal battery life is necessary for this project as the suits are used by performers.
 

 Now that I had figured out how to make the system wireless, the fun began – I was ready to build my project.  Of course, I needed additional parts like a cool project case, wire, LEDs, sensors – back online.


I now had everything – I thought - and so began building the suit.  My daughter, who is a talented dancing student, patiently acted as guinea pig.  Everything ran smoothly for now . . . until I had to add the project into a performance costume.

To help me in figuring out how best to add the system to a costume, I turned to my good friend and performance collaborator Heidi Kirchofer.  Heidi is a very gifted performance artist and costume designer so her input was invaluable as I figured out how to turn my cool project into something that would function in a real-life performance.  The system had to function without inhibiting the movements of the dancer – quite a challenge for a system that includes numerous wires, a project box and a good-size battery.

We had a June 2010 performance date as a deadline (more details on the performance are included in the section “The Body Sound Suit in Action”).  It was February when we started talking about this – plenty of time to find a solution.  We decided to use catsuits for the Body Suit costumes during our first performance and figured we would run channels throughout the suits to hold the wiring system.  

However, other issues soon took precedence over the actual construction of the suits – non-technical issues related to the development of a show and technical issues involving the sensors.

Choosing sensors turned into another exercise in experimentation.  The sensor that I originally had in mind to use was the Flex Sensor.  Because of the shape, these would be easy to add to a costume.  However, during rehearsals they worked well for five minutes before breaking in two.  I continued to use these sensors for joints like the neck and hip, but joints at the fingers, elbows and knees required a new solution.

We tried the Force Sensitive Resistor with greater success:

 

This sensor added the interactive piece to the body sound suit

 I wrapped the sensors in my favorite product, duct tape, as an extra support.  I hoped to reduce the stress on the sensors and prevent breakage.  On the first try, I went a little overboard and the performers were barely able to move the sensors.    Through trials, we found a happy medium between flexibility and strength.


Now we were ready to put the system into a suit.  As I wrote before, we assumed we would run it through channels either inside or outside the suit, depending more on our artistic vision rather than the technical needs.  The performers had different ideas on how their suits and sensors should look so I figured it was a great opportunity for me to experiment with various methods.  

One performer sewed channels into a shirt by folding the cloth of the shirt itself and sewing it up to create the channels.  He then sewed a patch onto the back of the shirt to hold the project case.  I installed the system into his shirt and he wore this under his costume.  It worked well except for two problems – extreme overheating during the performance and, though it made it through a couple of performances, during the most recent performance it was ripped in several places and just barely holding together.

The other performers turned to Heidi for solutions.  At first she created an intricate system of channels sewed onto the outside of the costumes using long strips of lycra.  This process was painstaking and time-consuming as the fabric was averse to falling correctly.  The other issue was that, as the visual artist, I really wanted at least some of the costumes to have a good amount of wires visible and hanging out of the suit.

Just a couple of weeks before the show – our cushion of time now completely used up - Heidi hit upon the solution that carried us through the show.  We cut holes in several catsuits and routed the wires – simple and it worked as an initial solution.  We even used tape at the last minute on the day of the show to refine the positioning of sensors and wires.  

Future suits will involve a more complex system; our numerous trials have given us valuable information on the integration of technology and costumes.

 

The Software

I use the open source software “Serial Midi” as a base; this program was developed by Mark Demers.  “Serial Midi” translates the information from the sender into midi notes, which can then be translated into sound by programs like Garage Band that read midi notes.

Because of the nature of open source software, I was able to customize it exactly to my needs by utilizing the program 'Processing'.  As an example of one of the customizations I implemented, I edited the software to automatically set the baud rate to the one used by the Xbees and to choose the first serial port available.  This would simplify set up time during performances, and during a live performance, every second is critical. So the simpler the set up the better.  Without this change, I would have had to set the baud rate manually each time I used the Sound Suits.  Baud rates affect the speed of data communication between the Xbees so they must be exact.

I decided to use Garage Band for the Suit sounds but wanted a more vintage mini-moog sound to emit from the suits.  The program Krystal Synth allowed me to customize these sounds.

 

 Part of the code I used for my art and technology project

The Body Sound Suit in Action

“The Circuit: Beta Version” premiered at “The Tank” in New York City in June 2010.  Excerpts from the Body Sound Suit portion are located below.  The exceptionally, talented performers, in order of appearance, are Jade Soto, Heidi Kirchofer, Edwin Olvera and Joel Melendez.

 

 

 

Body Sound Suit in Action

Watch that video ...