In Emovere, the bio-data sensors would be part of the extended touch, providing the performer with the ability to break the physical limits of their body and become part of a whole together with the scenography. This is where our creative objective is introduced, under the following premise:
The extension of corporality to the stage space through media transforms this space in a new organism.
Emovere explores movement from its origin. The brain sends orders, which travel in the form of electrical impulses through our nervous system. Once these impulses reach the muscle, the electrical variations that generate the movement are captured by the sensors. At this moment, the impulses are transformed into bio-data, creating an extension of the dancer’s movement towards the computers that generate the sound and light used in the scenography. These elements compose a new bio-technological system: dance connected to sound and light.
This system works in deep codependency. The variations in sound and light determine how far the movements of the performers can go. There are some movements where the muscle is not necessarily flexed enough to generate data. For this reason, the dancers had to rehearse connected to the sensors for several months in order to recognize the movements that generated good results. Therefore, the way in which they move in the stage space and their dance are affected because their corporality has extended towards mediality. Between the performer and the stage space, a network of stimuli is created that crosses from the biological to the digital in order to subsequently transform into something physical-audiovisual. The performers become the center of this new stage organism, and the sound and projections behave as new organs that react to the impulses sent by the brain. The four performers become creators of their own stage environment.
When managing visual and sound elements, the performer shows symptoms that are similar to the phenomenon called synesthesia. Synesthesia is a condition possessed by some human beings, which under certain conditions induces them to assimilate different senses in the same perceptive act. For example, mentally visualizing colors that appear spontaneously when they hear a melody. In Emovere, the process is different; dancing and emotion generate bio-data, which translates into sound and image. Therefore, it is a digital synesthesia.
“The perception of some fact excites the mental affection called the emotion, and this latter state of mind gives rise to the bodily expression.”
William James and Lange (in Bloch, 2007)
The Visual Design
The graphic aspect of the performance piece starts with an investigative process, mainly aimed at the idea of this new stage organism and the work related to dance and emotions. The human body is taken as the main reference, and three creative exploration lines are designed:
Line A: The representation of the human body through history.
Line B: The perception of the human body.
Line C: The functioning of the human body
Based on these three creative exploration lines, visual experiments are conducted with combinations of digital and photographic textures. First we work on the direct relationship between bio-data and visual results, creating textures and motifs that react to the movement of the dancers. This creation is a technical procedure that involves a connection between different software: Max/MSP, Quartz Composer (Apple Inc., 2015), Isadora (Troikatronix, 2015) and Adobe After Effects (AE) (Adobe Systems Inc., 2015).
Emovere’s visual design uses neural networks, electrical impulses and muscle tissues as an analogy in order to feed from a visual universe appropriate for this stage organism. It takes this connection as an inspiration to create textures and environments that accompany the dance.
We can divide Emovere’s visual design in three experimentation stages:
1. Reactive Graphics
Based on the visual research of bodily textures, we experimented with y Max/MSP. Max/MSP received the bio-data from the computer that previously cleaned the signal and then sent it to QC, where compositions were generated based on muscle fibers and bone structures.
Results:
After the sessions, it was determined that the reactive graphics was too distracting. It is already exhausting for the performers to pay attention to what they create in terms of sound without giving them the responsibility of creating the visual environment. So we determined that the graphic animation would be reactive and not interactive like in the case of the sound environment.
2. Reactive Animation
Using the experimentations generated in QC and dance recordings, we started to compose animations in After Effects (AE), creating fibers, organic networks and organisms. Then these animations were loaded in Isadora, which was also connected to Max/MSP by OSC, so that their speed, coloring and transparency, among other attributes, could be modified by the movements of the dancers.
Results:
We concluded that the conjunction of movement, reactive sound and reactive image provided a result that was too mechanical. So the animation took on a new role within the piece. We decided that the reactive image would be used only in certain moments when the stage, sound and visual composition can highlight their synesthetic dimension so it doesn’t become recurrent or mechanical. The visual projection acquired a role of lighting setting within the piece, without discarding the use of bio-data to modify these atmospheres.
3. Environmental animation (final application)
The design was reconsidered and oriented to a graphics based on the chiaroscuro technique. In order to set an atmosphere for the entire piece, we decided to zenithally project on the entire stage space. Environments were composed in AE and then processed in Isadora so the animations could vary, in real time, their rhythm, duration and position according to certain movements of the performers. The idea was to build environments that referred to the interior organics of the human body, neural networks, cellular multiplication, muscle fibers, bacteria and arterial cavities.
The projection is done with two zenithal devices on the entire dance surface, with dimensions of 9 x 6 meters. They are positioned in such a way that the performers are always under the beam of light of one or another. The audience can see another group of spectators on the other side of the stage with whom they occasionally exchange glances or witness reactions to the same thing that they are witnessing, turning them into accomplice observers of this performance. Little clothing is used in order to highlight the use of technology, as the intention is to show the cables and sensors used in the piece. Images are projected on the dance floor, strengthening the stage composition through video-mapping. A visual narrative is added that differentiates from what is communicated by the body or the sounds, creating a setting that is transversal to these other two stage components.
Reflection
The project Emovere shows the integration of artistic disciplines, technology and science for the use of bio-data as a creation resource. The data extracted from the organism is translated in order to create sound and visual environments in real time. This forces the team to face various situations and creative dilemmas that weren’t considered, which are the product of the bio-technological combination.
The bio-data is a variable source of parameters, which provides a degree of spontaneity and dynamism to the graphic result, so it is always different depending on the state the performer is in. External factors such as fatigue, stress and the personal issues of the dancers affect each and every rehearsal, as well as future presentations, so the results from one day cannot be deemed absolute because the next day they will change to some extent. The design must be conceived with a margin for error and chance. For this reason, the visual operator must maintain a degree of manual control over the projection. In some cases, it happens that in specific moments of the piece in which the bio-data must trigger a visual event, the performer’s gesture does not reach the necessary threshold, so this threshold must be forced in Max/MSP or the event must be directly triggered manually in Isadora so the fluency and coordination is not lost.
The performance goes through a stage narrative that initially does not make the technology evident. As the piece develops, the technology becomes an additional element of the scenography, which is lost in the conjunction and becomes a part of the sound and visual creation. Then the creation becomes visible – absorbing the technological factor – making it part of the stage organism. Visible Technology > Invisible Creation – Visible Creation > Invisible Technology.
In Emovere, the graphic product is granted a different use. The use of animations designed with the purpose of creating textures that evoke movements and sensations is fostered. The use of projection is revised and placed at the service of lighting. Using the versatility of the graphic animation software (AE), the possibilities for creation are widened in order to place the performers in lighting atmospheres that are different from the traditional ones, which are also used as a graphic support in order to demonstrate and strengthen their emotional state in accordance with the narrative. Because it is part of the modulation through bio-data, the graphics is fully integrated into the scenic system, leaving it – to some extent – at the mercy of spontaneity.
The synesthetic element occurs in selected moments of the piece in order to demonstrate the existing interconnection between dance, music and visuals. In some moments, it becomes necessary for the spectator to directly perceive what is happening at a technical level in order for them to find an interdisciplinary sense to this. On the contrary, if this demonstration occurs in every moment of the piece, the effect becomes mechanical and monotonous, as was seen in the experimentations; if there is a graphics overload, the effect becomes invisible and a little overwhelming.
