Projects and Performances

Moments of Ma Mere l’Oye

a realtime music visusalization for the LA Philharmonic
developed by Ars Electronica Futurelab
in collaboration with ADMAF, the Abu Dhabi Music&Arts Foundation
participating artists (Roland Aigner, Florian Berger, Horst Hörtner, Gerfried Stocker)
performed live at the Walt Disney Music Hall, Feb. 2016

laphil-1-cut

generative realtime graphic
14 individual graphical pieces have been developed for a performance of Ravels “Ma Mere l’Oye” under conductor Esa-Pekka Salonen.

virtual shadow play
In addition to these graphical pieces, the four interludes of Ma Mere l’Oye are visualized with virtual shadow plays. 7 kinect cameras are positioned among the orchestra and create virtual 3D shadows of the musicians and the conductor, which are then animated by the music and projected.

spatial multi screen installation for the Walt Disney Music Hall
7 large projections are being used to display the visual interpretations. A tryptic projection in the back of the orchestra, a duo-screen in the front and two single-screens on the side.

6 graphic-pcs are generating the visuals in realtime based on parameters derived from the live music with custom made sound analysing tools.

the performance at Ars Electronica Festival 2016 in Linz
the production was also performed at the Ars Electronica Festival in Linz with the Bruckner Orchestra under Dennis Russell Davies.


The general concept for the visualization

In the special version for ballet, Ma Mere L’Oye is mainly the story of Sleeping Beauty, a princess which, as we all know, was to be killed by an evil fairy and only by last minute charms of a good fairy instead was sent into a 100 years sleep from which she only could be woken up by the kiss of a prince. That’s the story line which Maurice Ravel spanned in his ballet composition, from the prelude to the apotheosis. He considered the original fairy tale episodes from the piano version, as the dreams, given by the good fairy to the sleeping princess, in order to entertain her during her 100 years of sleep.
So, what would be more appropriate than using this metaphor of dream and deep sleep as the guiding line for the visualization. We take the fairy tale sequences as the REM phases of sleep, in which abstract visualization and associative graphic elements always give some references to the fairy tales. In the “Interludi”, which Ravel composed for the ballet and inserted between the original parts, we emerge from this deep sleep, yet without quite awakening, without fully conscious to come.

The deep dream sequences consist of computer-generated graphics, which are derived and formed in real time from the sound parameters of the orchestra. The prevailing method here is a sort of sound-paninting, based on an elaborate sound analysis and algorithms which bring the actual sound into visual form. In a certain way the orchestra directly moves the digitally created brushes and pens.

The visualizations of the Interludi, on the other hand, are derived from 3D cameras directed directly at the conductor and individual orchestral musicians. Their digital images are used to create non-realistic images, but schematic representations such as shadow fissures or light-drawn outlines in the dark, which correspond in their effect to this moment, in which reality mixes with the last traces of our dream experiences.


How the story and the music come alive in the visualizations

I. Prélude
“Thickets of the Inevitable”

In the Prelude, the narrative strings are introduced with their musical motifs, and immediately after the first awakening and joyfully expectant mood, it is already clear that the story also has dark sides.
The inevitably unfolding destiny of sleeping beauty announces itself and this mood takes up the visualization of the first part.
With the first sounds of the orchestra “awake” also the Screens and in a gray / green dawn dawn pull the first fine strokes, which constrict themselves and to a scrub. Again and again, like creepy bird swarms, cascades of new lines emerge, the order of which is soon overlaid by violent orchestral tones. In the quiet moments of the prelude, this thicket comes to rest, and one can trace the development of the music in the layers and traces of the lines. In the middle triptych, the undergrowth now presents itself as a wave in the surf, which, like frozen, is waiting to collapse in a foamy manner.

prelude (13)

The harsh contrasts and matted lines that dominate the picture, especially on the side screens, deliberately contrast the overly deceptive harmony of the music. And when at the end of the prelude the effervescence of the orchestra announces the arrival of the evil fairy, black spiderlike lines like a mushroom braid overgrow the whole picture.

prelude on wing screens (3)prelude on wing screens (8)

 


II. Premier tableau – Danse du rouet et scène

In the second tableau three different visual motifs are being developed, each dealing with different aspects of the storyline:
“Gambol of the Fairies – En Arriere”
“Spinning Wheel and the Red Fairy”
“Frisking Spindles”


1. “Gambol of the Fairies – En Arriere”
Insect-like geometric creatures appear on the central projection tryptichon, hunt in a flare over the picture surfaces, pursue themselves, sometimes attack themselves, in order to prance the next moment coquettishly.

Danse Critters 04

Danse Critters 01

vvvv 2016-01-31 17-00-52-45

vvvv 2016-01-31 16-59-58-34 heller

2. “Spinning Wheel and the Red Fairy”
On the front screens, the graphics resembles the rotating spindles. Tens of thousands of pixels form in rotation to amorphous nuclei and in their dark red color, the moment when sleeping beauty is stinging on the spindle is picked up.

vvvv 2016-01-31 18-26-59-31

vvvv 2016-01-31 18-26-06-66

3. “Frisking Spindles”
The side screens display very minimal graphics which follow in their movements mainly the melodies of the reed instruments.


III. Deuxième tableau – Pavane de la belle au bois dormant


1. “Quiescent Flow”
While in the first two tableaus the graphics are created out of the overall spectrum of the orchestral sound, in the Pavane the solo instruments come to the fore. At first only on the central projection surface. Fine lines are being drawn, following the sounds of the clarinet, out of which figures arise reminding us of archaic animal skeletons.

Pavane 01-20-53-60

Pavane 01-43-01-36

Pavane 01-51-29-10


2. “Ariel and Oberon”
Following the figurines of the solo instruments, fleeting air ghosts emerge on the side as well as the front screens, gliding through the air in elegant twists and leaving fleeting traces. In the end they also take over the triptych and fadewith the last sounds of the pavanne.

petit poucet 245

petit poucet 242

petit poucet 187


IV. Interlude 1

After this first dream sequence, full of abstract associations, we emerge from the deep REM sleep to the surface but without really getting awake. The conductor and individual orchestral musicians become visible as vague and moving shadow. The real world penetrates our dream world without letting us wake up completely.


V. Troisième tableau – Les entretiens de la belle et de la bête


1. “Landscapes of Ambiguity”
The contrast of beauty and the beast is not interpreted as a discriminatory delimitation but as two opposite yet connected parts of the whole. To this end, a landscape of nature is created: mountains, hills, above it a sky, sometimes gently soft green hills and valleys, then once again sharply dangerous rocks and glaciers.
Several hundred thousand pixels are grouped into 21 brushes, which in real-time, only controlled by the sounds of the orchestra, create these landscapes.
Sound-painting, which has been repeatedly chosen to describe Ravel’s music, is implemented here in the literal sense with the help of the computer.

belle et bete 14-37-00-69

        belle et bete 14-40-59-59

Belle et Bete 03-43-56-78


2. “Magic Mirror”
The eyes of a mysterious being, looking out of the darkness, look at us from the side and front screens.
Particle systems, colored spray mist, which condenses and thus forms and color gradients arise and pass away.

Belle et Bete 12-09-33-44 cutout

vvvv 2016-01-31 23-26-55-39


VI. Interlude 2

Following the course of the scenic sequence of Ravels Balette-suite, we emerge from the deep dreams and the 3D infrared cameras let the orchestra appear on the screens ….


VII. Quatrième tableau – Petit Poucet


1. “Dark Forrest”
The triptych is dominated by an abstracted forest of geometrical elements, similar to tree trunks, from whose tops the higher sound parts of the music always replace fragments, which, recalling the birds of the story, surround the tree crowns.
A gloomy oppressive mood arises when real photos of a nightly forest light appear in the background, mixed in color with torches. The flurring thicket of the geometric shapes always blocks the view of the clearing behind it.

petit 19-20-07-94

petit 23-55-08-24

petit 23-54-43-91

petit 19-18-47-91


2. “Kodamas”
At the beginning we only see fine lines on the side screens. As detached branches and branches against a white sky, they dance to the deep sounds of the orchestra. Into this expanse, the forest ghosts, wounded shaped bodies from thousands of individual leaves, which in the wind of the orchestral sounds weaken and flare up. They watch over the forest and observe the tragic story that develops in it, unmoved and unimpeded
.

petit 50s108

petit dancer 18s212

petit dancer 48s204


VIII. Interlude 3

After the sad disappearance of the little thumbling in the fairy tale forest, it is absolutely dark and only with the first sounds of the third interlude the reality of the concert hall again reaches the surface. In the fine, hand-drawn lines we see the live images of the musicians and the conductor, which oscillate from the darkness.


IX. Cinquième tableau – Laideronnette, impératrice des Pagodes


1. “Laideronettes Island”
Abstract in depiction, but quite concretely referring to the story behind this part, we see inhabitants of Laideronettes island in a joyous swinging dance in honor of their empress.


vlcsnap-2017-01-09-01h01m42s457

Sketched, landscape motifs of the fairy tale island emerge on the side screens, mountains grow out of the blue of the sea and imaginative flowers fly through the air, until the arrival of the Green Snake with the big gong is announced. The dancing couples solidify and shed and thorn-like elements penetrate the flowing green thicket of the side screens.

laideronettes island 01

Laideronette 19-59-16-43

Laideronette 18-03-46-91

With the beginning of the flute motif, movement returns to the scene, the fear goes away, and the joyful dance is resumed.
From the landscape motifs of the side screens, a swarm of colorful birds breaks loose and the orchestra’s last sounds immerse the scenery into a threatening red.


X. Interlude 4

The deep red spreads out from the center over all projection surfaces. But a dream comes to an end and we return to reality for the last interlude. Only a thin silk curtain of fine light is interspersed, and this wraps the movements of conductor and orchestra, soothing the scenery, and letting us dive for the last time into the fairy world of ma Mere L’oye.


XI. Sixième tableau – Le jardin féerique


1. “Swaying in the Wind”
With the gentle beginning of the last part, we dive into a deep and formless dream world. Only the distant light, a sun surrounded by fog, receives us and movements of flowing form and color lie down over all the picture surfaces.
Behind them emerge dusky shrubs. In reference to the Dornenhain, which the prince must overcome in order to arrive at sleeping beauty.

vlcsnap-2017-01-09-01h27m19s706

jardin feerique 15-51-28-93

jardin feerique 15-50-41-94


2. “Marangonis Tears”
On the sidescreens colorful, uplifting elements celebrate the end of one hundred years of sleep and the union of prince and princess.

jardin feerique II 02

jardin feerique II 03


3. “Prince and Princess”
On the central screens, the main motif breaks with the idyll and moves the encounter of prince and princess from the distant fairy world into the beginning of the 20th century. Plays with original photos of the bold conquerors of the air, among many young women. The precursors of a new image of women, as well as precursors of modernism in the Ravel’s music.

vlcsnap-2017-01-09-01h27m52s289

vlcsnap-2017-01-09-01h28m18s039

Organ Concerto / Francis Poulenc

This realtime visualization was part of the concert evening at Walt Disney Music Hall in Feb 2016 and was performed in the first part of the program.
we used 7 large HD-projection screens distributed on five different locations in the big hall. A tryptic projection in the back of the orchestra, a duo-screen in the front and two single-screens on the side.
(The video you can see here is a kind of remix of these different visualisations on one screen)

a commissioned work for the LA Philharmonic
developed by Ars Electronica Futurelab
in collaboration with ADMAF, the Abu Dhabi Music&Arts Foundation
participating artists (Roland Aigner, Florian Berger, Horst Hörtner, Gerfried Stocker)
performed live at the Walt Disney Music Hall, Feb. 2016
LA Philharmonic, Conductor Esa-Pekka Salonen

LA Phil
Esa Pekka Salonen
Ars Electronica Futurelab
ADMAF

about the music

Ge Xu – Antiphony / Chen Yi

Here are two excerpts from the visualisation for the performance at Große Konzertnacht at Ars Electronica 2015.
Brucknerorchestra Linz conducted by Dennis Russell Davies.

We used a super wide panoramic projection with 6 HD projectors side by side (about 50m of screen).

Excerpt of part one – Landscape


Exceprt of part two – Expression

(Sorry for some sound problems of the recording)


Chen Yi
Bruckner Orchestra Linz
Dennis Russell Davies