Experiments in Visual Music

01.31

Since ancient times artists have longed to create with moving lights a music for the eye comparable to the effects of sound for the ear. (Moritz)

Still from Kreise (Circles) by Oskar Fischinger, 1933Still from Kreise (Circles) by Oskar Fischinger, 1933

.
Read Towards and Aesthetics of Visual Music by William Moritz (online)
Explore The Center for Visual Music (site)
Read at least sections 2.1 & 2.2 of this paper about Mary Hallock-Greenewalt

.

mary_hallock_01

Mary Hallock-Greenewalt was an early composer and performer of audiovisual music. Around 1910, she created some of the first films designed as musical accompaniment by painting on strips of film, a technique explored later by Len Lye and Norman McLaren, among others. Hallock-Greenewalt also developed the Sarabett, a color organ for musical performance. We will look at color organs and audiovisual instruments again later this semester.

Paul Friedlander distinguishes between 3 types of visual music…

  1. Visual Music is a means of converting music to images using a system or set of rules which can be implemented as machine or computer code.
  2. Visual Music is a means of expressing music in visual form requiring the active involvement of an artist, designer or director to interpret the music and find the means to express it visually.
  3. Visual music is about creating visual relationships which change over time.

Take a look at a few pieces with these descriptions in mind… comment about these pieces and/or the idea/genre of visual music more generally.

McLaren describes his process in a 1951 BBC interview:

I draw a lot of little lines on the sound-track area of the 35-mm. film. Maybe 50 or 60 lines for every musical note. The number of strokesto the inch controls the pitch of the note: the more, the higher the pitch; the fewer, the lower is the pitch. The size of the stroke con- trols the loudness: a big stroke will go “boom,” a smaller stroke will give a quieter sound, and the faintest stroke will be just a little “m-m-m.” A black ink is another way of making a loud sound, a mid-gray ink will make a medium sound, and a very pale ink will make a very quiet sound. The tone quality, which is the most difficult ele- ment to control, is made by the shape of the strokes. Well-rounded forms give smooth sounds; sharper or angular forms give harder, harsher sounds. Sometimes I use a brush instead of a pen to get very soft sounds. By drawing or exposing two or more patterns on the same bit of film I can create harmony and textural effects.



Synchromy N2 Abstarct film 1935/36, by Mary Ellen Bute


From By Brakhage: The Act of Seeing . . .

While Brakhage’s films are replete with other oppositions, on the levels of both style and subject matter, his editing is not merely—or even mostly—oppositional. Most frequently, a sequence will be followed by a kind of lateral move; rather than answering one set of visual forms with its opposite, he understood that opposition is a form of affirmation, because it accepts the terms of what is being opposed, and instead sought to shift the very grammar of the film’s discourse.


Gondry actually plotted out the synchronization of the song on graph paper before creating the video, eventually “modelling” the scenery with oranges, forks, tapes, books, glasses and tennis shoes. (wikipedia)


Driven by the primary principles of datamatics, but objectively deconstructing its original elements – sound, visuals and even source codes – this new work creates a kind of meta–datamatics. Ikeda employs real–time programme computations and data scanning to create an extended new sequence that is a further abstraction of the original work. The technical dynamics of the piece, such as its extremely fast frame rates and variable bit depths, continue to challenge and explore the thresholds of our perceptions. (from Ikeda’s Website)


The Night After I Kicked It from Jane Cassidy on Vimeo.


With the heart of a video synthesizer and the brain of a videogame console, Ming Mecca is the first of its kind: an ontological toy, a videogame easel, and a love letter to all things retrofuture. Designed for use in Eurorack format synthesizers but built to meet the demands of even the most seasoned micro-galactic interlopers, Ming Mecca modules will take your rig where no rig has gone before. (from the Special Stage systems website)

18 Thoughts

  • Susan Grochmal

    It would be interesting to know what kind of sounds people imagine for Black Ice. I imagine something like Vordhosbn by Aphex Twin (https://youtu.be/-iEl7OKrLGI). The song does sound nice with the glitchy color thing going on. But the video does speak for itself

    Ikeda’s video was really cool, immersive like living in a computer- I like the parts where it goes black with lines and does the doorway thing

    The Night After I Kicked It was like being hypnotized , all the sounds made so much sense with the video, synesthetic combination

    And the video for Ming Mecca, that was a great exciting trailer made me want to use it (very cool idea)

    In visual music, the rhythm is important as it is in music, in these videos there is a lot of movement and pattern, Synchromy is slower than the others, pretty&sculptural

  • corrinne

    While reading the Moritz article, I imagined animation/film to play a really intense role in these videos and was surprised to see how minimal the visual work was. I definitely prefer the McLaren piece because the environment created is unique and functions as its own universe but also think The Chemical Brothers video is really creative. I don’t know though, it’s probably just personal taste but I really crave something more dynamic…not a narrative but maybe some sort of repetitive symbol or something. I’m wondering when animation begins to overpower sound though~~~eh, but I also think it’s really powerful when there’s a super dynamic visual piece with a really cool lo-fi soundtrack or something

  • Lona

    First off, I really enjoyed a) the diversity of sounds/visuals between each piece and b) that it is arranged in quasi chronological order. Its nice to see how visual music has progressed.

    I liked the minimal design of McLaren’s piece.. sticking with one shape and a static background color makes it so that you have to focus on the movement of the dots. I also really appreciate his v constructed process. I can respect the principle of this piece (the process is creative).

    Star Guitar blew my mind. Woah. Since this was the fourth video in the queue, I found that I was searching for the patterns pretty hard by this point and when I found them in this piece.. wow just wow!!! Its SOOO satisfying when the images and sounds line up and I love how he utilized different objects to represent sound. Two of my favorite moments were (1) at 1:26 when the envelope filter on the synth (?) matched with the transformation from light to dark and (2) at 1:40 when humans are first introduced into the composition and it also happens to be when we first hear vocals. :)

    Although the Data Patch Installation was v hypnotizing to watch, I felt like it was too distracting for me to have to experience through a video. I just feel like its probably so different in person that analyzing it through video seems obsolete imo. I wonder how the sound is different when you are really there… I also wondered if the right and left walls are supposed to represent something in pan space maybe?

    Overall, I think I was most drawn to the pieces that use a system/set of rules BUT also warrant active involvement from the artist to determine how sound should be visualized.

  • Gillian

    I was really discouraged by how Synchromy No. 2 didn’t, in my opinion, stand on its own. I watched the video on mute and did not enjoy it nearly as much as I would the song by itself. The questions of “what is visual music” and indeed “is visual music possible” remain for me unanswered. However, I want to bring up the importance of the human voice. I realized that one factor keeping me interested in the musical piece was the voice of the singer. (keep in mind please that I have studied visual art and film on a theoretical level, but not music, so this might be old hat) So my interest in the music was not only based in sound but in a personal connection to the singer, as I could identify with the expression in the voice. I suppose the visual equivalent of that would be dance.

    When it comes to abstract, i.e. not human-produced, sounds, I still draw blanks on what an exact, or what the closest, visual analog would be.

    • Gillian

      I SPOKE TOO SOON. “BLACK ICE” IS SUPER ENGAGING WITH NO SOUND. SHOUTOUT STAN BRAKHAGE, ALWAYS A FAVE. But what about slow visuals? Slow music is engaging, but the slow visuals of Synchromy No. 2 were hard to pay attention to. Is this just an issue of my attention span?

      • Amber

        yeahhh I think there is something about black and white which also makes us less likely to keep looking at something (from our modern every screen every color everywhere perspective). Also yeah it real didn’t seem like the audio and visuals were matching at all to me.

    • Sophie Fields

      Agree about Synchromy N2…The visuals didn’t provide an important effect or seem like they had a strong correlation to the music being played, in my opinion. Or maybe the “Moods through the Ear” part rings true > “moods for the eye”. Maybe it’s because the visuals are in black and white? Not sure.

    • Michelle

      I find it really interesting that it was the human voice in Synchromy No. 2 that you felt engaged with, because for me it was the complete opposite. Once the human voice entered, I felt like there was no room left for me to be present in the space of the film, like I was just witnessing the experience of another rather than experiencing something myself. I found the abstract sounds/images to be more inviting and engaging because I wasn’t distracted by trying to identify with things that I had previous experiences with/notions about. Like in your follow up comment below, I also found Black Ice to be really enveloping without sound!

  • Sophie Fields

    “Dots” was really fun to watch, especially considering the thought that went into the illustrations/animation that accompanies the noises. If I didn’t have McClaren’s disclaimer about how exactly the visuals go with the sounds, however, it wouldn’t have been nearly as interesting for me. Just wondering what other people think about this.

    I really enjoyed the Star Guitar video. This one, (differing from the Synchromy N2 vid) left me with the impression that the sounds were on par with the visuals, both being equally important and engaging in this case.

    I wish I could see the “Ryoji Ikeda data.path installation” in real life! It seems really engrossing!

    Echoing Corrinne, the minimal nature of some of the videos, mostly “The Night After I Kicked It” was off-putting for me. I found myself bored by them, and I imagined that there was potential for much more visually stimulating images/illustrations/animations. However, maybe it’s just a matter of personal taste or how the artist themselves is visualizing the music.

    Ok I know I said I enjoyed a lot of these but definitely LOVED the Ming Mecca trailer the most!!!!! What the heck ! The moody colorful lighting is the *best* part. It made me really excited…where can I buy this? Also I want to use a synthesizer so bad. This is the kind of exciting visuals and music I’m TALKING about. Yeah

  • Amber

    —Dots: I do NOT understand how this was made. In the description it says that he drew lines on the soundtrack strip of the film.. But I don’t get that, then he also goes on to describe things that seem pretty metaphorical (like the relationships of the colors to sounds, which is doesn’t seem like that could be generated by something he did on film). It is incredible how synchronized all of the sounds and shapes are especially considering 1940 (!!!!!!)
    —Synchronomy: The video combined abstract and “real” images interestingly, but I had trouble connecting the music to the image. It would be fun to go back and make a new score for this.
    —Black Ice: Another scenario where I have no idea how this was made and would really like to know. The fact that it had no sound caused me to make up my own soundscapes for it in my head, but at the same time this was disturbing because I kept trying to discern images that weren’t actually there from the chaos. It felt like I should have been glimpsing into something but I could never catch it.
    —Star Guitar: took me a while to realize this was not real video. This seems really labor intensive, but I think I would be more intrigued if a real life video came first and then the sound was coordinated to work with what was happening. But in this scenario I guess I don’t know which came first either way, but I was assuming it to be the song.
    —Ikeda installation: WOW!!!!! I love this. I wish I had been able to watch it on a bigger screen, or better yet actually be in this hallway. These are the most interesting audiovisual “environments” to me—spaces people can actually be totally immersed in experiencing. This is truly “Screen as portal”. Again, how did this happen? Are these two giant screens, or projections from behind?
    Also I am really curious about how this installation Is experienced in real life. Are there multiple people walking through the hallway? Can you see others? Is there talking and rustling of clothes? In its ideal form I feel like this is something which should be experienced completely alone, but in an “art” context, it is not really feasible to have each person inside alone for ten minutes. This is crazy.
    -Cassidy: It’s hard to explain why but I feel like the “texture” of these sounds ad images go better together than most of the pure “videos” we have seen.

  • Michelle

    I personally found the pieces that contained traces of the physical world to be less immersive than the others. For example, when I watched “Dots,” “Black Ice,” and “The Night After I Kicked It” I really felt like I was experiencing an environment of sound and image, whereas the human voice in “Synchromy” and the landscapes in “Star Guitar” distracted me as I felt I created narratives in my head from these elements. The more abstract pieces seemed to me to accomplish creating music for the eye more so than the others, which (while I did enjoy them as well) came across more as songs with lyrics, if that makes sense.

  • Meesh

    A comment that helped me contextualize these videos was one in Mary Hallock Greenwalt’s paper: “Why shouldn’t light help the song sing?” I found this simple phrase relevant because it seemed to encompass the relationship between light and audio. While it’s easy to put together that audio enhances video and video enhances audio, I found it helpful to think of audio and light/visuals as a healthy friendship. They can exist independently, but, when combined, they are a strong force. The dynamics behind these audiovisual works is not simply the processes in which they are created, but they are intrinsic to the piece itself (if that makes sense?!). I especially enjoyed the date.path installation in Madrid because of it’s ability to create a physical environment. The sound, as well as the visual, created a space that would not work as well if the elements were not combined. Though it’s almost hard to tell where exactly this is and what kind of space it is, the light and audio give the space context (and obscure it at the same time!)

  • Isabella

    I found the simple pattern to the McLaren video to be refreshing. The sounds and images were fairly minimal, yet they worked well together to keep me intrigued. This video showed me the power that a straight forward concept without too many embellishments can have. Videos do not necessarily need over the top audio and visuals in order to be effective.

    …going off of that, The Night After I Kicked It was one of my favorite videos, and it left me confused and astounded. I really enjoyed the sounds in this video. They were crisp, articulated, and accompanied the visuals beautifully. I am very interested in how the visual aspect of this video was created. What program did Jane Cassidy use? These images appear to be computer generated and I am curious as to how the various textures and patterns were created. Watching Black Ice was also a mesmerizing experience and I would like to know how these images were made as well.

    The super 80’s vibe of the Ming Mecca Trailer was also very appealing to me in both a comical and an entirely serious way. The sounds and the video quality were nostalgic, as I often wish that I could have experienced this time period. I am also very interested in what exactly a Ming Mecca is/does. I’m pretty sure that I want one.

  • Matt

    I particularly liked Star Guitar, and since I didn’t see it get a lot of attention, I’d like to unpack it a little bit. While, besides the Wagner piece I think it is the most “traditional music”, I don’t think that this means that it is any less formed in partnership with the visuals. I think what is most powerful about the video is that it takes an environment very familiar to most people and harnesses it to demonstrate how the human mind searches for pattern and meaning. Whenever I am on a train or riding in a car, I usually end up looking out the window at the surroundings and click my teeth in rhythm as objects go by. So for me, this video was a very effective commentary on how the brain, especially when deprived of stimulation, will try to ascribe pattern to its surroundings. I think this video resonated with me so much because its setting brought me mentally back to times spent hungering for stimuli. Instead of merely being a quirky premise for a fun video, I think Star Guitar is a very interesting commentary on the brain and boredom.

  • Ori

    After reading about a lot of the early work on “Visual Music” I’m still not convinced that a visual piece without audio accompaniment can have the same “replay value” as musical pieces without visual accompaniment. While black ice was definitely visually interesting, it is hard to imagine a setting/situation where I would desire to watch such a piece over and over again.
    In a way, I feel like music is a more primal and universal translation of emotions (well, actually I stole that from Nietzsche who borrowed from Schopenhauer) while images are often context-specific (i.e. not universal) translations of ideas/concepts. In this way, music gives images depth while images give music clarity of meaning. Music can stand alone because universal emotions can stand alone. Ideas, on the other hand, need a grounding in particular contexts, be it their usefulness for uniting a country or simply the passions of their beholders, to be sustained. While this distinction is definitely not clear cut— of course, you can convey concepts with audible speech and emotions in pictures– I think it is useful to consider the complementary strengths of each medium when mixing and melding them, especially if you are trying to reverse the relation in interesting ways. I just wonder if Mary Hallock-Greenewalt’s project, with its disregard for musical accompaniment, was doomed to fail from the start.

  • Liza

    In reading the Moritz article, although pretty outdated, I think there are some important take aways. Specifically when he talks about gaining a sense of design, color, etc. and he asserts that no machinery can “teach” you this sense. I think this is important for us to remember when going forward in making our own work: knowing technique isn’t simply enough, in my opinion, and I think Moritz would agree. He posits that there might be rules of Visual Music, but those rules aren’t clearly outlined. Going forward and looking at this topic from a contemporary perspective, I think we can ask ourselves in the 20+ years since the publication of this article if there are terms or rules of visual music that we can meditate on when creating such work. I’m not sure that I know the answer for this, but I think it’s important to discuss/ponder.

    In general, I’m really excited by the concept of visual music, but I think it’s really important to grasp what this means and also recognize that this is flexible. I’m not even sure if I know what I mean by that, but it seems pretty nebulous but I also think that some of the language surrounding this topic are trying too much to pin it down.

    • Liza

      I will say though that I love the relationships that visual music creates and I think (echoing Moritz) it is important to think about it from a inter-disciplinary mindset.

  • Will

    Paul Friedlander’s three different ways of categorizing visual music was very interesting. It’s important to consider the discipline with which a creator is converting sound to image. The first method of making visual music involves the application of a strict set of rules that dictate how qualities of sounds correspond to qualities in visuals. The second method is centered around the vision of an artist or designer who interprets the music. The third method is more abstract and has no direct relation to music. With the third method of visual music, each viewing can be interpreted differently. Mary Hallock-Greenewalt’s “Sarabet” is a color organ that converts color into sound. The color organ allows music to be generated from visuals. Unlike “Symphony No2” in which the music existed first and the visuals were designed later.

    Stan Brakhage’s “Black Ice” is abstract enough that it could match many different types of music.

thoughts?