Tag Archives: Visual motion -> sound synaesthesia

Sound married to vision is the completely normal and sometimes entertaining way of perceiving the world

I’ve got to laugh on the odd time that I read a description of synaesthesia that makes it sound like some kind of mental disorder or abnormality of sensory perception. Sure enough, synaesthesia concurrents are perceptions of sensory experiences that are not triggered by things happening outside of the mind. They are experiences (not always sensory) triggered by activity happening within the mind (just like the meanderings of your normal constant train of thought). A synaesthete can experience sound as a synaesthesia concurrent triggered by a visual experience (the synaesthesia inducer), and might also experience a visual concurrent triggered by a sound inducer. Coloured music and visual animations that make sound are commonly-reported experiences in people who are, to borrow a phrase from Galton, “sane persons”. It might sound psychedelic to a non-synaesthete, but it is not far at all from normal perception, because life is full of events in which movement or some other visual stimuli is accompanied by a sound sensory stimuli: clapping hands, wind that rustles leaves, lips that speak, impacts that bang, or an explosion that is huge visually, sonically and physically. This pairing of sound and sight is so much a part of normal perception (in humans and other creatures) that it is commonly exploited in live entertainment.

I’ve had the opportunity to work in the past in the live entertainment industry and I also recently enjoyed the rehearsals of the Arcadia musical and special effects spectacular show currently at Elizabeth Quay in Perth. I know that there is a most startling loud roaring sound through the stage speakers that is typically created to coincide with a visual effect of an explosion of flames. It’s like some bloke presses a button somewhere and all hell breaks loose for a second or two. Arcadia uses this flame-roar sound to add sonic spectacle to the flame-thrower, and in the past while working I’ve also heard that sound used in complete isolation from music in a sound check of another spectacular stage show. At the risk of ruining the magic, I’m revealing that the sound that goes with the flames is an artistic artifice. I guess that any real sound that the flame effect makes has been judged to be not sufficiently loud and spectacular enough, and a suitably awesome sound (a recording of what I can only guess) was created to go along with the visual effect of flames from hell. I think this shows just how important crossmodal experiences are to live entertainment shows that are based on spectacular sensory experiences. The sound must equal the visual spectacle.

There are also many other ways in which sound and sight are linked in stage shows and special effects in entertainment. Musicolour lighting effects have been around since my Dad created disco equipment in our lounge using it back in the 1970s, and similar but much more developed lighting effects can be seen in the body of the Arcadia spider. Technology is not always required to artificially marry lighting and sound, as the amazing red and blue man electricity show features electrical discharges that look like tamed lightning that naturally give off a crackling sound along with the white light. But then again, I’m now wondering whether that sound is for real. Anyway, it’s wonderful, mad, sensory fun. I love it!

Postscript January 2017

I think the phenomenon of “quiet fireworks” adds more support to my point that spectacular public entertainment special effects often include the deliberate timing of sound and visual effects to happen at the same time to create a form of artistic synaesthesia, because while fireworks typically have bangs and flashes at the same time, the bang part of the spectacle is not essential or inseparable.

Arcadia Spider

Red man and blue man

Flaming Spider

Synaesthesia-related current and upcoming arts events in Australia

MONA in Tasmania will be revisiting the theme of synaesthesia in Synaesthesia+, a musical, visual and gustatory festival of the psychological phenomenon. It is happening this weekend and tickets will set you back quite a lot.

In Perth, Western Australia PICA have been hosting an exhibition of sound art, What I See When I Look at Sound, featuring the works of artists Lyndon Blue, Lauren Brown, Matthew Gingold, Cat Hope and Kynan Tan. This show will be on until the end of this month and it is free, or at least we didn’t get charged when we went to look and listen to it a while ago.

You might think from considering the title of the exhibition that it might have the theme of synaesthesia, and indeed the works are described each as a “synaesthetic offering”, but actually I believe that the theme of the exhibition, “the relationship between looking and hearing” is actually about binding, which is a broader term that can encompass normal or average sensory perception and also some types of synaesthesia that are similar to or more consciously-experienced variants of normal mental sensory perception. I think this exhibition is about binding more than it is about synaesthesia. If a multi-sensory arts event was “about synaesthesia” I’d expect to see lots of colour and hear music and maybe see or feel letters of the alphabet, or see calendars suspended in space, and maybe even experience smells and flavours. I might look at a “synaesthesia art” painting and as a direct result “feel” motion or “hear” rhythms.The painting Upward by synaesthete artist Vassily Kandinskii or the painting Broadway Boogie Woogie by probable synaesthete artist Piet Mondrian are both pretty clear examples of what I mean by synaesthesia art. I have written about both artists previously in posts at this blog.

Binding is a term used in psychology, the philosophy of mind, neuroscience and cognitive science. It is certainly related to synaesthesia and is central to scientific understanding of synaesthesia as a phenomenon in neuroscience, but it isn’t the same thing. As far as I understand binding is about the perception of the many different sensory characteristics of an object or an event as a unified thing or event. A clear example would be the installation Filament Orkestra by Matthew Gingold. It grabs and holds attention and causes reflection even though the idea is no more complicated than (simple) sound and (plain white) light being presented (or not presented) both at the same points in time. I found the effect to be quite reminiscent of flamenco dancing and tap dancing, which I guess shows how the sensory binding of sight and sound is an engaging effect that is used in a diverse range of art forms, high arts and popular arts, modern and traditional, even including firework displays. Have you ever had the experience of viewing from an elevated location a fireworks display that is happening a distance away, and the wind is blowing in such a direction that the sound waves never reach where you are standing, so that the sight has no soundtrack? It’s the strangest thing to see (and not hear).

According to some online festival programs, tomorrow (Saturday August 16th 2014), as a part of the Perth Science Festival which is a part of National Science Week there will be a free event in the Central Galleries at PICA titled Sounds Symbols and Science at 1.00pm, which will be “a special live concert of “Cat Hope’s End of Abe Sade in the What I See When I Look at Sound exhibition”” and this will somehow involve digital graphic notation, which is a concept that very much overlaps with many synaesthetes’ experiences of listening to music, including my own at times, so I’m happy to categorize this planned event as synaesthetic, which is more than enough to provoke my curiosity.

http://www.pica.org.au/view/Sounds%2C+Symbols+and+Science/1891/

https://www.facebook.com/events/686307634740051/

http://www.scienceweek.net.au/perth-science-festival/

http://www.scitech.org.au/events/1583-perth-science-festival

The Art of Fashion and the Sound of Fashion

I’ve had the opportunity to have a good browse (while the kids are at school) of an exhibition of some interesting works of wearable art. Fashion isn’t one of my major interests, so I’m not sure just how new or original the idea of fashion garments as works of art might be. I’m guessing that the extreme fashion which the pop music icon Lady Gaga has become famous for could be considered wearable art, and I’ve also got to wonder whether this singer-songwriter’s engagement with such extreme originality in an area of visual art is in some way connected with her coloured music synaesthesia. Some synaesthesia researchers believe there is a link between creativity and synaesthesia, a theory that must surely be difficult to test, and they never seem to explain exactly how this connection might work. Living in Perth, Western Australia I doubt that I’ll ever get to view items from Gaga’s wardrobe,  but I did get to see the Art of Fashion exhibition at Lakeside Joondalup Shopping Centre, which is a part of the annual Joondalup Festival, which is organized by the City of Joondalup and is happening this weekend.

Does a synaesthete create differently, and does a synaesthete perceive works of artistic creation in ways that non-synaesthetes do not? I doubt that a clear-cut answer to that question is possible, but I suspect that a synaesthete might experience a more conscious awareness of cross-sensory effects. The unusual coloured asymmetric frill at one hip of a predominantly black dress designed by Kasia Kolikow in the Joondalup exhibition has a full and contrasting appearance which evokes the idea of expansion or air blowing, a movement which would seem odd to me if it were not accompanied by a sound. What type of sound? The transparent, airy frill with its day-glow yellows and salmon pinks (contrasting against the black of the dress titled “Never Sleep Again”) has colours that I have always associated with falsetto singing and other high-pitched musical sounds. This dress whistles. There is another outfit in the exhibition which has a title that brings to mind the notion of sound “Summer Pop Fizz” by Cynthia Chong, but my visual perception of the work  evokes extra-modal motion more than sound. A translation of sound and touch and temperature into a visual expressive art form must have been the origin of this whimsical brightly coloured top and shorts, inspired by ice-cold bubbling lemonade, but it doesn’t give me a chill. When I look at the squiggly shapes on the surface of these garments I see motion typical of the surface of turbulent liquid.

It doesn’t take much thought to figure out why the dress named after the species of fish Chelmonops truncatus designed by April Richards evokes a rhythmic sound, as the scalloped edges in contrasting colours spiralling around the dress are visually striking and highly rhythmic, but it’s less clear to me why this rhythmic sound should be an electronic keyboard sound like something out of a 1970’s pop tune by a girl singer. The idea of a dress that looks a bit like a fish or even a mermaid is perhaps an idea typical of pop culture from a more innocent age, and maybe this is why my unconscious mind makes this association. It’s surprising how noisy an exhibition of fashion garments and jewellery can be, so it is some respite that the one outfit in the group of Celene Bridge’s works on display which makes a noise only whispers. I believe Bridge should have thought twice about naming one of her outfits Leap of the Rabbit, because whenever I looked at it I could not help thinking of the French word “lapin” spoken in the softest whisper, repeated over and over. Everything about this amazing outfit has a soft quality – the fabric looks soft and lustrous, the outlines of the dress are feminine and gentle curves, the gorgeous rabbit-shaped sculptural details at the back of the skirt of the dress are soft curved shapes, the shoulder-hugging limpness of the fabric in the short cape and even the headpiece though grim in theme has curving lines. I think an outfit like this demands to have a name with sound symbolism that sounds as soft as the outfit looks, but sadly the English-language word “rabbit” is all wrong. It is a jagged, hard-sounding word, not appropriate as a name for an animal with a soft pelt. The French have more of a clue. I can think of no animal in the world softer to touch than a rabbit, so I’d say a rabbit deserves to be called a lapin.

I’m a little bit surprised that my unconscious mind has spontaneously offered up a French word to my conscious mind as a comment on the fashion outfit, because I don’t consider myself in any way proficient in the French language. I dropped out of French classes early in year 8 of high school, and year 8 was the extent of my formal teaching in that language, but I suspect that most people have a broader vocabulary in foreign languages than they realise.

The Art of Fashion exhibition will be on display up to the 31st of March 2012 (tomorrow) at Lakeside Joondalup Shopping City. It’s worth a look (and a listen) so don’t wait till it’s too late.

Urban Couture. City of Joondalup. http://www.joondalup.wa.gov.au/Explore/artsandevents/JoondalupFestival/UrbanCouture.aspx

Urban Couture Gallery.  http://www.joondalup.wa.gov.au/Explore/artsandevents/JoondalupFestival/UrbanCouture/UrbanCoutureGallery.aspx

Are you a synaesthete? There are tests

Recommended  –  The Synesthesia Battery from the laboratory of Dr David Eagleman at Baylor College of Medicine.     http://www.synesthete.org/     (you will need to set aside a bit of time for this, and it only identifies some types of synaesthesia)

A Spanish synaesthesia test? http://www.artecitta.es/ARTECITTA/sinestesia/test/index.html

A screensaver which causes motion to sound synesthesia in some people, from New Scientist magazine’s YouTube channel  http://www.youtube.com/watch?v=hLhuRIeHj6Q

Revised Test of Genuineness (TOG-R) – 2006 journal paper about it  http://www.autismresearchcentre.com/docs/papers/2006_Asher.pdf     http://www.cortexjournal.net/article/S0010-9452(08)70337-X/abstract

A synaesthesia screening questionnaire used by researchers at the University of Cambridge  https://docs.google.com/spreadsheet/viewform?formkey=dDFwb0tWem5kYW9IOHE2RXMzNXpvV1E6MQ#gid=0

Some thoughts after viewing a simple little painting – Upward by Wassily Kandinsky

A while ago I had the opportunity to view first-hand some famous paintings when the Guggenheim collection toured Australia. It was a wonderful opportunity to see some artworks that I had only ever been able to see in books, which isn’t the same thing as seeing a picture displayed on a wall. I probably shouldn’t look at paintings or listen to music while knowing too much about the painter or the composer, because when an artistic creation of a synaesthete evokes a synaesthetic response from my mind, I’m then left wondering if this is due to the power of expectation rather than a discovery that I would have made in any circumstance. I already knew the painter Wassily Kandinsky, a pioneer of abstract art, was a synaesthete, and after looking at Kandinsky’s Upward for a while, it started making noises at me. Why shouldn’t a painting that has a type of movement as it’s title evoke a bit of visual movement -> sound synaesthesia? As my eye followed the curved lines of the painting upward and downward in a bumpy, interrupted cycle, the times when the curved line met an abrupt end against a straight line caused a “bonk” or “boink”  type noise.

This movement synaesthesia wasn’t the only thing about this painting that made me wonder about atypical neurological processes. As I looked at this painting I realised that I was seeing examples of types of things that my mind seems to be unusually good at perceiving, or unusually focused upon. A face can be discerned in this painting, if you use your imagination a bit (blue eye, red lip, black lip). Last year I shocked myself by getting some perfect scores in two face recognition tests. After doing some research, I’ve come to the conclusion that this is due to some hyperconnectivity in my brain and it is surely related to my synaesthesia. In this deceptively simple painting I can also see some letters of the alphabet (graphemes), one inverted. I appear to be the carrier of a gene “for” being unusually capable in the areas of reading, writing and languages, all areas of learning that involve letters and alphabets.

There is something else grapheme-related that I find very interesting about in this painting. I have personification synaesthesia, and one of the two letters of the alphabet in this painting is the one that is the most strongly personified in my mind; the letter “E”. It is also the most commonly used letter in the English language. I’m not sure if this is related. I do know that how commonly a letter of the alphabet is used is in the English language does influence another type of synaesthesia – grapheme -> colour synaesthesia. The most commonly used letters of the alphabet tend to be associated with the more basic and primary colours, while the less commonly used colours of the alphabet like “V” or “Z” tend to be linked to secondary and more complex colours like purple or gray. Anyhow, to me the letter “e” (upper and lower case), the most common letter of the alphabet looks like the happiest letter of the alphabet. In my mind it looks like a face with a big smile that is facing toward the right, somewhat like a smiling face in profile. Do synaesthesia researchers know that personified letters and numbers can have a physical orientation as well as having characteristics like gender, age and personality? If they don’t, they should.  In the painting Upward the capital letter “E” is one half of a platform that the face sits on top of, and the face and the “E” both have bits beside them that balance them out, with the face and the “E” facing in opposite directions, giving the picture a kind of balance. Looking at the way the elements in this painting are arranged, I find it very hard to believe that Kandinsky didn’t see a face in the letter “E” the way I do. Did Kandinsky have ordinal linguistic personification?

Another thing that is noticeable in this painting is the play with colours. There are colours varying in saturation and colours blended in graded adjacent segments and similar colours grouped together. Colour was clearly very important in Kandinsky’s work. When I was a child I was fascinated with colours, and I loved to make pictures with the large metal trays of watercolour paints that I was given. I believe a study has found that synaesthetes have an unusual ability to discriminate colours.

I can see a face, a facial expression, some graphemes and a focus on colours in this painting. Is it just a coincidence that faces, graphemes and colour are things that are processed in one part of the brain, the fusiform gyrus? Was there something unusual about Kandinsky’s fusiform gyrus? We already know that he experienced colour-related synaesthesia, so we know something was “up” with Wassily’s brain. Was Kandinsky more creative because of his synaesthesia? A lot of people believe the two traits are connected. Did Wassily Kandinsky have a mind that was unusually focused on, or perceptive of, or mixed up about colours, faces, visual motion and letters (graphemes)? I’m just not motivated enough to wade through his voluminous writings about art theory to find out. I only know that Kandinsky never tired of writing about his synaesthesia and other esoteric matters. I think we would have had a lot in common.

Upward by Wassily Kandinsky 1929 http://www.wassilykandinsky.net/work-203.php

YouTube video that can evoke hearing motion synaesthesia http://www.youtube-nocookie.com/v/hLhuRIeHj6Q?fs=1&hl=en_US

Wikipedia contributors Fusiform gyrus. Wikipedia, The Free Encyclopedia. http://en.wikipedia.org/w/index.php?title=Fusiform_gyrus&oldid=419089814