METAMORPHOSIS AND MIND
Animated metamorphosis is a unique visual phenomenon – a unique perceptual event for a viewer, able to conjure unique narrative and conceptual implications both within an animated experience, and for the medium of animation as a whole. My prior work as animator and researcher has included both theoretical and practical work with animated morphs, particularly using digital sculpture and CGI for creating morphing objects. This paper first describes the unique nature of animated metamorphosis, and then explains a preliminary test using brain-computer interface technology (BCI) to create, or to trigger a morph of a digital object through mental intention. While early experiments have failed to reliably produce the desired effect, this contribution clarifies the probable causes and directions for refining the system.
Complementing the basic ability of animation as a space of movement within the frame, Dan Torre describes animated metamorphosis as animation’s ‘other’ movement, conveying the concepts of growth and evolution (Torre, 2017). This ‘other’ movement appears at the dawn of animation with early examples such as Emile Cohl’s Fantasmagorie (1908) demonstrating that even in its simplest forms, animation has a unique potential to display transformation. Metamorphosis is so fundamental to the medium of animation, that it has been referred to as “the constituent core of animation itself” (Wells, 2015).
For the purpose of this contribution, I would consider even the fundamental ‘squash and stretch’ principle of animation to be a simple type of metamorphosis, as objects change through distortion of form, rather than simple change of position, and this distortion through time tells us something about the nature of the entity, its character, or its behaviour. More complex forms of metamorphosis can include fully integrated ‘realistic’ objects that transform from fully resolved, even humanoid forms into other entities entirely, for example in the perennial example – the T1000 in Terminator 2 (1991). In all cases, the visual morph always relates more than is shown directly – cartoon squash and stretch indicates the rubbery potential of cartoon physics, while the T1000’s adaptations stand in contrast to the singular identity of John Conner, deliberately highlighting the power of the individual in the machine-threat trope.
Not dissimilar to the use of metaphor in literature, the animated morph can induce a unique experience of conceptual transfer whereby the conceptual attachments and connotations of one identity are inherited by another. But the visual morph is more explicit than the metaphor; when a morph occurs, Popeyes hands are not merely like hammers, they are hammers[i]. And this moment of transformation of hands into hammers and arms into anvils is a strangely compelling conceptual experience.
Reflecting on the elasticity in Disney’s Alice In Wonderland as Alice grows and shrinks, Eisenstein describes reacting to this unique visual behaviour:
“What’s strange is not the fact that it exists.
What’s strange is that it attracts!
And you can’t help but arrive at the conclusion that a single, common prerequisite of attractiveness shows through in all these examples: a rejection of a once-and-forever allotted form, freedom from ossification, the ability dynamically assume any form.
An ability that I’d call ‘plasmaticness’”
(Eisenstein, 1986)
The ability to elude static identity is not mere visual trickery, but an important device that allows animation to operate almost purely on the level of concept, enhancing its fantastic nature and encouraging the suspension of reality, transferring this fluidity from the screen space, to the conceptual space of the viewer (see Lepot, 2013). At times, the conceptual transfer is an obvious device; as Merlin battles Madam Mim in a contest of shape-shifting, the relative physical advantages and characteristics of the animal kingdom are brought forth, such as strength, cunning, resilience and speed[ii]. Merlin defeats Mim by transforming into a tiny germ – a disease able to defeat her newly acquired dragon form. This is a metaphorical victory of knowledge over brawn, a conceptual victory that enhances Merlin’s character development and the themes of the film and the mythology of King Arthur.
As morphing pervades all aspects of animate potential, it can include transformations not only of a discrete object or character into another, but also transformations of the entire screen space and the dissolution of figure and ground relations (Pierson, 2015). The disorienting experience of this radical type of transformation is noted by the ‘pink elephants on parade’ in Dumbo, who equate animation with intoxication, and flirt with the fourth wall, noting that “…seein’ things you know that ain’t can certainly give you an awful fright” (Armstrong et al., 1941).
PERCIEVING MORPHS
The conceptual instability and play with metaphor and connotation shifting is used by animators either explicitly or intuitively as a type of visual, metaphorical game. On the other side of this game is the viewer of the morphing image, who experiences not only an interesting conceptual device, but the unique perceptual experience described by Eisenstein as strangely attractive. There are few naturally occurring visual experiences that present similar opportunities for the direct witnessing of physical or visual transformation in this way. Perhaps molten lava, changing cloudscapes or distorted reflections in the surface of moving water come to mind as counter examples, but the fluid shifting from one entity into another is at least atypical.
This atypical transformational movement defies efficient mental classification. It resists the chain of visual processing where typically, once detected, perceived and classified, objects are filed into mental schemata, a process that accelerates future mental reference by attaching identity properties to the object-concept (Solso, 2001)[iii]. This tendency of categorisation has effects beyond mere efficiency; it informs our sense of the structure of the world as being constituted of predictable objects and their properties. Some of the ‘uniqueness’ of the morphing experience then is likely due to expectation breach – we do not expect that the entities in view will transform into entirely new forms, with new sets of concept attachments in the schematic reference system of the mind.
The morph demands ongoing reconsideration of its being, its ontological status, and its behaviour. In the words of Vivian Sobchack, animated morphing “destabilises historically dominant western metaphysics of essences, categories and identities” (Sobchack, 2000b) and this creates a new metaphysics of animation (Sobchack, 2000a), one based on the transformational potential of the medium. So it is concurrently visual, conceptual and metaphysical in effect.
If an encounter with animated metamorphosis is actually a unique perceptual experience, does this encounter generate or evoke a unique mental event? In 1865, it was theorised that “Every psychical event corresponds to a physical event and vice versa” (Mach, 1965), an idea known as ‘Mach’s principle of equivalence’. This principle implies that if witnessing an entity transform is a unique experience, there will be a corresponding, and potentially unique mental occurrence. Might this occurrence be detectable?
DETECTING A MORPHING THOUGHT
A number of practical approaches could be pursued to attempt this detection, for example through neuro-imaging (such as FMRI), comparing results of a subject viewing image sequences to the viewing of morphing images, and noting the differences. This type of speculative testing is unfortunately prohibitively expensive. As an animator with an interest in making morphing images, I was also drawn more to the possibility that there is a unique mental event that could, once defined, actually be used to create (or at least trigger) a morphing image.
To this end, a brief experiment was devised, a brain computer interface (BCI)[iv] system. These systems are often used to control prosthetics or to interface with computer systems for typing, directional control, selections and a wide range of other basic functions, especially for users who are unable to use traditional systems (like keyboards, for example).
The BCI includes a detection device – in this case an electroencephalogram (EEG) headset and amplification hardware that can detect mental activity by measuring electrical frequencies at defined locations on the exterior of the user’s head. The BCI system is trained by capturing stimulus responses, then using the signature of certain thought patterns to evoke pre-defined visual outputs on a computer display system. In this case, the input stimulus, and the outputs are versions of a simple, digitally rendered spheroid which can be either squashed, or stretched. To summarize the intent of this preliminary test in its most simple terms: this was an attempt to train a computer system to recognize when the user intends the ball to be squashed, and when it should be stretched, and to evoke this response through detecting those intentions. If this is possible, then the next step could be to attempt to train the system to detect a different concept signature and generate the corresponding animated morph as an output. For example, a 3D model that could morph into an apple/banana.
This test was based on an existing BCI paradigm known as the ‘motor imagery’ paradigm (Brunner et al., 2012) which gathers data from the motor cortex area of the brain. Signals from this area are comparatively easily detected and relate to the movement of the body. Differences in signal between left and right limb movement are often used in these systems because the differences in left/right limb movement are detectable due to being hemispheric distribution of that brain activity. The method of gathering signals in response to motor-equivalent stimulus and using that response pattern to infer subsequent signals is known as the ‘Graz’[v] system.
The test used the open-source software platform OpenViBE (2020) designed for EEG processing which includes a series of modules for Graz Motor Imagery BCI, which were re-built to use squash/stretch, rather than left/right as both training stimulus and result. Th protocol consists of 4 stages, each of which serves a particular role in the interpretation of brain signal into visual output[vi].
- Signal Check
This stage confirms the signal is properly acquired, checking for known signal triggers, such as blinks and jaw clench.
- Training Data Acquisition
Presents stimulus to user, usually randomised alternations of a left or right arrow, in this case alternations of squashed and stretched versions of the sphere. The user concentrates on the direction or concept and EEG responses to the stimuli are recorded for a few seconds for 8-10 channels, located over the motor cortex areas.
- Classifier Trainer
This stage processes the stimulus response data and produces a configuration file, defining the individual mental ‘signature’ for each stimulus.
- Real Time Feedback Testing
This is the goal stage, used once the classifier is trained. In this stage, the same 8-10 motor cortex channels are monitored and when a correspondence to the trained signature is detected, a command is sent to the system to execute left/right (or in this case, squash/stretch).
PRELIMINARY RESULTS
The result of these experiments was disappointing. The system seems to work at some moments, and at other times seems to switch between the squash and stretch image seemingly at random. There are a number of areas that have contributed to the poor outcome of the experiment. First, the principle of EEG BCI, while fairly robust, is quite sensitive and very imperfect. Users must remain very still, and highly concentrated both during the training phase, and the testing phase. Many users are simply unable to use this system effectively, and this test subject (the author) may be one. There is some evidence for this, since the BCI works reliably in left/right mode when the hand is actually moved, but is unreliable (~65%) when using only mental intention. This could be tested by having others test the system, but this involves considerable investment, as a user improves their results over time. It may also be the case that some refinements can be made to the training algorithms to improve the result.
While more complex movements are possible, the Graz Motor Imagery paradigm primarily focusses on left/right differentiation. The areas of the motor cortex are hemispherically lateralised – one can detect which hand is being imagined by detecting activity on one side of the brain, rather than the other. Squash and stretch however, do not necessarily lend themselves to this localisation. It may be that there is a detectable signature, but that much more resolution (localised detection) would be needed. There may of course be no such detectable signature, with the holistic and highly complex patterns of thought being simply too complicated to detect and reproduce reliably – as a phenomenon that evokes complex conceptual attachments, there will not be a reliable ‘area’ for more complex morphs, so the idea is limited to the most basic movements anyway.
Future possibilities include more artistic interpretations of the brain imagery into morphing forms. It is possible to use a wide range of signals as inputs for real-time animation of any number of features of a 3D mesh. However this moves away from the concept of isolating the phenomenon of metamorphosis as a unique, discreet and reproducible mental event. There may also be other aspects of animation that can be generated by BCI systems, most obviously for abstract or generative digital animation.
[i] See (Sparber & Tendlar, 1944)
[ii] See (Reitherman et al., 1963)
[iii] This is a contested principle, and by necessity this comment is too brief. The “serial bucket brigade” view of perception has been criticized as an over-simplification of a holistic and multi-directional process involving mind, body and world in inseparable and ongoing relations.
[iv] (Brunner et al., 2012) provides an accessible overview of a range of BCI methods.
[v] ‘Graz’ refers to the University of Graz, Austria, where this model was developed.
[vi] Other platforms take slightly different approaches that link these stages together for convenience, or utilise machine learning rather than some of these separated training stages, but the principle is similar.
REFERENCES
Armstrong, S., Ferguson, N., Jackson, W., Kinney, J., Roberts, B., Sharpsteen, B., & Elliotte, J. (1941, October 31). Dumbo [Animation, Drama, Family, Musical]. Walt Disney Productions, Walt Disney Animation Studios.
Brunner, C., Andreoni, G., Bianchi, L., Blankertz, B., Breitwieser, C., Kanoh, S., Kothe, C. A., Lécuyer, A., Makeig, S., Mellinger, J., Perego, P., Renard, Y., Schalk, G., Susila, I. P., Venthur, B., & Müller-Putz, G. R. (2012). BCI Software Platforms. In B. Z. Allison, S. Dunne, R. Leeb, J. Del R. Millán, & A. Nijholt (Eds.), Towards Practical Brain-Computer Interfaces (pp. 303–331). Springer Berlin Heidelberg. https://doi.org/10.1007/978-3-642-29746-5_16
Cameron, J. (1991, July 3). Terminator 2: Judgment Day [Action, Sci-Fi]. Carolco Pictures, Pacific Western, Lightstorm Entertainment.
Cohl, E. (1908). Fantasmagorie.
Eisenstein, S. (1986). Eisenstein on Disney (J. Leyda, Ed.; A. Y. Upchurch, Trans.). Seagull Books.
Lepot, C. (2013). Stop motion: From plastic to plasmatic cinema. 107–121.
Mach, E. (1965). On the effect of the spatial distribution of the light stimulus on the retina. Mach Bands, 253–271.
OpenViBE (3.0.0). (2020). [Computer software]. Inria Hybrid Team. http://openvibe.inria.fr/
Pierson, R. (2015). Whole-Screen Metamorphosis and the Imagined Camera (Notes on Perspectival Movement in Animation). Animation, 10(1), 6–21. https://doi.org/10.1177/1746847715570812
Reitherman, W., Geronimi, C., & Hand, D. (1963, December 18). The Sword in the Stone [Animation, Adventure, Comedy, Family, Fantasy, Musical]. Walt Disney Productions, Walt Disney Animation Studios.
Sobchack, V. C. (Ed.). (2000a). At the Still Point of the Turning World. In Meta-morphing: Visual transformation and the culture of quick-change (pp. 131–158). University of Minnesota Press.
Sobchack, V. C. (Ed.). (2000b). Meta-morphing: Visual transformation and the culture of quick-change. University of Minnesota Press.
Solso, R. L. (2001). Cognition and the visual arts (5. print). MIT Press.
Sparber, I., & Tendlar, D. (1944, May 26). The Anvil Chorus Girl [Animation, Comedy, Short, Family]. Famous Studios.
Torre, D. (2017). Animation: Process, cognition and actuality. Bloomsbury Academic.
Wells, P. (2015). Understanding animation (2nd ed). Routledge.