In this part of the lesson the concept of multimodal literacy will be introduced by an interactive learning process. A series of images, some with links to audio, appear in a vertical list. Hovering over each image with a cursor reveals a short text that is hidden 'behind' the image. All the short texts together are part of a larger text about multimodality. After this first presentation of multimodal content, three exercises follow that help the learner make sense of the content.

The idea behind these exercises is that the learner will be able to better grasp the theoretical grounding of the concept of multimodality in the second part of this lesson. As such, these exercises sensitize the learner in a playful m anner, demonstrating the diverse multimodal ways in which information can be presented. As a result, engaging in these exercises results in an increased awareness about non formalized types of knowledge.

It will help better understand why media-theorist Gunther Kress argues that signs as basic units in non-textual communication are the equivalent of sentences as basic units in language. This is key for understanding why students in the 21st century have to complement textual literacy with visual and aural literacy.

Exercise: Hover over the following series of images

The following series of images form the study material on which the three exercises in 1.1.2-1.1.4 will be based. As a preparation for these exercises, look at each image and read each text attentively, trying to make sense of the overall message. Think about the argument that is being made in the text and determine if you are able to overcome the distraction that may be caused by movement, the different sizes and fonts, and the images that are competing for your attention.

Image 1

Image 2

Image 3

Image 4

Image 5

Image 6

Image 7- Click here

Image 9

Click on the first line of text to listen to a soundfile

Image 11

Image 12- Click here for sound

Image 13

Image 14

References

• Arnold, K. (n.d.)Spring garden scene through the open window [Photo]. https://www.publicdomainpictures.net/pictures/50000/velka/spring-garden-window-frame-view.jpg
• Askjem, S. (1959, Spring). Fritz Moen “listens” to radio. [Photo].
  https://commons.wikimedia.org/wiki/File:FritzLytterRadio.jpg
• Erdt, H. R. (1913). Den Berliner Börsen-Courier lesen Mann und Frau [Illustration]. Den Berliner Börsen-Courier Lesen Mann Und Frau. https://commons.wikimedia.org/wiki/File:Hans_Rudi_Erdt_1913_Druck_Hollerbaum_%26_Schmidt_Berlin_Berliner_Börsenkurier.jpg
• IBM Market Insights (2013). Expected wave of data showing the growth of audiovisual data (video, images, audio).
• ID, I. (2016, January 3). Social media icons collection [Illustration]. Ibrahim.ID - CC BY-SA 4.0 , via Wikimedia Commons. https://upload.wikimedia.org/wikipedia/commons/b/bf/Socialmedia-pm.png
• iStockphoto LP. (n.d.). Retro or vintage technical seventies items on colored background [Purchased illustration]. https://www.istockphoto.com/nl/
• Scagliola, S. (April 2017). Clip created with the music app Ditty   that can turn any text into a song that is mixed with random video-shots. It was launched in March 2015, and taken down in October 2015.
• Scagliola, S. (2017, May).Personal handwritten letter [Scan].
• Scagliola, S (June 2021). Screenshot of a text that has been converted into sign language through the web service: https://funtranslations.com/sign-language
• Scagliola, S (June 2021). Screenshot of digitally produced text that has been converted into braille through the webservice: https://www.brailletranslator.org
• Scagliola, S (April 2017). Screenshot of digitally produced text that has been converted into a typewriter font through the web service:
  https://lingojam.com/TypewriterFontTranslator%3AD
• Scagliola, S (April 2017). Screenshot of typed text.
• Szymik, B.(2011, May 09). What's Your Brain Doing?. ASU - Ask A Biologist. Retrieved June 8, 2021 from https://askabiologist.asu.edu/brain-regions
• NASA, & Hart, D. (n.d.). Electrodes used in subvocal speech recognition research [Photo]. https://commons.wikimedia.org/wiki/File:Subvocal_speech_recognition.jpg
• Novikov, A. (n.d.). Set of people of different generations [Illustration]. https://www.dreamstime.com/stock-illustration-set-people-different-generations-vector-icons-male-image51360057