Skip to main content

When multimodal speech speaks a thousand images and words

In today's communication landscape, multimodal speech, based on text, emojis, links, pictures and more arguably augments the nature of our messages 

As part of a literature review I've been working on the use of speech acts on social media, I came across the distinction that authors of a study make between "Speech and Image Acts" in branding messages on social media. 

Having for sometime settled for the idea that speech acts are multimodal in today's hybrid polymedia system, I wander whether the need to distinguish between speech and image acts is necessary. 

Multimodal speaks a thousand images and words:
replacing the old adage (Created with ChatGPT)

The way we communicate online defies simple categorization. A single social media post can be a demand, a plea, an informative statement, and a source of visual delight – all at once. 

Does it make sense, then, to stick with the traditional distinction between "speech acts" (focused on words) and "image acts"?  

While methodologically impressive, the study by Villarroel Ordenes and colleagues let me to think such distinction might be limiting our understanding.

The Study's Insight

In the study in question, the authors found that consumer sharing of brand messages isn't just about the linguistic content. 

Assertive speech acts (offering information) coupled with visually engaging images were far more likely to be shared than directives (demands for action), even if those directives were visually appealing. 

This suggests a complex interplay: the factual nature of the message and its visual presentation together create something compelling and share-worthy.

The Challenge of Multimodal Communication

The article I reference invites us to question whether we can truly isolate speech acts residing only in the words when our online world is so visually driven. 

Shouldn't we be focusing on the combined way in which images, text, and even the layout of a post perform actions on us? 

Think of how a meme uses humor, a dash of text, and a recognizable image format to go viral. None of those elements would have the same impact alone.

Why This Matters

Deeper understanding: Analyzing messages as multimodal speech acts lets us see how different elements reinforce, complement, or even contradict one another. We grasp why certain posts succeed and others fall flat.

Practical Applications: This approach has value for anyone creating content online. Want to design an effective campaign? Consider how your words and visuals work together to persuade, inform, or entertain - although I will need to consider this in relation to my ongoing research on healthcare messaging (more soon).

A Work in Progress

Admittedly, moving to a multimodal framework has challenges. How do we neatly code and analyze the vast array of content forms? Still, to truly grasp the persuasive power of digital communication, we need tools that reflect its inherently messy, multimedia nature.

While the answer may not be straightforward, we can still say with confidence that the old adage that "an image speaks a thousand words" has now been replaced by multimodal speech — if only because it takes us beyond images and words.

Labels:

Comments

Post a Comment

Popular posts from this blog

Overcoming Media Reluctance: How a Diverse Messaging Strategy Can Boost Vaccination Advocacy

As the UK grapples with the resurgence of whooping cough, it becomes increasingly important for public health messaging to adopt a nuanced and multi-strategy approach to vaccination appeals. In recent weeks, the UK has witnessed a notable surge in whooping cough cases, a severe infection particularly dangerous to children and pregnant women. Public health messaging has strongly emphasized vaccination as a crucial measure to combat this infection. Naturally, this is the case in the official communication of the UK Health Security Agency. BBC's coverage highlights worrying rise of cases However, a quick examination of the mainstream media coverage reveals what could be described as a hesitancy to include direct vaccination appeals, as can be surmised from a BBC’s report  which emphasized the threatening aspect, and a local news report which highlighted its regional spread . While mentioning a decline in vaccination rates, the coverage avoided direct calls to vaccinate and mere
Post a Comment
Read more

Organ transplants: why so many people are put off donating

Organ transplants: why so many people are put off donating irinabdw/Shutterstock (I'm republishing this article from The Conversation partly as an opportunity to acknowledge the progress that the UK has made towards reducing the number of cases in which families refuse to donate the organs of loved ones after they die. With the consent/authorisation rate going up 2% in 2018/19 to 67%, the country is much closer to the 80% target that would dramatically improve transplant outcomes, per the latest   Organ Donation and Transplantation Activity Report . But there is more work to be done) Gabriel Moreno Esparza , Northumbria University, Newcastle and Stephen Clark , Northumbria University, Newcastle It’s well known that there’s a worldwide shortage of organ donors. More than 100,000 organ transplants have taken place around the world every year since 2008 but this is way below what’s needed . In the UK , for example, figures
Post a Comment
Read more