Skip to main content

When multimodal speech speaks a thousand images and words

In today's communication landscape, multimodal speech, based on text, emojis, links, pictures and more arguably augments the nature of our messages 

As part of a literature review I've been working on the use of speech acts on social media, I came across the distinction that authors of a study make between "Speech and Image Acts" in branding messages on social media. 

Having for sometime settled for the idea that speech acts are multimodal in today's hybrid polymedia system, I wander whether the need to distinguish between speech and image acts is necessary. 

Multimodal speaks a thousand images and words:
replacing the old adage (Created with ChatGPT)

The way we communicate online defies simple categorization. A single social media post can be a demand, a plea, an informative statement, and a source of visual delight – all at once. 

Does it make sense, then, to stick with the traditional distinction between "speech acts" (focused on words) and "image acts"?  

While methodologically impressive, the study by Villarroel Ordenes and colleagues let me to think such distinction might be limiting our understanding.

The Study's Insight

In the study in question, the authors found that consumer sharing of brand messages isn't just about the linguistic content. 

Assertive speech acts (offering information) coupled with visually engaging images were far more likely to be shared than directives (demands for action), even if those directives were visually appealing. 

This suggests a complex interplay: the factual nature of the message and its visual presentation together create something compelling and share-worthy.

The Challenge of Multimodal Communication

The article I reference invites us to question whether we can truly isolate speech acts residing only in the words when our online world is so visually driven. 

Shouldn't we be focusing on the combined way in which images, text, and even the layout of a post perform actions on us? 

Think of how a meme uses humor, a dash of text, and a recognizable image format to go viral. None of those elements would have the same impact alone.

Why This Matters

Deeper understanding: Analyzing messages as multimodal speech acts lets us see how different elements reinforce, complement, or even contradict one another. We grasp why certain posts succeed and others fall flat.

Practical Applications: This approach has value for anyone creating content online. Want to design an effective campaign? Consider how your words and visuals work together to persuade, inform, or entertain - although I will need to consider this in relation to my ongoing research on healthcare messaging (more soon).

A Work in Progress

Admittedly, moving to a multimodal framework has challenges. How do we neatly code and analyze the vast array of content forms? Still, to truly grasp the persuasive power of digital communication, we need tools that reflect its inherently messy, multimedia nature.

While the answer may not be straightforward, we can still say with confidence that the old adage that "an image speaks a thousand words" has now been replaced by multimodal speech — if only because it takes us beyond images and words.

Labels:

Comments

Post a Comment

Popular posts from this blog

Rethinking Blood Donation: The Economics, Messaging, and Challenges

Blood transfusion is a critical medical procedure, yet the approach to securing a safe and sufficient blood supply involves complex economic, communicative, and logistical challenges which are difficult to understand when the messaging around it is haunted by catastrophic precedent such as the one that destroyed the lives of thousands in the UK in the 1970s and 1980s.  Reflecting on these aspects reveals the importance of treating blood as a public good , rather than a commodity, central to preventing future crises like the one the news media will abundantly be reflecting when findings from the Infected Blood Inquiry are finally published. The Power of Messaging in Blood Donation Campaigns Effective communication in blood donation campaigns is crucial to ensuring a stable blood supply. Research indicates that the nature of these messages significantly influences public perception and willingness to donate.  Traditional slogans like "Save Lives" are common, but recent studies ...
Post a Comment
Read more

Decoding NHS’s tweets with speech act theory and the encoding/decoding model

 In the era of social media, organizations like the Newcastle upon Tyne Hospitals NHS Foundation Trust are leveraging platforms like Twitter to communicate with publics in multifaceted ways. A fascinating case in point is how the NHS trust used Twitter to communicate about  #GeordieHospital, a documentary series  filmed on various of its premises and broadcast on Channel 4. Analysing these tweets through the lens of speech act theory is a fascinating if laborious task which is of value to tackle one of the many challenges of understanding human communication in the digital age. NHS digital speech acts are multifunctional ( created via OpenArt ) Breaking tweets into speech acts Speech act theory, a cornerstone of linguistic philosophy, breaks down communication into three types of acts: locutionary (the act of saying something), illocutionary (the intent behind saying something), and perlocutionary (the effect of saying something). In the context of the NHS trust's tweets,...
Post a Comment
Read more