'Events in The World' Research Project, Multimodal Analysis Laboratory, Interactive Digital Media Institute (IDMI), National University Of Singapore

Members:

  • Kay O’Halloran (Department of English Language & Literature)

About this project:

Developing & Using Interactive Digital Media (IDM) for Multimodal Discourse Analysis: Research in the Multimodality Lab, Interactive Digital Media Institute (IDMI), Singapore

The aim of the research in multimodal discourse analysis (MDA) to be undertaken in the Multimodality Laboratory in the Interactive Digital Media Institute (IDMI) at the National University of Singapore is to develop and use prototype IDM software MDA-Events to dynamically model, analyse, search and retrieve meaning from visual images, video texts, IDM sites and 3-D models of reality.  MDA-Events will be based on multimodal social semiotic theory and computer science techniques for multimedia analysis.  MDA-Events will be used to analyse the construction of world events in print, television and digital media with view of investigating the portrayal and orientation to events across cultures, and the relative affordances and constraints of different forms of technology.  The development of MDA-Events to model and analyse event construction is necessary, given the complexity of tracking meaning in dynamic multimodal social semiotic activity.  MDA-Events will involve the development of semiotic grammars for the linguistic, visual and audio tracks, through which the analysis will be entered using specially designed tools and interactive platforms which permit conflation of the layers of analysis. The analysis will proceed via manual, semi- and fully-automated modelling and coding procedures.  An assessment of the potential to fully automate the analytical techniques in MDA-Events will be undertaken.  MDA-Events has the potential to create a new research paradigm for multimodal and multimedia discourse analysis.