This article demonstrates how a digital environment offers new opportunities for transforming qualitative data into quantitative data in order to use data mining and information visualization for mixed methods research. The digital approach to mixed methods research is illustrated by a framework which combines qualitative methods of multimodal discourse analysis with quantitative methods of data mining and information visualization in a multilevel, contextual model that will result in an integrated, theoretically well-founded, and empirically evaluated technology for analyzing large data sets of multimodal texts. The framework is applicable to situations in which critical information needs to be extracted from geotagged public data: for example, in crisis informatics, where public reports of extreme events provide valuable data sources for disaster management.
10 November 2015 in Learning, Media and Technology
This paper proposes a pedagogical approach for teaching and learning critical thinking through multimodal analysis – that is, ‘multimodal analysis for critical thinking'. The approach builds on the conviction that students require competencies that move beyond traditional notions of literacy to meet the changing demands posed by media and technology in the twenty-first century. The approach takes a social semiotic view toward critical multimodal literacy, which aims to provide students with an analytical metalanguage for the systematic analysis of multimodal texts and videos. The pedagogical approach is facilitated through the use of purpose-built software applications with comprehensive analytical frameworks designed to support the systematic teaching and learning of multimodal analysis, with a view to developing the critical literacy skills needed for life in the digital age. The potential efficacies of the approach are illustrated via the exposition of software functionalities and the sample analyses of a printed text with image components, and a short advocacy video.
18 October 2016 in Terrorism and Political Violence
This article presents a mixed methods approach for analysing text and image relations in violent extremist discourse. The approach involves integrating multimodal discourse analysis with data mining and information visualisation, resulting in theoretically informed empirical techniques for automated analysis of text and image relations in large datasets. The approach is illustrated by a study which aims to analyse how violent extremist groups use language and images to legitimise their views, incite violence, and influence recruits in online propaganda materials, and how the images from these materials are re-used in different media platforms in ways that support and resist violent extremism. The approach developed in this article contributes to what promises to be one of the key areas of research in the coming decades: namely the interdisciplinary study of big (digital) datasets of human discourse, and the implications of this for terrorism analysis and research.
This study employs a multimodal social semiotic approach to the analysis of text and image relations in material produced by the violent extremist organisation known as Islamic State in Iraq and al-Sham (ISIS). The study focuses on iconisation, where meanings are condensed and interpersonally charged through ‘bonding icons’ which embody the organisation's world view and values. A sample of issues in the online magazine Dabiq produced by ISIS are analysed using Systemic Functional Multimodal Discourse Analysis (SF-MDA). The ISIS world view is shown to be internally cohesive, based on a narrow, fundamentalist interpretation of selected Islamic scripture and violently opposed to any other world view. To represent, synthesise and justify its values and world view, ISIS uses bonding icons constructed from combinations of artefacts, supported by references to selected Islamic scripture. The study is a prelude to more detailed investigation of bonding icons in materials produced by ISIS, the attraction of SIS to potential mujahideen and ‘copy-cat’ jihadist groups and the recontextualisation of materials from Dabiq in different media platforms. Such studies would provide deeper insights into the workings of organisations such as ISIS, and facilitate the further development of multimodal social semiotic approaches to image and text relations.
The paper discusses the challenges faced by researchers in developing effective digital interfaces for analyzing the meaning-making processes of multimodal phenomena. The authors propose a social semiotic approach as the underlying theoretical foundation, because interactive digital technology is the embodiment of multimodal social semiotic communication. The paper outlines the complex issues with which researchers are confronted in designing digital interface frameworks for modeling, analyzing, and retrieving meaning from multimodal data, giving due consideration to the multiplicity of theoretical frameworks and theories which have been developed for the study of multimodal text within social semiotics, and their impact on the development of a computer-based tool for the exploration, annotation, and analysis of multimodal data.
15 August 2015 in Educational Technology Research and Development