With the widespread availability and pervasiveness of artificial intelligence (AI) in many application areas across the globe, the role of crowdsourcing has seen an upsurge in terms of importance for scaling up data-driven algorithms in rapid cycles through a relatively low-cost distributed workforce or even on a volunteer basis. However, there is a lack of systematic and empirical examination of the interplay among the processes and activities combining crowd-machine hybrid interaction. To uncover the enduring aspects characterizing the human-centered AI design space when involving ensembles of crowds and algorithms and their symbiotic relations and requirements, a Computer-Supported Cooperative Work (CSCW) lens strongly rooted in the taxonomic tradition of conceptual scheme development is taken with the aim of aggregating and characterizing some of the main component entities in the burgeoning domain of hybrid crowd-AI centered systems. The goal of this article is thus to propose a theoretically grounded and empirically validated analytical framework for the study of crowd-machine interaction and its environment. Based on a scoping review and several cross-sectional analyses of research studies comprising hybrid forms of human interaction with AI systems and applications at a crowd scale, the available literature was distilled and incorporated into a unifying framework comprised of taxonomic units distributed across integration dimensions that range from the original time and space axes in which every collaborative activity take place to the main attributes that constitute a hybrid intelligence architecture. The upshot is that when turning to the challenges that are inherent in tasks requiring massive participation, novel properties can be obtained for a set of potential scenarios that go beyond the single experience of a human interacting with the technology to comprise a vast set of massive machine-crowd interactions.

Recently, citizen involvement has been increasingly used in urban disaster prevention and management, taking advantage of new ubiquitous and collaborative technologies. This scenario has created a unique opportunity to leverage the work of crowds of volunteers. As a result, crowdsourcing approaches for disaster prevention and management have been proposed and evaluated. However, the articulation of citizens, tasks, and outcomes as a continuous flow of knowledge generation reveals a complex ecosystem that requires coordination efforts to manage interdependencies in crowd work. To tackle this challenging problem, this paper extends to the context of urban emergency management the results of a previous study that investigates how crowd work is managed in crowdsourcing platforms applied to urban planning. The goal is to understand how crowdsourcing techniques and quality control dimensions used in urban planning could be used to support urban emergency management, especially in the context of mining-related dam outages. Through a systematic literature review, our study makes a comparison between crowdsourcing tools designed for urban planning and urban emergency management and proposes a five-dimension typology of quality in crowdsourcing, which can be leveraged for optimizing urban planning and emergency management processes.