10 January 2023 in Group Decision and Negotiation
Crowdsourcing holds great potential: macro-task crowdsourcing can, for example, contribute to work addressing climate change. Macro-task crowdsourcing aims to use the wisdom of a crowd to tackle non-trivial tasks such as wicked problems. However, macro-task crowdsourcing is labor-intensive and complex to facilitate, which limits its efficiency, effectiveness, and use. Technological advancements in artificial intelligence (AI) might overcome these limits by supporting the facilitation of crowdsourcing. However, AI’s potential for macro-task crowdsourcing facilitation needs to be better understood for this to happen. Here, we turn to affordance theory to develop this understanding. Affordances help us describe action possibilities that characterize the relationship between the facilitator and AI, within macro-task crowdsourcing. We follow a two-stage, bottom-up approach: The initial development stage is based on a structured analysis of academic literature. The subsequent validation & refinement stage includes two observed macro-task crowdsourcing initiatives and six expert interviews. From our analysis, we derive seven AI affordances that support 17 facilitation activities in macro-task crowdsourcing. We also identify specific manifestations that illustrate the affordances. Our findings increase the scholarly understanding of macro-task crowdsourcing and advance the discourse on facilitation. Further, they help practitioners identify potential ways to integrate AI into crowdsourcing facilitation. These results could improve the efficiency of facilitation activities and the effectiveness of macro-task crowdsourcing.
28 December 2012 in KI - Künstliche Intelligenz
1 June 2011 in Corporate Reputation Review
28 June 2025 in Group Decision and Negotiation
Complex problems such as climate change pose severe challenges to societies worldwide. To overcome these challenges, digital innovation contests have emerged as a promising tool for idea generation. However, assessing idea quality in innovation contests is becoming increasingly problematic in domains where specialized knowledge is needed. Traditionally, expert juries are responsible for idea evaluation in such contests. However, experts are a substantial bottleneck as they are often scarce and expensive. To assess whether expert juries could be replaced, we consider two approaches. We leverage crowdsourcing and a Large Language Model (LLM) to evaluate ideas, two approaches that are similar in terms of the aggregation of collective knowledge and could therefore be close to expert knowledge. We compare expert jury evaluations from innovation contests on climate change with crowdsourced and LLM’s evaluations and assess performance differences. Results indicate that crowds and LLMs have the ability to evaluate ideas in the complex problem domain while contest specialization—the degree to which a contest relates to a knowledge-intensive domain rather than a broad field of interest—is an inhibitor of crowd evaluation performance but does not influence the evaluation performance of LLMs. Our contribution lies with demonstrating that crowds and LLMs (as opposed to traditional expert juries) are suitable for idea evaluation and allows innovation contest operators to integrate the knowledge of crowds and LLMs to reduce the resource bottleneck of expert juries.