Thomas W. Malone

Thomas W. Malone

Auto-generated profile, based on publicly available information

Thomas W. Malone

Massachusetts Institute of Technology, Cambridge, MA, USA

Research Focus & Metrics

Academic Career Length

33Years (1994-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
28 October 2024 in Nature Human Behaviour
Verified
Journal Article
Open access
PDF file
MVMichelle Vaccaro
Michelle Vaccaro
AAAbdullah Almaatouq
Abdullah Almaatouq
TMThomas Malone
Thomas Malone
Inspired by the increasing use of artificial intelligence (AI) to augment humans, researchers have studied human–AI systems involving different tasks, systems and populations. Despite such a large body of work, we lack a broad conceptual understanding of when combinations of humans and AI are better than either alone. Here we addressed this question by conducting a preregistered systematic review and meta-analysis of 106 experimental studies reporting 370 effect sizes. We searched an interdisciplinary set of databases (the Association for Computing Machinery Digital Library, the Web of Science and the Association for Information Systems eLibrary) for studies published between 1 January 2020 and 30 June 2023. Each study was required to include an original human-participants experiment that evaluated the performance of humans alone, AI alone and human–AI combinations. First, we found that, on average, human–AI combinations performed significantly worse than the best of humans or AI alone (Hedges’ g = −0.23; 95% confidence interval, −0.39 to −0.07). Second, we found performance losses in tasks that involved making decisions and significantly greater gains in tasks that involved creating content. Finally, when humans outperformed AI alone, we found performance gains in the combination, but when AI outperformed humans alone, we found losses. Limitations of the evidence assessed here include possible publication bias and variations in the study designs analysed. Overall, these findings highlight the heterogeneity of the effects of human–AI collaboration and point to promising avenues for improving human–AI systems.

16 December 2014 in PLOS ONE
Verified
Journal Article
Open access
PDF file
DEDavid Engel
David Engel
AWAnita Williams Woolley
Anita Williams Woolley
LJLisa X. Jing
Lisa X. Jing
CCChristopher F. Chabris
Christopher F. Chabris
TMThomas W. Malone
Thomas W. Malone
Recent research with face-to-face groups found that a measure of general group effectiveness (called “collective intelligence”) predicted a group’s performance on a wide range of different tasks. The same research also found that collective intelligence was correlated with the individual group members’ ability to reason about the mental states of others (an ability called “Theory of Mind” or “ToM”). Since ToM was measured in this work by a test that requires participants to “read” the mental states of others from looking at their eyes (the “Reading the Mind in the Eyes” test), it is uncertain whether the same results would emerge in online groups where these visual cues are not available. Here we find that: (1) a collective intelligence factor characterizes group performance approximately as well for online groups as for face-to-face groups; and (2) surprisingly, the ToM measure is equally predictive of collective intelligence in both face-to-face and online groups, even though the online groups communicate only via text and never see each other at all. This provides strong evidence that ToM abilities are just as important to group performance in online environments with limited nonverbal cues as they are face-to-face. It also suggests that the Reading the Mind in the Eyes test measures a deeper, domain-independent aspect of social reasoning, not merely the ability to recognize facial expressions of mental states.

17 May 2021 in Proceedings of the National Academy of Sciences
Verified
Journal Article
Closed access
PDF file
CRChristoph Riedl
Christoph Riedl
YKYoung Ji Kim
Young Ji Kim
PGPranav Gupta
Pranav Gupta
TMThomas W. Malone
Thomas W. Malone
AWAnita Williams Woolley
Anita Williams Woolley
Significance: Collective intelligence (CI) is critical to solving many scientific, business, and other problems. We find strong support for a general factor of CI using meta-analytic methods in a dataset comprising 22 studies, including 5,279 individuals in 1,356 groups. CI can predict performance in a range of out-of-sample criterion tasks. CI, in turn, is most strongly predicted by group collaboration process, followed by individual skill and group composition. The proportion of women in a group is a significant predictor of group performance, mediated by social perceptiveness.

5 November 2023
Verified
Conference Paper
Open access
ACAlice Cai
Alice Cai
SRSteven R Rick
Steven R Rick
JHJennifer L Heyman
Jennifer L Heyman
YZYanxia Zhang
Yanxia Zhang
AFAlexandre Filipowicz
Alexandre Filipowicz
TMThomas Malone
Thomas Malone
Designers often struggle to sufficiently explore large design spaces, which can lead to design fixation and suboptimal outcomes. Here we introduce DesignAID, a generative AI tool that supports broader design space exploration by first using large language models to produce a range of diverse ideas expressed in words, and then using image generation software to create images from these words. This innovative combination of AI-based capabilities allows human-computer pairs to rapidly create a diverse set of visual concepts without time-consuming drawing. In a study with 87 crowd-sourced designers, we found that designers rated the automatic generation of images from words as significantly more inspirational, enjoyable, and useful than a conventional baseline condition of image search using Pinterest. Surprisingly, however, we found that automatically generating highly diverse ideas had less value. For image generation, the high diversity condition was somewhat better in inspiration but no better in the other dimensions, and for image search it was significantly worse in all dimensions.

11 May 2024
Verified
Conference Paper
Open access
JGJie Gao
Jie Gao
SGSimret Araya Gebreegziabher
Simret Araya Gebreegziabher
KCKenny Tsu Wei Choo
Kenny Tsu Wei Choo
TLToby Jia-Jun Li
Toby Jia-Jun Li
SPSimon Tangi Perrault
Simon Tangi Perrault
TMThomas W Malone
Thomas W Malone
With ChatGPT’s release, conversational prompting has become the most popular form of human-LLM interaction. However, its effectiveness is limited for more complex tasks involving reasoning, creativity, and iteration. Through a systematic analysis of HCI papers published since 2021, we identified four key phases in the human-LLM interaction flow—planning, facilitating, iterating, and testing—to precisely understand the dynamics of this process. Additionally, we have developed a taxonomy of four primary interaction modes: Mode 1: Standard Prompting, Mode 2: User Interface, Mode 3: Context-based, and Mode 4: Agent Facilitator. This taxonomy was further enriched using the “5W1H” guideline method, which involved a detailed examination of definitions, participant roles (Who), the phases that happened (When), human objectives and LLM abilities (What), and the mechanics of each interaction mode (How). We anticipate this taxonomy will contribute to the future design and evaluation of human-LLM interaction.

1 January 1999
Verified
Book Chapter
Closed access
PDF file
TMThomas W. Malone
Thomas W. Malone
RLRobert J. Laubacher
Robert J. Laubacher