Objective: Three different team configurations are compared with the goal of better understanding human-autonomy teaming (HAT). Background: Although an extensive literature on human-automation interaction exists, much less is known about HAT in which humans and autonomous agents interact as coordinated units. Further research must be conducted to better understand how all-human teams compare to HAT. Methods: In an unmanned aerial system (UAS) context, a comparison was made among three types of three-member teams: (1) synthetic teams in which the pilot role is assigned to a synthetic teammate, (2) control teams in which the pilot was an inexperienced human, and (3) experimenter teams in which an experimenter served as an experienced pilot. Ten of each type of team participated. Measures of team performance, target processing efficiency, team situation awareness, and team verbal behaviors were analyzed. Results: Synthetic teams performed as well at the mission level as control (all human) teams but processed targets less efficiently. Experimenter teams performed better across all other measures compared to control and synthetic teams. Conclusion: Though there is potential for a synthetic agent to function as a full-fledged teammate, further advances in autonomy are needed to improve team-level dynamics in HAT teams. Application: This research contributes to our understanding of how to make autonomy a good team player.
This study aims to better understand trust in human–autonomy teams, finding that trust is important for team performance. A Wizard of Oz approach was used to simulate an autonomous agent team member, in a remotely piloted aircraft system research environment, to study the relationship between trust and team performance in human–autonomy teams. Results show that (1) there are lower levels of trust in the autonomous agent in low-performing teams compared with medium- or high-performing teams; (2) there is a loss of trust in the autonomous agent over time across low-, medium-, and high-performing teams; and (3) both low- and medium-performing teams indicated lower levels of trust in their human team members. These findings indicate that trust in a teammate (autonomous or human) is associated with team performance and that trust may evolve over time irrespective of team performance.
5 July 2021 in IEEE Transactions on Human-Machine Systems
There are many unknowns regarding the characteristics and dynamics of human-AI teams, including a lack of understanding of how certain human-human teaming concepts may or may not apply to human-AI teams and how this composition affects team performance. This article outlines an experimental research study that investigates essential aspects of human-AI teaming such as team performance, team situation awareness, and perceived team cognition in various mixed composition teams (human-only, human-human-AI, human-AI-AI, and AI-only) through a simulated emergency response management scenario. Results indicate dichotomous outcomes regarding perceived team cognition and performance metrics, as perceived team cognition was not predictive of performance. Performance metrics like team situational awareness and team score showed that teams composed of all human participants performed at a lower level than mixed human-AI teams, with the AI-only teams attaining the highest performance. Perceived team cognition was highest in human-only teams, with mixed composition teams reporting perceived team cognition 58% below the all-human teams. These results inform future mixed teams of the potential performance gains in utilizing mixed teams' over human-only teams in certain applications, while also highlighting mixed teams' adverse effects on perceived team cognition.
7 November 2018 in IEEE Transactions on Human-Machine Systems
In the past, team coordination dynamics have been explored using nonlinear dynamical systems (NDS) methods, but the relationship between team coordination dynamics and team performance for all-human teams was assumed to be linear. The current study examines team coordination dynamics with an extended version of the NDS methods and assumes that its relationship with team performance for human-autonomy teams (HAT) is nonlinear. In this study, three team conditions are compared with the goals of better understanding how team coordination dynamics differ between all-human teams and HAT and how these dynamics relate to team performance and team situation awareness. Each condition was determined based on manipulation of the pilot role: in the first condition (synthetic) the pilot role was played by a synthetic agent, in the second condition (control) it was a randomly assigned participant, and in the third condition (experimenter) it was an expert who used a role specific coordination script. NDS indices revealed that synthetic teams were rigid, followed by experimenter teams, who were metastable, and control teams, who were unstable. Experimenter teams demonstrated better team effectiveness (i.e., better team performance and team situation awareness) than control and synthetic teams. Team coordination stability is related to team performance and team situation awareness in a nonlinear manner with optimal performance and situation awareness associated with metastability coupled with flexibility. This result means that future development of synthetic teams should address these coordination dynamics, specifically, rigidity in coordination.
1 August 2020 in International Journal of Human-Computer Studies