9 November 2024 in Philosophical Studies
The dominant practice of AI alignment assumes (1) that preferences are an adequate representation of human values, (2) that human rationality can be understood in terms of maximizing the satisfaction of preferences, and (3) that AI systems should be aligned with the preferences of one or more humans to ensure that they behave safely and in accordance with our values. Whether implicitly followed or explicitly endorsed, these commitments constitute what we term a preferentist approach to AI alignment. In this paper, we characterize and challenge the preferentist approach, describing conceptual and technical alternatives that are ripe for further research. We first survey the limits of rational choice theory as a descriptive model, explaining how preferences fail to capture the thick semantic content of human values, and how utility representations neglect the possible incommensurability of those values. We then critique the normativity of expected utility theory (EUT) for humans and AI, drawing upon arguments showing how rational agents need not comply with EUT, while highlighting how EUT is silent on which preferences are normatively acceptable. Finally, we argue that these limitations motivate a reframing of the targets of AI alignment: Instead of alignment with the preferences of a human user, developer, or humanity-writ-large, AI systems should be aligned with normative standards appropriate to their social roles, such as the role of a general-purpose assistant. Furthermore, these standards should be negotiated and agreed upon by all relevant stakeholders. On this alternative conception of alignment, a multiplicity of AI systems will be able to serve diverse ends, aligned with normative standards that promote mutual benefit and limit harm despite our plural and divergent values.
29 August 2017 in Scientific Reports
Rumor spreading can have a significant impact on people’s lives, distorting scientific facts and influencing political opinions. With technologies that have democratized the production and reproduction of information, the rate at which misinformation can spread has increased significantly, leading many to describe contemporary times as a ‘post-truth era’. Research into rumor spreading has primarily been based on either model of social and biological contagion, or upon models of opinion dynamics. Here we present a comprehensive model that is based on information entropy, which allows for the incorporation of considerations like the role of memory, conformity effects, differences in the subjective propensity to produce distortions, and variations in the degree of trust that people place in each other. Variations in the degree of trust are controlled by a confidence factor β, while the propensity to produce distortions is controlled by a conservation factor K. Simulations were performed using a Barabási–Albert (BA) scale-free network seeded with a single piece of information. The influence of β and K upon the temporal evolution of the system was subsequently analyzed regarding average information entropy, opinion fragmentation, and the range of rumor spread. These results can aid in decision-making to limit the spread of rumors.
14 October 2016 in Scientific Reports
Organisms with environmental sensors that guide survival are considered more likely to be favored by natural selection if they possess more accurate sensors. In this paper, we develop a theoretical model which shows that under certain conditions of environmental stochasticity, selection actually favors sensors of lower accuracy. An analogy between this counter-intuitive phenomenon and the well-known Parrondo’s paradox is suggested.
1 July 2018 in Communications in Nonlinear Science and Numerical Simulation
24 July 2018 in Journal of Physics A: Mathematical and Theoretical
How do movements and coalitions which engage with multiple social issues succeed in cross-issue solidarity, and when do they instead become fragmented? To address this, the mechanisms of cross-issue interaction have to be understood. Prior work on opinion dynamics and political disagreement has focused on single-issue consensus and polarization. Inspired by practices of cross-issue movement building, we have developed a general model of multi-issue opinion dynamics where agreement on one issue can promote greater inclusivity in discussing other issues, thereby avoiding the pitfalls of exclusivist interaction, where individuals engage only if they agree sufficiently on every issue considered. Our model shows that as more issues come into play, consensus and solidarity can only be maintained if inclusivity towards differing positions is increased. We further investigate whether greater inclusivity and compromise across issues lead people towards or away from normative truth, thereby addressing concerns about the non-ideal nature of political consensus.