Machines incorporating techniques from artificial intelligence and machine learning can work with human users on a moment-to-moment, real-time basis to generate creative outcomes, performances and artefacts. We define such systems collaborative, creative AI systems, and in this article, consider the theoretical and practical considerations needed for their design so as to support improvisation, performance and co-creation through real-time, sustained, moment-to-moment interaction. We begin by providing an overview of creative AI systems, examining strengths, opportunities and criticisms in order to draw out the key considerations when designing AI for human creative collaboration. We argue that the artistic goals and creative process should be first and foremost in any design. We then draw from a range of research that looks at human collaboration and teamwork, to examine features that support trust, cooperation, shared awareness and a shared information space. We highlight the importance of understanding the scope and perception of two-way communication between human and machine agents in order to support reflection on conflict, error, evaluation and flow. We conclude with a summary of the range of design challenges for building such systems in provoking, challenging and enhancing human creative activity through their creative agency.

Programming sound synthesizers is a complex and time-consuming task. Automatic synthesizer programming involves finding parameters for sound synthesizers using algorithmic methods. Sound matching is one application of automatic programming, where the aim is to find the parameters for a synthesizer that cause it to emit as close a sound as possible to a target sound. We describe and compare several sound matching techniques that can be used to automatically program the Dexed synthesizer, which is a virtual model of a Yamaha DX7. The techniques are a hill climber, a genetic algorithm, and three deep neural networks that have not been applied to the problem before. We define a sound matching task based on six sets of sounds, which we derived from increasingly complex configurations of the Dexed synthesis algorithm. A bidirectional, long short-term memory network with highway layers performed better than any other technique and was able to match sounds closely in 25% of the test cases. This network was also able to match sounds in near real time, once trained, which provides a significant speed advantage over previously reported techniques that are based on search heuristics. We also describe our open source framework, which makes it possible to repeat our study, and to adapt it to different synthesizers and algorithmic programming techniques.

To speak comfortably of the machine artist (as outlined in the call for papers for this Special Issue) makes key assumptions about what it is to be an artist. It assumes, for instance, that the experience of living as an artist, which includes the socialisation, hard work, single-mindedness, and focused energy of creative activity, is incidental rather than essential since these aspects are not comfortably applicable to machines. Instead, it supposes that what is essential is the artistic product, and it is the similarity of human and machine products that makes it possible to speak of machine artists. This definition of art in terms of products is supported by modern psychological theories of creativity, defined as the generation of novel ideas which give rise to valuable products. These ideas take place in the mind or brain, regarded as a closed system within whose workings the secret of creativity will eventually be revealed. This is the framework of what is widely referred to as “cognitivism”. This definition in terms of novel ideas and valuable products has been widely assumed by artificial intelligence (AI) and computational creativity (CC), and this has been backed up through a particular version of the Turing Test. In this, a machine can be said to be a creative artist if its products cannot be distinguished from human art. However, there is another psychological view of creativity, that of John Dewey, in which a lived experience of inquiry and focus is essential to being creative. In this theory, creativity is a function of the whole person interacting with the world, rather than originating in the brain. This makes creativity a Process rather than a Cognitivist framework. Of course, the brain is crucial in a Process theory, but as part of an open system which includes both body and environment. Developments in “machine art” have been seen as spectacular and are widely publicised. But there may be a danger that these will distract from what we take to be the most exciting prospect of all. This is the contribution of computer technology to stimulate, challenge, and provoke artistic practice of all forms.

There is increasing interplay between humans and artificial intelligent (AI) entities in online environments. With the growing autonomy and sophistication of these AI systems, the hybrid communities which are formed start to behave like the more-familiar, human-only social systems. This sets up the challenge to find systematic ways to ensure reliable governance for these interactions just as we do in human communities. This article proposes a novel approach to build governance for hybrid communities using what we call Conscientious Design (CD). There are two key aspects to CD: 1) the introduction of value categories that guides the identification of relevant stakeholder values, coupled with 2) a tripartite model for online institutions that serves to describe the interactions of hybrid communities of humans and artificial entities in a way that is consistent with the values of all stakeholders.