La mise en mouvement du corps à travers la danse, la présence d’une pulsation régulière qui rythme ces mouvements et le fait de contrecarrer cette pulsation par des événements musicaux placés à côté, ce qu’on appelle la « contramétricité », sont des points communs à la plupart des musiques africaines traditionnelles. Ils témoignent à la fois d’une disposition universelle du système cognitif (capacité du corps humain de se synchroniser avec des signaux sonores périodiques) et d’une dimension profondément culturelle dans la mesure où la contramétricité s’est développée dans ces musiques beaucoup plus qu’ailleurs. On montrera comment cet aspect contramétrique contredit les théories de la perception du rythme inspirées des hypothèses de Lerdahl et Jackendoff. On comparera également ces musiques avec le jazz, car celui-ci a une particularité qui le distingue des musiques africaines, à savoir l’existence de niveaux hiérarchiques dans l’organisation métrique, dont on montrera qu’elle n’est pas aussi discriminante que ce qui est généralement admis.
Recently, the field of musical co-creativity has gained some momentum. In this context, our goal is twofold: to develop an intelligent listening and predictive module of chord sequences, and to propose an adapted evaluation of the associated Music Information Retrieval (MIR) tasks that are the real-time extraction of musical chord labels from a live audio stream and the prediction of a possible continuation of the extracted symbolic sequence. Indeed, this application case invites us to raise questions about the evaluation processes and methodology that are currently applied to chord-based MIR models. In this paper, we focus on musical chords since these mid-level features are frequently used to describe harmonic progressions in Western music. In the case of chords, there exists some strong inherent hierarchical and functional relationships. However, most of the research in the field of MIR focuses mainly on the performance of chord-based statistical models, without considering music-based evaluation or learning. Indeed, usual evaluations are based on a binary qualification of the classification outputs (right chord predicted versus wrong chord predicted). Therefore, we present a specifically-tailored chord analyser to measure the performances of chord-based models in terms of functional qualification of the classification outputs (by taking into account the harmonic function of the chords). Then, in order to introduce musical knowledge into the learning process for the automatic chord extraction task, we propose a specific musical distance for comparing predicted and labeled chords. Finally, we conduct investigations into the impact of including high-level metadata in chord sequence prediction learning (such as information on key or downbeat position). We show that a model can obtain better performances in terms of accuracy or perplexity, but output biased results. At the same time, a model with a lower accuracy score can output errors with more musical meaning. Therefore, performing a goal-oriented evaluation allows a better understanding of the results and a more adapted design of MIR models.
19 July 2024 in Proceedings of the ACM on Computer Graphics and Interactive Techniques
Critical Climate Machine is a cutting-edge media art installation that critically exposes and quantifies mechanisms of climate change misinformation. Utilizing computational aesthetics across data, imagery, and sound, this work processes real-time data from X (Twitter) through a natural language processing learning model derived from cognitive sciences. It not only renders the statistical aspects of this data visually but also manifests its thermal effects. A unique audio dimension is introduced through dialogues between climate skeptics and climate advocates, processed by the generative machine learning (ML) algorithm Dicy2. These elements collectively shape the installation, each unveiling its distinctive algorithmic aesthetics and technical underpinnings. This paper concentrates on the dual application of ML algorithms: one for dissecting extensive online misinformation streams, and the other for creating climate-related dialogues. This dual approach opens a discussion on the mediation of climate, at the convergence of computational and physical realms. Our aim is to critically examine the role of ML technologies in crafting aesthetic experiences that resonate within scientific discourse and public debate on climate issues.
30 October 2014 in Techniques et Sciences Informatiques
This article focuses on learning the hierarchical structure of what we call a “temporal scenario” (for instance, a chord progression) to perform automatic improvisation consistently over several different time scales. We first present a way to represent hierarchical structures with a phrase structure grammar. Such a grammar enables us to analyze a scenario at several levels of organization, creating a “multilevel scenario.” We then develop a method to automatically induce this grammar from a corpus, based on sequence selection with mutual information. We applied this method to a corpus of transcribed improvisations based on the chord sequence, also with chord substitutions, from George Gershwin's “I Got Rhythm.” From these we obtained multilevel scenarios similar to the analyses performed by professional musicians. We then present a novel heuristic approach, exploiting the multilevel structure of a scenario to guide the improvisation with anticipatory behavior in an improvisation paradigm driven by a factor oracle. This method ensures consistency of the improvisation with regard to the global form, and it opens up possibilities when playing on chords that do not exist in memory. This system was evaluated by professional improvisers during listening sessions and received excellent feedback. This article focuses on learning the hierarchical structure of what we call a “temporal scenario” (for instance, a chord progression) to perform automatic improvisation consistently over several different time scales. We first present a way to represent hierarchical structures with a phrase structure grammar. Such a grammar enables us to analyze a scenario at several levels of organization, creating a “multilevel scenario.” We then develop a method to automatically induce this grammar from a corpus, based on sequence selection with mutual information. We applied this method to a corpus of transcribed improvisations based on the chord sequence, also with chord substitutions, from George Gershwin's “I Got Rhythm.” From these we obtained multilevel scenarios similar to the analyses performed by professional musicians. We then present a novel heuristic approach, exploiting the multilevel structure of a scenario to guide the improvisation with anticipatory behavior in an improvisation paradigm driven by a factor oracle. This method ensures consistency of the improvisation with regard to the global form, and it opens up possibilities when playing on chords that do not exist in memory. This system was evaluated by professional improvisers during listening sessions and received excellent feedback. This article focuses on learning the hierarchical structure of what we call a “temporal scenario” (for instance, a chord progression) to perform automatic improvisation consistently over several different time scales. We first present a way to represent hierarchical structures with a phrase structure grammar. Such a grammar enables us to analyze a scenario at several levels of organization, creating a “multilevel scenario.” We then develop a method to automatically induce this grammar from a corpus, based on sequence selection with mutual information. We applied this method to a corpus of transcribed improvisations based on the chord sequence, also with chord substitutions, from George Gershwin's “I Got Rhythm.” From these we obtained multilevel scenarios similar to the analyses performed by professional musicians. We then present a novel heuristic approach, exploiting the multilevel structure of a scenario to guide the improvisation with anticipatory behavior in an improvisation paradigm driven by a factor oracle. This method ensures consistency of the improvisation with regard to the global form, and it opens up possibilities when playing on chords that do not exist in memory. This system was evaluated by professional improvisers during listening sessions and received excellent feedback.
Guider ou composer l'improvisation musicale homme-machine à l'aide de scénarios temporels: Cette thèse propose l’introduction de scénarios temporels pour guider ou composer l’improvisation musicale homme-machine. Ce travail étudie la dialectique entre planification et réactivité dans les systèmes interactifs dédiés à l’improvisation : des systèmes informatiques pouvant générer de la musique en relation directe avec le contexte produit par une situation de concert. On cherche ici à appréhender l'improvisation pulsée et dite « idiomatique ». En s’appuyant sur l’existence d’une structure formalisée antérieure à la performance dans de nombreux répertoires improvisés (une « grille d’accords » par exemple) ces travaux proposent : un modèle d’improvisation guidée par un « scénario » introduisant des mécanismes d’anticipation ; une architecture temporelle hybride combinant anticipation et réactivité et permettant la synchronisation du rendu multimédia avec une pulsation non métronomique ; et un cadre pour composer des sessions d’improvisation idiomatique ou non à l’échelle du scénario en exploitant la généricité des modèles. Ces recherches ont été menées en interaction constante avec des musiciens experts, en intégrant pleinement ces collaborations au processus itératif de conception des modèles et architectures. Ceux-ci ont été implémentés dans le système ImproteK, utilisé à de nombreuses reprises lors de performances avec des improvisateurs. Au cours de ces collaborations, les sessions d'expérimentations ont été associées à des entretiens et séances de réécoute afin de recueillir de nombreuses appréciations formulées par les musiciens pour valider et affiner les choix technologiques.