Word embeddings have recently been applied to detect and explore changes in word meaning on large historical corpora. While word embeddings are useful in many Natural Language Processing tasks, there are a number of questions that need to be addressed concerning accuracy and applicability of these methods for historical data. There is a scarce literature on the stability and replicability of these embeddings, especially on small corpora, which are common in historical work. It also remains unclear whether methods used to evaluate embeddings in contemporary data can also be used for historical data sets. Our overarching goal is to use word embeddings for investigating semantic shifts in the history of Spanish. In the work presented here, we focus on methodological questions that arise. We first examine the stability and applicability of three commonly used word embeddings models on a small corpus of medieval and classical Spanish. Comparing our results with a study on the word algo as a test case, we show that a rank-averaging method can produce more stable results from the embeddings. We corroborate previous theoretical work while demonstrating the applicability of our method when training word embeddings on small corpora for the analysis of semantic change. Second, we investigate how best to evaluate different embeddings models. We show that an existing analogy test cannot be used without modification. Our new analogy test, consisting of roughly 10,000 questions for medieval and classical Spanish, will be released with the article.
7 January 2016 in Natural Language & Linguistic Theory
The development of periphrastic past constructions in Romance, including those that do not take a reflex of Latin habĒre as an auxiliary, has been analyzed along a continuum from a resultative construction to a perfect, and in some cases to a perfective (see Harris 1982; Fleischman 1983). This paper argues that the development of the Pretérito Perfeito Composto (PPC) in Portuguese does not adhere to the proposed typologies of periphrastic past evolution in Romance. Using diachronic corpus data, we revisit the proposed resultative > perfect grammaticalization path (see Bybee et al. 1994) and contend that the developmental trajectory of the Portuguese PPC is distinct from other cases of periphrastic past evolution in Romance languages, specifically Spanish. We demonstrate that the iterative meaning unique to the PPC in contemporary Portuguese arises in morphosyntactically ambiguous contexts in which the ter + Past Participle construction co-occurs with semantically plural complements.
25 April 2022 in Journal of Historical Linguistics
This study uses word embeddings to investigate the semantic changes underlying the creation of two adversative connectives in Portuguese,porémandmas‘but, however’. Forporém, we chart its development from an original PP formed by a preposition with a causal meaning (por) and a demonstrative pronoun that referred anaphorically to a previous proposition (en(de)). Formas, we trace its change from an adverb meaning ‘more’. Adopting a distributional semantics approach, we use word embedding models trained on two corpora, the CIPM (Corpus Informatizado do Português Medieval, containing texts from the 12th–16th centuries) and COLONIA (containing texts from the 16th–20th centuries). We produce a measure of change based on the similarity scores ofporémandmaswith respect to words in relevant semantic categories in each corpus, representing the source and the target meanings. This paper, which constitutes the first computational study of semantic change in Portuguese, also discusses challenges and outlines steps to be taken into consideration when choosing embedding algorithms for small historical corpora.
This paper analyzes the syntactic and semantic changes undergone by the PP sin embargo ‘without obstacle/impediment’ as it develops its clause-taking properties in Spanish from the 12th to the 16th centuries, essential for its further reanalysis as a concessive connective. We argue against an account that explains this change through a metaphor from the lack of a barrier in the sociophysical world to the epistemic world. Instead, we show that the syntactic and semantic change stems from the subcategorization properties of the noun embargo, which in the 1400s selects for the preposition de and later for the complementizer que, with scope over a proposition. The selection of a clausal complement, and hence the increase in scope of the original prepositional phrase, underlies the creation of the sentence connective.