Our objective was to review the characteristics, current applications, and evaluation measures of conversational agents with unconstrained natural language input capabilities used for health-related purposes. We searched PubMed, Embase, CINAHL, PsycInfo, and ACM Digital using a predefined search strategy. Studies were included if they focused on consumers or healthcare professionals; involved a conversational agent using any unconstrained natural language input; and reported evaluation measures resulting from user interaction with the system. Studies were screened by independent reviewers and Cohen’s kappa measured inter-coder agreement. The database search retrieved 1513 citations; 17 articles (14 different conversational agents) met the inclusion criteria. Dialogue management strategies were mostly finite-state and frame-based (6 and 7 conversational agents, respectively); agent-based strategies were present in one type of system. Two studies were randomized controlled trials (RCTs), 1 was cross-sectional, and the remaining were quasi-experimental. Half of the conversational agents supported consumers with health tasks such as self-care. The only RCT evaluating the efficacy of a conversational agent found a significant effect in reducing depression symptoms (effect size d = 0.44, p = .04). Patient safety was rarely evaluated in the included studies. The use of conversational agents with unconstrained natural language input capabilities for health-related purposes is an emerging field of research, where the few published studies were mainly quasi-experimental, and rarely evaluated efficacy or safety. Future studies would benefit from more robust experimental designs and standardized reporting. The protocol for this systematic review is registered at PROSPERO with the number CRD42017065917.

A number of methods for deciding when a systematic review should be updated have been proposed, yet little is known about whether systematic reviews are updated more quickly when new evidence becomes available. Our aim was to examine the timing of systematic review updates relative to the availability of new evidence. We performed a retrospective analysis of the update timing of systematic reviews published in the Cochrane Database of Systematic Reviews in 2010 relative to the availability of new trial evidence. We compared the update timing of systematic reviews with and without signals defined by the completion or publication of studies that were included in the updates. We found 43% (293/682) systematic reviews were updated before June 2017, of which 204 included an updated primary outcome meta-analysis (median update time 35.4 months; IQR 25.5–54.0), 38% (77/204) added new trials, and 4% (8/204) reported a change in conclusion. In the 171 systematic reviews with reconcilable trial reporting information, we did not find a clear difference in update timing (p = 0.05) between the 15 systematic reviews with a publication signal (median 25.3 months; IQR 15.3–43.5) and the 156 systematic reviews without a publication signal (median 34.4 months; IQR 25.1–52.2). In the 145 systematic reviews with reconcilable trial completion information, we did not find a difference in update timing (p = 0.33) between the 15 systematic reviews with a trial completion signal (median 26.0 months; IQR 19.3–49.5) and the 130 systematic reviews without a trial completion signal (median 32.4 months; IQR 24.1 to 46.0). A minority of 2010 Cochrane reviews were updated before June 2017 to incorporate evidence from new primary studies, and very few updates led to a change in conclusion. We did not find clear evidence that updates were undertaken faster when new evidence was made available. New approaches for finding early signals that a systematic review conclusion is at risk of change may be useful in allocated resources to the updating of systematic reviews.

Studies measuring the completeness and consistency of trial registration and reporting rely on linking registries with bibliographic databases. In this systematic review, we quantified the processes used to identify these links. PubMed and Embase databases were searched from inception to May 2016 for studies linking trial registries with bibliographic databases. The processes used to establish these links were categorised as automatic when the registration identifier was available in the bibliographic database or publication, or manual when linkage required inference or contacting of trial investigators. The number of links identified by each process was extracted where available. Linear regression was used to determine whether the proportions of links available via automatic processes had increased over time. In 43 studies that examined cohorts of registry entries, 24 used automatic and manual processes to find articles; 3 only automatic; and 11 only manual (5 did not specify). Twelve studies reported results for both manual and automatic processes and showed that a median of 23% (range from 13 to 42%) included automatic links to articles, while 17% (range from 5 to 42%) of registry entries required manual processes to find articles. There was no evidence that the proportion of registry entries with automatic links had increased (R 2 = 0.02, p = 0.36). In 39 studies that examined cohorts of articles, 21 used automatic and manual processes; 9 only automatic; and 2 only manual (7 did not specify). Sixteen studies reported numbers for automatic and manual processes and indicated that a median of 49% (range from 8 to 97%) of articles had automatic links to registry entries, and 10% (range from 0 to 28%) required manual processes to find registry entries. There was no evidence that the proportion of articles with automatic links to registry entries had increased (R 2 = 0.01, p = 0.73). The linkage of trial registries to their corresponding publications continues to require extensive manual processes. We did not find that the use of automatic linkage has increased over time. Further investigation is needed to inform approaches that will ensure publications are properly linked to trial registrations, thus enabling efficient monitoring of trial reporting.

Systematic reviews of clinical trials could be updated faster by automatically monitoring relevant trials as they are registered, completed, and reported. Our aim was to provide a public interface to a database of curated links between systematic reviews and trial registrations. We developed the server-side system components in Python, connected them to a PostgreSQL database, and implemented the web-based user interface using Javascript, HTML, and CSS. All code is available on GitHub under an open source MIT license and registered users can access and download all available data. The trial2rev system is a web-based interface to a database that collates and augments information from multiple sources including bibliographic databases, the ClinicalTrials.gov registry, and the actions of registered users. Users interact with the system by browsing, searching, or adding systematic reviews, verifying links to trials included in the review, and adding or voting on trials that they would expect to include in an update of the systematic review. The system can trigger the actions of software agents that add or vote on included and relevant trials, in response to user interactions or by scheduling updates from external resources. We designed a publicly-accessible resource to help systematic reviewers make decisions about systematic review updates. Where previous approaches have sought to reactively filter published reports of trials for inclusion in systematic reviews, our approach is to proactively monitor for relevant trials as they are registered and completed.