Implementation research necessitates a shift from clinical trial methods in both the conduct of the study and in the way that it is evaluated given the focus on the impact of implementation strategies. That is, the methods or techniques to support the adoption and delivery of a clinical or preventive intervention, program, or policy. As strategies target one or more levels within the service delivery system, evaluating their impact needs to follow suit. This article discusses the methods and practices involved in quantitative evaluations of implementation research studies. We focus on evaluation methods that characterize and quantify the overall impacts of an implementation strategy on various outcomes. This article discusses available measurement methods for common quantitative implementation outcomes involved in such an evaluation-adoption, fidelity, implementation cost, reach, and sustainment-and the sources of such data for these metrics using established taxonomies and frameworks. Last, we present an example of a quantitative evaluation from an ongoing randomized rollout implementation trial of the Collaborative Care Model for depression management in a large primary healthcare system.

Long waiting times in the emergency department (ED) are associated with decreased patient satisfaction and increased morbidity and mortality. Triage may be a contributing factor to prolonged wait times in the ED. At Alhada Armed Forces Hospital (Taif, Saudi Arabia), patients other than level 1 and 2 on the Canadian Triage and Acuity Scale are requested to wait until triage. During peak hours (08:00−22:00), the waiting time prior to triage is prolonged, and several patients leave the ED before triage. In this project, a multidisciplinary team was assembled to revise patient flow from the time of arrival at the ED to the time of triage. Lean methodology was used to identify the redundancies and design a seamless flow process for ED patients. Through reorganising the triage area using minimal additional resources, the project team devised a novel floor plan for the triage area which provided a unique patient flow in the ED. The median patient wait time from arrival to triage was reduced from 27 min to 4.09 min and the percentage of patients leaving the ER before triage was reduced to 0%. This project is the first of its kind in Saudi Arabia, as well as in the Gulf region, and provides a radical solution to the problem of patient waiting in the ED during peak hours.

Background: Despite advances in computer-assisted knee arthroplasty (CAKA), little is known about the uptake of this technology in recent years. We aimed to explore the utilization trends and practice variation of CAKA from 2010 to 2017 and investigate the predictors of CAKA adoption. Methods: Patients undergoing conventional knee arthroplasty and CAKA were identified from the states of New York and Florida's administrative databases using the International Classification of Diseases version 9 and 10 procedure codes. Quarterly proportions of CAKA were calculated over the study period, and logistic regression was used to estimate predictors of CAKA utilization. Results: Between 2010 and 2017, quarterly proportion of CAKAs increased from 4.89% in 2010Q1 to 9.45% in 2017Q3 in New York and from 4.03% in 2010Q1 to 5.73% in 2017Q3 in Florida. The general CA code was used to code most of the procedures (81%). Being Black (odds ratio [OR]: 0.63, 95% confidence interval [CI], 0.60 to 0.67), Hispanic (OR: 0.45, CI, 0.41 to 0.50), and having Medicaid coverage (OR: 0.46, CI, 0.40 to 0.53) were associated with lower likelihood of receiving CAKA in New York; similar findings were found in Florida. Conclusion: Utilization of CAKA has increased substantially in both New York and Florida from 2010 to 2017; however, with most CAKAs reported using the general code, understanding adoption rates of various modalities was not possible. Black and Hispanic patients and those with Medicaid insurance are least likely to receive this high-precision technology, illustrating the presence of disparities in the adoption of CAKA.

Background: TKA is performed only after the patient and the surgeon agree that the potential benefits of surgery outweigh potential risks. However, as many as 20% of patients are dissatisfied after TKA, suggesting that the shared decision-making process does not consistently serve our patients well. Expected outcomes are increasingly used in clinical practice, as an addendum to preoperative clinical and demographic factors, to inform this decision; however, the added value of this information in aligning patients and surgeons regarding the benefit and risk of TKA remains poorly understood. Questions/purposes: (1) How do patients’ ratings of appropriateness of TKA (based on benefits and risks as appropriate, inappropriate, or inconclusive) compare with clinicians’ ratings of appropriateness for 279 hypothetical patient scenarios that include both preoperative factors and expected outcomes? (2) Did expected outcomes drive the patients’ ratings? (3) What additional factors, not accounted for in the hypothetical patient scenarios, also affected the patients’ ratings of appropriateness of TKA for these scenarios? Methods: TKA appropriateness ratings for 279 hypothetical patient scenarios were previously developed using the RAND/UCLA Delphi method and rated by a panel of clinicians as either inappropriate, inconclusive, or appropriate. These scenarios are composed of clinically plausible permutations of five preoperative factors and three expected postoperative outcome factors. Expected outcomes were shown to be the most important drivers of physicians’ classification. In this study, a panel of eight patients (four with end-stage osteoarthritis [OA] without TKA and four who recently underwent TKA) independently rated the appropriateness of TKA for the same 279 scenarios on a scale from 1 to 9 (1 to 3 were considered inappropriate, 4 to 6 inconclusive, and 7 to 9 appropriate). These patients were identified by the Arthritis Foundation with equal representation of men and women who had TKA and those who did not and were educated on the clinical meaning and relevance of each factor. The median score rating was then classified into either “inappropriate,” “inconclusive,” or “appropriate,” and categories were compared with those of the clinicians for the same scenarios. Classification tree (CART) analysis was applied to the patient-assessed TKA appropriateness categories to identify the most influential of the eight factors. After completing the appropriateness ratings, the same eight patients were convened in two focus groups to elucidate qualitatively any additional factors that influenced their ratings. Semistructured interviews were conducted, and qualitative methods were applied to the narrative to determine these additional factors. Results: Based on patients’ median score for the 279 scenarios, 15% (43) of scenarios were classified as inappropriate, 53% (148) of scenarios as inconclusive, and 32% (88) of scenarios as appropriate. Overall, concordance between the patient and the surgeon classifications was 68% (weighted κ = 0.58 [95% confidence interval 0.51 to 0.66]; p < 0.001). None of the scenarios that the clinicians classified as appropriate were classified as inappropriate by the patients and vice versa. Preoperative symptom severity, OA radiographic severity, expected rate of serious complications, and expected improvement at 2 years were crucial factors that discriminated among the patient-assessed TKA appropriateness classification categories. The semistructured qualitative interviews with the same eight patients from focus groups identified risks of opioid addiction and fear of functional deterioration as additional important factors for patients to consider TKA earlier. Conclusion: When expected outcomes were included as part of a clinical scenario, patients generally agreed with the clinicians when a TKA was appropriate, suggesting that expected outcomes may further increase concordance between patients and surgeons. Clinical Relevance: Incorporating expected outcomes into clinical practice and engaging patients to understand all the factors affecting their decisions may aid surgeons in guiding the shared decision-making process and ultimately reduce dissatisfaction with the outcome of surgery.

Background Systematic reviews (SRs) of computer-assisted (CA) total knee arthroplasty (TKA) and total hip arthroplasty (THA) report conflicting evidence on its superiority over conventional surgery. Little is known about the quality of these SRs; variability in their methodological quality may be a contributing factor. We evaluated the methodological quality of all published SRs to date, summarized and examined the consistency of the evidence generated by these SRs. Methods We searched four databases through December 31, 2018. A MeaSurement Tool to Assess systematic Reviews 2 (AMSTAR 2) was applied to assess the methodological quality. Evidence from included meta-analyses on functional, radiological and patient-safety outcomes was summarized. The corrected covered area was calculated to assess the overlap between SRs in including the primary studies. Results Based on AMSTAR 2, confidence was critically low in 39 of the 42 included SRs and low in 3 SRs. Low rating was mainly due to failure in developing a review protocol (90.5%); providing a list of excluded studies (81%); accounting for risk of bias when discussing the results (67%); using a comprehensive search strategy (50%); and investigating publication bias (50%). Despite inconsistency between SR findings comparing functional, radiological and patient safety outcomes for CA and conventional procedures, most TKA meta-analyses favored CA TKA, whereas most THA meta-analyses showed no difference. Moderate overlap was observed among TKA SRs and high overlap among THA SRs. Conclusions Despite conclusions of meta-analyses favoring CA arthroplasty, decision makers adopting this technology should be aware of the low confidence in the results of the included SRs. To improve confidence in future SRs, journals should consider using a methodological assessment tool to evaluate the SRs prior to making a publication decision.