Background: Lung ultrasound (LUS) has received considerable interest in the clinical evaluation of patients with COVID‐19. Previously described LUS manifestations for COVID‐19 include B‐lines, consolidations, and pleural thickening. The interrater reliability (IRR) of these findings for COVID‐19 is unknown.Methods: This study was conducted between March and June 2020. Nine physicians (hospitalists: n = 4; emergency medicine: n = 5) from 3 medical centers independently evaluated n = 20 LUS scans (n = 180 independent observations) collected from patients with COVID‐19, diagnosed via RT‐PCR. These studies were randomly selected from an image database consisting of COVID‐19 patients evaluated in the emergency department with portable ultrasound devices. Physicians were blinded to any patient information or previous LUS interpretation. Kappa values (κ) were used to calculate IRR.Results: There was substantial IRR on the following items: normal LUS scan (κ = 0.79 [95% CI: 0.72–0.87]), presence of B‐lines (κ = 0.79 [95% CI: 0.72–0.87]), ≥3 B‐lines observed (κ = 0.72 [95% CI: 0.64–0.79]). Moderate IRR was observed for the presence of any consolidation (κ = 0.57 [95% CI: 0.50–0.64]), subpleural consolidation (κ = 0.49 [95% CI: 0.42–0.56]), and presence of effusion (κ = 0.49 [95% CI: 0.41–0.56]). Fair IRR was observed for pleural thickening (κ = 0.23 [95% CI: 0.15–0.30]).Discussion: Many LUS manifestations for COVID‐19 appear to have moderate to substantial IRR across providers from multiple specialties utilizing differing portable devices.The most reliable LUS findings with COVID‐19 may include the presence/count of B‐lines or determining if a scan is normal. Clinical protocols for LUS with COVID‐19 may require additional observers for the confirmation of less reliable findings such as consolidations.

Objective: To assess usability and usefulness of a machine learning-based order recommender system applied to simulated clinical cases. Materials and Methods: 43 physicians entered orders for 5 simulated clinical cases using a clinical order entry interface with or without access to a previously developed automated order recommender system. Cases were randomly allocated to the recommender system in a 3:2 ratio. A panel of clinicians scored whether the orders placed were clinically appropriate. Our primary outcome included the difference in clinical appropriateness scores. Secondary outcomes included total number of orders, case time, and survey responses. Results: Clinical appropriateness scores per order were comparable for cases randomized to the order recommender system (mean difference -0.11 order per score, 95% CI: [-0.41, 0.20]). Physicians using the recommender placed more orders (median 16 vs 15 orders, incidence rate ratio 1.09, 95%CI: [1.01-1.17]). Case times were comparable with the recommender system. Order suggestions generated from the recommender system were more likely to match physician needs than standard manual search options. Physicians used recommender suggestions in 98% of available cases. Approximately 95% of participants agreed the system would be useful for their workflows. Discussion: User testing with a simulated electronic medical record interface can assess the value of machine learning and clinical decision support tools for clinician usability and acceptance before live deployments. Conclusions: Clinicians can use and accept machine learned clinical order recommendations integrated into an electronic order entry interface in a simulated setting. The clinical appropriateness of orders entered was comparable even when supported by automated recommendations.

Objectives: Point‐of‐care ultrasound (POCUS) detects the pulmonary manifestations of COVID‐19 and may predict patient outcomes.Methods: We conducted a prospective cohort study at four hospitals from March 2020 to January 2021 to evaluate lung POCUS and clinical outcomes of COVID‐19. Inclusion criteria included adult patients hospitalized for COVID‐19 who received lung POCUS with a 12‐zone protocol.Each image was interpreted by two reviewers blinded to clinical outcomes. Our primary outcome was the need for intensive care unit (ICU) admission versus no ICU admission. Secondary outcomes included intubation and supplemental oxygen usage.Results: N = 160 patients were included. Among critically ill patients, B‐lines (94 vs 76%;P < .01) and consolidations (70 vs 46%;P < .01) were more common. For scans collected within 24 hours of admission (N = 101 patients), early B‐lines (odds ratio [OR] 4.41 [95% confidence interval, CI: 1.71–14.30];P < .01) or consolidations (OR 2.49 [95% CI: 1.35–4.86];P < .01) were predictive of ICU admission. Early consolidations were associated with oxygen usage after discharge (OR 2.16 [95% CI: 1.01–4.70];P = .047). Patients with a normal scan within 24 hours of admission were less likely to require ICU admission (OR 0.28 [95% CI: 0.09–0.75];P < .01) or supplemental oxygen (OR 0.26 [95% CI: 0.11–0.61];P < .01). Ultrasound findings did not dynamically change over a 28‐day scanning window after symptom onset.Conclusions: Lung POCUS findings detected within 24 hours of admission may provide expedient risk stratification for important COVID‐19 clinical outcomes, including future ICU admission or need for supplemental oxygen. Conversely, a normal scan within 24 hours of admission appears protective. POCUS findings appeared stable over a 28‐day scanning window, suggesting that these findings, regardless of their timing, may have clinical implications.

There is insufficient knowledge about how personal access to handheld ultrasound devices (HUDs) improves trainee learning with point-of-care ultrasound (POCUS). To assess whether HUDs, alongside a yearlong lecture series, improved trainee POCUS usage and ability to acquire images. Internal medicine intern physicians (n = 47) at a single institution from 2017 to 2018 were randomized 1:1 to receive personal HUDs (n = 24) for patient care/self-directed learning vs no-HUDs (n = 23). All interns received a repeated lecture series on cardiac, thoracic, and abdominal POCUS. Main outcome measures included self-reported HUD usage rates and post-intervention assessment scores using the Rapid Assessment of Competency in Echocardiography (RACE) scale between HUD and no-HUD groups. HUD interns reported performing POCUS assessments on patients a mean 6.8 (SD 2.2) times per week vs 6.4 (SD 2.9) times per week in non-HUD arm (P = .66). There was no relationship between the number of self-reported examinations per week and a trainee's post-intervention RACE score (rho = 0.022, P = .95). HUD interns did not have significantly higher post-intervention RACE scores (median HUD score 17.0 vs no-HUD score 17.8; P = .72). Trainee confidence with cardiac POCUS did not correlate with RACE scores. Personal HUDs without direct supervision did not increase the amount of POCUS usage or improve interns' acquisition abilities. Interns who reported performing more examinations per week did not have higher RACE scores. Improved HUD access and lectures without additional feedback may not improve POCUS mastery.