Abstract
Background
Tumor board documentation is challenging due to complexity of multidisciplinary input. Ambient artificial intelligence (AI) for medical dictation uses generative methods to summarize voice data. Strategic prompting can improve model output. We evaluated prompt-directed ambient AI for automated tumor board documentation and compared ambient AI with manual documentation.
Patients and methods
We performed a prospective quality improvement study evaluating ambient AI for documentation of multidisciplinary head and neck cancer tumor board discussions at a tertiary institution in 2025. The primary intervention was verbal prompting during case discussions to improve documentation quality. Primary cohorts were unprompted and prompt-directed ambient AI tumor boards. A paired comparator sub analysis included manual and ambient AI documentation from the same meetings. We validated our prompted ambient AI workflow with additional tumor boards. Documentation was evaluated using a modified MODe (Metric for Observation of Decision Making)-based rubric across 5 domains: demographics, tumor characteristics, radiology findings, pathology findings, and discussion, with criteria rated on a 1–3 scale (1=incorrect/omitted, 2=partial, 3=complete documentation).
Results
We analyzed results of 12 tumor board across 4 cohorts: unprompted ambient AI, prompt-directed ambient AI, manual documentation, and internal validation. In the primary cohorts, prompt-directed ambient AI improved Tumor Characteristics domain score (q = 0.042) and total score (p = 0.005) versus unprompted ambient AI, with no difference in per-patient discussion time (p = 0.158). In the paired comparator analysis, ambient AI improved total score (p < 0.001) and Discussion domain score (q < 0.001) versus manual documentation, while reducing administrative time per patient (p < 0.001). The internal validation cohort showed statistically comparable scores across all domains.
Conclusions
Directed prompting during head and neck tumor board improved generated summaries, without increasing discussion times. Compared with manual documentation, ambient AI improved documentation quality, while diminishing administrative burden.
Keywords: Artificial intelligence, ambient intelligence, tumor board, documentation, head neck, documentation
Graphical Abstract
Introduction
Tumor boards provide a setting for multidisciplinary review of patient-specific factors to support precision-based oncological management.1–4 These forums can also facilitate clinical trial recruitment, support medico-legal compliance, and guide survivorship planning.5–7 Therefore, it is essential to have accurate and thorough documentation for ongoing communication and reference by the care team. However, current manual documentation is time-intensive and is often delegated to trainees, which can lead to inconsistent quality and loss of critical information.8,9 Previously explored documentation strategies rely on standardized human summarization.9,10
Ambient artificial intelligence (AI) platforms serve to capture and structure information utilizing natural language processing and generative artificial intelligence.11 Such programs are currently deployed as an alternative to scribes in outpatient settings.12 These platforms have been associated with improvement in clinician experience, burnout risk, and efficiency.13 However, tumor board documentation represents a significant challenge due to the complex, multimodal information discussed by a large multidisciplinary team of providers in real time. The role of ambient AI in this setting has not been well explored.
Ambient AI for medical dictation platforms function by capturing conversation between patient and medical professionals, applying natural language processing to produce a transcript, and synthesize the script into a structured clinical note, typically with links back to the transcript for reference.11,12 ‘Prompting’ refers to the creation and refinement of instructions to guide models to produce desired outputs and improve performance14, and prompts to Ambient AI platforms can be delivered via verbal cues. Within tumor board documentation, prompt-directed ambient AI may improve capture of clinically relevant elements beyond what is achieved with unprompted ambient AI alone.
We present a prospective quality improvement study that aims to provide a framework for the implementation of ambient AI in multidisciplinary head and neck tumor board documentation without substantial workflow impediments. We perform an evaluation consisting of a primary comparison of unprompted versus prompt-directed ambient AI documentation, a paired sub analysis comparing ambient AI with manual documentation, and an internal validation cohort to assess reproducibility.
Methods
Study Design
We performed a prospective quality improvement study evaluating ambient AI for documentation of multidisciplinary head and neck cancer tumor board discussions at a tertiary institution from February to July 2025. This study included multiple components (Figure 1). First, a primary comparison cohort evaluated sequential implementation of unprompted ambient AI followed by prompt-directed ambient AI. Second, a paired sub analysis compared ambient AI with manual documentation, allowing within-tumor board meeting comparison. Finally, we used an internal validation cohort to assess the reproducibility of our finalized prompt-directed workflow.
Figure 1.
Workflow diagram
Workflow evaluating head and neck cancer tumor board documentation. Twelve recorded tumor board meetings were processed across 3 cohorts. The primary cohort compared baseline ambient AI documentation (n=4) versus prompted ambient AI documentation (n=4). A paired sub analysis (n=2, drawn from primary cohort) compared ambient AI with manual documentation by a trainee for matched patient discussions. An internal validation cohort (n=4) assessed performance of prompted ambient AI in a separate sample. Generated summaries were evaluated using a modified quality rubric. AI = artificial intelligence
We deployed this quality improvement study to all eligible cases during the study period to maximize precision of estimates. Tumor board meetings occurred weekly during the study period and included a consistent team of core multidisciplinary providers including surgical oncologists, medical oncologists, radiation oncologists, speech language pathologists, pathologists, radiologists, nutritionists, physical therapists, and nursing coordinators. Historically, results of tumor board discussions were relayed to the patient by individual providers and were documented briefly in a clinical note in the patient’s electronic health record.
Tumor Board Documentation
We implemented the use of an ambient artificial intelligence platform, Abridge (Abridge AI Inc., Pittsburgh, PA), to capture and summarize the individual patient discussion.15 Abridge utilizes machine learning and natural language processing to capture and summarize clinical conversations in real-time. Abridge is developed containing over 10,000 hours of de-identified, patient-clinical conversations with SOAP-structured annotations and expert fine tuning. It is trained on healthcare-specific data from over 50 specialties.15
Tumor board meetings were held virtually on Zoom (Zoom Communications, Inc., San Jose, CA) and were saved using the record function and stored temporarily on an institutionally maintained encrypted storage device. The individual patient recordings were processed with Abridge to generate a structured note (Figure 2; Supplemental Figure 1).
Figure 2.
Example of recorded transcript and summarized note
Patient with T4N3 HPV-related squamous cell carcinoma of the right tonsil with extensive lymphadenopathy. Discussion included directed prompting. The right panel shows the transcript of the treatment recommendation discussion, while the left panel presents a comprehensive summarized note depicting the complexity of tumor board deliberations. An error was noted in capturing ‘right submandibular gland space’ as the ‘right stem endovascular space.’
The study was conducted as a quality improvement initiative. The primary comparison used a sequential design. Unprompted ambient AI was performed to establish baseline documentation performance from February to April 2025. Preliminary results helped develop a standardized prompting template (Figure 3). The second phase of the study (June – August) used prompt-directed ambient artificial intelligence as a tool to improve documentation quality. During the prompted phase of the study, verbal prompts were incorporated into the tumor board workflow. Prompts were delivered aloud by the presenting clinician at designated points during each case presentation. Presenting physician was patient’s primary oncologic provider. While the prompts were adapted to the details of each patient, presenters followed a standardized prompt framework with the goal of increasing accuracy and consistency of captured details, including demographics, tumor characteristics, imaging findings, pathology results, and multidisciplinary recommendations. Per-case discussion time was recorded for each tumor board case.
Figure 3.
Tumor board directed prompting
Directed prompts used to guide tumor board discussion. Prompts emphasized consistency in capturing important elements of discussion including demographics, tumor characteristics, imaging, pathology, and multidisciplinary recommendations
Abbreviations: AI, artificial intelligence
For the paired sub analysis, a trainee generated standard manual documentation for tumor boards. These were compared with paired ambient AI generated documentation for the same case discussions. The trainee conducting the manual documentation was blinded from the rubric. The summaries were scored independently using the same rubric and reference recordings. After development of the prompting workflow in the primary cohort, the finalized prompt-directed ambient AI framework was applied to an internal validation dataset conducted from September to October. Prompting was consistent in the validation dataset.
Administrative time for documentation was recorded. For the human documentation arm, this included time taken to manually write up the tumor board discussion while referencing the recordings. For the ambient artificial intelligence arm, administrative time included Abridge processing time and reviewer time required to validate the generated summary while referencing the recordings before finalization.
Outcomes
Documentation was evaluated using a rubric shown in table 1. The rubric was modified from an observational tool named MODe (Metric for Observation of Decision Making) initially developed by Lamb et al, in 2011 to evaluate decision-making processes in tumor boards.16 We adapted the content areas derived from MODe to create a 5-domain rubric that focuses on documentation quality. The rubric includes demographics, tumor characteristics, radiology and pathology findings, and treatment discussion. The adapted rubric serves as a practical tool to assess documentation completeness. The individual criteria are detailed in table 1. Tumor board content was scored on an ordinal scale: 1 indicates incorrect or omitted information, despite discussion at tumor board; 2 indicates partial documentation; and 3 indicates complete and comprehensive documentation. We selected this instrument and 1–3 scoring system as a practical and reproducible solution for comparing documentation across different cohorts. Mean ± standard deviation scores were calculated across criteria and domains to assess overall documentation quality. We consider scores ≥2.5 to indicate that documentation was on average between partial and complete across all criteria. This threshold was used for comparison and not for clinical input. Scoring was performed by physicians (A.M. and S.A.) who underwent a training session to review the scoring rubric and criteria to ensure consistent evaluations. Raters were blinded to whether cases were documented manually or with ambient AI and to whether tumor boards were discussed with or without prompting. Each case was reviewed by 1 rater. Raters reviewed both prompted and unprompted cases to minimize rater-based confounding effect. Scoring occurred after tumor board using the zoom recording as reference.
Table 1.
Rubric developed to measure quality in tumor board documentation.
| Domain | Criteria | Documentation requirement |
|---|---|---|
| Demographics | Age Comorbidities |
Demographics, if presented, documented in summary |
| Tumor characteristics |
Primary site Histology TNM Staging HPV status |
Tumor characteristics, if presented, documented in summary |
| Radiology findings |
Locoregional findings Distant findings |
Imaging modality and findings documented |
| Pathology findings |
Perineural invasion Lymphovascular invasion Extranodular extension |
Pathology results and features documented |
| Discussion | Surgical oncology Medical oncology Radiation oncology Speech language pathology Nursing coordinator Pathology Radiology Treatment recommendation |
Involvement and recommendations from
multidisciplinary team members
documented Final treatment discussion & recommendation documented |
Scoring (applied to each criterion): 1 = incorrect or omitted despite discussion; 2 = partial documentation; 3 = complete and comprehensive documentation; NA = not mentioned during tumor board.
In the primary cohort analysis, the prespecified primary outcome was the mean domain-level rubric scores comparing prompt-directed versus unprompted ambient AI documentation. Secondary outcomes include mean total score and criterion-level rubric scores, per-case discussion duration, and documentation administrative times. In the paired sub analysis, prespecified outcomes include total, domain-level, and criterion-level differences between ambient AI and manual documentation for matched cases. Finally, in the internal validation cohort, outcomes included total, criterion, and domain-level rubric scores, discussion duration, and documentation time metrics.
Statistical analysis
Descriptive statistics were used to characterize cohort characteristics and workflow metrics. Data is displayed as mean ± standard deviation for continuous variables or count (percentage) for categorical variables.
No a priori sample size calculation was performed, as enrollment was determined by the patients who were presented during the quality improvement period. A sensitivity analysis was conducted to characterize the minimum effect size the study was powered to detect with baseline and prompt-directed ambient AI discussions at alpha = 0.05 and 80% power for the observed sample size.
A temporal trend analysis within the prompted ambient AI cohort was assessed to isolate the effect of prompting on the primary outcomes. This was done using Spearman rank correlation between total rubric score and discussion date. The magnitude of change was quantified with ordinary least squares regression for total score on days. Non-significance was interpreted as temporal stability within the prompted ambient AI cohort. For the primary comparison cohort, unprompted and prompt-directed ambient AI cases were compared using Mann-Whitney-U tests. For the paired sub analysis, manual and ambient AI documentation from matched cases were compared using paired Wilcoxon signed-rank test. The internal validation cohort was compared to the primary prompt-directed cohort using Mann-Whitney-U tests to assess reproducibility. For domain- and criterion-level analyses, p-values were adjusted using the Benjamini-Hochberg false discovery rate (FDR) procedure.
Clustered boxplots and stacked bar charts were used to display domain-level and criterion-level analysis in the primary cohort (prompted vs unprompted ambient AI cohorts) and the validation comparison (prompted vs validation cohort). Paired sub analysis comparing ambient artificial intelligence and manual documentation were displayed using a paired slope plot.
All statistical analysis was conducted using Python and included the pandas, scipy, statsmodels, and matplotlib packages.
Ethics and Data Security
This study was evaluated by the University of Chicago Institutional Review Board as a quality improvement initiative. Individual patient consent was not required as the study evaluated documentation processes during standard clinical care. Participating tumor board team members provided informed consent.
All audio recordings and documentation were handled in accordance with institutional regulations. Zoom recordings were stored on institutionally encrypted servers. The Abridge platform maintains HIPAA-compliant encryption for data and automatically deletes recordings after 30 days. Audio recordings were retained on secure servers for the duration of the study period and deleted after completion of scoring. All generated summaries were de-identified prior to scoring for this study.
All data presented in this manuscript were de-identified prior to analysis. The Abridge platform was designed to capture patient names as part of standard clinical documentation. For this study, patient names were required to match AI-generated summaries to appropriate medical records for clinical use.
Results
Cohort characteristics
Tumor board characteristics are displayed in Table 2. A total of 12 unique tumor boards were evaluated including 4 unprompted ambient AI tumor boards, 4 prompt-directed ambient AI tumor boards, and 4 additional tumor boards in the internal validation cohort. Two meetings were analyzed in the paired manual comparator sub analysis, so the manual documentation cohort was not mutually exclusive from the primary cohorts. There were 131 patients in the unprompted ambient AI cohort, 73 patients in the prompted ambient AI cohort, 48 patients in the manual documentation group, and 81 patients in the validation cohort.
Table 2.
Tumor board characteristics stratified by prompted discussion
| N (%) or mean (standard deviation) | Baseline ambient AI (N =131) | Prompted ambient AI (N=73) | Manual documentation (N=48) | Validation cohort (N=81) |
|---|---|---|---|---|
|
| ||||
| Primary tumor site | ||||
| Carotid body | 1 (0.8) | 0 (0.0) | 0 (0.0) | 0 (0.0) |
| Carcinoma unknown primary | 1 (0.8) | 1 (1.4) | 0 (0.0) | 0 (0.0) |
| Cutaneous | 11 (8.3) | 2 (2.7) | 1 (2.1) | 7 (8.6) |
| Hypopharynx | 3 (2.3) | 2 (2.7) | 2 (4.2) | 2 (2.5) |
| Larynx | 8 (6.1) | 11 (15.1) | 3 (6.2) | 4 (4.9) |
| Nasopharynx | 1 (0.8) | 1 (1.4) | 0 (0.0) | 2 (2.5) |
| Neck | 1 (0.8) | 0 (0.0) | 0 (0.0) | 1 (1.2) |
| Oral cavity | 46 (35.1) | 13 (17.8) | 17 (35.4) | 27 (33.3) |
| Oropharynx | 36 (27.3) | 25 (34.2) | 17 (35.4) | 16 (19.8) |
| Salivary | 9 (6.9) | 8 (11.0) | 3 (6.2) | 2 (2.5) |
| Sinonasal | 3 (2.3) | 4 (5.5) | 2 (4.2) | 3 (3.7) |
| Thyroid | 9 (6.9) | 6 (8.2) | 2 (4.2) | 11 (13.6) |
| Trachea | 2 (1.5) | 0 (0.0) | 1 (2.1) | 0 (0.0) |
| Number of tumor boards included | 4 | 4 | 2 | 4 |
|
| ||||
| Mean number of cases per tumor board | 32.75 (7.93) | 18.25 (5.68) p=0.114 |
24.0 (2.83) | 20.25 (2.22) p=0.886 |
|
| ||||
| Mean length of tumor board | 96m 34s (24m 4s) | 62m 35s (21m 9s) p=0.114 | 81m 30s (26m 58s) NA | 74m 53s (17m 16s) p = 0.343 |
|
| ||||
| Pathology reviewed | 63 (48.1) | 37 (50.7) | 21 (43.8) | 72 (88.9) |
|
| ||||
| Imaging reviewed | 85 (64.9) | 64 (87.7) | 44 (91.7) | 68 (84.0) |
|
| ||||
| Time spent per patient discussion (minutes) | 2m 57s (1m 58s) | 3m 26s (2m 33s) p=0.158 | 3m 24s (1m 55s) | 3m 42s (2m 15s) p = 0.292 |
|
| ||||
| Administrative time (minutes) | 3m 12s (1m 58s) | 3m 41s (2m 33s) p = 0.158 | 5m 36s (1m 58s) p < 0.001 | 4m 24s (2m 38s) p = 0.074 |
Abbreviations: AI, artificial intelligence
Baseline and prompted ambient AI cohorts were compared using the Mann-Whitney U testing. Manual documentation was compared with their ambient AI counterparts using Wilcoxon signed-rank testing. Validation cohort comparisons were performed against the prompted ambient AI cohort using Mann-Whitney U testing. P-values indicated are of that cohort compared to prompted ambient AI.
The most common primary tumor sites across cohorts were oral cavity in 86 (30.2%) patients and oropharynx in 76 (26.7%) patients.
Primary cohort comparisons
There were no significant differences in tumor board length (p=0.114), time spent per patient discussion, (p = 0.158) or administrative time spent per patient (p = 0.158) between the unprompted and prompt-directed ambient AI groups (Table 2). The standardized effect was Cohen’s d = 0.42 (95% CI, 0.22 – 0.61) and rank biserial r = 0.21 (95% CI, 0.08 – 0.35), consistent with the minimum effect size the study was designed to detect (d = 0.42 by Mann-Whitney U).
Documentation quality analyzed using the modified rubric is shown in Table 3 and Figure 4. Compared with the unprompted ambient AI, prompt-directed ambient AI demonstrated significantly higher scores in the48. For each domain, baseline ambient AI mean scores were greater than 2.5. There was a significant improvement in total score after prompting (2.95 ± 0.10 vs 2.88 ± 0.20, p = 0.005).
Table 3.
Comparison of documentation quality analyzed using mean rubric scores in patients discussed with and without prompting.
| Criterion Mean (STD) |
Baseline ambient AI (N=131) | Prompted ambient AI (N=73) | U | P-value | q-value |
|---|---|---|---|---|---|
|
| |||||
| Demographics | 2.77 (0.63) | 2.96 (0.26) | 3079 | 0.022 | 0.113 |
|
| |||||
| Age | 2.77 (0.64) | 2.96 (0.26) | 2854 | 0.036 | 0.156 |
| Comorbidities | 2.78 (0.55) | 3.00 (0.00) | 172.5 | 0.048 | 0.171 |
|
| |||||
| Tumor characteristics | 2.79 (0.42) | 2.94 (0.18) | 3726.5 | 0.005 | 0.042 |
|
| |||||
| Primary site | 2.80 (0.51) | 2.94 (0.29) | 3927 | 0.023 | 0.113 |
| Histology | 2.80 (0.46) | 2.99 (0.12) | 2420.5 | <0.001 | 0.021 |
| TNM Staging | 2.73 (0.50) | 2.89 (0.38) | 858.5 | 0.055 | 0.173 |
| HPV status | 2.74 (0.65) | 3.00 (0.00) | 168 | 0.066 | 0.183 |
|
| |||||
| Radiology findings | 2.95 (0.22) | 2.98 (0.14) | 1936.5 | 0.351 | 0.487 |
|
| |||||
| Locoregional findings | 2.94 (0.24) | 3.00 (0.00) | 1527.5 | 0.096 | 0.219 |
| Distant findings | 3.00 (0.00) | 2.94 (0.25) | 221 | 0.22 | 0.393 |
|
| |||||
| Pathology findings | 3.00 (0.00) | 2.83 (0.41) | 52.5 | 0.14 | 0.292 |
|
| |||||
| Perineural invasion | 3.00 (0.00) | 3.00 (0.00) | 24 | 1 | 1 |
| Lymphovascular invasion | 3.00 (0.00) | 3.00 (0.00) | 10 | 1 | 1 |
| Extranodular extension | 3.00 (0.00) | 2.50 (1.00) | 17.5 | 0.257 | 0.401 |
|
| |||||
| Discussion | 2.92 (0.19) | 2.94 (0.13) | 4628 | 0.611 | 0.743 |
|
| |||||
| Surgical oncology | 2.99 (0.09) | 3.00 (0.00) | 4087 | 0.467 | 0.615 |
| Medical oncology | 3.00 (0.00) | 3.00 (0.00) | 1903.5 | 1 | 1 |
| Radiation oncology | 2.96 (0.20) | 3.00 (0.00) | 705 | 0.273 | 0.402 |
| Speech language pathology | 2.00 (0.00) | 3.00 (0.00) | 0 | 0.096 | 0.219 |
| Nursing coordinator | 2.93 (0.27) | 3.00 (0.00) | 225 | 0.256 | 0.401 |
| Pathology | 2.84 (0.41) | 2.95 (0.23) | 1061 | 0.171 | 0.329 |
| Radiology | 2.94 (0.28) | 2.97 (0.18) | 2676 | 0.624 | 0.743 |
| Treatment | |||||
| recommendation | 2.84 (0.37) | 2.81 (0.43) | 4791 | 0.751 | 0.854 |
|
| |||||
| Total | 2.88 (0.20) | 2.95 (0.10) | 3763.5 | 0.005 | |
Abbreviation: STD, standard deviation; AI, artificial intelligence
Figure 4.
Domain and criterion-level rubric score performance between baseline and prompted ambient artificial intelligence documentation
A) Clustered boxplot bars show mean (standard error) rubric scores for each domain and total scores prior to and after implementation of directed prompting at tumor board discussions B) Stacked bar plots display the proportion of ordinal rubric scores (1= incorrect, 2= partial, 3= comprehensive) for each criterion across all domains. Asterisks (*) denote unadjusted p <0.05 statistically significant differences in documentation quality between prompting conditions.
Of note, total rubric score did not show significant temporal trend in the prompted ambient AI group. Ordinary least squares regression yielded a slope of 0.0008 points/day (95% CI, 0.0011 – 0.0027; p = 0.41). The prompt-directed ambient AI documentation quality remained stable across the prompt-directed ambient AI cohort, with no drift.
Paired sub analysis
There was a significant increase in administrative time spent per patient discussion in the manual documentation vs ambient AI groups (5m 36s ± 1m 58s vs 3m 39s ± 1m 55s, p < 0.001). Documentation quality comparing the ambient AI and manual documentation group using the modified rubric is shown in Table 4 and Figure 5. In the domain-level paired analysis, the Discussion domain exhibited statistically significantly improved rubric scores in the ambient AI group compared to the manual documentation group (2.94 ± 0.10 vs 2.69 ± 0.29, q < 0.001). At the criterion-level, the ambient AI group was better able to capture pathology input (q = 0.016), and radiology input (q < 0.001). The ambient AI group achieved a statistically significantly higher total score (2.89 ± 0.16 vs 2.76 ± 0.23, p < 0.001) than the manual documentation group.
Table 4.
Paired comparison of documentation quality analyzed using mean rubric scores in ambient AI and manually documented tumor board discussions.
| Criterion Mean (STD) N=48 |
Ambient AI | Manual documentation | P-value | q-value |
|---|---|---|---|---|
|
| ||||
| Demographics | 2.95 (0.30) | 3.00 (0.00) | 0.18 | 0.359 |
|
| ||||
| Age | 2.96 (0.30) | 3.00 (0.00) | 0.317 | 0.448 |
| Comorbidities | 2.91 (0.30) | 3.00 (0.00) | 1 | 1 |
|
| ||||
| Tumor characteristics | 2.75 (0.49) | 2.77 (0.38) | 0.641 | 0.81 |
|
| ||||
| Primary site | 2.78 (0.59) | 2.87 (0.34) | 0.26 | 0.415 |
| Histology | 2.77 (0.48) | 2.75 (0.49) | 0.564 | 0.752 |
| TNM Staging | 2.69 (0.62) | 2.77 (0.59) | 0.317 | 0.448 |
| HPV status | 3.00 (0.00) | 3.00 (0.00) | 1 | 1 |
|
| ||||
| Radiology findings | 2.97 (0.18) | 2.77 (0.58) | 0.039 | 0.135 |
|
| ||||
| Locoregional findings | 2.97 (0.18) | 2.77 (0.57) | 0.034 | 0.135 |
| Distant findings | 3.00 (0.00) | 2.80 (0.63) | 1 | 1 |
|
| ||||
| Pathology findings | 3.00 (0.00) | 1.67 (0.58) | 0.25 | 0.415 |
|
| ||||
| Perineural invasion | 3.00 (0.00) | 2.00 (1.41) | 1 | 1 |
| Lymphovascular
invasion Extranodular extension |
3.00 (0.00) | 1.33 (0.58) | 0.25 | 0.415 |
|
| ||||
| Discussion | 2.94 (0.10) | 2.69 (0.29) | <0.001 | <0.001 |
|
| ||||
| Surgical oncology | 3.00 (0.00) | 2.94 (0.24) | 0.083 | 0.182 |
| Medical oncology | 3.00 (0.00) | 2.76 (0.50) | 0.011 | 0.055 |
| Radiation oncology | 2.95 (0.23) | 2.62 (0.67) | 0.059 | 0.167 |
| Speech language pathology | 2.67 (0.58) | 2.67 (0.58) | 1 | 1 |
| Nursing coordinator | 3.00 (0.00) | 2.31 (0.85) | 0.062 | 0.167 |
| Pathology | 2.86 (0.36) | 2.43 (0.51) | 0.003 | 0.016 |
| Radiology | 2.98 (0.15) | 2.57 (0.55) | <0.001 | <0.001 |
| Treatment | ||||
| recommendation | 2.83 (0.38) | 2.71 (0.46) | 0.083 | 0.182 |
|
| ||||
| Total | 2.89 (0.16) | 2.76 (0.23) | <0.001 | |
Abbreviation: STD, standard deviation; AI, artificial intelligence
Figure 5.
Paired sub analysis of ambient artificial intelligence versus manual hand documentation.
A) Paired comparison of total rubric scores for patient discussions documented using ambient AI versus hand documentation. Bold line indicates mean paired difference. B) Paired comparison of domain-level rubric scores for paired patient discussions. Bold line indicates mean paired difference. Asterisks denote unadjusted P <0.05 based on two-sided Wilcoxon signed-rank tests.
Internal validation
Internal validation performance is shown in Figure 6. There was no significant difference in tumor board length (p = 0.343), time spent per patient discussion, (p = 0.292) and administrative time spent per patient (p = 0.074) between the prompted ambient AI cohort and the validation cohort (Table 2). Documentation quality was similar between both cohorts. After FDR correction, there was no significant differences in domain-level rubric scores. The total score was statistically similar between the prompted ambient AI cohort and the validation cohort (2.95 ± 0.10 vs 2.96 ± 0.09, p = 0.948).
Figure 6.
Domain and criterion-level rubric score validation cohort performance compared to the primary prompted ambient artificial intelligence cohort
A) Clustered boxplot bars show mean (standard error) rubric scores for each domain and total scores prior to and after implementation of directed prompting at tumor board discussions B) Stacked bar plots display the proportion of ordinal rubric scores (1= incorrect, 2= partial, 3= comprehensive) for each criterion across all domains. Asterisks (*) denote unadjusted p <0.05 statistically significant differences in documentation quality between groups.
Discussion
Ambient artificial intelligence technology has made significant strides in improving the efficiency of clinical documentation, allowing physicians to focus on patient care and improve workflow. This study was the first prospective quality improvement study to evaluate ambient AI in tumor board documentation across a primary cohort analyzing unprompted vs prompt-directed ambient AI, a paired sub analysis comparing manual documentation to ambient AI documentation, and an internal validation cohort to assess reproducibility. Prompting improved Tumor Characteristics and overall documentation quality in the primary cohort. Mixed-model sensitivity. In the paired sub analysis, ambient AI demonstrated significant improvement in total score and Discussion domain documentation compared to manual documentation, while significantly decreasing administrative time spent per patient. Internal validation performance was statistically similar to the prompt-directed cohort.
Ambient AI platforms have demonstrated success as an alternative to scribes in clinical settings. The use of this technology has been limited to a patient and provider in the outpatient or emergency room setting. An oncological study by Baldwin-Medsker et al, revealed that 94% of providers found the generated summaries usable and 61% of providers indicated likelihood for continued use of technology for integration into clinical workflow.17 The integration of ambient artificial intelligence has made significant advancements in reducing providers’ mental effort and time spent on documentation, leading to improved satisfaction and decreased burn out.12,13,18 A study by Stults et al, demonstrated a significant improvement in mental demand and effort (measured using the NASA Task Load Index ) when using ambient artificial intelligence for documentation. This was associated with a decrease of 0.91 minutes in time spent note-taking per appointment (95% CI, −1.20 to −0.62 minutes, p < .001). Nevertheless, there continues to be challenges surrounding acceptance of this tool.19 Errors and inaccuracies in generated summaries require continued oversight by physicians and failure to correct these can have clinical and medicolegal implications. A study using ChatGPT-4, a generative artificial intelligence chatbot, showed a mean of 23.6 errors per clinical case in simulated patient-provider encounters representing varying specialties.19 These errors included omission (86%), addition errors (10.5%), and inclusion of incorrect facts (3.2%). Importantly transcript length and data complexity were inversely correlated with note accuracy (p = 0.05). The authors underscore the platform’s limitation in handling complex medical cases.
As such we aimed to evaluate the use of ambient AI in the setting of complex case discussions. Multidisciplinary tumor board documentation is particularly challenging due to the nature of oncological decision making involving multiple providers. Unprompted ambient artificial intelligence was able to create highly comprehensive summaries with excellent rubric scores. Baseline mean scores for the tumor characteristics, radiology findings, pathology findings, and discussion domains were excellent (mean >2.5). Nevertheless, prompt-directed ambient AI documentation showed a significant improvement in Tumor characteristics domain documentation, histology criterion documentation, and total score. A standardized prompting framework improves captured content by making clinical elements more explicit during discussion. This is important because tumor board discussions are dense in information and require participation of multiple people, potentially limiting performance of ambient AI platforms that were originally meant for a conversation between one provider and their patient. Importantly, there was no difference in time spent in discussion when directed prompting was applied during tumor board (2m 57s ± 1m 58s unprompted vs 3m 26s ± 2m 33s in prompted discussions (p = 0.158)). It is possible that the standardized prompting framework improved not only ambient AI-generated documentation but the tumor board structure itself. The prompting may have encouraged clearer presentations by having the presenter systematically address demographics, tumor characteristics, imaging and pathology findings, and recommendations. This secondary effect could promote multidisciplinary communication independent of the model’s performance. Nevertheless, these prompts provide a standardized scalable solution for enhancing ambient AI-assisted documentation without need for new software or model fine tuning. The internal validation cohort further strengthened the reproducibility of our prompt-directed ambient AI workflow outside the constrains of a quality improvement study.
Current documentation largely relies on active listening to tumor board discussions and manually documenting the final diagnostic or treatment recommendations only. This cumbersome task largely falls on junior trainees and can lead to trainee burnout.9 The manual nature of this workflow can lead to variable quality and often lacks standardization. In comparison with manual documentation, ambient AI demonstrated a statistically significant improvement in Discussion domain documentation leading to an improved total rubric score while requiring less administrative time spent per patient. This is particularly notable because ambient AI does not only record isolated facts regarding demographics, tumor characteristics etc, but also captures the complex reasoning that leads to subsequent patient care. Ambient AI provides a scalable advantage in tumor board documentation because it better captures elements of discussion while reducing time spent doing so.
This study has multiple limitations. The sequential, non-randomized design was chosen given the practical constraints of continuing our usual tumor board, which introduces the risk for period effects, selection bias, and learning contamination. An additional methodological concern is that our scoring rubric assigned equal weight to all five documentation elements, despite components such as treatment recommendation having more clinical consequence than patient demographics. We chose this unweighted approach to provide comprehensive assessment across all documentation domains and to provide a transparent and reproducible framework to compare generated summaries. A weighed scoring system can assign relatively greater importance to domains that more directly impact clinical care, and this can be incorporated in future iterations of our study. Cases were not scored by both raters, limiting the assessment of inter-rater reliability. Nevertheless, the raters were trained and blinded to cohort and reviewed cases from all cohorts. We also did not perform a separated structured audit of error types or downstream safety implications. The documentation quality was scored against source recording and administrative time was measured, which included reviewer validation time for ambient AI generated summaries. Importantly, our study does not evaluate ambient AI as a fully autonomous documentation tool. Rather, it was assessed with a human-in-the-loop model for verification of generated summaries, which reflects safe implementation in the context of multidisciplinary setting. An important limitation is that we used the Abridge ambient AI documentation platform without performing any institution-specific fine-tuning. This decision was made to demonstrate the feasibility of implementing an already existing ambient AI platform without model retraining. Our goal was to evaluate real-world implementation of a commercially available tool.
While our study demonstrates that prompt-directed ambient AI enhances the completeness of tumor board documentation, it remains unknown whether these improvements translate into improved communication with patients or patient outcomes. We aim to scale our study to multiple tumor boards across the institution and integrate this into tumor board workflow. Eventually, we aim to generalize our findings to community cancer centers, other tumor types with different discussion patterns or terminology, and other healthcare systems. Ambient AI has the potential to be integrated to the electronic health records, with human oversight, to improve accessibility of clinical documentation for providers and patients. Currently, the manual aspect of recording tumor board discussions and passing each patient discussion individually into Abridge limits widespread use. Whether this application of tumor board documentation is correlated with improved oncological outcomes is yet to be elucidated. Future work should also include clinician satisfaction of generated summaries in a blinded fashion to assess real-world usability.
Our study is a novel application of ambient artificial intelligence in a highly complex multidisciplinary setting. Tasks that were previously laborious and for which documentation could be highly variable can now be mitigated with the use of this technology. We provide a head and neck specific modification to a validated rubric to assess quality of tumor board documentation. Finally, we have created a prompting template to standardize and improve documentation. This results in accurate summarizations of complex medical discussions in real-time, which preserves the precision of discussions and potentially reduces the load of robust documentation.
Conclusion
Ambient AI software, Abridge, provides real-time comprehensive summaries for tumor board documentation. Directed prompting during head and neck tumor board provides significant improvements to the AI-generated summaries with no statistically significant difference in time spent on patient discussions. In comparison to manual documentation, Ambient AI documentation improves the capture of complex tumor board discussion elements while diminishing administrative burden. We deliver a scalable, generalizable workflow for the use of ambient AI for tumor board documentation.
Supplementary Material
Highlights.
Ambient AI was feasible for documentation of multidisciplinary head and neck tumor board discussions in routine workflow.
Directed prompting during tumor board significantly improves quality of generated summaries using the ambient AI platform.
Ambient AI reduces administrative documentation time compared with manual handwritten documentation.
Ambient AI captures tumor board domains, particularly discussion elements, more comprehensively than manual documentation.
Internal validation showed similar performance, supporting a reproducible workflow for prompted AI documentation.
Funding:
Funding for this work was provided by Biohub Chicago, LLC, NIH R01CA293995–01, and generous donations by Tom Wehmeier and Mike Sekulic.
Footnotes
Declaration of generative AI and AI-assisted technologies in the manuscript preparation process
During the preparation of this work the author(s) used ChatGPT (OpenAI) to improve readability and language consistency of the manuscript sections. The tool was not used to generate scientific content, analyze data, interpret results, or draw conclusions. After using this tool/service, the author(s) reviewed and edited the content as needed and take(s) full responsibility for the content of the published article.
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
References
- 1.Quero G, Salvatore L, Fiorillo C et al. The impact of the multidisciplinary tumor board (MDTB) on the management of pancreatic diseases in a tertiary referral center. ESMO Open 2021; 6:100010. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Habermann TM, Khurana A, Lentz R et al. Analysis and impact of a multidisciplinary lymphoma virtual tumor board. Leuk Lymphoma 2020; 61:3351–3359. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Nguyen NP, Vos P, Lee H et al. Impact of tumor board recommendations on treatment outcome for locally advanced head and neck cancer. Oncology 2008; 75:186–191. [DOI] [PubMed] [Google Scholar]
- 4.Algwaiz G, Salam Y, Bustami R, Ferwana M, Jazieh AR. Do Multidisciplinary Tumor Board Discussions Correlate With Increase in 5-Year Survival? A Meta-Analysis Study. Glob J Qual Saf Healthc 2021; 4:3–10. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Brink LVD, Ruiter AEC, Lagerveld BW et al. The Impact of a Multidisciplinary Tumor Board (MTB) on Treatment Decision Making for Patients With Renal Cell Carcinoma (RCC): 5-Year Data Analysis. Clin Genitourin Cancer 2024; 22:610–617.e611. [DOI] [PubMed] [Google Scholar]
- 6.Karas PL, Rankin NM, Stone E. Medicolegal Considerations in Multidisciplinary Cancer Care. JTO Clin Res Rep 2020; 1:100073. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Mobley EM, Swami U, Mott S, Ounda A, Milhem M, Monga V. A Retrospective Analysis of Clinical Trial Accrual of Patients Presented in a Multidisciplinary Tumor Board at a Tertiary Health Care Center and Associated Barriers. Oncol Res Treat 2020; 43:196–203. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.El Saghir NS, Keating NL, Carlson RW, Khoury KE, Fallowfield L. Tumor boards: optimizing the structure and improving efficiency of multidisciplinary management of patients with cancer worldwide. Am Soc Clin Oncol Educ Book 2014:e461–466. [DOI] [PubMed] [Google Scholar]
- 9.Yalamanchi P, Casper KA, Evans J et al. Facilitating Automated Data Analytics Through Structured Head and Neck Oncology Tumor Board Documentation. JCO Clinical Cancer Informatics 2022:e2100168. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Farrugia DJ, Fischer TD, Delitto D, Spiguel LR, Shaw CM. Improved Breast Cancer Care Quality Metrics After Implementation of a Standardized Tumor Board Documentation Template. J Oncol Pract 2015; 11:421–423. [DOI] [PubMed] [Google Scholar]
- 11.van Buchem MM, Boosman H, Bauer MP, Kant IMJ, Cammel SA, Steyerberg EW. The digital scribe in clinical practice: a scoping review and research agenda. NPJ Digit Med 2021; 4:57. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Albrecht M, Shanks D, Shah T et al. Enhancing clinical documentation with ambient artificial intelligence: a quality improvement survey assessing clinician perspectives on work burden, burnout, and job satisfaction. JAMIA Open 2025; 8:ooaf013. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Balloch J, Sridharan S, Oldham G et al. Use of an ambient artificial intelligence tool to improve quality of clinical documentation. Future Healthc J 2024; 11:100157. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Meskó B Prompt Engineering as an Important Emerging Skill for Medical Professionals: Tutorial. J Med Internet Res 2023; 25:e50638. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Abridge AI I Generative AI for Clinical Conversations. Available at:https//www.abridge.com. Accessed March 29, 2026. [Google Scholar]
- 16.Lamb BW, Wong HW, Vincent C, Green JS, Sevdalis N. Teamwork and team performance in multidisciplinary cancer teams: development and evaluation of an observational assessment tool. BMJ Qual Saf 2011; 20:849–856. [DOI] [PubMed] [Google Scholar]
- 17.Baldwin-Medsker AE, Chimonas S, Goyal SP et al. Ambient Artificial Intelligence Scribes in Oncology: Adoption, Feasibility, Acceptability, and Appropriateness. Appl Clin Inform 2025; 16:995–1004. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Stults CD, Deng S, Martinez MC et al. Evaluation of an Ambient Artificial Intelligence Documentation Platform for Clinicians. JAMA Netw Open 2025; 8:e258614. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Kernberg A, Gold JA, Mohan V. Using ChatGPT-4 to Create Structured Medical Notes From Audio Recordings of Physician-Patient Encounters: Comparative Study. J Med Internet Res 2024; 26:e54419. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.