Table 4.
Performance of 12 language models on Deauville score prediction
| Models | 5-Class Accuracy (%) | Weighted Cohen’s κ |
|---|---|---|
| PGN | 53.5 [46.9, 60.2] | 0.528 [0.445, 0.605] |
| BERT2BERT | 69.7 [64.9, 74.3] | 0.762 [0.716, 0.805] |
| BART | 75.3 [70.6, 79.7] † | 0.806 [0.760, 0.846] † |
| BioBART | 73.9 [69.7, 78.1] † | 0.802 [0.761, 0.840] † |
| PEGASUS | 76.7 [72.0, 81.0] * | 0.811 [0.767, 0.852] † |
| T5 | 76.3 [72.0, 80.6] † | 0.814 [0.772, 0.853] * |
| Clinical-T5 | 72.5 [67.7, 77.0] † | 0.788 [0.745, 0.829] † |
| FLAN-T5 | 72.6 [68.0, 77.2] † | 0.798 [0.757, 0.837] † |
| GPT2 | 71.3 [65.8, 76.4] | 0.768 [0.715, 0.817] † |
| OPT | 63.1 [57.7, 68.6] | 0.718 [0.665, 0.767] |
| LLaMA-LoRA | 62.9 [56.8, 68.7] | 0.708 [0.647, 0.763] |
| Alpaca-LoRA | 70.6 [64.9, 75.8] | 0.754 [0.696, 0.805] |
Note that data are shown as mean [2.5th percentile, 97.5th percentile]. “*” denotes the best model for each metric, and “†” denotes the other models that have no statistically significant difference (P > 0.05) with the best model