Table 4:
Correlation coefficients between automated metrics and human evaluation facets. There is weak to no correlation between metrics and human-assessed facets (aside from between PIO-overlap and PIO). Statistical significance at α = 0.05 is marked with *, 0.01 with **, though these thresholds for significance do not account for multiple hypothesis testing.
| Metric | Flu. | PIO | Dir. | Str. |
|---|---|---|---|---|
| ROUGE | −0.014 | −0.010 | 0.007 | −0.035 |
| BERTScore | −0.000 | 0.022 | 0.036 | −0.033 |
| Delta-EI | 0.066 | −0.080 | −0.060 | −0.054 |
| ClaimVer | −0.051 | 0.142** | −0.017 | −0.093* |
| NLI | −0.026 | 0.053 | −0.011 | −0.063 |
| STS | −0.042 | 0.066 | 0.001 | −0.056 |
| PIO-Overlap | 0.043 | 0.358** | 0.033 | 0.050 |