Limitations of ML Functional Performance Metrics CT-AI Practice Questions (Page 4)

Questions 16–20

1. 16foundation · easy

   A model for a rare event has a precision of 0.9 and a recall of 0.2. Which statement best describes why recall alone is incomplete for evaluating this model?

   • ARecall does not consider the number of false positives, so a model with high recall might produce many false alarms.
   • BRecall is always lower than precision in imbalanced datasets.
   • CRecall is not a valid metric for imbalanced datasets.
   • DRecall only measures the model's performance on the negative class.
   Reveal Solution Discussion Select an answer first
2. 17foundation · easy

   Which statement best describes a limitation of the F1-score?

   • AF1-score is always lower than both precision and recall.
   • BF1-score is not affected by class imbalance.
   • CF1-score is the harmonic mean of precision and recall, so it gives equal weight to both, which may not reflect the relative importance of precision and recall in a specific application.
   • DF1-score is only defined for binary classification problems.
   Reveal Solution Discussion Select an answer first
4. 18expert · hard

   A hospital uses an ML model to predict patient readmission within 30 days. The readmission rate is 8%. The model achieves 92% accuracy, but the hospital's goal is to reduce readmissions by identifying high-risk patients for intervention. The cost of a false negative (missing a patient who will be readmitted) is high, while the cost of a false positive (unnecessary intervention) is low. Which metric should the team prioritize to align with the hospital's goal?

   • ARecall for the readmission class
   • BAccuracy
   • CF1-score
   • DPrecision for the readmission class
   Reveal Solution Discussion Select an answer first
5. 19foundation · easy

   A multi-class classifier has an overall accuracy of 90%. Which limitation of confusion matrix metrics is most relevant when evaluating this model?

   • AThe overall accuracy is always higher than the per-class accuracy.
   • BThe confusion matrix cannot be used for multi-class problems.
   • CThe overall accuracy may hide that the model performs poorly on a specific class, such as a rare but important class.
   • DThe confusion matrix metrics are only valid if the classes are balanced.
   Reveal Solution Discussion Select an answer first
6. 20foundation · easy

   Which statement best describes a limitation of metrics computed on a static test set in a non-stationary environment?

   • AStatic test set metrics are not affected by data drift.
   • BStatic test set metrics are only valid for regression problems.
   • CThe metrics may not reflect the model's performance when the data distribution shifts over time.
   • DStatic test set metrics are always higher than metrics on new data.
   Reveal Solution Discussion Select an answer first