Limitations of ML Functional Performance Metrics CT-AI Practice Questions (Page 3)

Questions 11–15

1. 11foundation · easy

   A binary classifier has a precision of 0.8 and recall of 0.6 at a threshold of 0.5. If the threshold is lowered to 0.3, what is the most likely effect on precision and recall?

   • ABoth precision and recall will increase.
   • BBoth precision and recall will decrease.
   • CRecall will likely increase, and precision will likely decrease.
   • DPrecision will likely increase, and recall will likely decrease.
   Reveal Solution Discussion Select an answer first
2. 12foundation · easy

   Why is the Root Mean Squared Error (RMSE) not directly comparable across datasets with different target variable scales?

   • ARMSE is only defined for classification problems.
   • BRMSE is scale-dependent because it is in the same units as the target variable, so a model predicting house prices in dollars will have a different RMSE magnitude than one predicting in thousands of dollars.
   • CRMSE is always lower than MAE.
   • DRMSE is not affected by outliers.
   Reveal Solution Discussion Select an answer first
4. 13expert · hard

   A model classifies customer support tickets into 'urgent' and 'non-urgent'. The overall precision is 0.85 and recall is 0.80. However, the model performs poorly on tickets written in non-English languages, with a recall of only 0.30 for those tickets. The team is considering deploying the model. Which approach would BEST address the limitation of aggregate metrics?

   • ACompute and report metrics separately for each language subgroup
   • BUse F1-score instead of precision and recall
   • CUse a higher decision threshold to improve overall precision
   • DDeploy the model because the overall metrics are acceptable
   Reveal Solution Discussion Select an answer first
5. 14foundation · easy

   Which statement best describes a limitation of aggregate metrics like overall accuracy or macro-F1?

   • AAggregate metrics provide a single number that can obscure performance differences across data segments or classes.
   • BAggregate metrics are not affected by class imbalance.
   • CAggregate metrics are only useful for binary classification.
   • DAggregate metrics are always computed on the training set.
   Reveal Solution Discussion Select an answer first
6. 15foundation · easy

   In a spam detection system, false negatives (spam that is not caught) are considered much more harmful than false positives (legitimate emails marked as spam). Which limitation of F1-score is most relevant?

   • AF1-score does not allow assigning different weights to precision and recall.
   • BF1-score is only useful for regression problems.
   • CF1-score automatically adjusts for unequal error costs.
   • DF1-score is not computable when the dataset is imbalanced.
   Reveal Solution Discussion Select an answer first