Limitations of ML Functional Performance Metrics CT-AI Practice Questions (Page 2)

Questions 6–10

1. 6foundation · easy

   Which statement best describes a limitation of confusion matrix metrics?

   • AConfusion matrix metrics always reflect the model's performance on each individual class.
   • BConfusion matrix metrics provide an aggregate view that can hide performance differences across classes or subgroups.
   • CConfusion matrix metrics cannot be computed for multi-class problems.
   • DConfusion matrix metrics are only useful for binary classification.
   Reveal Solution Discussion Select an answer first
2. 7application · medium

   A bank deploys a fraud detection model that flags only 0.1% of transactions as fraudulent. The model achieves 99.9% accuracy on a held-out test set. The bank's compliance team is concerned about the high number of false negatives, which allow fraudulent transactions to go undetected. Which metric would BEST reveal the model's poor performance on the minority class?

   • AF1-score for the non-fraud class
   • BOverall accuracy on the test set
   • CROC-AUC for the fraud class
   • DRecall (sensitivity) for the fraud class
   Reveal Solution Discussion Select an answer first
4. 8foundation · easy

   A model for loan approval has an overall accuracy of 95%. Why might this aggregate metric be misleading?

   • AThe model is biased because the accuracy is not 100%.
   • BThe model is overfitting because the accuracy is too high.
   • CAccuracy is not a valid metric for loan approval because it is a regression problem.
   • DThe model might have very low accuracy for a specific subgroup, such as applicants from a certain region, but this is hidden by the overall average.
   Reveal Solution Discussion Select an answer first
5. 9foundation · easy

   Which statement best describes a limitation of the Mean Squared Error (MSE) as a regression metric?

   • AMSE is not affected by the scale of the target variable.
   • BMSE is always lower than MAE.
   • CMSE is only defined for binary classification.
   • DMSE is heavily influenced by outliers because errors are squared, so a few large errors can dominate the metric.
   Reveal Solution Discussion Select an answer first
6. 10foundation · easy

   Which statement best describes a limitation of PR-AUC?

   • APR-AUC is sensitive to the class distribution, so values from datasets with different positive-to-negative ratios are not directly comparable.
   • BPR-AUC is always higher than ROC-AUC.
   • CPR-AUC cannot be computed for binary classification.
   • DPR-AUC is not affected by the choice of decision threshold.
   Reveal Solution Discussion Select an answer first