A team is developing a supervised learning model. They split the data into training and test sets, but they accidentally use the test set to tune hyperparameters. What is the likely consequence?

The model will underfit the training data

The test set will become larger

The model's performance on the test set will be optimistically biased

The model will perform better on new data

A data scientist is comparing two models for a binary classification task. Model A achieves 90% accuracy on the test set, while Model B achieves 85% accuracy but has much higher recall for the positive class. The cost of missing a positive case is high. Which model should be chosen?

Model A, because higher accuracy is always better

Model B, because higher recall is more important when the cost of false negatives is high

Model A, because it has higher precision

Model B, because it has lower accuracy, which is better

What does 'accuracy' measure in a classification model?

The proportion of correct predictions out of all predictions made.

The difference between predicted and actual values.

The proportion of positive predictions that are actually correct.

The proportion of actual positive cases that are correctly identified.

A model achieves 95% accuracy on the test set, but the test set contains 95% non-spam emails and only 5% spam. The model classifies all emails as non-spam. What is the main problem with using accuracy as the evaluation metric?

Accuracy is misleading with imbalanced classes

Accuracy is too complex to compute

Accuracy only works for regression

Accuracy is not a valid metric for classification

A bank wants to build a model that predicts whether a loan applicant will default. The historical data contains customer attributes and a 'defaulted' flag for each past loan. The data science team is deciding which ML approach to use. Which approach is most appropriate?

Supervised learning with the 'defaulted' flag as the target variable

Semi-supervised learning using only the customer attributes

Unsupervised clustering to group similar customers

Reinforcement learning to optimize loan decisions over time

Certified Tester AI Testing

Domain 3Objective 1

Supervised Learning CT-AI Practice Questions (Page 3)

Part of the Domain 3: Machine Learning (ML) - Overview domain, which makes up ~15% of our current practice bank. ISTQB does not publish an official question count, but from its 60-minute exam (~25–40 total, ~4–6 in this domain), expect 1–1 from this objective — we provide 27 practice questions to prepare you well beyond it. (estimate)

27questions here

6free pages

6concepts

Questions 11–15

11expert · hard
A team is developing a supervised learning model. They split the data into training and test sets, but they accidentally use the test set to tune hyperparameters. What is the likely consequence?
Select an answer first
12expert · hard
A data scientist is comparing two models for a binary classification task. Model A achieves 90% accuracy on the test set, while Model B achieves 85% accuracy but has much higher recall for the positive class. The cost of missing a positive case is high. Which model should be chosen?
Select an answer first
13foundation · easy
What does 'accuracy' measure in a classification model?
Select an answer first
14application · medium
A model achieves 95% accuracy on the test set, but the test set contains 95% non-spam emails and only 5% spam. The model classifies all emails as non-spam. What is the main problem with using accuracy as the evaluation metric?
Select an answer first
15application · medium
A bank wants to build a model that predicts whether a loan applicant will default. The historical data contains customer attributes and a 'defaulted' flag for each past loan. The data science team is deciding which ML approach to use. Which approach is most appropriate?
Select an answer first

Finished these 5 questions?

Review the revealed explanations, or continue through the curriculum.

Next 5 questions →Back to CT-AI

Free Basic Practice is a study aid with revealable answers — not a scored exam. Examers.io is independent and not affiliated with or endorsed by ISTQB. “CT-AI” is a trademark of its owner, used for identification only.