Test Creation CERTIFIED-DATA-SCIENCE-PRACTITIONER Practice Questions (Page 4)

Questions 16–20

1. 16expert · hard

   A team is designing test cases for a model that predicts whether a transaction is fraudulent. They have a test set with 1,000 fraudulent and 99,000 legitimate transactions. They want to ensure the model is tested on rare fraud patterns. What is the most effective approach?

   • AUse a random sample of the test set.
   • BUse only the fraudulent transactions for testing.
   • CCreate synthetic fraudulent transactions that mimic rare patterns.
   • DUse the natural distribution and rely on the model's ability to detect rare patterns.
   Reveal Solution Discussion Select an answer first
2. 17application · medium

   A team is testing a model that predicts customer lifetime value. They have a dataset with 1 million customers, but they only have enough computational resources to test on 10,000 customers. They randomly sample 10,000 customers from the dataset. What is the primary risk of this approach?

   • AThe test set may not be large enough to detect rare but important customer segments.
   • BThe test set will be biased because it is not the full dataset.
   • CThe test set will be too large for the model to handle.
   • DThe test set will be independent of the training data.
   Reveal Solution Discussion Select an answer first
4. 18expert · hard

   A team is building a test harness for a model that is retrained weekly. They want to compare the performance of each new model version against the previous one. They also want to ensure that any regression is caught early. What is the most effective way to design the test harness?

   • ARun tests on a random sample of data each week.
   • BRun tests only when a major version is released.
   • CRun a different set of test cases each week to cover more scenarios.
   • DRun the same test suite on each version and store results in a versioned database.
   Reveal Solution Discussion Select an answer first
5. 19application · medium

   A team is setting up a test harness for a model that is updated frequently. They want to automatically run a predefined set of test cases and generate a report after each update. Which component is essential for this automation?

   • AA dedicated testing team.
   • BA script that runs the test cases and saves results to a file.
   • CA manual checklist for testers to follow.
   • DA larger test dataset.
   Reveal Solution Discussion Select an answer first
6. 20application · medium

   A team has completed testing a model and needs to document the results for stakeholders. They have recorded the test cases, metrics, and any issues encountered. What is the most important element to include in the documentation to ensure the results are reproducible?

   • AThe exact version of the model and the test data used.
   • BThe number of test cases that passed.
   • CThe names of the team members who ran the tests.
   • DThe total time spent on testing.
   Reveal Solution Discussion Select an answer first