Hands-On Exercise: Exploratory Testing and Exploratory Data Analysis (EDA) CT-AI Practice Questions (Page 1)

Questions 1–5

1. 1application · medium

   A data analyst is performing EDA on a dataset used to train a spam-detection model. The analyst notices that the feature 'email_length' has a highly skewed distribution, with most values between 50 and 200 characters, but a few values exceeding 10,000 characters. What is the most appropriate initial action during EDA?

   • AInvestigate the extreme values to determine if they are valid long emails or data errors.
   • BIgnore the skew because the model will handle it automatically.
   • CApply a log transformation to the feature to normalize the distribution.
   • DRemove all emails longer than 1,000 characters to simplify the dataset.
   Reveal Solution Discussion Select an answer first
2. 2expert · hard

   A team is developing a sentiment-analysis model for product reviews. During exploratory testing, they find the model misclassifies reviews that contain sarcasm. In parallel, EDA reveals that the training data contains very few examples of sarcastic reviews. The team has limited time and budget. What is the most effective integrated approach to address this issue?

   • AConduct more exploratory testing to find all possible failure cases before making any changes.
   • BAdd a rule-based pre-processing step that flags sentences containing exclamation marks as sarcastic.
   • CImmediately retrain the model with a larger, more complex neural network architecture to improve its ability to learn sarcasm.
   • DUse the EDA findings to quantify the under-representation of sarcasm, then use exploratory testing to identify specific failure patterns, and prioritize collecting more sarcastic examples.
   Reveal Solution Discussion Select an answer first
4. 3application · medium

   A tester is exploring a new AI-based fraud-detection system. The tester has a list of known fraud patterns from previous systems. What is the most effective way to use this knowledge in exploratory testing?

   • AAsk the developer to write test cases for the known patterns.
   • BIgnore the known patterns because exploratory testing should be completely unscripted.
   • CUse the known patterns as a starting point, but also probe for novel fraud patterns that the system might miss.
   • DFocus exclusively on the known patterns to ensure the system catches them.
   Reveal Solution Discussion Select an answer first
5. 4application · medium

   A tester is exploring a voice-assistant AI. The tester notices that the assistant responds correctly to commands in a quiet room but fails when there is background noise. What is the most appropriate exploratory testing technique to investigate this further?

   • ARepeat the same command in a quiet room multiple times to check for consistency.
   • BAsk the developer to explain how the noise-cancellation algorithm works.
   • CIgnore the noise issue and focus on testing other features like weather queries.
   • DVary the type and volume of background noise (e.g., music, conversation, traffic) while issuing the same commands to map the failure boundary.
   Reveal Solution Discussion Select an answer first
6. 5application · medium

   A quality engineer is exploratory-testing a chatbot that answers customer questions about billing. The engineer notices that the chatbot performs well on standard queries but gives a confusing response when a user asks a follow-up question that refers to a previous topic. What is the most appropriate next step in exploratory testing?

   • AImmediately modify the chatbot's training data to include more examples of follow-up questions.
   • BRun a pre-defined regression test suite to see if the issue was already known.
   • CDocument the observed behavior, create a minimal reproducible example, and file a defect report with the development team.
   • DIgnore the issue because it only occurs in a rare conversational pattern.
   Reveal Solution Discussion Select an answer first