Experimental Design CERTIFIED-DATA-SCIENCE-PRACTITIONER Practice Questions (Page 2)

Questions 6–10

1. 6foundation · easy

   When analyzing experimental results, what does a p-value less than the significance level (e.g., 0.05) indicate?

   • AThe null hypothesis is definitely true
   • BThe observed effect is unlikely to have occurred by chance alone
   • CThe experiment was poorly designed
   • DThe effect size is large
   Reveal Solution Discussion Select an answer first
2. 7foundation · easy

   Which experimental design allows a data scientist to study the effect of two or more factors simultaneously, including their interactions?

   • AA/B testing
   • BFactorial design
   • CCohort study
   • DRandomized controlled trial
   Reveal Solution Discussion Select an answer first
4. 8application · medium

   A team is planning an experiment to test a new pricing model. They want to detect a 2% increase in conversion rate with 80% power and a 5% significance level. They estimate the baseline conversion rate is 10%. Which step is most critical to determine the required sample size?

   • AIncrease the significance level to 0.10 to reduce the required sample size.
   • BRun a pilot study to estimate the variance of the conversion rate.
   • CRandomly assign users to groups without calculating sample size to avoid bias.
   • DUse a sample size calculator with the baseline conversion rate, effect size, power, and significance level.
   Reveal Solution Discussion Select an answer first
5. 9application · medium

   A marketing team wants to test the effect of two different email subject lines and two different call-to-action buttons on click-through rate. They believe the optimal combination might depend on the user's segment (new vs. returning). They have enough users to run all four combinations. Which experimental design is most appropriate?

   • ARun a separate A/B test for each user segment, testing only the subject line, and ignore the call-to-action.
   • BA randomized controlled trial with only one factor (subject line) and no consideration of the call-to-action.
   • CA 2x2 factorial design with random assignment to each of the four combinations.
   • DA/B test only the subject lines, holding the call-to-action constant, to keep the experiment simple.
   Reveal Solution Discussion Select an answer first
6. 10foundation · easy

   In experimental planning, what is the purpose of determining the sample size?

   • ATo guarantee the model will have high accuracy
   • BTo reduce the number of variables in the experiment
   • CTo eliminate the need for randomization
   • DTo ensure the experiment has enough power to detect a meaningful effect if it exists
   Reveal Solution Discussion Select an answer first