Describe the Theory, Process, and Analysis of Results for Hypothesis Testing and Its Use in Evaluating a Model DATA-SCIENCE-FOUNDATIONS Practice Questions (Page 2)

Questions 6–10

1. 6application · medium

   A data analyst is designing a hypothesis test to compare the mean response time of two servers. He wants to ensure that the probability of falsely declaring a difference when there is none is limited to 5%. What should he set as the significance level?

   • A0.95
   • B0.01
   • C0.05
   • D0.10
   Reveal Solution Discussion Select an answer first
2. 7application · medium

   A team is evaluating whether a new recommendation algorithm has a higher click-through rate (CTR) than the current one. They plan to run an A/B test. Which sequence of steps correctly follows the hypothesis testing process?

   • ACollect data, then define the hypotheses based on what the data shows.
   • BCompute the p-value first, then set the significance level to match the result.
   • CDefine null and alternative hypotheses, collect data, compute a test statistic, determine the p-value, and make a decision.
   • DState the alternative hypothesis, run the test, and if the p-value is high, accept the alternative.
   Reveal Solution Discussion Select an answer first
4. 8foundation · easy

   If a hypothesis test yields a p-value of 0.03 and the significance level is 0.05, what is the correct decision?

   • AAccept the null hypothesis
   • BAccept the alternative hypothesis
   • CFail to reject the null hypothesis
   • DReject the null hypothesis
   Reveal Solution Discussion Select an answer first
5. 9foundation · easy

   In hypothesis testing, what does the p-value represent?

   • AThe probability that the alternative hypothesis is true
   • BThe probability that the null hypothesis is true
   • CThe probability of obtaining the observed results, or more extreme, assuming the null hypothesis is true
   • DThe probability of making a Type I error if the null hypothesis is rejected
   Reveal Solution Discussion Select an answer first
6. 10expert · hard

   A data science team is comparing a new model to a baseline. They run a hypothesis test and obtain a p-value of 0.03, which is below their significance level of 0.05. However, they used the same validation set for both models, and the test assumes independent samples. What is the main issue with this approach?

   • AThe p-value is still valid because the models are different.
   • BThe result is still significant, so the new model is better.
   • CUsing the same validation set violates the independence assumption, so a paired test should be used instead.
   • DThe significance level should be lowered to 0.01 to account for the shared data.
   Reveal Solution Discussion Select an answer first