Compute Summary Statistics on a Spark DataFrame Using .summary() or Dbutils Data Summaries MACHINE-LEARNING-ASSOCIATE Practice Questions (Page 2)

Questions 6–10

1. 6foundation · easy

   Which statistic in the summary() output describes the spread of the data around the mean?

   • Amean
   • Bstddev
   • Ccount
   • Dmax
   Reveal Solution Discussion Select an answer first
2. 7expert · hard

   A data engineer is using a Databricks notebook to explore a Spark DataFrame `transactions` with columns `transaction_id`, `amount`, `timestamp`, and `status`. They run `dbutils.data.summarize(transactions)` and see that the `amount` column has a histogram that is heavily right-skewed, with a long tail of high values. They also notice that the `status` column has a high percentage of missing values. What is the most appropriate next step for data cleaning?

   • AApply a log transformation to the `amount` column before handling missing values.
   • BIgnore the missing values in `status` because they are not important for the analysis.
   • CInvestigate the missing values in the `status` column and decide whether to impute or drop them.
   • DImmediately drop all rows with missing `status` values without further analysis.
   Reveal Solution Discussion Select an answer first
4. 8foundation · easy

   What is the primary purpose of dbutils.data.summarize() in a Databricks notebook?

   • ATo persist a DataFrame to a table
   • BTo convert a DataFrame to a pandas DataFrame
   • CTo generate an interactive summary of a Spark DataFrame
   • DTo optimize the query plan of a DataFrame
   Reveal Solution Discussion Select an answer first
5. 9application · medium

   A data scientist is exploring a new Spark DataFrame `customer_events` in a Databricks notebook. They want to quickly understand the data types, missing values, and basic distribution of all columns without writing multiple commands. They also want an interactive visualization to share with their team. What should they do?

   • ACall `dbutils.data.summarize(customer_events)`.
   • BCall `customer_events.summary()`.
   • CCall `customer_events.describe()`.
   • DCall `display(customer_events)`.
   Reveal Solution Discussion Select an answer first
6. 10application · medium

   A data scientist has a Spark DataFrame `prices` with a column `price`. They run `prices.select('price').summary().show()` and see the following output: `count` = 1000, `mean` = 150.5, `stddev` = 75.2, `min` = 10, `25%` = 95, `50%` = 140, `75%` = 190, `max` = 500. Which conclusion about the `price` column is most appropriate?

   • AThe distribution is right-skewed, with a long tail of high prices.
   • BThe distribution is symmetric, with no outliers.
   • CThe distribution is uniform, with prices evenly spread.
   • DThe distribution is left-skewed, with a long tail of low prices.
   Reveal Solution Discussion Select an answer first