Data Aggregation and Joins ASSOCIATE-DEVELOPER-APACHE-SPARK Practice Questions (Page 1)

Questions 1–5

1. 1application · medium

   A retail company has a `sales` DataFrame with columns `transaction_id`, `product_id`, and `amount`. They also have a `products` DataFrame with `product_id` and `product_name`. The company needs a report that lists every transaction with its product name, but only for transactions where the product is still in the current catalog. Some transactions reference products that have been discontinued and are not in the `products` DataFrame. Which join type should be used?

   • Across join followed by a filter on `product_id`
   • Bleft join on `product_id`
   • Cright join on `product_id`
   • Dinner join on `product_id`
   Reveal Solution Discussion Select an answer first
2. 2application · medium

   A data scientist has a `users` DataFrame with columns `user_id` and `name`, and a `permissions` DataFrame with `permission_id` and `description`. They need to generate all possible combinations of users and permissions for a matrix. The Spark configuration `spark.sql.crossJoin.enabled` is set to `false`. What should they do?

   • AUse a union of the two DataFrames
   • BUse a cross join and set `spark.sql.crossJoin.enabled` to `true`
   • CUse an inner join on a common column
   • DUse a left join on a common column
   Reveal Solution Discussion Select an answer first
4. 3foundation · easy

   What is the result of joining two DataFrames on multiple keys?

   • ARows are matched based on the first key only
   • BRows are matched only when all specified keys are equal
   • CRows are matched based on the last key only
   • DRows are matched when any of the specified keys are equal
   Reveal Solution Discussion Select an answer first
5. 4expert · hard

   A data engineer is working with a very large `events` DataFrame (billions of rows) and needs to report the number of unique `user_id` values. The exact count is required for a compliance report, but the computation is taking too long. Which approach should they take?

   • AUse `countDistinct` on the `user_id` column
   • BUse `approx_count_distinct` with a very small relative error
   • CUse `summary` on the `user_id` column
   • DUse `count` on the `user_id` column
   Reveal Solution Discussion Select an answer first
6. 5application · medium

   An HR analyst has an `employees` DataFrame with `employee_id` and `name`, and a `salaries` DataFrame with `employee_id` and `salary`. They need to compute the average salary for all employees, including those who may not have a salary record yet (e.g., new hires). Which approach should they take?

   • APerform an inner join on `employee_id`, then compute the mean of the `salary` column
   • BPerform a right join on `employee_id`, then compute the mean of the `salary` column
   • CPerform a cross join, then compute the mean of the `salary` column
   • DPerform a left join on `employee_id`, then compute the mean of the `salary` column
   Reveal Solution Discussion Select an answer first