Utilize Common Data Sources Such as JDBC, Files, Etc., to Efficiently Read from and Write to Spark DataFrames Using Spark SQL, Including Overwriting and Partitioning by Column. ASSOCIATE-DEVELOPER-APACHE-SPARK Practice Questions (Page 1)

Questions 1–5

1. 1foundation · easy

   Which Spark SQL write mode should you use to replace the contents of an existing table with new data?

   • Aappend
   • Berror
   • Coverwrite
   • Dignore
   Reveal Solution Discussion Select an answer first
2. 2application · medium

   A team writes a DataFrame to a JDBC table using the overwrite mode. The target table has foreign key constraints from other tables. What is the most likely outcome?

   • AThe overwrite truncates the table and inserts new rows, preserving constraints.
   • BThe overwrite succeeds, and the foreign key constraints are preserved.
   • CThe overwrite fails because the table cannot be dropped due to foreign key constraints.
   • DThe overwrite appends the new data to the existing table.
   Reveal Solution Discussion Select an answer first
4. 3foundation · easy

   Which practice can help avoid unnecessary data shuffles when writing a DataFrame?

   • AIncreasing the number of partitions
   • BUsing the same partition column as the existing data layout
   • CWriting to a single file
   • DUsing a random partition column
   Reveal Solution Discussion Select an answer first
5. 4application · medium

   A data engineering team needs to read a large PostgreSQL table into a Spark DataFrame for a nightly batch job. The table has 50 million rows and the team wants to minimize the load time and avoid overwhelming the database. They plan to use Spark SQL with the JDBC source. Which approach should they use?

   • ARead the full table with a single connection and let Spark parallelize the read across partitions.
   • BRead the table using the dbtable option with a subquery that pre-aggregates the data.
   • CSpecify partitionColumn, lowerBound, upperBound, and numPartitions to read the table in parallel.
   • DUse the fetchsize option set to a very large value to pull all rows in one round trip.
   Reveal Solution Discussion Select an answer first
6. 5expert · hard

   A team writes a DataFrame to a JDBC table using mode("overwrite"). The target table has a large number of rows, and the team is concerned about the time it takes to replace the data. They notice that the overwrite operation is slow. Which factor is the most likely cause?

   • AThe overwrite mode uses a single transaction, which locks the table and slows down the write.
   • BThe overwrite mode truncates the table and then inserts the new data, and the insert is slow due to the large number of rows.
   • CThe overwrite mode requires a full shuffle of the data before writing.
   • DThe overwrite mode drops and recreates the table, which is slow for large tables.
   Reveal Solution Discussion Select an answer first