Input/Output Operations ASSOCIATE-DEVELOPER-APACHE-SPARK Practice Questions (Page 2)

Questions 6–9

1. 6application · medium

   A team maintains a gold-layer table in Parquet format. They run a nightly job that produces a new version of the data. The team wants to ensure that the table is completely replaced with the new data, but they are concerned about the risk of data loss if the job fails. Which strategy minimizes the risk of data loss while still achieving a full replacement?

   • AWrite to a temporary location, validate the output, then use `overwrite` mode to move the data to the final location.
   • BUse `mode("overwrite")` directly on the final table path.
   • CUse `mode("append")` and then delete the old data manually.
   • DUse `mode("ignore")` to skip the write if the table already exists.
   Reveal Solution Discussion Select an answer first
2. 7foundation · medium

   Which method is used to write a DataFrame to a specific file format, such as Parquet or JSON?

   • Adf.write.saveAsTable("parquet", "/path/to/output")
   • Bdf.write.option("format", "parquet").save("/path/to/output")
   • Cdf.write.parquet("/path/to/output")
   • Ddf.write.format("parquet").save("/path/to/output")
   Reveal Solution Discussion Select an answer first
4. 8expert · hard

   A team writes a DataFrame to a Delta table. They want to ensure that the write operation is atomic, meaning that either the entire write succeeds or the table remains unchanged. Which feature of Delta Lake ensures this?

   • AThe `overwrite` mode.
   • BThe transaction log.
   • CThe `append` mode.
   • DThe `ignore` mode.
   Reveal Solution Discussion Select an answer first
5. 9foundation · medium

   A data engineer needs to read a CSV file into a Spark DataFrame. They want to ensure that the 'age' column is read as an integer and the 'name' column as a string, regardless of the actual values in the file. Which approach should they use?

   • AUse the `inferSchema` option set to true and rely on Spark's automatic type detection.
   • BUse the `mode` option set to `PERMISSIVE` to handle any type mismatches gracefully.
   • CUse the `header` option set to true to read column names from the file and let Spark assign types based on the header.
   • DUse the `schema` option to provide a `StructType` that defines the exact data types for each column.
   Reveal Solution Discussion Select an answer first