Create an Append-Only Data Pipeline Capable of Handling Both Batch and Streaming Data Using Delta. DATA-ENGINEER-PROFESSIONAL Practice Questions (Page 1)

Questions 1–5

1. 1foundation · easy

   Which output mode should be used with Spark Structured Streaming when writing to a Delta table to ensure only new rows are appended?

   • Aupdate
   • Boverwrite
   • Cappend
   • Dcomplete
   Reveal Solution Discussion Select an answer first
2. 2application · medium

   A company has a batch job that reads from an external database and writes to a Delta table. They also have a streaming job that reads from a Kafka topic and writes to the same Delta table. The schemas of the two sources are slightly different: the batch source has an extra column 'source_system'. The team wants to store all data in one table without losing the extra column. What should they do?

   • ACreate two separate tables, one for each source.
   • BUse a UNION in the streaming job to add the 'source_system' column as null.
   • CEnable schema evolution on the Delta table to allow the batch job to add the 'source_system' column.
   • DDrop the 'source_system' column from the batch source before writing.
   Reveal Solution Discussion Select an answer first
4. 3expert · hard

   A financial services company maintains a Delta table of transaction records that must be strictly append-only for regulatory compliance. They have set 'delta.appendOnly' to true. However, they need to correct a batch of erroneous transactions that were accidentally ingested. The correction must be auditable and must not violate the append-only guarantee. What is the best approach?

   • ATemporarily disable the append-only property, perform the correction, and re-enable it.
   • BUse the 'DELETE' command to remove the erroneous records and re-append the correct ones.
   • CCreate a new table with the corrected data and update the pipeline to read from the new table.
   • DUse the 'MERGE' command to update the erroneous records.
   Reveal Solution Discussion Select an answer first
5. 4foundation · easy

   Which Delta Lake operation is explicitly blocked when the 'delta.appendOnly' table property is enabled?

   • ASELECT
   • BCREATE TABLE
   • CINSERT
   • DMERGE
   Reveal Solution Discussion Select an answer first
6. 5foundation · easy

   What happens if you attempt to run a DELETE operation on a Delta table with 'delta.appendOnly' set to 'true'?

   • AThe operation succeeds but logs a warning
   • BThe operation is silently ignored
   • CThe table property is automatically disabled
   • DThe operation fails with an error
   Reveal Solution Discussion Select an answer first