Use Auto Loader with Schema Enforcement and Schema Evolution in Batch Modes (for Example, Directory Listing or File Notification) to Land Data into UnityCatalog–governed Tables. DATA-ENGINEER-ASSOCIATE Practice Questions (Page 3)

Questions 11–15

1. 11foundation · easy

   What is the default behavior of Auto Loader regarding schema inference?

   • AIt requires the schema to be specified in the options
   • BIt infers the schema automatically from the data files
   • CIt uses a fixed schema for all file formats
   • DIt uses the schema of the target table if it exists
   Reveal Solution Discussion Select an answer first
2. 12application · medium

   A data engineer is configuring Auto Loader to ingest Parquet files from a GCS bucket. The engineer wants to use directory listing mode and process only new files that arrive after the query starts. Which option should be set?

   • A`cloudFiles.includeExistingFiles = 'false'`
   • B`cloudFiles.includeExistingFiles = 'true'`
   • C`cloudFiles.useNotifications = 'true'`
   • D`cloudFiles.maxFilesPerTrigger = '10'`
   Reveal Solution Discussion Select an answer first
4. 13foundation · easy

   Which option is used to specify the file format for Auto Loader?

   • A`cloudFiles.format`
   • B`inputFormat`
   • C`format`
   • D`fileFormat`
   Reveal Solution Discussion Select an answer first
5. 14foundation · easy

   How does Auto Loader discover new files in file notification mode?

   • ABy reading a log file that records all file additions
   • BBy querying a database of file metadata
   • CBy using cloud storage event services like AWS SQS or Azure Event Grid
   • DBy scanning the directory structure at regular intervals
   Reveal Solution Discussion Select an answer first
6. 15application · medium

   A logistics company receives shipment tracking files from partners in an S3 bucket. Each partner drops a file every few minutes, and the files are immutable once written. A data engineer needs to ingest these files into a Unity Catalog table in the `bronze` schema of the `analytics` catalog, processing them in micro-batches as they arrive. The engineer wants to minimize the infrastructure required and avoid writing custom file-watching code. Which approach should the engineer use?

   • AUse a scheduled Databricks job that runs `spark.read.format('json').load(path)` every 5 minutes and overwrites the target table.
   • BUse Auto Loader with `cloudFiles.format = 'json'` in a streaming query, writing to `analytics.bronze.shipments` with a checkpoint location.
   • CUse Auto Loader in batch mode with `cloudFiles.format = 'json'` and a trigger that runs every minute.
   • DUse `spark.readStream.format('json').load(path)` and write to a Delta table without specifying a checkpoint.
   Reveal Solution Discussion Select an answer first