Use Auto Loader with Schema Enforcement and Schema Evolution in Batch Modes (for Example, Directory Listing or File Notification) to Land Data into UnityCatalog–governed Tables. DATA-ENGINEER-ASSOCIATE Practice Questions (Page 6)

Questions 26–30

1. 26foundation · easy

   Which of the following best describes the role of Auto Loader in a data ingestion pipeline?

   • AIt automatically discovers and processes new data files from cloud storage without requiring manual tracking of file names
   • BIt requires users to manually specify each new file name before processing
   • CIt is a visualization tool for data quality metrics
   • DIt only works with data stored in on-premises HDFS
   Reveal Solution Discussion Select an answer first
2. 27foundation · easy

   What is schema evolution in Auto Loader?

   • AThe ability to automatically adapt to changes in the schema of incoming data, such as new columns
   • BThe process of manually updating the target table schema
   • CThe ability to revert to a previous schema version
   • DThe process of deleting old columns from the target table
   Reveal Solution Discussion Select an answer first
4. 28foundation · easy

   What happens if the checkpoint information is lost in Auto Loader?

   • AAuto Loader will use the file modification time to determine processed files
   • BAuto Loader may reprocess previously processed files, leading to duplicate data
   • CAuto Loader will fail to start until a new checkpoint is created
   • DAuto Loader will automatically recover the checkpoint from the target table
   Reveal Solution Discussion Select an answer first
5. 29foundation · easy

   What is the effect of configuring `failOnNewColumns` in Auto Loader?

   • ANew columns are automatically added to the schema
   • BThe schema is reset to its initial state
   • CThe ingestion fails if new columns are detected in the incoming data
   • DNew columns are ignored and not processed
   Reveal Solution Discussion Select an answer first
6. 30application · medium

   A data engineer is ingesting JSON files into a Delta table using Auto Loader. The upstream system recently added a new field `device_type` to the JSON files. The engineer wants Auto Loader to automatically add this new column to the target table without failing the ingestion. Which configuration should be used?

   • ASet `cloudFiles.schemaEvolutionMode = 'addNewColumns'`.
   • BSet `cloudFiles.schemaEvolutionMode = 'none'`.
   • CSet `cloudFiles.schemaLocation` to a new path.
   • DSet `cloudFiles.schemaEvolutionMode = 'failOnNewColumns'`.
   Reveal Solution Discussion Select an answer first