Ingest Semi-Structured and Unstructured Data (for Example, JSON and Nested Data) via Lakeflow Connect and Other Managed Connectors into UnityCatalog–governed Delta Tables. DATA-ENGINEER-ASSOCIATE Practice Questions (Page 3)

Questions 11–15

1. 11application · medium

   A media company stores video metadata as JSON files in an Azure Data Lake Storage (ADLS) Gen2 account. They want to ingest this data into a Unity Catalog-managed Delta table using a managed connector. The data is updated incrementally. Which connector and configuration should they use?

   • AUse the Azure Blob Storage connector with 'fileFormat' set to 'parquet'.
   • BUse the Azure SQL Database connector to read the JSON files.
   • CUse the Auto Loader connector with 'fileFormat' set to 'json' and 'cloudFiles' options for incremental ingestion.
   • DUse the JDBC connector to connect directly to ADLS Gen2.
   Reveal Solution Discussion Select an answer first
2. 12foundation · easy

   Which Spark SQL function is commonly used to flatten a nested JSON structure into separate columns?

   • Ato_json
   • Bexplode
   • CselectExpr with dot notation
   • Dfrom_json
   Reveal Solution Discussion Select an answer first
4. 13expert · hard

   A company ingests JSON files from an external partner into a Unity Catalog-managed Delta table using Lakeflow Connect. The partner occasionally sends files with a completely different schema (e.g., new columns and changed data types). The team wants to avoid pipeline failures but also wants to maintain data quality. What is the best approach?

   • AEnable schema evolution with 'mergeSchema' and set up a data quality monitor to alert on unexpected schema changes.
   • BDisable schema evolution and manually update the table schema whenever the partner changes the schema.
   • CUse 'overwriteSchema' to replace the table schema with each new file.
   • DSet the pipeline to fail on any schema change and require manual approval.
   Reveal Solution Discussion Select an answer first
5. 14application · medium

   A retail company receives daily JSON files from its point-of-sale system, each containing a nested 'items' array. The data team wants to use Lakeflow Connect to ingest these files into a Unity Catalog-managed Delta table. The JSON schema occasionally adds new optional fields. Which configuration should the team use in Lakeflow Connect to ensure the ingestion handles schema changes without manual intervention?

   • AUse 'overwriteSchema' mode to replace the table schema with each new file.
   • BSet the target table to 'failOnSchemaMismatch' to stop ingestion when schema changes are detected.
   • CDisable schema inference and manually define the table schema in the target catalog.
   • DEnable schema evolution and set the target table to 'mergeSchema' mode.
   Reveal Solution Discussion Select an answer first
6. 15application · medium

   A company uses a PostgreSQL database as a source for customer data. They want to ingest this data into a Unity Catalog-managed Delta table using a managed connector. The source table has a 'last_updated' timestamp column. Which connector and configuration should they use to perform incremental ingestion?

   • AUse the JDBC connector with a query that filters on 'last_updated' greater than the last ingestion timestamp.
   • BUse the Auto Loader connector with 'fileFormat' set to 'jdbc'.
   • CUse the PostgreSQL connector with 'incremental' mode enabled.
   • DUse the Azure SQL Database connector to connect to PostgreSQL.
   Reveal Solution Discussion Select an answer first