Enable and Detail Data Ingestion Patterns, Including Batch, Streaming, and Incremental Loading, and Import Data from Sources Such as Local Files, Lakeflow Connect Standard Connectors, and Lakeflow Connect Managed Connectors. DATA-ENGINEER-ASSOCIATE Practice Questions (Page 4)

Questions 16–20

1. 16application · medium

   A financial services firm needs to ingest real-time stock ticker data from a third-party market data feed into Databricks for immediate analysis. The data arrives as JSON messages over a message queue. Which ingestion approach best meets the low-latency requirement?

   • AUse a Lakeflow Connect standard connector to pull the data from the message queue in batches every 15 minutes.
   • BManually upload the JSON files to DBFS using the Databricks CLI on a daily basis.
   • CUse a streaming ingestion pipeline that reads from the message queue and writes to a Delta table in near real time.
   • DSet up a scheduled Databricks job that runs every hour to read the JSON files from a cloud storage location.
   Reveal Solution Discussion Select an answer first
2. 17foundation · easy

   A company needs to ingest clickstream events from a website into a lakehouse and make them available for dashboards within seconds of each event. Which ingestion pattern is most appropriate?

   • AManual upload of log files
   • BStreaming ingestion with low-latency processing
   • CIncremental loading with a nightly job
   • DBatch ingestion on an hourly schedule
   Reveal Solution Discussion Select an answer first
4. 18foundation · easy

   Which Lakeflow Connect connector type is designed for automated, scalable ingestion with managed infrastructure?

   • AManaged connectors
   • BCustom connectors
   • CStandard connectors
   • DPartner connectors
   Reveal Solution Discussion Select an answer first
5. 19foundation · easy

   A data engineer needs to load a daily snapshot of a customer table into a Databricks lakehouse. The snapshot is generated by an upstream system every night at 2:00 AM. Which data ingestion pattern best fits this requirement?

   • AIncremental loading, because only new or changed rows should be loaded each night.
   • BBatch ingestion, because the data arrives on a fixed schedule and does not require immediate processing.
   • CStreaming ingestion, because the data is generated automatically and should be processed as soon as it is available.
   • DManual file upload, because the snapshot is small and can be uploaded through the UI.
   Reveal Solution Discussion Select an answer first
6. 20application · medium

   A manufacturing company has a sensor readings table that grows by millions of rows each day. They run a nightly batch job to load new readings. Currently, the job loads all readings from the beginning of time, which is becoming slow. Which change would most effectively reduce the processing time?

   • ASwitch to a streaming ingestion pipeline to process sensor readings as they arrive.
   • BIncrease the cluster size to handle the full load faster.
   • CUse a Lakeflow Connect managed connector to replicate the entire sensor database.
   • DUse a watermark column (e.g., reading_timestamp) to load only readings newer than the last run.
   Reveal Solution Discussion Select an answer first