Data Acquisition GMLE Practice Questions (Page 7)

Questions 31–35

1. 31application · medium

   A company collects IoT sensor readings from thousands of devices. The data arrives as JSON messages over a message broker. The data science team needs to analyze historical trends and also run real-time anomaly detection. Which ingestion architecture best supports both requirements?

   • AStore all raw messages in a data lake and run batch analytics nightly for both historical and real-time needs.
   • BUse a stream processing engine to analyze messages in memory and discard them after processing.
   • CWrite messages directly to a relational database and query it for both real-time and historical analysis.
   • DConsume messages into a stream processing engine that writes to both a time-series database for real-time queries and a data lake for historical analysis.
   Reveal Solution Discussion Select an answer first
2. 32expert · hard

   A data engineer is extracting data from a database that is updated continuously. The extraction job runs every hour and uses a timestamp column to identify new records. The engineer notices that some records are missed because the timestamp is set when the record is created, but updates to existing records do not change the timestamp. Which extraction strategy should be used to capture both new and updated records?

   • AUse a larger timestamp window to catch delayed records.
   • BUse a change-data-capture (CDC) mechanism that tracks inserts, updates, and deletes.
   • CIncrease the extraction frequency to every minute.
   • DQuery the entire table each hour and replace the target dataset.
   Reveal Solution Discussion Select an answer first
4. 33expert · hard

   A data engineering team is ingesting user activity logs from a web application. The logs are sent to a cloud storage bucket as JSON files. The team notices that some files contain records with missing user IDs and some records are duplicated across files. The team needs to ensure data quality before loading into the analytics database. What is the most effective approach?

   • AIgnore the missing user IDs and duplicates because they are rare
   • BRun a validation job that checks for missing user IDs and deduplicates records before loading
   • CLoad all data into the database and run a cleanup script afterward
   • DAsk the web application team to fix the log generation
   Reveal Solution Discussion Select an answer first
5. 34application · medium

   A machine learning team maintains a training dataset that is updated weekly. They need to trace which version of the dataset was used to train each model so that results can be audited. Which approach best supports this requirement?

   • AUse a database view that always returns the current data when training runs.
   • BStore the dataset in a versioned data store and record the dataset version ID in the model training metadata.
   • CCopy the dataset to a shared folder each week and overwrite the previous version.
   • DKeep only the latest version of the dataset and rely on the training timestamp to identify the data.
   Reveal Solution Discussion Select an answer first
6. 35application · medium

   A data team is ingesting data from a third-party API that occasionally returns records with inconsistent date formats (e.g., '2023-01-01' vs '01/01/2023'). The team wants to ensure consistency in the data warehouse. What should they do during ingestion?

   • AReject all records with non-standard date formats
   • BNormalize the date fields to a standard format during the ingestion process
   • CStore the data as-is and handle inconsistencies in the analytics layer
   • DAsk the API provider to fix their data
   Reveal Solution Discussion Select an answer first