Describe Data Pipeline Best Practices DATA-ENGINEERING-OPTIMIZE Practice Questions (Page 5)

Questions 21–25

1. 21foundation · easy

   What is an example of a data quality check in a pipeline?

   • ALogging pipeline execution time
   • BVerifying that required fields are not null
   • CCompressing data to save storage
   • DParallelizing data processing
   Reveal Solution Discussion Select an answer first
2. 22foundation · easy

   Why is logging important in a data pipeline?

   • AIt speeds up data processing
   • BIt helps with debugging and monitoring pipeline execution
   • CIt automatically fixes errors in the pipeline
   • DIt replaces the need for error handling
   Reveal Solution Discussion Select an answer first
4. 23application · medium

   A pipeline runs daily and the team wants to be alerted if the pipeline takes significantly longer than usual, as this may indicate a problem. What should they monitor?

   • AThe number of rows processed, but not the runtime.
   • BThe CPU usage of the server, but not the pipeline runtime.
   • CThe runtime of the pipeline and set an alert for deviations from the historical average.
   • DOnly the final success or failure of the pipeline.
   Reveal Solution Discussion Select an answer first
5. 24expert · hard

   A team is designing a new data pipeline that will process streaming data and batch data. They have conflicting requirements: the streaming part needs low latency, while the batch part needs high throughput. They also want to minimize operational overhead. Which approach best balances these requirements?

   • AUse only batch processing and schedule it more frequently to approximate streaming.
   • BUse only streaming processing and handle batch data as a special case.
   • CUse a unified pipeline framework that supports both streaming and batch processing, such as Apache Beam or Spark Structured Streaming.
   • DBuild two separate pipelines: one optimized for streaming and one for batch.
   Reveal Solution Discussion Select an answer first
6. 25foundation · easy

   What practice helps ensure a data pipeline is reproducible?

   • AHardcoding file paths in the code
   • BStoring data in temporary locations
   • CPinning dependency versions in a requirements file
   • DUsing the latest version of all libraries
   Reveal Solution Discussion Select an answer first