Describe Data Pipeline Best Practices DATA-ENGINEERING-OPTIMIZE Practice Questions (Page 4)

Questions 16–20

1. 16foundation · easy

   What is a recommended strategy for handling errors in a data pipeline?

   • ACatch exceptions and log them with context
   • BIgnore exceptions to keep the pipeline running
   • CLog errors only after the pipeline completes
   • DTerminate the pipeline on any error
   Reveal Solution Discussion Select an answer first
2. 17expert · hard

   A pipeline runs a critical job every night. The team wants to ensure that if the job fails, it is retried automatically, but they also want to avoid duplicate processing if the job partially completes. What is the best approach?

   • AImplement idempotent processing and use a retry mechanism with a dead-letter queue for persistent failures.
   • BRetry the entire job from the beginning every time, regardless of partial completion.
   • CUse a simple retry loop without any checks for partial completion.
   • DDo not retry automatically; instead, alert the team to manually rerun the job.
   Reveal Solution Discussion Select an answer first
4. 18foundation · easy

   What is a best practice for securing sensitive data in a data pipeline?

   • AStoring credentials in plain text in the code
   • BDisabling access controls to simplify processing
   • CSharing data with all team members by default
   • DEncrypting data at rest and in transit
   Reveal Solution Discussion Select an answer first
5. 19expert · hard

   A pipeline ingests data from multiple sources, and the team has observed that some sources occasionally send data with incorrect schemas. They want to prevent schema mismatches from breaking downstream jobs, but they also want to avoid rejecting valid data. What is the best strategy?

   • AImplement schema validation at ingestion, and route invalid records to a quarantine area for review.
   • BIgnore schema validation and rely on downstream jobs to handle any issues.
   • CAutomatically coerce all fields to strings to avoid schema errors.
   • DReject the entire batch if any record has a schema mismatch.
   Reveal Solution Discussion Select an answer first
6. 20foundation · easy

   How can a data pipeline help ensure compliance with regulations?

   • ABy avoiding all data retention policies
   • BBy making all data publicly accessible
   • CBy storing all data in a single location
   • DBy implementing data governance and audit trails
   Reveal Solution Discussion Select an answer first