Task 1.3: Orchestrate Data Pipelines DEA-C01 Practice Questions (Page 1)

Questions 1–5

1. 1foundation · easy

   A data pipeline uses an SQS queue to buffer messages between a producer and a consumer. To ensure that no messages are lost if the consumer fails permanently, what should be configured?

   • AA Lambda function to delete failed messages
   • BAn SNS topic to notify the producer
   • CA dead-letter queue (DLQ) on the SQS queue
   • DA second SQS queue for backup
   Reveal Solution Discussion Select an answer first
2. 2application · medium

   A logistics company has a Step Functions workflow that processes delivery updates. The workflow reads from an SQS queue, processes each message, and writes to a database. The team wants to ensure that if a message fails to process, it is not lost and can be retried later. They also want to alert the operations team after a message has failed 3 times. What is the best approach?

   • AConfigure the SQS queue with a dead-letter queue and set the maximum receives to 3, then subscribe the DLQ to an SNS topic
   • BEnable redrive policy on the SQS queue and use CloudWatch alarms on the queue depth
   • CStore failed messages in an S3 bucket and use a scheduled Lambda to reprocess them
   • DUse a Lambda function with a retry policy and send an SNS notification after each failure
   Reveal Solution Discussion Select an answer first
4. 3foundation · easy

   A data engineer is designing an ETL workflow using AWS Glue workflows. Which component is used to define the dependency between Glue jobs in the workflow?

   • AJobs
   • BCrawlers
   • CWorkflow graph
   • DTriggers
   Reveal Solution Discussion Select an answer first
5. 4expert · hard

   A media company has a data pipeline that processes video files. The pipeline uses a Step Functions state machine that calls a Lambda function to transcode each video. The transcoding can take up to 20 minutes, which exceeds the Lambda timeout. The team wants to keep using Step Functions for orchestration but needs to handle long-running tasks. What is the best approach?

   • AUse a Lambda function that invokes an EC2 instance to do the transcoding
   • BUse a Glue job for transcoding and call it from Step Functions
   • CUse AWS Batch instead of Lambda for the transcoding task and integrate it into the Step Functions workflow
   • DIncrease the Lambda timeout to 30 minutes
   Reveal Solution Discussion Select an answer first
6. 5foundation · easy

   A data engineer wants to monitor the execution of an AWS Step Functions state machine. Which AWS service can be used to capture logs of state transitions and errors?

   • AAmazon EventBridge
   • BAWS CloudTrail
   • CAmazon S3
   • DAmazon CloudWatch Logs
   Reveal Solution Discussion Select an answer first