Task 1.3: Orchestrate Data Pipelines DEA-C01 Practice Questions (Page 5)

Questions 21–25

1. 21application · medium

   A company runs a batch pipeline that loads data into a data warehouse every hour. The operations team wants to receive an email if the pipeline fails, and a separate team wants to receive a Slack message for the same failure. The pipeline currently publishes a message to an SNS topic on failure. What is the most efficient way to deliver these notifications?

   • AConfigure a CloudWatch alarm on the pipeline's error metric and send to both email and Slack
   • BCreate two SNS subscriptions: one with an email endpoint and one with an AWS Chatbot endpoint for Slack
   • CUse an SQS queue subscribed to the SNS topic and have two Lambda functions poll the queue
   • DCreate two separate SNS topics and have the pipeline publish to both
   Reveal Solution Discussion Select an answer first
2. 22foundation · easy

   A data engineer is optimizing a Spark-based ETL job in AWS Glue. Which technique is most effective for improving performance when processing large datasets?

   • AUsing a single large instance
   • BIncreasing the number of partitions
   • CDisabling dynamic frame partitioning
   • DDecreasing the number of partitions
   Reveal Solution Discussion Select an answer first
4. 23application · medium

   A startup runs a data pipeline that ingests data from a Kinesis stream, processes it with a Lambda function, and stores results in DynamoDB. The traffic is spiky, with occasional bursts. The team wants to ensure the pipeline can handle sudden increases in data volume without losing data and without manual intervention. Which architecture best meets these requirements?

   • AIncrease the number of shards in the Kinesis stream and configure Lambda with reserved concurrency
   • BUse a Kinesis Data Firehose delivery stream to buffer data and deliver to S3, then trigger a Lambda function
   • CUse Kinesis Data Streams with a Lambda consumer and configure the Lambda function with a DLQ and retries
   • DUse a Lambda function with an SQS queue as an event source and enable auto-scaling on the queue
   Reveal Solution Discussion Select an answer first
5. 24foundation · easy

   A data engineer wants to use Amazon SQS to decouple two components of a data pipeline. What is the primary benefit of using a queue between the producer and consumer?

   • AIt allows the producer and consumer to operate independently
   • BIt increases the speed of data processing
   • CIt ensures that messages are processed in exactly the same order they were sent
   • DIt reduces the cost of data transfer
   Reveal Solution Discussion Select an answer first
6. 25expert · hard

   A financial company runs a Step Functions workflow that processes real-time stock trades. The workflow is triggered by an EventBridge rule that matches trade events. The team wants to ensure that the workflow can handle a sudden spike in trade volume without losing events. The workflow currently uses a Lambda function that can be throttled. What is the best way to make the pipeline more resilient?

   • AIncrease the Lambda function's reserved concurrency to a high number
   • BEnable the Step Functions workflow to retry on Lambda throttling errors
   • CUse a Kinesis stream instead of EventBridge to ingest trade events
   • DAdd an SQS queue between EventBridge and Step Functions to buffer events
   Reveal Solution Discussion Select an answer first