Objective: Apply Data Collection Methods from the Test Automation Solution and the System Under Test CTAL-TAE Practice Questions (Page 4)

Questions 16–20

1. 16expert · hard

   A team is testing a real-time application and needs to collect SUT metrics with millisecond precision. The monitoring tool they are evaluating samples every 10 seconds. What is the best course of action?

   • ARun the test multiple times and average the results.
   • BUse a different tool that samples at 1-second intervals.
   • CUse a custom instrumentation approach that logs events with timestamps at the application level.
   • DAccept the 10-second sampling and interpolate the data.
   Reveal Solution Discussion Select an answer first
2. 17foundation · easy

   Which data is typically collected from the test automation solution to assess its performance?

   • AThe SUT's CPU usage during test execution
   • BThe application's response time to user inputs
   • CThe number of test cases executed per hour
   • DThe number of defects found in the application
   Reveal Solution Discussion Select an answer first
4. 18expert · hard

   A team is running a performance test on a critical production-like environment. They need to collect detailed metrics from the SUT, but the monitoring agent adds 15% CPU overhead, which could skew results. They also need to collect data from the automation framework. What is the best approach to minimize interference while still collecting necessary data?

   • ARun the performance test on a separate environment with the agent installed.
   • BUse a sampling-based monitoring tool that collects metrics at a lower frequency.
   • CCollect only framework logs and skip SUT metrics.
   • DAccept the overhead and adjust the results mathematically.
   Reveal Solution Discussion Select an answer first
5. 19expert · hard

   A team needs to collect data from a cloud-based SUT that is ephemeral (instances are created and destroyed frequently). They need to ensure no data is lost. Which approach is most suitable?

   • ARely on the cloud provider's built-in logging service.
   • BStream the data to a centralized storage service in real-time.
   • CStore data locally on the instance and retrieve it before termination.
   • DUse a snapshot of the instance's disk after termination.
   Reveal Solution Discussion Select an answer first
6. 20application · medium

   A test automation framework runs thousands of Selenium tests nightly. The team needs to collect detailed execution logs and performance metrics from the framework itself, but the volume is causing storage issues. Which approach best addresses the storage concern while maintaining data integrity?

   • AUse a single log file for all tests to simplify storage management.
   • BEnable log rotation and archive logs to a low-cost storage tier after a retention period.
   • CStore all logs in memory and only write to disk when a test fails.
   • DDisable logging for all tests that pass to reduce data volume.
   Reveal Solution Discussion Select an answer first