ISTQBRetiringCertified Tester AI Testing
The ISTQB Certified Tester AI Testing (CT-AI) certification equips testing professionals to design and execute tests for AI-based systems, including machine learning and generative AI. It addresses the unique challenges of probabilistic behavior, non-determinism, and data reliance, and introduces AI-specific quality characteristics. This certification is essential for testers, data scientists, and developers working with modern AI systems.
1587 practice questions · Updated 2026-07-30
11Domains
63Objectives
395Concepts
1587Questions
CT-AI Curriculum
Every domain, objective, and concept the CT-AI exam measures.
- Definition of AI
- Types of AI
- AI-Based vs Conventional Systems
- AI Technologies Overview
- Machine Learning and Deep Learning
- Natural Language Processing and Computer Vision
- AI Development Frameworks
- Framework Selection Criteria
- Hardware for AI-Based Systems
- Hardware Considerations
- Definition of AI as a Service
- Contractual aspects of AIaaS
- Examples of AIaaS offerings
- Definition and Purpose of Pre-Trained Models
- Common Pre-Trained Models
- Definition and Process of Transfer Learning
- Types of Transfer Learning
- Benefits of Transfer Learning
- Risks and Limitations of Pre-Trained Models
- Risks and Limitations of Transfer Learning
- Mitigation Strategies for Risks
- Overview of AI Standards
- Key AI Standards Organizations
- AI Regulations and Legal Frameworks
- Compliance and Governance
- Definition of Flexibility and Adaptability
- Importance of Flexibility and Adaptability
- Scenarios Requiring Flexibility
- Scenarios Requiring Adaptability
- Testing for Flexibility
- Testing for Adaptability
- Trade-offs with Other Quality Characteristics
- Definition of Autonomy
- Levels of Autonomy
- Human Oversight
- Autonomy and Quality Characteristics
- Testing Challenges for Autonomous Systems
- Verification and Validation of Autonomy
- Evolution of AI-based systems
- Impact of evolution on quality characteristics
- Testing strategies for evolving systems
- Define bias in AI-based systems
- Identify sources of bias
- Recognize types of bias
- Explain impact of bias
- Apply bias mitigation techniques
- Ethical principles in AI
- Ethical risks and harms
- Ethical guidelines and standards
- Ethical considerations in testing
- Definition of Side Effects
- Definition of Reward Hacking
- Causes of Side Effects
- Causes of Reward Hacking
- Examples of Side Effects
- Examples of Reward Hacking
- Distinguishing Side Effects from Reward Hacking
- Impact on Quality Characteristics
- Mitigation Strategies for Side Effects
- Mitigation Strategies for Reward Hacking
- Transparency
- Interpretability
- Explainability
- Relationship among concepts
- Levels of interpretability
- Techniques for interpretability
- Techniques for explainability
- Trade-offs
- Stakeholder perspectives
- Regulatory and ethical considerations
- Safety Definition in AI
- Safety Risks in AI
- Safety Requirements
- Safety Assurance Techniques
- Safety and Quality Characteristics
- Safety Standards and Regulations
- Definition of Supervised Learning
- Labeled Data
- Training and Test Sets
- Common Supervised Learning Algorithms
- Classification vs. Regression
- Model Evaluation
- Definition of Unsupervised Learning
- Clustering
- Dimensionality Reduction
- Anomaly Detection
- Association Rule Learning
- Applications of Unsupervised Learning
- Reinforcement Learning Definition
- Agent and Environment
- States, Actions, and Rewards
- Policy and Value Function
- Exploration vs. Exploitation
- Applications of Reinforcement Learning
- ML Workflow Overview
- Data Preparation
- Model Training
- Model Evaluation
- Model Deployment and Monitoring
- Definition of ML
- Types of ML
- Supervised Learning
- Unsupervised Learning
- Reinforcement Learning
- Selection Criteria
- Algorithm Selection Criteria
- Data Characteristics
- Problem Type and Complexity
- Performance Metrics
- Interpretability vs. Accuracy Trade-off
- Computational Constraints
- Overfitting and Generalization
- Ensemble Methods Consideration
- Definition of Overfitting
- Causes of Overfitting
- Symptoms of Overfitting
- Bias-Variance Tradeoff
- Techniques to Prevent Overfitting
- Definition of Underfitting
- Causes of Underfitting
- Symptoms of Underfitting
- Detection of Underfitting
- Mitigation Strategies for Underfitting
- Define overfitting
- Define underfitting
- Identify causes of overfitting
- Identify causes of underfitting
- Recognize symptoms of overfitting
- Recognize symptoms of underfitting
- Demonstrate overfitting in practice
- Demonstrate underfitting in practice
- Compare overfitting and underfitting
- Apply techniques to mitigate overfitting
- Apply techniques to mitigate underfitting
- Data Quality Issues
- Data Cleaning Techniques
- Data Transformation
- Data Splitting Strategies
- Handling Imbalanced Data
- Feature Engineering Basics
- Data Leakage Prevention
- Bias and Fairness in Data
- Data Preparation Overview
- Data Cleaning
- Data Transformation
- Data Splitting
- Feature Engineering
- Hands-On Implementation
- Identify Training and Test Data
- Create an ML Model
- Evaluate Model Performance
- Data Quality Dimensions
- Impact of Data Quality on ML Models
- Common Data Quality Issues
- Data Quality Assessment Techniques
- Data Cleaning Strategies
- Data Quality Monitoring
- Define data quality
- Identify data quality dimensions
- Explain data quality issues
- Assess data quality impact
- Apply data quality improvement techniques
- Evaluate data quality for ML
- Data labelling overview
- Types of data labelling
- Manual labelling
- Automated labelling
- Semi-automated labelling
- Crowdsourcing for labelling
- Labelling quality and consistency
- Labelling tools and platforms
- Labelling for specific data types
- Ethical and privacy considerations in labelling
- Definition of mislabeled data
- Sources of mislabeled data
- Impact of mislabeled data on ML models
- Detection techniques for mislabeled data
- Handling and correction strategies
- Confusion Matrix Definition
- Reading a Confusion Matrix
- Deriving Metrics from Confusion Matrix
- Confusion Matrix for Multi-class
- Identify limitations of accuracy
- Identify limitations of precision and recall
- Identify limitations of F1-score
- Identify limitations of confusion matrix metrics
- Identify limitations of ROC-AUC
- Identify limitations of PR-AUC
- Identify limitations of regression metrics
- Identify limitations of threshold-dependent metrics
- Identify limitations of aggregate metrics
- Identify limitations of metrics in non-stationary environments
- Evaluate ML model with functional performance metrics
- Interpret functional performance metrics results
- Compare model performance against requirements
- Document evaluation findings
- Purpose of benchmark suites
- Common benchmark datasets
- Benchmark metrics
- Benchmark limitations
- Selecting appropriate benchmarks
- Perceptron Model
- Perceptron Learning Rule
- Binary Classification with Perceptron
- Testing Perceptron Implementation
- Neural Network Coverage Metrics
- Applying Coverage Measures
- Limitations and Challenges
- Specification of AI-Based Systems
- Input Data Testing
- ML Model Testing
- Test Data for Testing AI-based Systems
- Selecting a Test Approach for an ML System
- Component Testing
- Component Integration Testing
- System Testing
- Acceptance Testing
- Automation Bias Definition
- Identifying Automation Bias Risks
- Mitigation Strategies for Automation Bias
- Documentation Requirements for AI Components
- Documentation Standards and Formats
- Documentation for Traceability and Maintenance
- Concept Drift Definition
- Detecting Concept Drift
- Testing for Concept Drift Impact
- Handling Concept Drift in Testing
- Self-learning systems definition
- Challenges in testing self-learning systems
- Non-deterministic behavior
- Data dependency and drift
- Test oracles for self-learning systems
- Continuous learning and adaptation
- Test environment and simulation
- Monitoring and feedback loops
- Ethical and safety considerations
- Autonomy Levels
- Testing Challenges
- Simulation-Based Testing
- Real-World Testing
- Safety and Risk Assessment
- Verification and Validation
- Ethical and Regulatory Considerations
- Define algorithmic bias
- Define sample bias
- Define inappropriate bias
- Identify sources of bias
- Explain impact of bias
- Apply testing techniques for bias
- Definition of probabilistic and non-deterministic systems
- Sources of non-determinism in AI systems
- Challenges in testing probabilistic systems
- Statistical testing methods
- Test oracles for non-deterministic systems
- Handling flaky tests
- Evaluating quality characteristics
- Reporting and communicating test results
- Identify challenges in testing AI-based systems
- Explain the impact of data quality on AI testing
- Describe the difficulty of defining expected outcomes
- Analyze the role of model complexity in testing
- Discuss the challenge of test data selection and coverage
- Explain the difficulty of reproducing and debugging AI failures
- Evaluate the need for specialized testing skills and tools
- Explainability techniques overview
- Applying explainability methods
- Interpreting explanations
- Evaluating explainability results
- Definition and Purpose of Test Oracles
- Challenges in Defining Oracles for AI Systems
- Types of Test Oracles
- Human-Based Oracles
- Heuristic Oracles
- Statistical Oracles
- Model-Based Oracles
- Selecting Appropriate Oracles
- Combining Multiple Oracles
- Evaluating Oracle Quality
- Define test objectives for AI systems
- Define acceptance criteria for AI testing
- Align test objectives with AI quality characteristics
- Derive acceptance criteria from requirements
- Evaluate testability of acceptance criteria
- Document test objectives and acceptance criteria
- Definition of adversarial attacks
- Types of adversarial attacks
- Threat models
- Adversarial examples generation
- Impact on AI systems
- Defense mechanisms
- Testing for adversarial robustness
- Definition of Data Poisoning
- Types of Data Poisoning Attacks
- Attack Vectors and Entry Points
- Impact on Model Performance and Security
- Detection Techniques for Data Poisoning
- Mitigation and Prevention Strategies
- Testing for Data Poisoning Resilience
- Pairwise Testing Fundamentals
- Identifying Input Parameters and Values
- Generating Pairwise Combinations
- Applying Pairwise Testing to AI Systems
- Evaluating Pairwise Test Coverage
- Definition of Back-to-Back Testing
- Purpose and Benefits
- Applicability to AI-Based Systems
- Process and Workflow
- Comparison and Analysis of Outputs
- Handling Non-Determinism
- Limitations and Challenges
- A/B testing fundamentals
- Hypothesis formulation
- Test design and randomization
- Metrics selection
- Statistical significance
- Sample size determination
- Handling biases and confounders
- Interpreting results
- Limitations and pitfalls
- Purpose of Metamorphic Testing
- Metamorphic Relations
- Source and Follow-up Test Cases
- Executing Metamorphic Tests
- Verifying Metamorphic Relations
- Interpreting Results
- Exploratory Testing Fundamentals
- Exploratory Data Analysis (EDA) Fundamentals
- Applying Exploratory Testing to AI Systems
- Applying EDA to AI Datasets
- Integrating Exploratory Testing and EDA
- Selection criteria for AI test techniques
- Mapping techniques to AI system characteristics
- Comparing black-box and white-box testing for AI
- Selecting techniques for data validation
- Selecting techniques for model evaluation
- Selecting techniques for AI-specific testing
- Considering regulatory and ethical constraints in technique selection
- Purpose of Test Environments
- Components of AI Test Environments
- Data Management in Test Environments
- Model Deployment and Serving
- Simulation and Synthetic Data
- Monitoring and Logging
- Version Control and Reproducibility
- Security and Compliance
- Integration with CI/CD
- Challenges and Best Practices
- Purpose of Virtual Test Environments
- Types of Virtual Test Environments
- Simulation Environments
- Emulation Environments
- Digital Twins
- Data Generation in Virtual Environments
- Validation of Virtual Environments
- Integration with Test Processes
- Hands-On Exercise Setup
- Applying AI to a Testing Task
- Evaluating AI-Based Testing Results
- Defect Analysis Automation
- Defect Triage and Prioritization
- Natural Language Processing for Defect Reports
- Defect Clustering and Pattern Recognition
- Predictive Defect Analysis
- Integration with Defect Management Tools
- AI-based test case generation techniques
- Input data generation for testing
- Coverage optimization in test generation
- Evaluation of AI-generated test cases
- Integration of AI test generation into test processes
- Regression Test Suite Optimization
- AI Techniques for Test Suite Reduction
- Test Case Prioritization
- Test Case Selection
- Evaluation of Optimization Effectiveness
- Defect prediction fundamentals
- Data collection for defect prediction
- Data preprocessing for defect prediction
- Feature selection and engineering
- Model training for defect prediction
- Model evaluation and validation
- Interpreting and using predictions
- GUI-based testing with AI
- AI-driven GUI test generation
- GUI element recognition and interaction
- Handling dynamic and complex GUIs
- AI for GUI test oracles
- Evaluating AI-based GUI testing
- GUI Testing Fundamentals
- AI Techniques for GUI Testing
- Automated GUI Test Generation
- GUI Element Recognition and Interaction
- Self-Healing GUI Tests
- Visual Validation and Defect Detection
- AI-Based Test Oracles for GUI
- Challenges and Limitations of AI in GUI Testing
Ready to practice?Test your knowledge with exam-style questions or take an intelligent quiz tailored to your level.
Percentages reflect share of the current practice bank, not official exam weightings — no structured per-skill weight is published for CT-AI, so none is invented.