Use Tools and Metrics to Evaluate Retrieval Performance GENERATIVE-AI-ENGINEER-ASSOCIATE Practice Questions (Page 2)

Questions 6–10

1. 6foundation · easy

   Which of the following is a Python library commonly used to compute retrieval evaluation metrics such as precision, recall, and NDCG?

   • Apandas
   • Bscikit-learn
   • Cpytrec_eval
   • Dmatplotlib
   Reveal Solution Discussion Select an answer first
2. 7expert · medium

   A team is evaluating a retrieval system for a job search platform. They have a set of 50 queries, each with a list of relevant job postings. The system returns a ranked list of 10 job postings per query. They compute the average precision@10 and get 0.4. They also compute recall@10 and get 0.6. They want to understand the relationship between these two metrics. Which statement is correct?

   • APrecision@10 measures the fraction of relevant job postings in the top 10, while recall@10 measures the fraction of all relevant job postings that are in the top 10
   • BPrecision@10 is more important than recall@10 for this application
   • CPrecision@10 and recall@10 are inversely related, so improving one will always hurt the other
   • DPrecision@10 and recall@10 are always equal when the number of relevant documents is less than 10
   Reveal Solution Discussion Select an answer first
4. 8expert · medium

   A team is evaluating a RAG pipeline for a medical diagnosis support system. They have a ground-truth set of 500 queries with relevance judgments. The retriever returns 10 documents per query. They compute recall@10 = 0.95 and NDCG@10 = 0.6. The medical team is concerned that the system sometimes retrieves relevant documents but ranks them too low, leading to incorrect answers. They want to improve the ranking of relevant documents. Which metric should they use to track improvements?

   • AMean Reciprocal Rank (MRR)
   • BNDCG@10
   • CPrecision@10
   • DRecall@10
   Reveal Solution Discussion Select an answer first
5. 9application · medium

   A data scientist is evaluating a RAG pipeline and wants to compute NDCG@10 for a set of queries. They have the retrieved document IDs and the relevance scores (0, 1, or 2) from human annotators. They are using a Python library that provides an `ndcg_score` function. What input format does the function typically expect?

   • AA list of retrieved document IDs and a separate list of relevant document IDs
   • BA dictionary mapping each query to a list of relevant document IDs
   • CA single list of relevance scores for all retrieved documents across all queries
   • DA 2D array of true relevance scores and a 2D array of predicted scores, where each row corresponds to a query
   Reveal Solution Discussion Select an answer first
6. 10foundation · easy

   Which retrieval evaluation metric measures the proportion of relevant documents that are successfully retrieved out of all relevant documents in the corpus?

   • ARecall
   • BNormalized discounted cumulative gain (NDCG)
   • CMean reciprocal rank (MRR)
   • DPrecision
   Reveal Solution Discussion Select an answer first