Scaling Online Model Serving PROFESSIONAL-MACHINE-LEARNING-ENGINEER Practice Questions (Page 6)

Questions 26–30

1. 26application · medium

   A media company serves a computer-vision model that processes images in batches. The model is deployed on a GPU endpoint, but GPU utilization is only 20% because requests arrive one at a time. The team wants to improve throughput without changing the model architecture. What should they do?

   • AMove the model to a CPU machine
   • BEnable dynamic batching on the serving container to group multiple requests into a single inference
   • CSwitch to a smaller GPU machine type
   • DIncrease the number of GPU replicas
   Reveal Solution Discussion Select an answer first
2. 27application · medium

   A team uses Vertex AI Feature Store to serve features for a real-time churn-prediction model. They notice that some predictions use stale features because the streaming pipeline occasionally fails to update the online store. What should they do to detect and address this issue?

   • ASwitch from streaming to batch ingestion for the features
   • BDisable the online serving endpoint and use batch serving instead
   • CMonitor the feature value staleness metric and set up an alert to notify the team when staleness exceeds a threshold
   • DIncrease the number of nodes in the Feature Store online serving cluster
   Reveal Solution Discussion Select an answer first
4. 28foundation · easy

   What is the primary advantage of using a managed inference platform like Gemini Enterprise Agent Platform?

   • AIt is always cheaper than self-managed serving
   • BIt allows you to use any programming language for the model
   • CIt provides full control over the underlying infrastructure
   • DIt reduces the operational overhead of managing serving infrastructure
   Reveal Solution Discussion Select an answer first
5. 29foundation · easy

   What is the main benefit of dynamic batching in model serving?

   • AIt reduces the model's memory footprint
   • BIt simplifies the model deployment process
   • CIt improves throughput by processing multiple requests together on the accelerator
   • DIt increases the accuracy of predictions
   Reveal Solution Discussion Select an answer first
6. 30foundation · easy

   What is the primary difference between a public and a private endpoint for model serving?

   • APublic endpoints require no authentication, while private endpoints require IAM
   • BPublic endpoints are faster than private endpoints
   • CPublic endpoints are accessible from the internet, while private endpoints are only accessible within a VPC
   • DPrivate endpoints support only CPU models, while public endpoints support GPUs
   Reveal Solution Discussion Select an answer first