What is the primary role of gradient synchronization in data parallelism?

To ensure all model replicas have the same updated weights after each step

To reduce the size of the training dataset

To increase the learning rate during training

To distribute the model layers across devices

A team is training a model on a dataset that requires high precision (FP64) for numerical stability. They are choosing between CPU, GPU, and TPU options. Which hardware is most suitable?

TPU, because it provides the highest throughput

CPU, because it provides the best FP64 support

GPU, because it provides good FP64 performance

Any of the above, because precision is not a factor

A data scientist wants to train a transformer model with 50 billion parameters on a single GPU with 16 GB of memory. The model does not fit in memory. Which approach should they take?

Reduce the batch size to fit the model in memory

Use data parallelism across multiple GPUs

Use model parallelism to split the model across multiple GPUs

What is the main purpose of model parallelism?

To reduce the number of training steps needed

To speed up training by using multiple devices on the same data

To increase the batch size without increasing memory usage

To split the model across devices to fit models that exceed a single device's memory

A team is training a model with a large embedding table (20 GB) and small dense layers. They have 4 GPUs with 16 GB memory each. The training data is also large. Which parallelism strategy would be most effective?

Hybrid: model parallelism for the embedding table and data parallelism for the dense layers

Use a single GPU with a larger memory

Google CloudProfessional Machine Learning Engineer

Domain 3Objective 3

Choosing Appropriate Hardware for Training PROFESSIONAL-MACHINE-LEARNING-ENGINEER Practice Questions (Page 4)

Part of the Scaling prototypes into ML models domain, which accounts for ~21% of the PROFESSIONAL-MACHINE-LEARNING-ENGINEER exam.

20questions here

4free pages

5concepts

~21%of the exam

Questions 16–20

16foundation · easy
What is the primary role of gradient synchronization in data parallelism?
Select an answer first
17application · medium
A team is training a model on a dataset that requires high precision (FP64) for numerical stability. They are choosing between CPU, GPU, and TPU options. Which hardware is most suitable?
Select an answer first
18application · medium
A data scientist wants to train a transformer model with 50 billion parameters on a single GPU with 16 GB of memory. The model does not fit in memory. Which approach should they take?
Select an answer first
19foundation · easy
What is the main purpose of model parallelism?
Select an answer first
20expert · hard
A team is training a model with a large embedding table (20 GB) and small dense layers. They have 4 GPUs with 16 GB memory each. The training data is also large. Which parallelism strategy would be most effective?
Select an answer first

Finished these 5 questions?

Review the revealed explanations, or continue through the curriculum.

No more pagesBack to PROFESSIONAL-MACHINE-LEARNING-ENGINEER

Free Basic Practice is a study aid with revealable answers — not a scored exam. Examers.io is independent and not affiliated with or endorsed by Google Cloud. “PROFESSIONAL-MACHINE-LEARNING-ENGINEER” is a trademark of its owner, used for identification only.