Language Modeling DATA-SCIENCE-OPTIMIZE Practice Questions (Page 3)

Questions 11–15

1. 11expert · hard

   A company is building a text autocomplete feature for a code editor. The feature must suggest completions for code snippets. The team is considering using a general-purpose language model trained on natural language, or a specialized model trained on code. Which consideration is most important?

   • AThe general-purpose model has lower perplexity on code
   • BThe general-purpose model is easier to integrate
   • CThe specialized model is always more expensive
   • DThe specialized code model is more likely to understand code syntax and patterns
   Reveal Solution Discussion Select an answer first
2. 12foundation · easy

   Which neural network architecture is specifically designed to handle sequential data by maintaining a hidden state that captures information from previous time steps?

   • AFeedforward Neural Network
   • BAutoencoder
   • CConvolutional Neural Network (CNN)
   • DRecurrent Neural Network (RNN)
   Reveal Solution Discussion Select an answer first
4. 13foundation · easy

   What key mechanism allows transformer-based language models to weigh the importance of different words in a sequence when processing each word?

   • ASelf-attention
   • BHidden state recurrence
   • CBackpropagation through time
   • DConvolutional filtering
   Reveal Solution Discussion Select an answer first
5. 14expert · hard

   A team is evaluating two language models for a speech recognition system. Model A is an n-gram model with Kneser-Ney smoothing, and Model B is a transformer-based model. On a test set, Model A has a perplexity of 150, and Model B has a perplexity of 80. However, Model B requires a GPU server, while Model A runs on a CPU. The system must run on a low-power device. Which model should the team choose?

   • AModel A, but only if the team accepts a 50% error rate
   • BModel B, because lower perplexity is always better
   • CModel B, but only if the team can upgrade the device hardware
   • DModel A, because it can run on the low-power device despite higher perplexity
   Reveal Solution Discussion Select an answer first
6. 15expert · hard

   A team is fine-tuning a pre-trained transformer language model for a domain-specific task. They have a small labeled dataset. They are considering two approaches: (1) fine-tune the entire model, or (2) freeze the transformer layers and only train a classification head. Which approach is more likely to avoid overfitting?

   • ANeither approach will overfit because transformers are robust
   • BBoth approaches are equally likely to overfit
   • CFreeze the transformer layers and train only the classification head
   • DFine-tune the entire model, because it will adapt better to the domain
   Reveal Solution Discussion Select an answer first