Multinomial Logistic Regression and Maximum Entropy DATA-SCIENCE-OPTIMIZE Practice Questions (Page 2)

Questions 6–10

1. 6application · medium

   A data scientist is training a multinomial logistic regression model to classify documents into three topics: Sports, Politics, and Technology. The model uses a bag-of-words representation with binary indicators for the presence of each word. After training, the data scientist wants to interpret the model's decision for a specific document. Which quantity directly determines the model's predicted class for that document?

   • AThe number of features present in the document, because more features indicate a stronger signal
   • BThe gradient of the loss function with respect to the document's features
   • CThe softmax output values, where the class with the highest probability is the predicted class
   • DThe sum of all feature weights for the document, regardless of class
   Reveal Solution Discussion Select an answer first
2. 7expert · hard

   A researcher is comparing two models for a 3-class classification problem. Model A is a multinomial logistic regression trained via maximum likelihood estimation. Model B is a maximum entropy model trained with the same set of feature constraints. Both models are trained on the same dataset and converge to their respective optima. Which statement about the two models is correct?

   • AModel A will generalize better to unseen data because MLE provides regularization, while maximum entropy does not
   • BModel A will have higher training accuracy because MLE directly optimizes classification accuracy, while maximum entropy optimizes entropy
   • CModel B will have higher entropy on the training data because it explicitly maximizes entropy, while Model A does not
   • DModel A and Model B will produce identical probability estimates for every input, because they are mathematically equivalent formulations of the same underlying model
   Reveal Solution Discussion Select an answer first
4. 8foundation · easy

   What is the relationship between multinomial logistic regression and maximum entropy models?

   • ALogistic regression is a special case of maximum entropy when there are no constraints
   • BMaximum entropy models are a special case of logistic regression with only binary features
   • CThey are unrelated; logistic regression is a discriminative model while maximum entropy is generative
   • DThey are equivalent when the feature constraints match the model's feature expectations
   Reveal Solution Discussion Select an answer first
5. 9expert · hard

   A team is building a maximum entropy model for a 3-class problem. They have two feature constraints: Feature A has an empirical expectation of 0.6, and Feature B has an empirical expectation of 0.4. The model is trained and the resulting distribution has entropy H. Now the team adds a third feature, Feature C, with an empirical expectation of 0.5. After retraining, the new model's entropy H' is compared to H. What is the most likely relationship between H' and H?

   • AH' will be less than or equal to H, because adding constraints cannot increase the maximum entropy
   • BH' will be greater than H, because more features provide more information
   • CH' will be exactly equal to H, because the maximum entropy is always the same regardless of constraints
   • DH' will be exactly half of H, because the number of constraints doubled
   Reveal Solution Discussion Select an answer first
6. 10foundation · easy

   What role do feature constraints play in a maximum entropy model?

   • AThey eliminate the need for training data
   • BThey restrict the set of possible probability distributions to those matching the observed feature expectations
   • CThey force the model to use only binary features
   • DThey determine the number of classes in the classification problem
   Reveal Solution Discussion Select an answer first