Text Preprocessing DATA-SCIENCE-OPTIMIZE Practice Questions (Page 8)

Questions 36–40

1. 36expert · hard

   A fraud-detection model for transaction descriptions must distinguish 'transfer $100' from 'transfer $10000'. The vocabulary is capped at 20,000 tokens, and numbers appear in many formats ('$100', '100.00', 'one hundred'). What is the best number-handling strategy?

   • AConvert all numbers to words and keep them as-is.
   • BReplace all numbers with a placeholder token like '<NUM>' and keep the currency symbol.
   • CRemove all numbers and currency symbols.
   • DKeep all numbers as-is in their original format.
   Reveal Solution Discussion Select an answer first
2. 37expert · hard

   A web-crawling pipeline collects forum posts that contain HTML tags, JavaScript code snippets, and URLs. The team wants to extract only the visible text for a topic classifier. They are considering two orders: (A) strip tags then remove URLs, or (B) remove URLs then strip tags. Which is correct and why?

   • ARemove JavaScript code before stripping tags.
   • BRemove URLs first, then strip HTML tags.
   • CDecode HTML entities before stripping tags.
   • DStrip HTML tags first, then remove URLs.
   Reveal Solution Discussion Select an answer first
4. 38foundation · easy

   Which of the following is an example of a tokenizer that splits text based on whitespace?

   • AWordNet lemmatizer
   • BPorter stemmer
   • CWhitespace tokenizer
   • DTF-IDF vectorizer
   Reveal Solution Discussion Select an answer first
5. 39application · medium

   A team is training an LSTM for document classification. Documents vary in length from 10 to 2,000 tokens. The model requires fixed-length input. What is the most appropriate strategy?

   • ATruncate all documents to 10 tokens to match the shortest.
   • BUse variable-length sequences and skip padding entirely.
   • CPad all documents to 2,000 tokens without truncation.
   • DTruncate documents longer than 500 tokens and pad shorter ones to 500 tokens.
   Reveal Solution Discussion Select an answer first
6. 40application · medium

   A team is building a document retrieval system for legal contracts. They need to match queries like 'termination' with documents containing 'terminate' and 'terminated'. They also need to distinguish 'saw' (tool) from 'saw' (past tense of see) in context. Which approach best balances recall and precision?

   • AApply lemmatization with part-of-speech tagging to each token.
   • BApply Porter stemming to all tokens without POS tagging.
   • CUse only tokenization and lowercase, keeping all word forms.
   • DApply stemming first, then lemmatization on the stemmed output.
   Reveal Solution Discussion Select an answer first