Drop numeric (house-number) tokens from token lists

Symmetric inverse of numeric_tokens(): removes pure-digit tokens (typically house numbers) from a token column. Operates on the list-of-character token vectors produced by earlier steps such as word_tokens(), mirroring filter_stopwords().

Useful in address pipelines where the street name carries the matching signal but the house number is noise (and fans out blocks): tokenize the street, then drop_numeric_tokens() to keep only the name tokens.

Usage

drop_numeric_tokens(tokens, keep_letters = TRUE)

Arguments

tokens

A list of character vectors.

keep_letters

Logical. If TRUE (default), number-letter tokens such as "12A" are retained; only pure-digit tokens like "12" are dropped. If FALSE, any token containing a digit is dropped.

Value

A list of character vectors with numeric tokens removed.

See also

numeric_tokens(), its inverse; filter_stopwords() for the same idea with a named word list.

Other token transformers: drop_short_tokens(), extract_initials(), filter_stopwords(), fuzzy_tokens(), token_shapes(), use_dictionary()

Examples

drop_numeric_tokens(list(c("MAIN", "12", "ST")))
#> [[1]]
#> [1] "MAIN" "ST"  
#> 
# list(c("MAIN", "ST"))

drop_numeric_tokens(list(c("MAIN", "12A")), keep_letters = FALSE)
#> [[1]]
#> [1] "MAIN"
#> 
# list("MAIN")