##### ###### ##### ### # # ### # # ###### ## ## ## ## ## ## ## # # # # # ## ##### #### ##### # # # # # # # #### ## # ## ## ## ## # # # # # ## ## # ###### ## ### # ### # ######
##### ###### ##### ### # # ### # # ###### ## ## ## ## ## ## ## # # # # # ## ##### #### ##### # # # # # # # #### ## # ## ## ## ## # # # # # ## ## # ###### ## ### # ### # ######
Tokenization splits text into searchable terms. "Quick brown fox" becomes ["quick", "brown", "fox"]. Analyzers chain: character filters → tokenizer → token filters.
Standard analyzer: lowercase, split on whitespace/punctuation. Language analyzers handle stemming. Custom analyzers for domain-specific needs (code search, product names).
