Package org.simmetrics.tokenizers

Interface Summary

Interface	Description
Tokenizer	A tokenizer divides an input string into tokens.

Class Summary

Class	Description
AbstractTokenizer	Convenience tokenizer.
QGram	Deprecated Use Tokenizers.qGram(int) instead
QGramExtended	Deprecated Use Tokenizers.qGramWithPadding(int, String)
Tokenizers	Utilities for tokenizers.
Whitespace	Deprecated use Tokenizers.whitespace() instead