Break of a string or other character sequence into a series of tokens.