tokens

Arguments

str

a character string. The string where to search the tokens.

delimiter

(optional) a character or a vector of characters. The tokens delimiters.

T

a column vector of found tokens.

Description

T = tokens(str [,delimiter]) searches the tokens included in the string str. The delimiter default value is [" ",<Tab>] where <Tab> stands for ascii(9).

Examples

tokens('This is a character string')

tokens('SCI/demos/scicos','/')

tokens('y=a+b*2',['=','+','*'])

See also

• strindex — search position of a character string in another string
• tokenpos — returns the tokens positions in a character string