Tokenize

POST {{baseUrl}}/tokenize

Request Body

{"text"=>"This is a test"}

RESPONSES

status: OK

{"text":"This is a test","tokens":[{"token":"▁This▁is▁a","textRange":{"start":0,"end":9}},{"token":"▁test","textRange":{"start":9,"end":14}}]}