Tokenize

POST {{baseUrl}}/tokenize

Request Body

{"model"=>"command", "text"=>"tokenize me! :D"}

RESPONSES

status: OK

{&quot;tokens&quot;:[10002,2261,2012,8,2792,43],&quot;token_strings&quot;:[&quot;token&quot;,&quot;ize&quot;,&quot; me&quot;,&quot;!&quot;,&quot; :&quot;,&quot;D&quot;],&quot;meta&quot;:{&quot;api_version&quot;:{&quot;version&quot;:&quot;1&quot;}}}