karaza
AI fundamentals

Token

What is a token?

A small unit of text, often a word or part of a word, that AI models read and generate. Usage limits, context windows and API pricing are all counted in tokens.

Example

In English, 1,000 tokens is roughly 750 words.

Reviewed by Mahmoud, founder of Karaza, L3 in Education and TeachingUpdated