I
Ishan
Mob programming an LLM Tokenizer in Haskell
Submitted Feb 15, 2025
In this mob programming session, we’ll build an LLM style tokenizer in Haskell. The primary focus will be:
- Applying fp concepts for this text processing task
- Thinking in Folds, Streams and PIpes using the Streamly library
- (Bonus) How LLM tokenizers work
{{ gettext('Login to leave a comment') }}
{{ gettext('Post a comment…') }}{{ errorMsg }}
{{ gettext('No comments posted yet') }}