Add tiktoken dependency to pyproject.toml and uv.lock

Browse Source

- Updated pyproject.toml to include "tiktoken>=0.12.0" in dependencies.
- Modified uv.lock to reflect the addition of tiktoken in both dependencies and requires-dist sections.

...

This commit is contained in:

Arthur Danjou

2026-02-18 18:57:26 +01:00

parent a8e15d66f7

commit ac389113da

3 changed files with 2128 additions and 0 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download Patch File Download Diff File Expand all files Collapse all files

2125

M2/Natural Language Process/TP2 tokenization.ipynb Normal file

File diff suppressed because one or more lines are too long