rustio rustio.net
63

tiktoken-rs

v0.12.0 Growing

Library for encoding and decoding with the tiktoken library in Rust

MIT Edition 2024 MSRV 1.85.0
#ai#openai#gpt#bpe

Quick Verdict

  • โœ“Actively maintained (updated 41d ago)
  • !Pre-1.0: API may have breaking changes
  • โœ“Massive adoption (6.6K crates depend on it)
  • โœ“Permissive license (MIT)

Security

Checking security advisories...
Downloads
11.7M
Dependents
6.6K
Releases
35
Size
3.8MB

Deep Insights

๐Ÿ“‰
Download decline

1.9M downloads in the last 30 days, down 32% from the previous period. May indicate migration to alternatives.

๐Ÿ”—
Widely adopted

6.6K crates depend on tiktoken-rs. Strong ecosystem adoption means battle-tested code and long-term stability.

โš ๏ธ
Single maintainer risk

tiktoken-rs has only 1 owner despite 6.6K dependents. This is a bus-factor concern โ€” consider the implications for long-term support.

๐Ÿ”ฌ
Pre-1.0 for over a year

Despite being 3+ years old, tiktoken-rs hasn't reached 1.0 yet. Expect potential API changes between versions.

๐Ÿ“
Large crate size

3.8MB is relatively large. May impact download and compile times.

๐ŸŒŸ
Used by top crates

Notable dependents include spider, text-splitter, llm-tokenizer, toon-format, langchain-rust. When high-quality crates choose tiktoken-rs, it's a strong quality signal.

Health Breakdown

Maintenance 16/25

Recency, release consistency, active ratio

Quality 12/25

Yanked ratio, deps, size, maturity, features

Community 12/20

Reverse deps, ownership, ecosystem

Popularity 8/15

Downloads, momentum, growth trend

Documentation 15/15

Docs, repo, license, metadata

Download Trend

Daily downloads ยท last 90 days
61K/day avg-37%
050K100K4/265/146/16/197/77/24

Top Dependents

Version Adoption

v0.7.0
36%
v0.9.1
18%
v0.11.0
17%
v0.5.9
16%
v0.6.0
14%

Release Timeline

10 releasessince 2023
J
F
M
A
M
J
J
A
S
O
N
D
2023
2
2024
2
2025
3
2026
3
Less
More

Feature Flags

dhat-heapasync-openai

README

Loading README...

Maintainers

Dependencies
9
direct dependencies
Dependents
6.6K
crates depend on tiktoken-rs

Similar Crates