rustio rustio.net

Decision Workspace

language-tokenizer

Side-by-side comparison of Rust crates

45
language-tokenizer
experimentalv0.1.0

Text tokenizer for linguistic purposes, such as text matching. Supports more than 40 languages, including English, French, Russian, Japanese, Thai etc.

Core Metrics

language-tokenizer
Health Score45
Total Downloads447
30d Downloads137
Dependents2
Releases1
Last Updated79d ago
Age2m

Health Breakdown

language-tokenizer
Maintenance
9
Quality
16
Community
4
Popularity
3
Documentation
13

Technical Details

language-tokenizer
Version0.1.0
Stable (≥1.0)✗ No
Licensenon-standard
Dependencies12
Crate Size27KB
Features11
Yanked %0.0%
Edition2021
MSRV1.83.0
Owners1

Links

language-tokenizer detail →GitHub ↗Docs ↗crates.io ↗

Quick Verdict

  • •language-tokenizer leads with a health score of 45/100, but none of the options score above 80.