rustio rustio.net

Decision Workspace

language-tokenizer

Side-by-side comparison of Rust crates

52
language-tokenizer
experimentalv0.3.0

Text tokenizer for linguistic purposes, such as text matching. Supports more than 40 languages, including English, French, Russian, Japanese, Thai etc.

Core Metrics

language-tokenizer
Health Score52
Total Downloads27.5K
30d Downloads11.2K
Dependents2
Releases3
Last Updated94d ago
Age6m

Health Breakdown

language-tokenizer
Maintenance
14
Quality
16
Community
4
Popularity
5
Documentation
13

Technical Details

language-tokenizer
Version0.3.0
Stable (≥1.0)✗ No
Licensenon-standard
Dependencies12
Crate Size27KB
Features11
Yanked %0.0%
Edition2024
MSRV1.88.0
Owners1

Links

language-tokenizer detail →GitHub ↗Docs ↗crates.io ↗

Quick Verdict

  • •language-tokenizer leads with a health score of 52/100, but none of the options score above 80.