Decision Workspace
rust_scraper vs robotxt vs halo-spider
Side-by-side comparison of Rust crates
48
rust_scraper
experimentalv1.0.0
Production-ready web scraper with Clean Architecture, TUI selector, and sitemap support
46
robotxt
growingv0.6.1
The implementation of the Robots.txt (or URL exclusion) protocol with the support of crawl-delay, sitemap and universal match extensions.
43
halo-spider
experimentalv0.0.5
A fast, async web crawling framework inspired by Scrapy, built in Rust
Core Metrics
| rust_scraper | robotxt | halo-spider | |
|---|---|---|---|
| Health Score | 48 | 46 | 43 |
| Total Downloads | 14 | 34.6K | 55 |
| 30d Downloads | 14 | 3.9K | 55 |
| Dependents | 0 | 34 | 0 |
| Releases | 1 | 14 | 5 |
| Last Updated | 17d ago | 750d ago | today |
| Age | 17d | 3y | 9d |
Health Breakdown
rust_scraper
Maintenance
12
Quality
14
Community
6
Popularity
1
Documentation
15
robotxt
Maintenance
6
Quality
12
Community
10
Popularity
5
Documentation
13
halo-spider
Maintenance
13
Quality
12
Community
6
Popularity
2
Documentation
10
Technical Details
| rust_scraper | robotxt | halo-spider | |
|---|---|---|---|
| Version | 1.0.0 | 0.6.1 | 0.0.5 |
| Stable (≥1.0) | ✓ Yes | ✗ No | ✗ No |
| License | MIT OR Apache-2.0 | MIT | MIT |
| Dependencies | 45 | 8 | 17 |
| Crate Size | 181KB | 16KB | 121KB |
| Features | 5 | 6 | 4 |
| Yanked % | 0.0% | 14.3% | 0.0% |
| Edition | 2021 | 2021 | 2024 |
| MSRV | 1.80 | — | — |
| Owners | 1 | 1 | 1 |
Links
Quick Verdict
- •rust_scraper leads with a health score of 48/100, but none of the options score above 80.
- •robotxt has the most downloads (34.6K), suggesting wider adoption.
- •⚠ robotxt has not been updated in over a year.
- •robotxt, halo-spider are pre-1.0 — API may change.