rustio rustio.net
54

crw-crawl

v0.37.2 Experimental

Async BFS web crawler with rate limiting and robots.txt support for CRW

AGPL-3.0 Edition 2024
Command line utilitiesWeb programming #mcp#web-crawler#llm#web-scraper

Quick Verdict

  • โœ“Actively maintained (updated 9d ago)
  • !Pre-1.0: API may have breaking changes
  • โœ“Trusted by 158 crates
  • !Heavy dependency tree (23 direct deps)

Security

Checking security advisories...
Downloads
3.0K
Dependents
158
Releases
75
Size
174KB

Deep Insights

๐Ÿ“Š
Stable downloads

322 downloads in the last 30 days (11/day). Volume is roughly flat compared to the previous period.

๐Ÿ”—
Moderate adoption

158 crates depend on crw-crawl. Reasonable ecosystem adoption, though not yet a core dependency.

๐Ÿ“ฆ
Heavy dependency tree

23 direct dependencies. Consider the impact on compile times and supply chain complexity.

๐ŸŒŸ
Used by top crates

Notable dependents include crw-server, crw-cli, crw-monitor. When high-quality crates choose crw-crawl, it's a strong quality signal.

Health Breakdown

Maintenance 18/25

Recency, release consistency, active ratio

Quality 10/25

Yanked ratio, deps, size, maturity, features

Community 12/20

Reverse deps, ownership, ecosystem

Popularity 4/15

Downloads, momentum, growth trend

Documentation 10/15

Docs, repo, license, metadata

Download Trend

Daily downloads ยท last 90 days
0/day avg
08/239/109/2810/1611/311/20

Top Dependents

Most downloaded crates that depend on crw-crawl

Version Adoption

v0.35.1
26%
v0.36.0
23%
v0.33.0
20%
v0.37.2
16%
v0.32.0
15%

Release Timeline

10 releasessince 2026
J
F
M
A
M
J
J
A
S
O
N
D
2026
10
Less
More

README

Loading README...

Maintainers

Dependencies
23
direct dependencies
Dependents
158
crates depend on crw-crawl

Similar Crates