rustio rustio.net
72

spider

v2.53.9 Stable

A web crawler and scraper, building blocks for data curation workloads.

MIT Edition 2021
Command line utilitiesWeb programming #spider#crawler#scraper

Quick Verdict

  • โœ“Actively maintained (updated 34d ago)
  • โœ“Stable API (2.x for 8+ years)
  • โœ“Massive adoption (5.8K crates depend on it)
  • !Heavy dependency tree (91 direct deps)
  • โœ“Permissive license (MIT)

Security

Checking security advisories...
Downloads
2.1M
Dependents
5.8K
Releases
2227
Size
784KB

Deep Insights

๐Ÿ“‰
Download decline

13.4K downloads in the last 30 days, down 40% from the previous period. May indicate migration to alternatives.

๐Ÿ”—
Widely adopted

5.8K crates depend on spider. Strong ecosystem adoption means battle-tested code and long-term stability.

โœ…
Stable for 8+ years

The API has been stable (1.x) for over 8 years with 2227 releases. This level of maturity means you can depend on it without worrying about breaking changes.

๐Ÿ“ฆ
Heavy dependency tree

91 direct dependencies. Consider the impact on compile times and supply chain complexity.

๐ŸŒŸ
Used by top crates

Notable dependents include spider_cli, spider_worker, spider_transformations, website_crawler, accessibility-rs. When high-quality crates choose spider, it's a strong quality signal.

Health Breakdown

Maintenance 17/25

Recency, release consistency, active ratio

Quality 17/25

Yanked ratio, deps, size, maturity, features

Community 17/20

Reverse deps, ownership, ecosystem

Popularity 8/15

Downloads, momentum, growth trend

Documentation 13/15

Docs, repo, license, metadata

Download Trend

Daily downloads ยท last 90 days
101/day avg-100%
05001K2K2K3K8/229/99/2710/1511/211/19

Top Dependents

Version Adoption

v2.53.6
59%
v2.53.9
26%
v2.53.4
13%
v2.53.7
1%
v2.53.8
1%

Release Timeline

10 releases2 yankedsince 2026
J
F
M
A
M
J
J
A
S
O
N
D
2026
10
Less
More|
Yanked

Feature Flags

default =["basic", "io_uring", "tcp_fastopen", "splice", "numa", "zero_copy"]

fscmdcrondiskglobnuma*simdsynctimewarcwreqagentbasic*bloomcachehedgeregexserdeservosmartsocksspoofchromecowboygeminiopenaisearchsplice*__basicadblockbalancecontrolcookiesheaderssitemaptracingagent_fsdisk_awsencodingfirewallio_uring*llm_jsonbasic_tlscache_memdns_cachewebdriverzero_copy*agent_fullchrome_cpuetag_cacherate_limitwait_guardchrome_simdflexbuffersinline-moreremote_addrsearch_bingagent_chromeagent_openaiagent_skillscache_geminicache_openaih2_multiplexreal_browserreqwest_jsonrobots_cachesearch_bravespider_cloudtcp_fastopen*tokio_io_stdua_generatorauto_throttlecache_requestchrome_headeddecentralizedsearch_serpersearch_tavilybench_jemallocbench_mimallocchrome_stealthfull_resourceswebdriver_edgeagent_skills_s3agent_webdriverdisk_native_tlsopenai_slim_fitadblock_easylistchrome_interceptrequest_coalescewebdriver_chromewebdriver_headedagent_search_bingchrome_screenshotchrome_store_pageextra_informationparallel_backendspriority_frontierreqwest_multipartwebdriver_firefoxwebdriver_stealthagent_search_bravereqwest_native_tlsreqwest_rustls_tlsrquest_hickory_dnsagent_search_serperagent_search_tavilycache_chrome_hybridchrome_headless_newchrome_remote_cachereqwest_hickory_dnsadaptive_concurrencychrome_serde_stackerwebdriver_screenshotchrome_tls_connectiondecentralized_headersparallel_backends_fullcache_chrome_hybrid_memchrome_remote_cache_memreqwest_native_tls_alpnchrome_remote_cache_diskpage_error_status_detailsreqwest_native_tls_vendoredstring_interner_bucket_backendstring_interner_buffer_backendstring_interner_string_backendreqwest_native_tls_manual_rootsreqwest_native_tls_native_rootsreqwest_native_tls_webpki_roots

README

Loading README...

Maintainers

Dependencies
91
direct dependencies
Dependents
5.8K
crates depend on spider

Similar Crates