30% off Lifetime Premium Access expires December 2nd!

30% off Lifetime Premium Access expires Dec 2nd!

Your Dashboard

Interview Coaching

Mock Interviews

1:1 Mentorship

Salary Negotiation

Practice

Learn

System Design

ML System Design

Code

Behavioral

Salary Negotiation

Interview Guides

Blog

Community

Interview Questions

Interview Experiences

Peer System Design Library

Ask The Community

Discord

Refer a Friend

Pricing

Become a Coach

Sign in / Sign up

Search

⌘K

Pricing Become a Coach

Search

⌘K

Get Premium

Design Wikipedia Crawler

Design a distributed web crawler system that can crawl and store Wikipedia pages at scale using multiple machines or edge devices, with focus on coordination, deduplication, and efficient resource utilization.

Asked at:

Meta

Lyft