Web Crawler Engineer Exa is an applied AI lab building a search engine unlike the world has ever seen. We build massive-scale infra to crawl the entire web, train state-of-the-art embedding models to ...
Web Crawler Engineer Exa is an applied AI lab building a search engine unlike the world has ever seen. We build massive-scale infra to crawl the entire web, train state-of-the-art embedding models to ...
Software Engineer, Web Crawling
San Francisco, CA · On-site
$180K - $350K/yr
As a Web Crawler engineer, you'd be responsible for crawling the entire web. Basically build Google-scale crawling! Who You Are * You have extensive experience building and scaling web crawlers, or ...
Software Engineer, Web Crawling
San Francisco, CA · On-site
$180K - $350K/yr
As a Web Crawler engineer, you'd be responsible for crawling the entire web. Basically build Google-scale crawling! Who You Are * You have extensive experience building and scaling web crawlers, or ...
Web Crawler Development: Design and implement efficient and scalable web crawlers in Java to collect data from various online sources * Experience with designing and building POCs as required for new ...
Web Crawler Development: Design and implement efficient and scalable web crawlers in Java to collect data from various online sources * Experience with designing and building POCs as required for new ...
$42.50 - $56.25/hr
\n \n \n \n \n Senior Fullstack Developer needed for an impressive Irish tech company building a world leading bot\/web crawler detection product. \n \n \n \n \n \n We need engineers who are ...
$42.50 - $56.25/hr
\n \n \n \n \n Senior Fullstack Developer needed for an impressive Irish tech company building a world leading bot\/web crawler detection product. \n \n \n \n \n \n We need engineers who are ...
\n \n \n \n \n Senior Fullstack Developer needed for an impressive Irish tech company building a world leading bot\/web crawler detection product. \n \n \n \n \n \n We need engineers who are ...
\n \n \n \n \n Senior Fullstack Developer needed for an impressive Irish tech company building a world leading bot\/web crawler detection product. \n \n \n \n \n \n We need engineers who are ...
Java Developer (Remote Permanent Opportunity)
$52.25 - $67.50/hr
Ingestion Pipeline: process data that comes from our web crawler which discovers and fetches ... Solid experience with Java programming (we also use Spring, Spring Webflux, Reactor, Netty) * Multi ...
Java Developer (Remote Permanent Opportunity)
$52.25 - $67.50/hr
Ingestion Pipeline: process data that comes from our web crawler which discovers and fetches ... Solid experience with Java programming (we also use Spring, Spring Webflux, Reactor, Netty) * Multi ...
Java Developer (Remote Permanent Opportunity)
$52.25 - $67.50/hr
Ingestion Pipeline: process data that comes from our web crawler which discovers and fetches ... Solid experience with Java programming (we also use Spring, Spring Webflux, Reactor, Netty) * Multi ...
Java Developer (Remote Permanent Opportunity)
$52.25 - $67.50/hr
Ingestion Pipeline: process data that comes from our web crawler which discovers and fetches ... Solid experience with Java programming (we also use Spring, Spring Webflux, Reactor, Netty) * Multi ...
Research Engineer, Generalist
San Francisco, CA · On-site
$180K - $350K/yr
We now power search for Cursor, Cognition, HubSpot, and over 400,000 developers and have raised ... Make our web crawler 3x faster Logistics * Location: This is an in-person opportunity in San ...
Research Engineer, Generalist
San Francisco, CA · On-site
$180K - $350K/yr
We now power search for Cursor, Cognition, HubSpot, and over 400,000 developers and have raised ... Make our web crawler 3x faster Logistics * Location: This is an in-person opportunity in San ...
Senior Java Engineer
$58.50 - $74.75/hr
Senior Java Engineer Position: Full time Location: Toronto, Ontario (Initially Remote) About Us: We ... Ingestion Pipeline: process data that comes from our web crawler which discovers and fetches ...
Senior Java Engineer
$58.50 - $74.75/hr
Senior Java Engineer Position: Full time Location: Toronto, Ontario (Initially Remote) About Us: We ... Ingestion Pipeline: process data that comes from our web crawler which discovers and fetches ...
Senior Java Engineer
$58.50 - $74.75/hr
Senior Java Engineer Position: Full time Location: Toronto, Ontario (Initially Remote) About Us: We ... Ingestion Pipeline: process data that comes from our web crawler which discovers and fetches ...
Senior Java Engineer
$58.50 - $74.75/hr
Senior Java Engineer Position: Full time Location: Toronto, Ontario (Initially Remote) About Us: We ... Ingestion Pipeline: process data that comes from our web crawler which discovers and fetches ...
Software Engineer, Data Infrastructure
San Francisco, CA · On-site
$350K - $475K/yr
About the Role We're looking for an engineer to join us and contribute to data infrastructure. You ... Have experience building a web crawler. * Have extensive experience understanding and scaling ...
Software Engineer, Data Infrastructure
San Francisco, CA · On-site
$350K - $475K/yr
About the Role We're looking for an engineer to join us and contribute to data infrastructure. You ... Have experience building a web crawler. * Have extensive experience understanding and scaling ...
Member of Engineering (Pre-training / Data Acquisition)
$117K - $140K/yr
As our first dedicated data acquisition engineer, you will spearhead and evolve systems that crawl ... Design, build, and operate a large-scale web crawler responsible for acquiring all openly ...
Member of Engineering (Pre-training / Data Acquisition)
$117K - $140K/yr
As our first dedicated data acquisition engineer, you will spearhead and evolve systems that crawl ... Design, build, and operate a large-scale web crawler responsible for acquiring all openly ...
Remote US or UK Crawl Engineer
Chicago, IL · On-site +1
We're seeking a Senior Crawl Engineer to be part of our professional services team. This role ... the modern web with a sophisticated crawler at its core. This job is at the heart of crawler ...
Remote US or UK Crawl Engineer
Chicago, IL · On-site +1
We're seeking a Senior Crawl Engineer to be part of our professional services team. This role ... the modern web with a sophisticated crawler at its core. This job is at the heart of crawler ...
Remote US or UK Crawl Engineer
Chicago, IL · On-site +1
We're seeking a Senior Crawl Engineer to be part of our professional services team. This role ... the modern web with a sophisticated crawler at its core. This job is at the heart of crawler ...
Remote US or UK Crawl Engineer
Chicago, IL · On-site +1
We're seeking a Senior Crawl Engineer to be part of our professional services team. This role ... the modern web with a sophisticated crawler at its core. This job is at the heart of crawler ...
Remote US or UK Crawl Engineer
Chicago, IL · On-site +1
We're seeking a Senior Crawl Engineer to be part of our professional services team. This role ... the modern web with a sophisticated crawler at its core. This job is at the heart of crawler ...
Remote US or UK Crawl Engineer
Chicago, IL · On-site +1
We're seeking a Senior Crawl Engineer to be part of our professional services team. This role ... the modern web with a sophisticated crawler at its core. This job is at the heart of crawler ...
Were seeking a Senior Crawl Engineer to be part of our professional services team. This role ... the modern web with a sophisticated crawler at its core. This job is at the heart of crawler ...
Quick apply
Were seeking a Senior Crawl Engineer to be part of our professional services team. This role ... the modern web with a sophisticated crawler at its core. This job is at the heart of crawler ...
Software Engineer, Data Infrastructure
San Francisco, CA · On-site
$350K - $475K/yr
Software Engineer, Data Infrastructure Thinking Machines Lab's mission is to empower humanity ... Have experience building a web crawler. * Have extensive experience understanding and scaling ...
Software Engineer, Data Infrastructure
San Francisco, CA · On-site
$350K - $475K/yr
Software Engineer, Data Infrastructure Thinking Machines Lab's mission is to empower humanity ... Have experience building a web crawler. * Have extensive experience understanding and scaling ...
Software Engineer, Data Infrastructure
San Francisco, CA · On-site
$350K - $475K/yr
We are scientists, engineers, and builders who've created some of the most widely used AI products ... Have experience building a web crawler. * Have extensive experience understanding and scaling ...
Software Engineer, Data Infrastructure
San Francisco, CA · On-site
$350K - $475K/yr
We are scientists, engineers, and builders who've created some of the most widely used AI products ... Have experience building a web crawler. * Have extensive experience understanding and scaling ...
Partner with web developers and Web Data Specialist on the technical foundation (schema markup, site speed, entity relationship) to ensure content is discoverable by both crawler and generative ...
Partner with web developers and Web Data Specialist on the technical foundation (schema markup, site speed, entity relationship) to ensure content is discoverable by both crawler and generative ...
Web Content Specialist I
Maryville, TN · On-site
Partner with web developers and Web Data Specialist on the technical foundation (schema markup, site speed, entity relationship) to ensure content is discoverable by both crawler and generative ...
Web Content Specialist I
Maryville, TN · On-site
Partner with web developers and Web Data Specialist on the technical foundation (schema markup, site speed, entity relationship) to ensure content is discoverable by both crawler and generative ...
Web Crawler Engineer information
See salary details
$11.54 - $18.36
4% of jobs
$18.36 - $25.17
0% of jobs
$25.17 - $31.99
0% of jobs
$31.99 - $38.81
6% of jobs
$38.81 - $45.63
5% of jobs
$50.89 is the 25th percentile. Wages below this are outliers.
$45.63 - $52.45
12% of jobs
The median wage is $59.11 / hr.
$52.45 - $59.27
23% of jobs
$65.74 is the 75th percentile. Wages above this are outliers.
$59.27 - $66.08
26% of jobs
$66.08 - $72.90
13% of jobs
$72.90 - $79.72
3% of jobs
$79.72 - $86.54
7% of jobs
$11
$59
$86
How much do web crawler engineer jobs pay per hour?
As of Jun 22, 2026, the average hourly pay for web crawler engineer in the United States is $59.01, according to ZipRecruiter salary data. Most workers in this role earn between $51.20 and $66.83 per hour, depending on experience, location, and employer.
Other
Medical, Dental, Vision
This job post has expired today. Applications are no longer accepted.
Job description
Web Crawler Engineer
Exa is an applied AI lab building a search engine unlike the world has ever seen. We build massive-scale infra to crawl the entire web, train state-of-the-art embedding models to process it, and design super high performant vector databases to retrieve over it. We now power search for Cursor, Cognition, HubSpot, and over 400,000 developers and have raised $350m from Lightspeed, Benchmark, and a16z.
Our ultimate goal is to build perfect search over all the world's information, far beyond Google. If you want to build massive-scale ML systems that will define the way the new AI world consumes information, this is the place for you.
As a Web Crawler engineer, you'd be responsible for crawling the entire web. Basically build Google-scale crawling!
Who You AreYou have extensive experience building and scaling web crawlers, or would be excited to ramp up very quickly
You have experience with some high performance language (C++, Rust, etc.)
You are familiar with TypeScript, Playwright, modern web design, CDP (Chrome DevTools Protocol)
You're comfortable optimizing a system to an exceptional degree
You care about the problem of finding high quality knowledge and recognize how important this is for the world
Build a distributed crawler that can handle 100M+ pages per day
Optimize crawl politeness and rate limiting across thousands of domains
Design systems to detect and handle dynamic content, JavaScript rendering, and anti-bot measures
Create intelligent crawl scheduling and prioritization algorithms for maximum coverage efficiency
Location: This is an in-person opportunity in San Francisco.
Visas: We're happy to sponsor international candidates (e.g., STEM OPT, OPT, H1B, O1, E3). While we cannot guarantee your visa, we have historically been successful in sponsoring candidates from all over the world. If you receive an offer, our team will work hard to get you a visa.
Benefits: We offer premium healthcare benefits (medical, dental, vision), fertility benefits, 16 weeks of fully paid parental leave for all new parents, and a monthly wellness stipend to all of our employees.