ZipRecruiter

Log In

1

Web Scraping Engineer Jobs in California (NOW HIRING)

SS&C Technologies

Sr. Data Engineer

San Francisco, CA · On-site

$134K - $162K/yr

Data Engineer Location : San Francisco, CA | Hybrid We're looking for a talented Sr. Data Engineer ... Expertise in building data ingestion tools using Python, including web scraping and external APIs ...

Salesforce, Inc.

Data Engineer - Senior/Lead

San Francisco, CA · On-site

$124K - $169K/yr

Job Category Software Engineering Job Details About Salesforce Salesforce is the #1 AI CRM, where ... Comfort with tasks such as writing scripts, web scraping, and pulling data from APIs. * Experience ...

SS&C

Sr. Data Engineer

San Francisco, CA · On-site

$134K - $162K/yr

Data Engineer Location : San Francisco, CA | Hybrid We're looking for a talented Sr. Data Engineer ... Expertise in building data ingestion tools using Python, including web scraping and external APIs ...

daydream Labs, Inc

Applied AI Engineer

San Francisco, CA · On-site

$160K - $260K/yr

Shravan (CTO) , formerly an Engineering Lead at Flixed and a Software Engineer at Facebook, where he worked on mass-scale web scraping for Facebook Jobs and Meta Reality Labs. * Nico (Product ...

daydream, inc.

Senior / Staff Engineer

San Francisco, CA · On-site

$180 - $280/hr

Shravan (CTO) , formerly an Engineering Lead at Flixed and a Software Engineer at Facebook, where he worked on mass-scale web scraping for Facebook Jobs and Meta Reality Labs. * Nico (Product ...

New

Anthropic

Threat Intelligence Engineer

San Francisco, CA · On-site +1

$320K - $405K/yr

This is a foundational engineering role on a small, high-impact team. You will take projects from ... Familiarity with web scraping and data extraction at scale * Experience with behavioral analytics ...

Dapi

Full Stack Developer

San Francisco, CA · On-site +1

We are looking to hire a passionate full-stack engineer who can contribute to the design ... Your primary focus will be developing, maintaining and supporting our novel in-house scraping ...

previous page

Showing results 41-60

All JobsWeb Scraping Engineer JobsWeb Scraping Engineer Jobs in California

Web Scraping Engineer information

What are the key skills and qualifications needed to thrive as a web scraping engineer, and why are they important?

To thrive as a Web Scraping Engineer, you need strong programming skills (especially in Python), knowledge of data extraction techniques, and familiarity with web protocols and HTML structure. Expertise with tools like Scrapy, BeautifulSoup, Selenium, and experience with APIs or anti-bot evasion techniques is typically required. Attention to detail, problem-solving, and strong analytical thinking are essential soft skills in this field. These skills ensure efficient, ethical, and reliable extraction of data from diverse web sources while navigating technical and legal complexities.

What does a web scraping engineer do?

A Web Scraping Engineer designs and develops software tools or scripts to automatically extract data from websites. They work with programming languages like Python and use libraries such as BeautifulSoup or Scrapy to gather and process web data efficiently. In addition to building scrapers, they also handle challenges such as dealing with website restrictions, CAPTCHAs, and ensuring compliance with legal and ethical guidelines. Their work is often used for market research, data analysis, or feeding information into business applications.

What is the difference between Web Scraping Engineer vs Data Engineer?

AspectWeb Scraping EngineerData Engineer
Primary FocusDeveloping and maintaining web scraping tools to extract data from websitesBuilding and managing data pipelines and infrastructure for data storage and processing
Skills & CertificationsPython, APIs, HTML, CSS, web scraping libraries (BeautifulSoup, Scrapy)SQL, ETL tools, cloud platforms, programming (Python, Java)
Work EnvironmentTech companies, data-driven startups, research projectsLarge enterprises, data warehouses, cloud environments
Industry UsageData collection for analytics, research, competitive analysisData integration, analytics, machine learning pipelines

While both roles involve working with data, a Web Scraping Engineer specializes in extracting data from websites using scraping tools, whereas a Data Engineer focuses on building data pipelines and infrastructure for storing and processing large datasets. The roles often overlap in skills like Python programming but serve different core functions within data ecosystems.

What are some common challenges faced by web scraping engineers when extracting data from dynamic websites?

Web Scraping Engineers often encounter challenges such as navigating websites that use JavaScript to load content dynamically, dealing with anti-bot measures like CAPTCHAs or IP blocking, and ensuring data accuracy as website structures frequently change. To address these issues, engineers typically use headless browsers, rotating proxies, and robust error handling strategies. Staying up-to-date with the latest web technologies and adapting scripts proactively are essential for long-term success in this role.

What are popular job titles related to Web Scraping Engineer jobs in California?

For Web Scraping Engineer jobs in California, the most frequently searched job titles are:

What job categories do people searching Web Scraping Engineer jobs in California look for?

The top searched job categories for Web Scraping Engineer jobs in California are:

What cities in California are hiring for Web Scraping Engineer jobs?

Cities in California with the most Web Scraping Engineer job openings:

Web Scraping Engineer jobs near you

Senior Technical Recruiter, Production Engineering (Contract)

Anduril Industries

Costa Mesa, CA • Hybrid

Full-time, Contractor

Re-posted 25 days ago

Anduril rating

9.1

Company rating: 9.1 out of 10

Anduril

Based on 17 frontline employees who took The Breakroom Quiz

7.4

Company rating compared to similar companies: 7.4 out of 10

Manufacturers average

Based on 75,298 frontline employees who took The Breakroom Quiz

Job description

ABOUT THE JOB

As a key member of Anduril's Talent Acquisition team, you will be responsible for the company's talent management as we continue to rapidly scale and grow. Your work will be hands-on and involve collaborating with Production leaders to define the hiring needs, influence process improvements and efficiencies, and set a high bar of talent and performance. To achieve this, you will be given the autonomy and tools to execute, build, and develop Anduril's world class team. We are looking for someone who can handle every aspect of recruiting and talent mapping, excited to be part of lean and nimble team on one mission together, where no task is too big or too small.

This role will be hybrid in Costa Mesa, CA. This is a 6 month contract with the ability to extend up to 18 months. Opportunity to be converted to a full-time employee based on performance, impact, and business need. 

WHAT YOU'LL DO
  • Act as a subject matter expert in prospecting techniques and tools used for information retrieval, data extraction, web-scraping, continuous process improvement, process automation, and candidate management
  • Own and drive hiring strategy full-cycle, providing effective, clear communication and reporting
  • Conduct a high volume of interviews, demonstrating ability to anticipate and influence hiring manager preferences through successful interview-to-offer conversion ratios
  • Engage and source passive candidates using LinkedIn Recruiter, Boolean strings, referrals and SOBO campaigns
  • Build talent maps to generate market insights to inform your engagement strategy
  • Drive diverse talent into the organization ensuring a positive candidate experience at every touchpoint
  • Represent the company's brand and recruiting team internally and externally at the highest caliber
  • Leverage internal resources, team mates, and cross functional partners to build strategy around selling our value proposition and impacting hiring practices

REQUIRED QUALIFICATIONS

  • 6+ years of high-volume talent acquisition experience evaluating and hiring top tier talent
  • 3+ years of experience working with senior level leaders and stakeholders, reporting on progress, creating and advising on talent strategies
  • Experience recruiting at an agency and fast-paced startup, hiring exempt roles for Hardware, Production, or Manufacturing Engineering
  • Ability to be onsite/hybrid at Costa Mesa, CA office
PREFERRED QUALIFICATIONS 
  • Knowledge of ITAR policies around hiring practices 
  • Experience acting in a consultative manner where your guidance has led to improved outcomes and positive candidate experiences
  • Clear and effective communication with candidates to understand motivation, drivers, and fit within the organization
  • Experience managing various high-volume pipelines of candidates and keeping track of their progress throughout the recruitment process
  • Highly autonomous, collaborative, and adaptable

Working at Anduril

About Anduril, in their own words

From Anduril

As the world enters an era of strategic competition, Anduril delivers cutting-edge autonomy, AI, computer vision, sensor fusion, and networking technology to the warfighter in months, not years.

What Anduril employees say

Pay

Benefits

Hours and flexibility

Workplace

Get the full story on Breakroom

Anduril Industries logo

About Anduril Industries

Sourced by ZipRecruiter

Anduril Industries is a trailblazer in the technology industry based in Costa Mesa, CA, US. Founded in 2017 by Palmer Luckey, the creator of Oculus VR, the company focuses on developing innovative technology to equip and empower those in the defense sector. Its primary products include cutting-edge autonomous systems and AI software that assist in combating threats to national and global security. The mission of Anduril Industries is to integrate technology and defense by building transformative, scalable solutions that ensure a safer world.

Industry

Guided missile and space vehicle manufacturing

Company size

501 - 1,000 Employees

Headquarters location

Costa Mesa, CA, US

Year founded

2017

Social media

FacebookTwitter

View All Anduril Industries Jobs

Apply