ZipRecruiter

Log In

1

Web Scraping Engineer Jobs in Washington (NOW HIRING)

CTC

Web Scraping Engineer- 3566726

Arlington, VA · On-site +1

$105K - $110K/yr

Computer Technologies Consultants (CTC) is seeking a Web Scraping Engineer to support the United States Department of State in Arlington, VA. This would be a 3+ month contract with the possibility of ...

10a Labs

Data Engineer, Web Scraping

Washington, DC · Remote

$105K - $125K/yr

... engineering or a closely related field * Ability to communicate complex technical ideas clearly to non-technical audiences * Proficiency in Python, SQL * Experience with web scraping/crawling (e.g ...

10a Labs

Data Engineer, Web Scraping

Washington, DC · Remote

$105K - $125K/yr

... engineering or a closely related field * Ability to communicate complex technical ideas clearly to non-technical audiences * Proficiency in Python, SQL * Experience with web scraping/crawling (e.g ...

10a Labs

Data Engineer, Web Scraping

Washington, DC · On-site

$105K - $125K/yr

... engineering or a closely related field * Ability to communicate complex technical ideas clearly to non-technical audiences * Proficiency in Python, SQL * Experience with web scraping/crawling (e.g ...

Babel Street

Software Engineer III

Reston, VA · On-site

We are looking for a Senior Python Engineer with a "hacker" mindset to join our team as a Software Engineer III. This role is dedicated to large-scale web scraping and data harvesting. If you have ...

Babel Street

Software Engineer III

Reston, VA · On-site +1

$59.75 - $80.25/hr

We are looking for a Senior Python Engineer with a "hacker" mindset to join our team as a Software Engineer III. This role is dedicated to large-scale web scraping and data harvesting. If you have ...

Prescient Edge

Full Stack Engineer

Mclean, VA · On-site

Perform and oversee data engineering tasks such as writing scripts, calling APIs, web scraping, and writing SQL queries to handle large datasets. * Design, prototype, and integrate distributed ...

AARP

Data Science Intern

Washington, DC · On-site

$28/hr

... web scraping, model topics via generative AI, create digital attributes based on modeled topics ... Pursuing a degree in computer science, statistics, science, technology, engineering, mathematics ...

InstantServe LLC

Data Inventory Analyst

Washington, DC · On-site

Experience with web scraping and APIs * Basic understanding of Cloud Networking * Fluency in SQL and NoSQL database technologies such as SQL Server, Oracle SQL * Fluency in programming languages such ...

next page

Showing results 1-20

People also search for

All JobsWeb Scraping Engineer JobsWeb Scraping Engineer Jobs in Washington

Web Scraping Engineer information

What are the key skills and qualifications needed to thrive as a Web Scraping Engineer, and why are they important?

To thrive as a Web Scraping Engineer, you need strong programming skills (especially in Python), knowledge of data extraction techniques, and familiarity with web protocols and HTML structure. Expertise with tools like Scrapy, BeautifulSoup, Selenium, and experience with APIs or anti-bot evasion techniques is typically required. Attention to detail, problem-solving, and strong analytical thinking are essential soft skills in this field. These skills ensure efficient, ethical, and reliable extraction of data from diverse web sources while navigating technical and legal complexities.

What are some common challenges faced by Web Scraping Engineers when extracting data from dynamic websites?

Web Scraping Engineers often encounter challenges such as navigating websites that use JavaScript to load content dynamically, dealing with anti-bot measures like CAPTCHAs or IP blocking, and ensuring data accuracy as website structures frequently change. To address these issues, engineers typically use headless browsers, rotating proxies, and robust error handling strategies. Staying up-to-date with the latest web technologies and adapting scripts proactively are essential for long-term success in this role.

What does a Web Scraping Engineer do?

A Web Scraping Engineer designs and develops software tools or scripts to automatically extract data from websites. They work with programming languages like Python and use libraries such as BeautifulSoup or Scrapy to gather and process web data efficiently. In addition to building scrapers, they also handle challenges such as dealing with website restrictions, CAPTCHAs, and ensuring compliance with legal and ethical guidelines. Their work is often used for market research, data analysis, or feeding information into business applications.

What is the difference between Web Scraping Engineer vs Data Engineer?

AspectWeb Scraping EngineerData Engineer
Primary FocusDeveloping and maintaining web scraping tools to extract data from websitesBuilding and managing data pipelines and infrastructure for data storage and processing
Skills & CertificationsPython, APIs, HTML, CSS, web scraping libraries (BeautifulSoup, Scrapy)SQL, ETL tools, cloud platforms, programming (Python, Java)
Work EnvironmentTech companies, data-driven startups, research projectsLarge enterprises, data warehouses, cloud environments
Industry UsageData collection for analytics, research, competitive analysisData integration, analytics, machine learning pipelines

While both roles involve working with data, a Web Scraping Engineer specializes in extracting data from websites using scraping tools, whereas a Data Engineer focuses on building data pipelines and infrastructure for storing and processing large datasets. The roles often overlap in skills like Python programming but serve different core functions within data ecosystems.

What are popular job titles related to Web Scraping Engineer jobs in Washington? For Web Scraping Engineer jobs in Washington, the most frequently searched job titles are:
What job categories do people searching Web Scraping Engineer jobs in Washington look for? The top searched job categories for Web Scraping Engineer jobs in Washington are:
What cities in Washington are hiring for Web Scraping Engineer jobs? Cities in Washington with the most Web Scraping Engineer job openings:
Web Scraping Engineer jobs near you
Web Scraping Engineer- 3566726

Web Scraping Engineer- 3566726

Computer Technologies Consultants, Inc.

Arlington, VA • On-site

Full-time

Posted 2 days ago

Job description

Computer Technologies Consultants (CTC) is seeking a Web Scraping Engineer to support the United States Department of State in Arlington, VA. This would be a 3+ month contract with the possibility of extension.
With offices in Washington DC and San Diego, CA, CTC is a leading technology company providing lifecycle IT, data analytics, cloud managed hosting services, agile software development, DevOps, Test Automation, Cyber Security, and infrastructure solutions. Additionally, we provide Professional Talent Acquisition Services as we proudly support the unique needs of U.S. Defense, Intelligence, and Federal Civilian agencies as well as Fortune 1000 companies.
Got the Government Contractor Blues? Looking for a company that cares and goes beyond just filling another contract billet? Well look no further! Experience this family-oriented company who takes pride in you and will help you grow where your passions lie. Holding many Defense & Federal government contracts around the globe, with our client you have the opportunity to take on new and evolving challenges, aim beyond what you think you are capable of and work in collaborative, dynamic, and high-tempo environments. Our clients' employees are their most valued asset and they invest in their people because they are in it for the long term. They are committed to your success and well-being and offer competitive benefits packages, salaries, bonus/award programs, and a high potential for professional growth and job opportunities world-wide.
Why Should You Be Interested?
  • Direct hire position
  • Competitive compensation and comprehensive benefits
  • Mid-size company with room for growth

Position Title: Web Scraping Engineer
Position Location: This position is currently remote, with occasional meetings in Arlington, VA.
Daily Responsibilities:
  • Position supports the development and maintenance of the agency's web scraping infrastructure. The position is responsible for extracting data from various websites and APIs, ensuring data quality and accuracy, and optimizing the scraping process for efficiency. Duties include:
  • Develop and maintain web scraping scripts and tools to extract data from websites and APIs.
  • Collaborate with cross-functional teams to understand data requirements and implement scraping solutions accordingly.
  • Monitor and troubleshoot scraping processes to ensure data quality and accuracy.
  • Optimize scraping scripts for performance and efficiency, considering factors such as speed, scalability, and resource utilization.
  • Stay up-to-date with the latest web scraping techniques, tools, and best practices.
  • Conduct data analysis and validation to ensure the integrity of scraped data.
  • Collaborate with data engineering and data science teams to integrate scraped data into our data pipelines and systems.
  • Document and communicate technical solutions, processes, and best practices to team members.

Required Years of Experience (min):
  • 3+ years of professional experience in web scraping or a similar role.

Required Degree:
  • BS/BA degree in Computer Science, Information Sciences, or related IT discipline OR Allowable Substitution: Additional ten (10) years of related professional experience can be substituted for a BS/BA degree.

Required Experience:
  • Proficiency in Python and Java and experience with web scraping libraries such as BeautifulSoup, Scrapy, or Selenium.
  • Knowledge of AI/machine learning techniques for data extraction and classification.
  • Experience incorporating AI into Quality Control solutions and AI/ML techniques on an open-source application to web scrape for data extraction and classification.
  • Understanding of HTML, CSS, and JavaScript to navigate and interact with websites.
  • Experience working with APIs and handling different data formats (JSON, XML, etc.).
  • Familiarity with database systems and SQL for data storage and retrieval.
  • Familiarity of data cleaning and preprocessing techniques to ensure data quality.
  • Strong problem-solving skills and ability to troubleshoot and debug scraping issues.
  • Excellent communication and collaboration skills to work effectively in a team environment.
  • Attention to detail and ability to handle large volumes of data efficiently.

Preferred Experience:
  • Experience with cloud platforms for scalable web scraping infrastructure.
  • Familiarity with data visualization tools and techniques.
  • Understanding of legal and ethical considerations related to web scraping.

Required Clearance:
  • Secret clearance

Pay Information
TBD
Please note: This range is based on our market pay structures. However, individual salaries are determined by a variety of factors including, but not limited to: business considerations, local market conditions, and internal equity, as well as candidate qualifications, such as skills, education, and experience.
Computer Technologies Consultants, Inc. is an Equal Opportunity Employer that provides employment opportunities for all qualified applicants without regard to race, color, religion, gender identity and/or expression, sexual orientation, age, mental or sensory differing abilities, protected veteran status, sex, national origin, or any other characteristic protected by applicable law. Computer Technologies Consultants, Inc. is devoted to diversity, equity, and inclusion.

Apply