Perform and oversee data engineering tasks such as writing scripts, calling APIs, web scraping, and writing SQL queries to handle large datasets. * Customize and oversee integration tools, warehouses ...
Perform and oversee data engineering tasks such as writing scripts, calling APIs, web scraping, and writing SQL queries to handle large datasets. * Customize and oversee integration tools, warehouses ...
Senior Data Scientist
Mclean, VA · On-site
Perform and oversee data engineering tasks such as writing scripts, calling APIs, web scraping, and writing SQL queries to handle large datasets. * Customize and oversee integration tools, warehouses ...
Senior Data Scientist
Mclean, VA · On-site
Perform and oversee data engineering tasks such as writing scripts, calling APIs, web scraping, and writing SQL queries to handle large datasets. * Customize and oversee integration tools, warehouses ...
Data Scientist
Mclean, VA · On-site
$107K - $195K/yr
Perform web scraping and apply various techniques for processing unstructured data. * Leverage automation and machine learning to manage data, predict scenarios, and make data-driven recommendations.
Data Scientist
Mclean, VA · On-site
$107K - $195K/yr
Perform web scraping and apply various techniques for processing unstructured data. * Leverage automation and machine learning to manage data, predict scenarios, and make data-driven recommendations.
Research Assistant
Arlington, VA · On-site
$22.25 - $30.50/hr
Collecting original data through various methods including public records/FOIA requests, contacting government agencies, web scraping, or systematic review of government documents, statutes ...
Quick apply
Research Assistant
Arlington, VA · On-site
$22.25 - $30.50/hr
Collecting original data through various methods including public records/FOIA requests, contacting government agencies, web scraping, or systematic review of government documents, statutes ...
Research Assistant
Arlington, VA · On-site
$22.25 - $30.50/hr
Collecting original data through various methods including public records/FOIA requests, contacting government agencies, web scraping, or systematic review of government documents, statutes ...
Research Assistant
Arlington, VA · On-site
$22.25 - $30.50/hr
Collecting original data through various methods including public records/FOIA requests, contacting government agencies, web scraping, or systematic review of government documents, statutes ...
Business Intelligence Consultant
Richmond, VA · On-site
$60 - $70/hr
Experience with web scraping or API integrations for data sourcing. * Understanding of data modeling concepts. * Familiarity with other popular BI tools. What We Offer: * A challenging and rewarding ...
Business Intelligence Consultant
Richmond, VA · On-site
$60 - $70/hr
Experience with web scraping or API integrations for data sourcing. * Understanding of data modeling concepts. * Familiarity with other popular BI tools. What We Offer: * A challenging and rewarding ...
Data Engineer
Chantilly, VA · On-site
$117K - $140K/yr
Develop and implement web scraping and data ingestion workflows to collect open-source data, integrating content and producing structured datasets and visualizations for analytics and stakeholder ...
Data Engineer
Chantilly, VA · On-site
$117K - $140K/yr
Develop and implement web scraping and data ingestion workflows to collect open-source data, integrating content and producing structured datasets and visualizations for analytics and stakeholder ...
Data Engineer with Security Clearance
Chantilly, VA · Hybrid
$118K - $142K/yr
Develop and implement web scraping and data ingestion workflows to collect open-source data, integrating content and producing structured datasets and visualizations for analytics and stakeholder ...
Data Engineer with Security Clearance
Chantilly, VA · Hybrid
$118K - $142K/yr
Develop and implement web scraping and data ingestion workflows to collect open-source data, integrating content and producing structured datasets and visualizations for analytics and stakeholder ...
Data Engineer
Centreville, VA · On-site +1
$113K - $136K/yr
Develop and implement web scraping and data ingestion workflows to collect open-source data, integrating content and producing structured datasets and visualizations for analytics and stakeholder ...
Data Engineer
Centreville, VA · On-site +1
$113K - $136K/yr
Develop and implement web scraping and data ingestion workflows to collect open-source data, integrating content and producing structured datasets and visualizations for analytics and stakeholder ...
Senior Research Analyst
Arlington, VA · On-site
$133K - $134K/yr
Experience with both original data collection (whether through public records requests, web scraping, document review, or other means) and use of datasets from government or non-governmental sources ...
Quick apply
Senior Research Analyst
Arlington, VA · On-site
$133K - $134K/yr
Experience with both original data collection (whether through public records requests, web scraping, document review, or other means) and use of datasets from government or non-governmental sources ...
Build web scraping algorithms for imagery curation in accordance with NGA Maven data priorities * Integrate multiple data and intelligence sources in various formats for imagery curation to address ...
Build web scraping algorithms for imagery curation in accordance with NGA Maven data priorities * Integrate multiple data and intelligence sources in various formats for imagery curation to address ...
Experience with API integrations, web scraping, and database management * Demonstrated experience working on client-facing activities * Experience supporting change management processes, especially ...
Experience with API integrations, web scraping, and database management * Demonstrated experience working on client-facing activities * Experience supporting change management processes, especially ...
Exploitation Specialist/Data Scientist support to MA VEN, EO and SAR (Senior)
Springfield, VA · On-site
$175K - $195K/yr
Build web scraping algorithms for imagery curation in accordance with NGA Maven data priorities * Integrate multiple data and intelligence sources in various formats for imagery curation to address ...
Exploitation Specialist/Data Scientist support to MA VEN, EO and SAR (Senior)
Springfield, VA · On-site
$175K - $195K/yr
Build web scraping algorithms for imagery curation in accordance with NGA Maven data priorities * Integrate multiple data and intelligence sources in various formats for imagery curation to address ...
Experience with API integrations, web scraping, and database management * Demonstrated experience working on client-facing activities * Experience supporting change management processes, especially ...
Experience with API integrations, web scraping, and database management * Demonstrated experience working on client-facing activities * Experience supporting change management processes, especially ...
Senior Research Analyst
Arlington, VA · On-site
$133K - $134K/yr
Experience with both original data collection (whether through public records requests, web scraping, document review, or other means) and use of datasets from government or non-governmental sources ...
Senior Research Analyst
Arlington, VA · On-site
$133K - $134K/yr
Experience with both original data collection (whether through public records requests, web scraping, document review, or other means) and use of datasets from government or non-governmental sources ...
Build web scraping algorithms for imagery curation in accordance with NGA Maven data priorities * Integrate multiple data and intelligence sources in various formats for imagery curation to address ...
Build web scraping algorithms for imagery curation in accordance with NGA Maven data priorities * Integrate multiple data and intelligence sources in various formats for imagery curation to address ...
Python / AWS Developer
Chantilly, VA · On-site
$96K - $192K/yr
Experience with AWS CDK, DevOps, and/or Web Scraping. Security Clearance : * Must have an active TS/SCI with Poly security clearance. As required by local law, Accenture Federal Services provides ...
Python / AWS Developer
Chantilly, VA · On-site
$96K - $192K/yr
Experience with AWS CDK, DevOps, and/or Web Scraping. Security Clearance : * Must have an active TS/SCI with Poly security clearance. As required by local law, Accenture Federal Services provides ...
Python / AWS Developer
$69K - $131K/yr
Experience with AWS CDK, DevOps, and/or Web Scraping Security Clearance : * Must have an active TS/SCI with Poly security clearance As required by local law, Accenture Federal Services provides ...
Python / AWS Developer
$69K - $131K/yr
Experience with AWS CDK, DevOps, and/or Web Scraping Security Clearance : * Must have an active TS/SCI with Poly security clearance As required by local law, Accenture Federal Services provides ...
Python / AWS Developer
$96K - $192K/yr
Experience with AWS CDK, DevOps, and/or Web Scraping. Security Clearance : * Must have an active TS/SCI with Poly security clearance. As required by local law, Accenture Federal Services provides ...
Python / AWS Developer
$96K - $192K/yr
Experience with AWS CDK, DevOps, and/or Web Scraping. Security Clearance : * Must have an active TS/SCI with Poly security clearance. As required by local law, Accenture Federal Services provides ...
Python / AWS Developer
Chantilly, VA · On-site
$69K - $131K/yr
Experience with AWS CDK, DevOps, and/or Web Scraping Security Clearance : * Must have an active TS/SCI with Poly security clearance As required by local law, Accenture Federal Services provides ...
Python / AWS Developer
Chantilly, VA · On-site
$69K - $131K/yr
Experience with AWS CDK, DevOps, and/or Web Scraping Security Clearance : * Must have an active TS/SCI with Poly security clearance As required by local law, Accenture Federal Services provides ...
Web Scraping information
See Virginia salary details
$11.44 - $18.20
4% of jobs
$18.20 - $24.96
0% of jobs
$24.96 - $31.72
0% of jobs
$31.72 - $38.48
6% of jobs
$38.48 - $45.24
5% of jobs
$50.45 is the 25th percentile. Wages below this are outliers.
$45.24 - $52
12% of jobs
The median wage is $58.60 / hr.
$52 - $58.76
23% of jobs
$65.18 is the 75th percentile. Wages above this are outliers.
$58.76 - $65.52
26% of jobs
$65.52 - $72.28
13% of jobs
$72.28 - $79.04
3% of jobs
$79.04 - $85.80
7% of jobs
$11
$58
$85
How much do web scraping jobs pay per hour?
Will AI replace web scraping?
What are some common challenges faced by web scraping professionals, and how are they typically addressed?
What is web scraping?
What is the difference between Web Scraping vs Data Analyst?
| Aspect | Web Scraping | Data Analyst |
|---|---|---|
| Required Skills | Programming, data extraction, scripting | Data interpretation, statistical analysis, visualization |
| Work Environment | Technical, often remote, involves coding | Office or remote, involves analysis and reporting |
| Tools & Technologies | Python, BeautifulSoup, Scrapy | Excel, SQL, Tableau, R |
| Industry Usage | Data collection for research, marketing, SEO | Business insights, reporting, decision-making |
Web Scraping focuses on extracting data from websites using programming skills, while Data Analysts interpret and analyze data to generate insights. Both roles require analytical thinking but differ in technical focus and end goals.
Is web scraping a job?
What are the key skills and qualifications needed to thrive as a Web Scraping Specialist, and why are they important?
Is web scraping illegal?
What Are Web Scraping Jobs?
Web scraping jobs involve using specialized software and web crawling tools to extract data from websites. This data is extracted for competitor analysis, market trends, pricing research, and other information that can help businesses improve their performance. As a data engineer, your responsibilities are to understand and use these tools or use manual techniques to collect information. Your duties also include creating or troubleshooting web scraping tools, analyzing the data you obtain, and passing the information on to the appropriate leaders of a company. You can find these positions with technology companies that offer web scraping services to clients.
What is the salary of web scraping engineer?
Other
Medical, Dental, Vision, Retirement
Posted 6 days ago
Job description
* Comprehensive healthcare benefits, including medical, vision, dental, and orthodontia coverage. * A substantial retirement plan with no vesting schedule. Career development opportunities, including on-the-job training, tuition reimbursement, and networking.
* A positive work environment where employees are respected, supported, and engaged. As a Senior Data Scientist, you will: * Conducts data analytics, data engineering, data mining, exploratory analysis, predictive analysis, and statistical analysis, and uses scientific techniques to correlate data into graphical, written, visual and verbal narrative products, enabling more infom1ed analytic decisions. * Proactively retrieves information from various sources, analyzes it for better understanding about the data set, and builds Al tools that automate certain processes.
* Duties typically include creating various ML based tools or processes, such as recommendation engines or automated lead scoring systems. * Performs statistical analysis, applies data mining techniques, and builds high quality prediction systems. * Should be skilled in data visualization and use of graphical applications, including Microsoft Office (Power BI) and Tableau; major data science languages, such as Rand Python; managing and merging of disparate data sources, preferably through R, Python, or SQL; statistical analysis; and data mining algorithms.
* Design, prototype, test, and maintain software and web-based applications and content solutions to satisfy customer requirements. * Support software and web architecture development, requirements analysis, process execution and evaluation, selection, and evaluation of COTS/GOTS tools, and integration (with both new and legacy systems). * Work with and design, prototype, & deployment of complex asynchronous frameworks.
* Perform and oversee data engineering tasks such as writing scripts, calling APIs, web scraping, and writing SQL queries to handle large datasets. * Customize and oversee integration tools, warehouses, databases, and analytical systems * Advanced experience with databases design for big data, such as management of large data sets, and/or design and implementation of data stores to support analysis and scalable processing and storage for high frequency/high volume data pipelines. * Configure and provide availability for data-access tools used by all data scientists.
* Work with and design, develop, and actively participate in open-source libraries and technologies such as the Netflix/Twitter stack or equivalent Microservices tooling sud1 as Service Discovery, Circuit Breakers, and Software Load Balancing. * Design, prototype, and integrate distributed caching frameworks such as Evcache and Redis Work with Platforms as a Service such as AWS, Cloud and OpenShift. * Follow a formal design process using formal specifications, data flow diagrams, and other accepted design techniques and also shall adhere to laws, standards, and established guidelines for prototyping and delivery of software and web tools (e.g., software reuse, modifiable, efficient, reliable, understandable, fault tolerant).
* Shall provide software process management and configuration management throughout the software/web prototyping lifecycle. * Collaborate between team members, participating in AGILE scrum for prototyping and product development, as needed. * Aid in the development of strategies, roadmaps, standards, and reference architecture / implementation of prototypes into Enterprise tools.
* Develop and maintain current and planned state architectural documents such as Domain roadmaps, Standards, Patterns, reference architecture/implementation documents for Enterprise Applications. * Develop full-stack and reference architecture designs for infrastructure across hybrid environments inclusive of cloud platforms. * Should have prior experience with large data Multi-INT analytics, ML, large language models, and automated predictive analytics.
* Shall solve complex problems with many variables using various security, network, storage, and computer platforms. * Shall participate in the development of governance processes to ensure that objectives are achieved, risks are managed appropriately, and the organization's resources are used responsibly for the infrastructure landscape. Job Requirements Desired experiences: * At least 8 years of professional experience with data science.
* Experience with software integration or testing, including analyzing and implementing test plans and scripts. * Experience with frequent scripting language use, such as Python and R and using packages commonly used in data science applications or advanced analytics. * Experience with data science, data mining, statistics, or graph algorithms to support analytics objectives.
* Demonstrates comprehensive mission knowledge and skills that affirms completion of all developmental training and experiences for the labor category. * Demonstrates ability to communicate understanding from information that may be incomplete, indirect, highly complex, seemingly unrelated, &/or technically advanced * Demonstrates ability to work independently with minimal oversight and direction. * Demonstrates ability to collaborate and work with other IC members on information sharing, driving collection, and addressing analytic disputes and conflict resolution.
* Experience applying Structured Query Language (SQL), Non-Structured Query Language (NoSQL), Application Program Interface (API) Building, Extract, Transform and Load (ETL) pipelines, Web Application Servers, or Search Index. * Experience using programming languages and products such as Python, Jupyter Notebook, Pandas, Numpy, Requests or Antigravity. * Experience applying complex mathematical and statistical concepts.
* Experience applying statistical and operations research methods and tools. * Experience employing spreadsheets for data manipulations and visualizations. * Relevant industry certification such as CompTIA Cloud Essentials/Microsoft Technical Associate (MTA)/Certificate of Cloud Security (CCSK)/CompTIA A+/CompTIA Security+/EMC Data Science Associates (EMCDSA)/Cloudera System Security Professional (CISSP)/Certified Cloud Professional (CCP) (Cloudera)/Microsoft Certified Professional Developer (MCPD)/Microsoft Certified Solution Developer (MCSD)/Microsoft Certified Solution Expert (MCSE)/Private Cloud/Certified Administrator for Apache Hadoop (CCAH) (Cloudera).
Typical education: * Bachelor's degree. Security Clearance: * Security clearance - Minimum of TS, with ability to get TS/SCI Location: * Remote, with possible Client Site Travel. Prescient Edge is a Veteran-Owned Small Business (VOSB) founded as a counterintelligence (CI) and Human Intelligence (HUMINT) company in 2008.
We are a global operations and solutions integrator delivering full-spectrum intelligence analysis support, training, security, and RD&E support solutions to the Department of Defense and throughout the intelligence community. Prescient Edge is an Equal Opportunity Employer (EEO). All applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, veteran status, or any other characteristic that is protected by law.
We strive to foster equity and inclusion throughout our organization because we believe that diversity of thought is critical for creating a safe and engaging work environment while also enabling the organization's success.
About Prescient Edge
Sourced by ZipRecruiter
Prescient Edge is a prominent name in the global security industry, based out of McLean, VA, US. Established with the primary mission of harnessing science and technology for safety and security purposes, the company specializes in providing a wide range of products and services. This includes research and development, consulting, global operations support, intelligence evaluation, and advanced data solutions. These offerings make Prescient Edge an integral factor in national defense and commercial innovations.
Industry
Guided missile and space vehicle manufacturing
Company size
51 - 200 Employees
Headquarters location
McLean, VA, US
Year founded
2008