Python Web Scraping Internship Jobs in Chicago, IL

This is a remote position; however, the candidate must reside within 30 miles of one of the following locations: Portland, ME; Boston, MA; Chicago, IL; Dallas, TX; San Francisco Bay Area, CA; and Seattle/WA.

About the Team/Role

As WEX continues to scale its Data-as-a-Service (DaaS) platform, the Data Acquisition Team plays a critical role in enabling secure, scalable, and reliable ingestion of data from hundreds of internal systems and external sources.

We are seeking a hands-on Senior Manager, Software Engineering - Data Acquisition to lead our team in acquiring and processing high-volume data, while simultaneously driving the evolution toward AI-augmented, spec-driven software development to enhance platform scalability and delivery speed.

This role requires a strong leader with deep technical expertise in data pipelines, distributed systems, and cloud architecture, who can drive technical excellence, foster a culture of innovation, and align the data acquisition strategy with overall business goals.

Responsibilities:

• Engineering Leadership & Team Development: Recruit, mentor, and lead a high-performing team of software engineers focused on data acquisition, fostering a collaborative and inclusive culture. Oversee performance management, career pathing, and top-tier talent acquisition.

• AI-Augmented Development Strategy: Pioneer the adoption of AI-assisted software development across engineering teams. Define metrics and implement AI-enabled development workflows to measurably enhance engineering productivity.

• Specification-Driven Development (SDD): Establish and enforce a specification-first development methodology. Standardize templates for all key artifacts (APIs, data contracts, ingestion pipelines, architecture) and ensure end-to-end traceability across implementation, validation, deployment, and observability.

• Architectural Transformation & Modernization: Drive the migration to automated, metadata-driven, and declarative engineering architectures. Develop reusable frameworks that translate technical specifications directly into generated code, deployment artifacts, and operational controls.

• Strategic Roadmap Execution: Define and execute the technical roadmap for all data acquisition pipelines and systems, ensuring the infrastructure is highly scalable, reliable, secure, and cost-effective to accommodate accelerating data volume and velocity.

• Technical Governance & Oversight: Provide authoritative technical direction on the design, development, and maintenance of mission-critical data ingestion frameworks. Mandate and enforce best practices for software engineering, data governance, and data quality.

• Stakeholder Collaboration: Partner closely with Product Management, Data Science, Data Governance, and other engineering teams to align data solutions with overarching business requirements and strategic data needs.

• Engineering Process Optimization: Institute and champion continuous improvement in engineering processes, tools, and methodologies, including CI/CD, automation, monitoring, and alerting practices.

• Sustained Operational Excellence: Guarantee the high availability and performance of all data acquisition systems, taking ownership of incident response, recovery, and thorough root cause analysis for major service disruptions.

• Resource & Financial Stewardship: Oversee budget allocation, resource management, and capacity planning to ensure the strategic growth of the data acquisition organization.

Qualifications:

• Education: Bachelor's or Master's degree in Computer Science, Engineering, or a related technical field.

• Experience: 10+ years of experience in software engineering, with at least 5+ years in a management role overseeing software engineering or data acquisition teams.

• Experience in leading virtual teams is highly desirable

• Technical Expertise:

  • Experience implementing AI-assisted engineering workflows in production software organizations.

  • Deep understanding of specification-driven engineering, declarative system design, or model-driven development.

  • Deep expertise in building and managing high-volume, real-time and batch data pipelines (e.g., Kafka, Kinesis, Pulsar).

  • Proficiency with cloud platforms (e.g., AWS, Azure, GCP) and experience designing scalable, serverless, or containerized data ingestion architectures (e.g., Kubernetes, EKS/AKS/GKE).

  • Strong knowledge of various data sources, integration patterns (APIs, web scraping, messaging queues), and ETL/ELT tools.

  • Expertise in programming languages such as Java, Python, Scala, or Go.

  • Solid understanding of database technologies (SQL, NoSQL, Data Warehouses like Snowflake, Redshift, etc.).

• Leadership Skills: Proven ability to lead, motivate, and manage multiple distributed teams. Excellent communication, presentation, and interpersonal skills.

• Problem Solving: Strong analytical and problem-solving skills, with the ability to define solutions for complex technical challenges.

The base pay range represents the anticipated low and high end of the pay range for this position. Actual pay rates will vary and will be based on various factors, such as your qualifications, skills, competencies, and proficiency for the role. Base pay is one component of WEX's total compensation package. Most sales positions are eligible for commission under the terms of an applicable plan. Non-sales roles are typically eligible for a quarterly or annual bonus based on their role and applicable plan. WEX's comprehensive and market competitive benefits are designed to support your personal and professional well-being. Benefits include health, dental and vision insurances, retirement savings plan, paid time off, health savings account, flexible spending accounts, life insurance, disability insurance, tuition reimbursement, and more. For more information, check out the "About Us" section.Pay Range: $175,600.00 - $204,300.00