Manager Web Scraping Jobs (NOW HIRING)

Pay:$90,000.00 - $120,000.00 per year

Job description:

Role Summary

We are seeking a hands-on IT Manager / Senior Software & Infrastructure Engineer who can write production code, architect distributed systems, and own our organization's IT and security posture. This is not a help-desk management role roughly half of the job is engineering.

The successful candidate will design and build internal applications on a microservice architecture backed by message queues and event-driven patterns, deploy and operate them on AWS, ship them through CI/CD pipelines, and enforce security and compliance standards appropriate to a healthcare non-profit handling PHI.

The ideal candidate is proactive, comfortable working independently, able to perform under pressure, and motivated to identify manual processes and automate them.

Key Responsibilities

Software Engineering & Distributed Systems

• Design, develop, and maintain internal applications and services using Python, Java (Spring Boot), Node.js, and React.js
• Architect and build microservices service decomposition, API contracts, versioning, inter-service communication, and failure isolation
• Design and operate distributed, asynchronous systems using message queues and event streaming (SQS, SNS, RabbitMQ, Kafka, or similar), including retries, dead-letter queues, idempotency, and eventual-consistency handling
• Build and maintain backend services, RESTful APIs, and automated workflows
• Containerize services (Docker) and orchestrate deployments (ECS/EKS/Fargate or equivalent)
• Write clean, tested, reviewable code; participate in code reviews and maintain engineering standards
• Proactively identify manual, repetitive, or inefficient processes and design automation solutions
• Develop and maintain web scraping and data extraction tools, including Selenium for dynamic content
• Design, query, and optimize SQL and NoSQL databases, including schema design, indexing, and performance tuning

Cloud Infrastructure & CI/CD

• Own and operate our AWS environment (EC2, S3, IAM, RDS, Lambda, SQS/SNS, CloudWatch, VPC) in a cost-conscious manner right-sizing, budget alarms, and spend review
• Build, maintain, and improve CI/CD pipelines (GitHub Actions, GitLab CI, or similar) for automated build, test, and deployment
• Manage infrastructure as code (Terraform, CloudFormation, or CDK) and keep environments reproducible
• Implement monitoring, logging, alerting, and performance tuning across services
• Work extensively in Linux environments using CLI tools and scripting to improve system efficiency

Security & Compliance

• Establish and enforce security policies and best practices across cloud, application, and endpoint layers IAM least privilege, secrets management, encryption at rest and in transit, network segmentation
• Support HIPAA-aligned safeguards for systems that touch PHI; maintain audit-ready documentation and access logs
• Manage vulnerability patching, dependency scanning, and incident response
• Administer identity, access, and endpoint security (M365/Entra ID, MFA, device management)
• Conduct periodic access reviews and support internal/external audits

IT Management & Leadership

• Oversee the organization's IT infrastructure, systems, vendors, and internal tools
• Provide hands-on IT management, including support for IT operations, vendor relationships, and internal users
• Establish and maintain IT best practices, technical documentation, and automation standards
• Troubleshoot and resolve issues across application, system, and infrastructure layers
• Manage multiple priorities under pressure and lead incident resolution
• Communicate clearly with leadership and non-technical stakeholders through status updates and reports
• Perform data mining, analysis, and reporting (including Excel) to support operational and compliance needs

Required Qualifications

• Strong proficiency in Python and Java
• Working knowledge of Node.js, React.js, and Java Spring Boot
• Demonstrated experience designing or building microservices and distributed systems
• Hands-on experience with message queues / event-driven architecture (SQS, SNS, Kafka, RabbitMQ, or similar)
• Practical, substantial AWS experience not just familiarity
• Experience building and maintaining CI/CD pipelines
• Working knowledge of Docker and containerized deployment
• Solid experience working in Linux environments; strong comfort with CLI tools, scripting, and automation
• Strong understanding of database design principles (SQL and NoSQL)
• Proficiency with Git (branching, pull requests, code reviews)
• Working knowledge of security best practices in cloud and backend systems (IAM, secrets management, encryption, least privilege)
• Hands-on experience with web scraping, including Selenium
• Experience with data mining and basic data analysis; strong Excel skills for reporting
• Demonstrated IT management skills, including ownership of systems, vendors, or small teams
• Strong independent problem-solving skills
• Ability to work under pressure, manage multiple priorities, and meet deadlines
• Strong written and verbal communication skills
• Flexible, adaptable, and willing to take on evolving responsibilities

Preferred Qualifications (Nice to Have)

• Infrastructure as Code experience (Terraform, CloudFormation, or CDK)
• Container orchestration (ECS, EKS, or Kubernetes)
• Experience with observability tooling (CloudWatch, Datadog, Grafana, ELK)
• Experience building internal tools, dashboards, or automation systems
• Familiarity with data pipelines, ETL processes, or analytics workflows
• HIPAA / PHI experience, or work in healthcare, non-profit, or other regulated environments
• Security certifications (Security+, AWS Security Specialty) or AWS Solutions Architect / Cloud Practitioner certification
• Hands-on experience with machine learning (ML) is a plus