$112.40K - $134.90K/yr
Other
This job posting has expired and is no longer accepting applications. Check out similar jobs
Job description
Location Malvern, PA Hybrid
Duration 6+ months
Interview Video
Job Description
Required Skills - Top 3-5 Key Words
Required Technical Skills: Python, SQL, AWS Web services (Glu, S3, Lambda)
Core Programming Skills:
- Expert proficiency in Python, with experience in building data pipelines and back-end
- systems.
- Advanced knowledge of SQL for querying and optimizing large datasets.
- AWS Cloud Services Expertise:
- DynamoDB, S3, Athena, GlueETL, Lambda, ECS, Glue Data Quality, EventBridge,
- Redshift Machine Learning, OpenSearch, and RDS.
- Proven expertise in designing fault-tolerant APIs using Swagger/OpenAPI, GraphQL,
- and RESTful standards.
- Robust understanding of distributed systems, load balancing, and failover strategies.
- Monitoring and Orchestration:
- Hands-on experience with Prometheus and Grafana for observability and monitoring.
- Senior Data Engineer 7+ Years of Experience
- We are seeking a highly experienced Senior Data Engineer with 7+ years of expertise in
- designing, building, and optimizing robust data solutions. The ideal candidate must
- possess top-tier skills in Python, AWS services, API development, and TypeScript, and
- have significant hands-on experience with anomaly detection systems.
- The candidate should have a proven ability to work at both strategic and tactical levels,
- from designing data architectures to implementing them in the weeds.
Job Requirements
Key Responsibilities:
- Data Pipeline Development
- Independently design, build, and maintain complex ETL pipelines, ensuring scalability
- and efficiency for large-scale data processing needs.
- Manage pipeline complexity and orchestration, delivering high-performance data
- products accessible via APIs for business-critical applications.
- Archive processed data products into data lakes (e.g., AWS S3) for analytics and
- machine learning use cases.
- Anomaly Detection and Data Quality
- Implement advanced anomaly detection systems and data validation techniques, ensuring data integrity and quality.
- Leverage AI/ML methodologies, including Large Language Models (LLMs), to detect and address data inconsistencies.
- Develop and automate robust data quality and validation frameworks.
- Cloud and API Engineering
- Architect and manage resilient APIs using modern patterns, including microservices,
- RESTful design, and GraphQL.
- Configure API gateways, circuit breakers, and fault-tolerant mechanisms for distributed
Most Popular Jobs Similar to Data Software Engineer
data engineer
senior data engineer
big data software engineer
software engineer
database software engineer
software engineer 3
software engineer 2
big data developer
senior data developer
software engineer 1
Other Helpful Pages Related To Data Engineer
Senior Data Engineer Salaries
Frequently asked questions
Q: What skills or qualities help someone succeed as a Data Software Engineer?
A: To succeed as a Data Software Engineer, key technical skills include proficiency in programming languages such as Python, Java, or C++, as well as expertise in data structures, algorithms, and software development methodologies like Agile. Additionally, strong soft skills like effective communication, problem-solving, and collaboration are crucial, as Data Software Engineers often work with cross-functional teams and stakeholders to design, develop, and deploy data-driven solutions. By combining technical expertise with strong soft skills, Data Software Engineers can effectively drive business outcomes, innovate, and adapt to the rapidly evolving landscape of data technology.
Q: What is the career path for a Data Software Engineer?
A: A Data Software Engineer's typical career progression involves starting as a Junior Software Engineer, where they focus on developing and maintaining data-driven software applications, and gradually advancing to roles such as Senior Software Engineer, Technical Lead, or Data Architect, where they oversee large-scale data systems and lead cross-functional teams. Key opportunities for skill development include learning programming languages like Python, SQL, and Java, as well as data science tools like Hadoop, Spark, and machine learning frameworks like TensorFlow and PyTorch. Long-term, Data Software Engineers may pursue leadership roles, such as Director of Engineering or Chief Technology Officer, or transition into related fields like data science, product management, or entrepreneurship.