Requirement - Sr. Data Engineer
Location-
• 1041 West Valley Road Wayne, Pennsylvania 19087
• 7900 Windrose Ave Plano, Texas 75024
• 2525 Water Ridge Pkwy, Charlotte, NC 28217
Contract W2
Top technologies:
We use AWS Services like Glue ETL, Lambda, Redshift, S3 etc., for our data pipelines. Skills required would be Python, SQL, Spark, AWS Infrastructure and CI/CD Pipelines (GitHub).
- ETL
- Python
- SQL
- Data pipeline design
- Data quality management
- Database architecture
- Data warehousing
- Historical data retention strategies
- Cloud-based data movement
- Ideally someone with 8+ years of experience in a large enterprise environment since they are the go to person for data engineering
- They will be laying the foundation and drawing out the road map as well as doing the work
- Coming from a large financial enterprise would be ideal but not required
Job Description:
The Data Engineer, Specialist is responsible for building and maintaining scalable data pipelines to ensure seamless integration and efficient data flow across various platforms. This role involves designing, developing, and optimizing ETL (Extract, Transform, Load) processes, managing databases and leveraging data technologies to support analytics and business intelligence initiatives.
Responsibilities:
- Writes ETL (Extract / Transform / Load) processes, designs database systems, and develops tools for real-time and offline analytic processing.
- Troubleshoots software and processes for data consistency and integrity. Integrates large scale data from a variety of sources for business partners to generate insight and make decisions.
- Translates business specifications into design specifications and code. Responsible for writing complex programs, ad hoc queries, and reports. Ensures that all code is well structured, includes sufficient documentation, and is easy to maintain and reuse.
- Partners with internal clients to gain an enhanced understanding of business functions and informational needs. Gains expertise in tools, technologies, and applications/databases in specific business areas and company-wide systems.
- Leads all phases of solution development. Explains technical considerations at related meetings, including those with internal clients and less experienced team members.
- Tests code thoroughly for accuracy of intended purpose. Reviews end product with the client to ensure adequate understanding. Provides data analysis guidance as required.
- Designs and conducts training sessions on tools and data sources used by the team and self provisioners. Provides job aids to team members and business users.
- Tests and implements new software releases through regression testing. Identifies issues and engages with vendors to resolve and elevate software into production.
- Participates in special projects and performs other duties as assigned.
Qualifications:
- Experience with Data Integration
- Machine Learning
- Data Governance
- Minimum of five years data analytics, programming, database administration, or data management experience.
- Undergraduate degree or equivalent combination of training and experience.