Pre ID for Sundaram Manickam
XXXX: XXXX
Job SummaryWe are seeking an experienced
Senior Data Engineer to design, build, optimize, and maintain scalable cloud-based data platforms and pipelines. The ideal candidate will have strong expertise in
Snowflake,
AWS Glue, cloud-native ETL/ELT architectures, and enterprise data warehousing solutions. This role will collaborate closely with business stakeholders, analytics teams, data architects, and application teams to deliver reliable, high-performance data solutions supporting analytics, reporting, AI/ML, and operational systems.
Key Responsibilities Data Engineering & Pipeline Development - Design and develop scalable ETL/ELT pipelines using AWS services and modern data engineering practices.
- Build and optimize data ingestion frameworks using AWS Glue, Spark, Python, SQL, and cloud-native services.
- Develop robust batch and near real-time data pipelines from multiple structured and unstructured sources.
- Implement metadata-driven and reusable pipeline frameworks.
Snowflake Data Warehouse Development - Design and maintain enterprise-grade Snowflake data warehouse solutions.
- Create optimized schemas, tables, views, tasks, streams, and stored procedures in Snowflake.
- Implement Snowflake best practices for:
- Performance tuning
- Clustering
- Data sharing
- Security and governance
- Cost optimization
- Support large-scale data transformations using Snowflake ELT patterns.
Cloud & AWS Expertise - Work extensively with AWS services including:
- AWS Glue
- S3
- Lambda
- IAM
- CloudWatch
- Step Functions
- Redshift (optional)
- SNS/SQS
- Automate deployment and infrastructure using Infrastructure-as-Code tools such as Terraform or CloudFormation.
Data Modeling & Architecture - Design dimensional and normalized data models for enterprise analytics.
- Support master data management and data governance initiatives.
- Ensure high data quality, lineage, consistency, and reliability across platforms.
Performance & Optimization - Tune SQL queries, Spark jobs, and Snowflake workloads for performance and scalability.
- Monitor and troubleshoot production data pipelines and resolve data issues proactively.
- Optimize AWS Glue jobs and Snowflake warehouse utilization to reduce operational costs.
Collaboration & Leadership - Work closely with Data Architects, BI teams, Data Scientists, and business stakeholders.
- Participate in architecture discussions, sprint planning, code reviews, and release management.
- Mentor junior data engineers and promote engineering best practices.
- Drive adoption of DevOps and CI/CD practices for data engineering solutions.
Required Qualifications - Bachelor's degree in Computer Science, Engineering, Information Systems, or related field.
- 7+ years of experience in Data Engineering or Data Warehouse development.
- 3+ years of hands-on experience with Snowflake.
- Strong expertise with Amazon Web Services services, especially AWS Glue and S3.
- Advanced SQL development and query optimization skills.
- Strong programming skills in Python and/or PySpark.
- Experience building scalable cloud data pipelines and enterprise data warehouses.
- Experience with Git-based version control and CI/CD pipelines.
- Strong understanding of data modeling concepts and ETL/ELT architecture.
Preferred Qualifications - Experience with:
- DBT
- Airflow
- Kafka
- Terraform
- Databricks
- Redshift
- Synapse
- Experience in Agile/Scrum delivery environments.
- Knowledge of data governance, security, and compliance frameworks.
- Snowflake or AWS certifications preferred.
Technical Skills Category Skills Cloud AWS, S3, Glue, Lambda, IAM, CloudWatch Data Warehouse Snowflake, Redshift, Synapse Programming Python, PySpark, SQL ETL/ELT AWS Glue, DBT, Airflow Databases SQL Server, Oracle, PostgreSQL DevOps GitHub, Jenkins, CI/CD Big Data Spark, Hadoop (preferred) Reporting/Analytics Power BI, Tableau (preferred)
Soft Skills - Strong analytical and problem-solving skills
- Excellent communication and stakeholder management
- Ability to lead technical discussions and mentor teams
- Strong project ownership and delivery mindset
- Ability to work in fast-paced enterprise environments
Nice-to-Have Domain Experience - Customer data platforms
- Marketing analytics
- Digital transformation initiatives
- Enterprise master data management
- Large-scale cloud migration projects