About VBeyond
Sourced by ZipRecruiter
Industry
Recruiting and staffing services
Company size
201 - 500 Employees
Headquarters location
Amwell, NJ, US
Year founded
2006
$113.70K - $136.50K/yr
Other
This job posting has expired and is no longer accepting applications. Check out similar jobs
Job description
Role Overview
The Lead Data Engineer will serve as a technical leader responsible for defining data architecture, leading the development of scalable and secure data platforms, and driving enterprise-wide data engineering best practices. This role combines hands-on engineering with technical leadership, focusing on building metadata-driven, automated ELT pipelines using Snowflake, Qlik Replicate, dbt Cloud, and AWS services.
The position plays a key role in data governance, quality, performance optimization, and release management while partnering closely with business stakeholders to translate data requirements into robust architecture solutions.
Key Responsibilities
The Lead Data Engineer will serve as a technical leader responsible for defining data architecture, leading the development of scalable and secure data platforms, and driving enterprise-wide data engineering best practices. This role combines hands-on engineering with technical leadership, focusing on building metadata-driven, automated ELT pipelines using Snowflake, Qlik Replicate, dbt Cloud, and AWS services.
The position plays a key role in data governance, quality, performance optimization, and release management while partnering closely with business stakeholders to translate data requirements into robust architecture solutions.
Key Responsibilities
- Design and lead scalable data architecture aligned with business strategy
- Build and optimize real-time and batch data pipelines for large-scale data environments
- Lead the development of ELT pipelines using Qlik Replicate, Snowflake, dbt Cloud, and CI/CD tools
- Drive data governance, security, compliance, and data quality initiatives
- Analyze and resolve complex performance and data reliability issues
- Review, approve, and manage code deployments across environments including production and DR
- Enforce testing, automation, monitoring, and observability best practices
- Mentor junior engineers and provide technical guidance to teams
- Collaborate with business and leadership teams to deliver data-driven solutions
- Deep expertise in Snowflake, including medallion architecture, Snowpipe, RBAC, and data masking
- Strong hands-on experience with Qlik Replicate, dbt Cloud, and GitLab CI/CD
- Advanced proficiency in Python, PySpark, and SQL
- Experience building pipelines across SQL Server, Oracle, Mainframe DB2, files, and Snowflake
- Strong AWS experience: S3, Lambda, Glue, SQS, SNS, RDS
- Experience with structured and semi-structured data formats (CSV, JSON, XML, Excel, VSAM)
- Orchestration using dbt Cloud and Astronomer Airflow
- Experience implementing logging, monitoring, alerting, observability (e.g., Dynatrace)
- Strong experience with schema drift detection and schema evolution
- Proven expertise in data security, governance, tokenization, and Snowflake masking policies
- Financial/banking domain experience is a plus
- One or more relevant technical certifications required
vbeyond.com
Most Popular Jobs Similar to Data Software Engineer
data engineer
senior data engineer
big data software engineer
software engineer
database software engineer
software engineer 3
software engineer 2
big data developer
senior data developer
software engineer 1
Other Helpful Pages Related To Lead Data Engineer
Senior Data Engineer Salaries
Frequently asked questions
Q: What skills or qualities help someone succeed as a Data Software Engineer?
A: To succeed as a Data Software Engineer, key technical skills include proficiency in programming languages such as Python, Java, or C++, as well as expertise in data structures, algorithms, and software development methodologies like Agile. Additionally, strong soft skills like effective communication, problem-solving, and collaboration are crucial, as Data Software Engineers often work with cross-functional teams and stakeholders to design, develop, and deploy data-driven solutions. By combining technical expertise with strong soft skills, Data Software Engineers can effectively drive business outcomes, innovate, and adapt to the rapidly evolving landscape of data technology.
Q: What is the career path for a Data Software Engineer?
A: A Data Software Engineer's typical career progression involves starting as a Junior Software Engineer, where they focus on developing and maintaining data-driven software applications, and gradually advancing to roles such as Senior Software Engineer, Technical Lead, or Data Architect, where they oversee large-scale data systems and lead cross-functional teams. Key opportunities for skill development include learning programming languages like Python, SQL, and Java, as well as data science tools like Hadoop, Spark, and machine learning frameworks like TensorFlow and PyTorch. Long-term, Data Software Engineers may pursue leadership roles, such as Director of Engineering or Chief Technology Officer, or transition into related fields like data science, product management, or entrepreneurship.