Job Summary:
VT Group (VTG) is seeking a Data & Software Engineer to work with a small team to build complex data flows for a custom application. The successful candidate will be responsible for building end-to-end data pipelines, deploying applications in cloud environments, and integrating AI/ML services and models while ensuring data quality and performance optimization.
Responsibilities:
• Building end-to-end data pipelines leveraging Python
• Using orchestration tools to deploy data pipelines, including configuring and updating Spark Jobs
• Containerizing and deploying applications in cloud environments like AWS.
• Working with MySQL and PostgreSQL including performance tuning, schema design, and query optimization for complex, analytical workloads.
• Leveraging industry standard tools for code control (Git, IaaC control, etc.)
• Working with data catalogs, tracking data lineage and handling a variety of data formats, including Geospatial.
• Using Bash scripting for automation and data processing tasks
• Integrating Al/ML services and models
• Work with stakeholders to understand data requirements, assess feasibility, and design appropriate solutions with minimal oversight
• Leverage strong problem-solving and debugging skills for data quality issues, pipeline failures, and performance bottlenecks
• Leverage a background in large-scale data migration or platform modernization efforts
• Contribute to data engineering documentation, best practices, and design patterns.
Qualifications:
Required:
• Active TS/SCI W/ Polygraph required.
• Bachelor's degree in Computer Science, Engineering, Finance, or a related technical field, or equivalent practical experience.
• Minimum of 5 years' experience with Apache Spark & PySpark.
• Minimum of 5 years' experience with advanced Python skills (including Pandas & NumPy).
• Minimum of 5 years' experience with Docker, Podman.
• Minimum of 5 years' experience with AWS S3, Lambda & Step functions.
• Minimum of 5 years' experience with Apache Iceberg, Airflow, etc.
• Minimum of 5 years' experience with SQL (with Trino).
• Minimum of 5 years' experience with NoSQL, DynamoDB.
• Minimum of 5 years' experience with Unity Catalog OSS, Apache Polaris.
• Minimum of 5 years' experience with Apache Superset.
• Minimum of 5 years' experience with Terraform or CloudFormation.
• Minimum of 5 years' experience with OpenLineage.
• Minimum of 5 years' experience with H3, PostGIS.
Company:
VTG delivers force modernization and digital transformation solutions that expand America’s competitive advantage in the modern battlespace. Founded in 1866, the company is headquartered in Arlington, USA, with a team of 1001-5000 employees. The company is currently Late Stage.