Job Summary:
Bespoke Technologies, Inc. is seeking a Data & Software Engineer to join their team in McLean, VA. The role involves building complex data flows for custom applications, requiring strong Python programming skills and experience in data pipeline development.
Responsibilities:
โข Work with stakeholders to understand data requirements, assess feasibility, and design appropriate solutions with minimal oversight
โข Leverage strong problem-solving and debugging skills for data quality issues, pipeline failures, and performance bottlenecks
โข Leverage a background in large-scale data migration or platform modernization efforts
โข Contribute to data engineering documentation, best practices, and design patterns.
Qualifications:
Required:
โข Active Poly clearance
โข Advanced Python programming skills
โข Familiarity with Java
โข Understanding of data security, privacy, governance and compliance principles
โข Demonstrated history of building production data pipelines and ETL workflows at scale
โข Building end-to-end data pipelines leveraging Python
โข Using orchestration tools to deploy data pipelines, including configuring and updating Spark Jobs
โข Containerizing and deploying applications in cloud environments like AWS
โข Working with MySQL and PostgreSQL including performance tuning, schema design, and query optimization for complex, analytical workloads
โข Leveraging industry standard tools for code control (Git, IaaC control, etc.)
โข Working with data catalogs, tracking data lineage and handling a variety of data formats, including Geospatial
โข Using Bash scripting for automation and data processing tasks
โข Integrating AI/ML services and models
โข Minimum of 5 years' experience with Apache Spark & PySpark
โข Minimum of 5 years' experience with Advanced Python skills (including Pandas & NumPy)
โข Minimum of 5 years' experience with Docker, Podman
โข Minimum of 5 years' experience with AWS S3, Lambda & Step functions
โข Minimum of 5 years' experience with Apache Iceberg, Airflow, etc.
โข Minimum of 5 years' experience with SQL (with Trino)
โข Minimum of 5 years' experience with NoSQL, DynamoDB
โข Minimum of 5 years' experience with Unity Catalog OSS, Apache Polaris
โข Minimum of 5 years' experience with Apache Superset
โข Minimum of 5 years' experience with Terraform or CloudFormation
โข Minimum of 5 years' experience with OpenLineage
โข Minimum of 5 years' experience with H3, PostGIS
Company:
Bespoke Technologies, Inc. Founded in , the company is headquartered in Aldie, USA, with a team of 51-200 employees. The company is currently Growth Stage.