Azure Big Data Engineer Jobs in Indiana (NOW HIRING)

The Data Engineer supports the execution of Allied's Data Strategy by focusing on data acquisition, curation, and transformation. This role involves working with cross-functional teams to implement efficient data models and contribute to the development of modern data architecture aligned with business and data initiatives. The Data Engineer will assist in building end-to-end data pipelines to enable business intelligence, analytics, and data-driven decision making. This position requires a solid understanding of SQL, data design principles, and data processing methods. The Data Engineer will also help implement and maintain data models, schemas, and participate in discussions around the evolution of Allied's Data Management and Strategy.Job Duties and Responsibilities:

Data Integration Development and Administration (75%)

• Identifies, troubleshoots, and resolves data and data processing errors in existing solutions.
• Works closely with the Data Intelligence team to understand data and reporting requirements and assists in implementing backend data solutions to meet business needs.
• Develops, maintains, and updates data sets while helping standardize data collection and processing practices.
• Assists in executing technical plans for extracting, integrating, and moving data to and from third-party providers.
• Supports the delivery of enterprise data solutions, including ETL processes, warehouse updates, and performance tuning.
• Mentors other ETL Developers and Junior Data Engineers.

Discovery, Design and Documentation (15%)

• Creates and maintains documentation of data warehouse structures, data stores and end-to-end data flows.
• Collaborates with technical teams and internal and external users to support the development of technical designs.
• Supports the refinement and maintenance of BI-focused data architecture under guidance from senior data engineers.
• Assists in conducting gap analysis on source system databases compared to the enterprise data warehouse to support Business Intelligence initiatives.
• Participates in technical discussions and provides input on technical decisions that impact cross-functional teams.
• Attends architecture and planning meetings to contribute to defining project scope, timelines, and resource estimates.
• Gains a working knowledge of data warehouse and data mart models and associated business information.
• Helps identify opportunities for process improvements or the adoption of new technologies that improve data delivery and business value.
• Works collaboratively with developers and subject matter experts to align technical solutions with business needs, focusing on usability and performance.

Enhance Skillsets (10%)

• Possess and demonstrate good communication skills to be able to run discovery sessions.
• Continue to develop data architecture skills by working with senior data engineers and architects.
• Continue to develop data modeling skills by working with senior data engineers.
• Enhance skills and knowledge around data management platforms and tools:
  • AWS (EC2, S3, Glue, Lambda, etc.)
  • Snowflake
  • Python
  • Orchestration (Apache Airflow, Step Functions, etc.)
• Enhance skills and knowledge around database platforms:
  • Snowflake
  • SQL Server
  • Postgres
• Enhance skills and knowledge around Data Governance
  • Data Analysis and Profiling
  • Data Quality
  • Data Governance
  • Data Modeling
• Enhance skills and knowledge around file formats and communication types:
  • APIs
  • JSON
  • XML
  • CSV (character separated values)
  • Unstructured

Qualifications (Education, Experience, Certifications & KSA):

• Bachelor's degree or equivalent combination of education and experience required.
• 3-5 years of experience in data engineering, ETL development, or a related role.
• Extensive hands-on experience with Informatica and AWS Glue for data integration and transformation.
• Strong proficiency in Python for data engineering and automation.
• Deep expertise in AWS Lambda and other cloud-based data services.
• Strong understanding of data warehousing concepts, data modeling, and SQL.
• Experience with big data technologies and distributed computing frameworks.
• Excellent problem-solving skills and ability to work in a fast-paced environment.
• Experience with cloud data platforms such as AWS Redshift, Snowflake, or BigQuery.
• Advanced knowledge of data governance, security, and compliance best practices.
• Proficiency in workflow orchestration tools (e.g., Apache Airflow, Step Functions).
• Familiarity with DevOps practices and CI/CD pipelines for data engineering.
• Experience with real-time data processing frameworks such as Kafka or Kinesis.

The above statements are intended to describe the general nature and level of work being performed by people assigned to this job. They are not intended to be an exhaustive list of all responsibilities, skills, efforts or working conditions associated with a job.

We offer our employees a robust compensation package! Our comprehensive benefits include: medical, dental and vision insurance coverage; 100% company-paid life and disability coverage, 401k options with company match, three weeks PTO by the end of the first year and much more. Allied proudly promotes from within as part of a strong commitment to providing career growth opportunities for employees of all levels. Our diverse business portfolio allows employees broad career options with the advantage of staying with the same organization.
All qualified candidates will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability status, protected veteran status, or any other characteristic protected by law.

To view our privacy statement click here

To view our terms and conditions click here