Job Summary:
OTR Solutions is an innovator in the transportation industry providing a suite of factoring, fuel, and business management solutions. As a Data Engineer – External Data Integrations, you will design, build, and maintain scalable batch-oriented data pipelines to enable secure and reliable data exchange between OTR and external systems.
Responsibilities:
• Design and build scalable batch ingestion pipelines from external systems, including APIs, SFTP sources, and databases
• Develop outbound data pipelines to securely publish structured data to external partners and third-party platforms
• Build, maintain, and optimize Azure Synapse pipelines and notebook-based transformations using PySpark and Spark SQL
• Orchestrate workflows using Azure Logic Apps, including scheduling, retries, monitoring, and alerting
• Design and implement data quality validation, observability, and monitoring frameworks to ensure pipeline reliability and SLA adherence
• Define and maintain schema standards, versioning strategies, and partner-facing data contracts
• Build secure integrations with third-party systems using modern authentication methods including OAuth and Azure Key Vault
• Partner with Engineering, Product, and Business Intelligence teams to define integration requirements and scalable data exchange patterns
• Troubleshoot complex integration issues and continuously improve performance, reliability, and maintainability of data pipelines
• Identify opportunities to automate and streamline data workflows across systems
Qualifications:
Required:
• 4–7+ years of experience in Data Engineering, Integration Engineering, or similar technical roles
• Strong experience designing and maintaining batch-oriented data pipelines at scale
• Hands-on experience with Azure Synapse Analytics, Spark/Python, and Azure Logic Apps
• Experience integrating with REST APIs, SFTP workflows, and external data providers
• Strong SQL skills with experience transforming and modeling large datasets
• Understanding of data observability, monitoring, and reliability best practices
• Experience designing secure integrations and implementing authentication mechanisms for external systems
• Strong problem-solving skills with experience troubleshooting complex distributed data workflows
• Excellent communication skills with ability to work across technical and non-technical stakeholders
• Experience working in Agile environments
Preferred:
• Experience with Azure Data Lake and Parquet or Delta file formats
• Familiarity with Azure API Management (APIM)
• Experience implementing CI/CD pipelines using Azure DevOps
• Knowledge of data governance, lineage, and metadata management concepts
• Experience in FinTech, transportation, logistics, or other high-volume transactional environments
Company:
OTR Solutions is a solution-driven firm that offers elevation, capital fuel card, transportation, and brokerage factoring. Founded in 2012, the company is headquartered in Roswell, USA, with a team of 201-500 employees. The company is currently Growth Stage.