Software Engineer IV - Big Data Engineer
Location: Durham, NC (Day 1 onsite)
Duration: Long Term
Responsibilities:
- Ability to assemble large, complex sets of data that meet non-functional and functional business requirements
- Ability to identify, design and implement internal process improvements including re-designing infrastructure for greater scalability, optimizing data delivery, and automating manual processes
- Build required infrastructure for optimal extraction, transformation and loading of data from various data sources using Azure and SQL technologies
- Building analytical tools to utilize the data pipeline, providing actionable insight into key business performance metrics including operational efficiency and customer acquisition
- Working with stakeholders including data, design, product and executive teams and assisting them with data-related technical issues
- Strong programming and analytical skills, problem determination and debugging, attention to detail, teamwork, and excellent communication skills.
- Knowledge of software development concepts/methodology/life cycle, especially Agile methodology
- Proficiency with code versioning tools like Git
- Proficiency with Windows- and Linux-based operating systems, including software installation, light administration and simple scripting.
Required Skills:
- Bachelorโs Degree in Computer Science or related field
- Strong analytical and Java programming skills
- Deep technical skills in software development technologies and tools such as socket communication, Python, DI, CI, Spring Boot, XML/JSON, Java, JavaScript, Docker, Dagger2, maven
- Experience in Cloud and Big Data technologies such as PowerShell, SQL, ADLS/Blob, Spark/SparkSQL, Databricks.
- 8+ years professional experience in software development
- 3+ years of cloud based or client/server development experience.
- 1+ years of Big Data development experience.
Preferred Skills:
- Experience implementing data migration and data processing using Azure services: ADLS, Azure Data Factory, Azure Functions, Synapse/DW, Azure SQL DB, Event Hub, IOT Hub, Azure Stream Analytics, Azure Analysis Service, HDInsight, Databricks Azure Data Catalog, Cosmo Db, ML Studio, AI/ML, etc.
- Experience in developing data ingestion, data processing and analytical pipelines for big data, relational databases, NoSQL and data warehouse solutions
- Experience in Big data partitioning strategies, Azure ADF, Azure Databricks.
- Knowledge of cloud computing technologies
- Knowledge of non-relational databases
- Experience working on Azure SQL Server.