Job Summary:
Stanford University is a prestigious institution seeking an Associate Data Engineer. The role involves executing programming tasks, developing solutions for complex systems, and leading projects while mentoring junior developers.
Responsibilities:
• Conceptualize design, implement and develop solutions for complex system/programs independently.
• Design and develop applications that may involve sophisticated data manipulation. Implementation and maintenance of complex Data Store/ Data Lake/Lake house and Data warehousing systems and data-intensive solutions that are scalable, optimized, and fault-tolerant.
• Document system builds and application configurations; maintain and update documentation as needed.
• Provide technical analysis, design, development, conversion and implementation work.
• Work as a project leader, as needed, for projects of moderate complexity.
• Serve as a technical resource for all Data Engineering applications.
• Compare, evaluate, and implement new features and technologies, and integrate them into the computing environment.
• Follow team software development methodology.
• Mentor lower-level software developers.
• Design and implement Data Migration and Data Integration across cloud and hybrid environments.
• Mastery and hands-on experience with Data Engineering technologies and scripting languages
• Ability to translate complex functional and technical requirements into detailed architecture, design and high-performing software.
• Design, build and optimize pipelines for data collection for storage, access and analytics.
Qualifications:
Required:
• Bachelor's degree and five years of relevant experience, or a combination of education, training and relevant experience.
• Expertise in designing, developing, testing, and deploying applications.
• Ability to define and solve logical problems for highly technical applications.
• Hands-on experience and training in Advanced SQL, Advanced Python programming, AWS Services (such as S3, Redshift, Glue ETL, IAM, DMS, Appflow, VPC and others), SnowFlake, FiveTran, KAFKA, Airflow, Oracle Cloud and other open source tools.
• Experience in writing reusable complex Python/PySpark scripts for ELT, Business Logic OR APIs.
• Hands-on development work on all aspects of data analysis, data provisioning, modeling, performance tuning and optimization.
• Experience in working on AWS cloud environment, using the marketplace for the right tool, efficient utilization of them to meet business requirements.
• Build scalable data pipelines for both real time and batch using best practices in data modeling, ETL/ELT processing using various technology stack.
• Experienced in Data Modeling, Data Migration and Data Integration.
• Know the pain points in Data integration across SaaS applications and implement the best solution that fits the organization.
• Strong communication skills with both technical and non-technical clients.
Preferred:
• Other Coding experience such as Scala and R Programming is a plus.
Company:
Stanford is a place of discovery, creativity and innovation located in the San Francisco Bay Area on the ancestral land of the Muwekma Ohlone Tribe. Founded in 2008, the company is headquartered in Stanford, USA, with a team of 10001+ employees. The company is currently Late Stage.