Data Engineer
Portland, Los Angeles, Las Vegas, Denver, Vancouver, BC
What You Will Do:
- Build new systems to provide real-time streaming analytics and event processing pipeline based on fast data architecture
- Build enterprise grade data lake to support both business analytical needs and next generation data infrastructure
- Building data integration toolkit for backend services
- Support our data science team in deploying new algorithms for matchmaking, fraud and cheat detection
- Find better ways to move massive amounts of data from a variety of sources to formats consumable by reporting systems and people
- Improve monitoring and alarms that impact data integrity replication lag
- Support our product development team in creating new events to measure/track
What You Need:
Basic Qualifications:
- At least 4-5 years of experience in Scala/Java or Python programming
- AWS data products (Data pipelines, Athena, Pinpoint, S3, etc)
- Experience deploying data infrastructure
- Experience with recognized industry patterns, methodologies, and techniques
Nice to Haves:
- Familiarity with Agile engineering practices
- 2+ years of experience on Kubernetes, Helm chart
- 4+ years of experience with Spark, Scala and/or Akka
- 4+ years of experience with Spark Streaming, Storm, Flink, or other Stream Processing technologies
- 2+ years of experience working with Kafka or similar data pipeline backbone
- 4+ years of experience with Unix/Linux systems with scripting experience in Shell, Perl or Python
- 3+ years' experience with NoSQL implementation (Elasticsearch, Cassandra, etc. a plus)
- At least 4-5 years of experience with Unix/Linux systems with scripting experience
- Familiarity with Snowflake or OLAP
- Familiarity with Kinesis, Lambda
- Prior experience in gaming/Esports
- Prior experience in finance
