$150 - $200/hr
We are building an innovative data processing engine that is transforming how Apache Spark, Apache Flink, Ray and others operate on diverse, large-scale data. Our team of engineers drive and adopt ...
$150 - $200/hr
We are building an innovative data processing engine that is transforming how Apache Spark, Apache Flink, Ray and others operate on diverse, large-scale data. Our team of engineers drive and adopt ...
$200 - $250/hr
## Principal Software Engineer, Real Time Data Enrichment Platform (Hybrid)Applylocations: USA - New ... Production experience building either Spark- or Flink-based self-service data platforms, or ...
$200 - $250/hr
## Principal Software Engineer, Real Time Data Enrichment Platform (Hybrid)Applylocations: USA - New ... Production experience building either Spark- or Flink-based self-service data platforms, or ...
$150 - $200/hr
Experience with modern data stack tools such as Spark/Flink, Kafka/Kinesis, Airflow/managed schedulers, and data warehouses (e.g., Snowflake, Redshift, BigQuery, Databricks) * Familiarity with DevOps ...
$150 - $200/hr
Experience with modern data stack tools such as Spark/Flink, Kafka/Kinesis, Airflow/managed schedulers, and data warehouses (e.g., Snowflake, Redshift, BigQuery, Databricks) * Familiarity with DevOps ...
$250/hr
... data lake or lakehouse technologies such as Iceberg, Delta Lake, Hudi, Spark, Trino, Presto, Flink ... Strong programming skills in Java, Scala, Go, Rust, C++, or similar systems‑oriented languages
$250/hr
... data lake or lakehouse technologies such as Iceberg, Delta Lake, Hudi, Spark, Trino, Presto, Flink ... Strong programming skills in Java, Scala, Go, Rust, C++, or similar systems‑oriented languages
$130 - $170/hr
Support data engineering and analytics teams * Support data governance and data quality * AWS ... Apache Flink * Apache Kafka * AI Data * AI data readiness * Access Control * Architecture ...
$130 - $170/hr
Support data engineering and analytics teams * Support data governance and data quality * AWS ... Apache Flink * Apache Kafka * AI Data * AI data readiness * Access Control * Architecture ...
$200 - $250/hr
Our platform helps data and engineering teams reduce storage and compute costs, improve performance ... Improve performance and cost efficiency across Spark, Trino, Presto, Flink, Databricks, and ...
$200 - $250/hr
Our platform helps data and engineering teams reduce storage and compute costs, improve performance ... Improve performance and cost efficiency across Spark, Trino, Presto, Flink, Databricks, and ...
$125 - $150/hr
Production experience with data engineering tools such as Apache Spark, Kafka, Flink, Airflow, and Pentaho PDI.* Ability to design data pipelines that integrate streaming, batch, transformation ...
$125 - $150/hr
Production experience with data engineering tools such as Apache Spark, Kafka, Flink, Airflow, and Pentaho PDI.* Ability to design data pipelines that integrate streaming, batch, transformation ...
$125 - $150/hr
... Engineer or Data Scientist building and productional machine learning solutions * Experience building real-time event-driven stream processing solutions with technologies such as Kafka, Flink, and ...
$125 - $150/hr
... Engineer or Data Scientist building and productional machine learning solutions * Experience building real-time event-driven stream processing solutions with technologies such as Kafka, Flink, and ...
$150 - $200/hr
... like Spark, Flink, Airflow, Ray, Notebooks, and LLM‑based agent platforms reliable at scale ... As an SRE on Apple Data Platform, you'll operate and support the team's full portfolio, from big ...
$150 - $200/hr
... like Spark, Flink, Airflow, Ray, Notebooks, and LLM‑based agent platforms reliable at scale ... As an SRE on Apple Data Platform, you'll operate and support the team's full portfolio, from big ...
$150 - $200/hr
... like Spark, Flink, Airflow, Ray, Notebooks and LLM-based agent platforms reliable at scale ... As an SRE on Apple Data Platform, you'll operate and support the team's full portfolio, from big ...
$150 - $200/hr
... like Spark, Flink, Airflow, Ray, Notebooks and LLM-based agent platforms reliable at scale ... As an SRE on Apple Data Platform, you'll operate and support the team's full portfolio, from big ...
$150 - $200/hr
Data Engineering: Spark, Kafka, Flink, BigQuery, streaming pipelines, ETL at scale. * Distributed Systems: High-volume scaling, fault tolerance, eventual consistency. * AI/ML: LLMs in production, RAG ...
$150 - $200/hr
Data Engineering: Spark, Kafka, Flink, BigQuery, streaming pipelines, ETL at scale. * Distributed Systems: High-volume scaling, fault tolerance, eventual consistency. * AI/ML: LLMs in production, RAG ...
$150 - $200/hr
This role is a senior engineer responsible for the architecture, engineering direction, scalability ... Strong experience with Kafka, Spark, Flink, Dataflow, Beam, or related real-time processing ...
$150 - $200/hr
This role is a senior engineer responsible for the architecture, engineering direction, scalability ... Strong experience with Kafka, Spark, Flink, Dataflow, Beam, or related real-time processing ...
$150 - $200/hr
... Flink, or similar * Experience with relational and non‑relational databases, data lakes e.g. Hive ... Software Engineer/Level I: $125,000.00 - $165,000.00/per year Software Engineer/Level II: $145,000 ...
$150 - $200/hr
... Flink, or similar * Experience with relational and non‑relational databases, data lakes e.g. Hive ... Software Engineer/Level I: $125,000.00 - $165,000.00/per year Software Engineer/Level II: $145,000 ...
$150 - $200/hr
Description As a Data Site Reliability Engineer in Apple Ads, you will focus on designing, building ... Flink, Kafka, Iceberg or other similar technologies * Strong expertise and experience with ...
New
$150 - $200/hr
Description As a Data Site Reliability Engineer in Apple Ads, you will focus on designing, building ... Flink, Kafka, Iceberg or other similar technologies * Strong expertise and experience with ...
New
$200 - $250/hr
We embed engineers directly with strategic customers and build the agents and integrations that ... Background in streaming or real-time data systems (Kafka, Flink, Redpanda, or similar)
$200 - $250/hr
We embed engineers directly with strategic customers and build the agents and integrations that ... Background in streaming or real-time data systems (Kafka, Flink, Redpanda, or similar)
$150 - $200/hr
As a Sr. Solutions Engineer, you will independently lead technical engagements for customers ... data systems: Apache Spark™, Delta Lake, or equivalent (Hadoop, Kafka, Flink) * Experience ...
$150 - $200/hr
As a Sr. Solutions Engineer, you will independently lead technical engagements for customers ... data systems: Apache Spark™, Delta Lake, or equivalent (Hadoop, Kafka, Flink) * Experience ...
$200 - $250/hr
To learn more about Discord's Data Platform, read our engineering blog, including how we built our ... Flink, Airflow, Dagster, or similar) * Proven track record of delivering complex, multi-phase ...
$200 - $250/hr
To learn more about Discord's Data Platform, read our engineering blog, including how we built our ... Flink, Airflow, Dagster, or similar) * Proven track record of delivering complex, multi-phase ...
$150 - $200/hr
Site Reliability Engineer, Apple Data Platform Austin, Texas, United States Software and Services ... Flink, Hive, Hadoop/HDFS, Trino, and/or Druid. * Proficiency in managing applications and infra on ...
$150 - $200/hr
Site Reliability Engineer, Apple Data Platform Austin, Texas, United States Software and Services ... Flink, Hive, Hadoop/HDFS, Trino, and/or Druid. * Proficiency in managing applications and infra on ...
$200 - $250/hr
Senior Software Engineer, Apple Services Engineering San Francisco, California, United States ... Integrate with large‑scale data processing engines (e.g., Spark, Trino, Flink) as part of a ...
$200 - $250/hr
Senior Software Engineer, Apple Services Engineering San Francisco, California, United States ... Integrate with large‑scale data processing engines (e.g., Spark, Trino, Flink) as part of a ...
$200 - $250/hr
Require the agility to troubleshoot open-source data processing engine, such as Apache Spark, Apache Storm and Apache Flink * Utilize analytical and programming skills andopen-source systems, such as ...
$200 - $250/hr
Require the agility to troubleshoot open-source data processing engine, such as Apache Spark, Apache Storm and Apache Flink * Utilize analytical and programming skills andopen-source systems, such as ...
Data Engineer Flink information
What is a data engineer Flink?
A Data Engineer Flink is a data engineering professional who specializes in using Apache Flink, an open-source stream processing framework, to build, maintain, and optimize systems that process large-scale data in real time. They design and implement data pipelines, ensure data quality and consistency, and collaborate with other engineering teams to deliver reliable and scalable data solutions. Their expertise allows organizations to process, analyze, and react to data as it is generated, enabling real-time insights and decision-making.
What are the key skills and qualifications needed to thrive as a data engineer Flink?
To thrive as a Data Engineer specializing in Flink, you need strong programming skills (especially in Java or Scala), a solid understanding of distributed data processing, and experience with data architecture. Familiarity with Apache Flink, stream processing frameworks, big data tools (like Kafka, Hadoop, or Spark), and cloud platforms is typically required, along with relevant certifications. Excellent problem-solving abilities, attention to detail, and effective teamwork and communication skills help you excel in complex data environments. These competencies are crucial for building reliable, scalable data pipelines that power real-time analytics and business decision-making.
What are some common challenges data engineers face when working with Apache Flink in a production environment?
Data Engineers working with Apache Flink often encounter challenges such as managing stateful stream processing at scale, ensuring fault tolerance, and optimizing resource usage for real-time data pipelines. Handling late-arriving data and tuning Flink jobs for low latency and high throughput are also frequent hurdles. Collaborating closely with data scientists and application developers is key to aligning data models and ensuring smooth data flow throughout the system.
Are data engineers still in demand?
Data engineers, including those skilled in Apache Flink, are in high demand due to the increasing need for real-time data processing and scalable data infrastructure. Organizations seek professionals with expertise in big data tools, cloud platforms, and programming languages like Java or Scala to build and maintain data pipelines. The role is expected to grow as data-driven decision-making becomes more critical across industries.
What are popular job titles related to Data Engineer Flink jobs in Kentucky?
For Data Engineer Flink jobs in Kentucky, the most frequently searched job titles are:
What job categories do people searching Data Engineer Flink jobs in Kentucky look for?
The top searched job categories for Data Engineer Flink jobs in Kentucky are:
$150 - $200/hr
Other
Posted 6 days ago
Key responsibilities
Lead the architecture, design, and implementation of core components of the data processing engine.
Drive the evolution of the parallel and distributed execution engine architecture, focusing on accelerated computing technologies.
Guide and mentor engineering team members, and collaborate with cross-functional teams to ensure high-quality product development.
Job description
Principal Data Processing Engineer
Mountain View, CAAbout DataPelago:
#J-18808-LjbffrMountain View, CAAbout DataPelago:
DataPelago is at the forefront of revolutionizing data processing for traditional analytics and cutting-edge GenAI preprocessing. We are building an innovative data processing engine that is transforming how Apache Spark, Apache Flink, Ray and others operate on diverse, large-scale data. Our team of engineers drive and adopt advances in hardware-accelerated computing, parallel processing of large-scale data, query optimization, distributed systems, compilers, machine learning, and cloud-native computing. We are looking for world-class engineers to join our team and shape the future of accelerated data processing.
The Role:As a Principal Data Processing Engineer, you will be a key technical leader in the development of the core execution components of our data processing engine. You will lead architecture, design, and implementation that will enhance the functional breadth, performance, scale, and reliability of the engine to deliver a product that will redefine how users extract intelligence from their data. This is a unique opportunity to make a significant impact on a category-defining product and work with a talented team of engineers.
What You’ll Do:- Drive the evolution of our parallel and distributed execution engine architecture, with a strong focus on leveraging accelerated computing technologies.
- End-to-End Ownership: Lead the execution engine team in the complete lifecycle of design,implementation, and rollout of an enterprise-grade product.
- Individually design, implement, test, and maintain critical components of the data processing execution engine.
- Innovation and Differentiation: Analyze technology advances from industry and academia to identify opportunities for the engine to enhance technology and product leadership.
- Collaboration: Partner effectively with engineering, product management,and customer success teams.
- Guide and mentor engineers on the execution engine team.
- Foster best practices in design and code reviews, testing, CI/CD, and issue resolution to maintain the highest product quality, security, efficiency, and productivity.
- BS/MS in Computer Science (or a related field) with 10+ years of relevant experience
- 7+ years of deep technical experience in developing core components of enterprise-grade database or analytics execution engines designed for large-scale data processing.
- Proven expertise in developing high-performance parallel implementations of data processing operators and functions on rich data types.
- Significant experience developing for platforms such as Apache Spark, Apache Flink, Apache Doris, Apache Gluten, Velox, Apache DataFusion/Comet preferred.
- Previous experience working as technical lead/architect with teams of 10+ engineers in the design, development, and successful release of high-performance data processing engines for large production deployments.
- Proficiency in C, C++, and Rust programming.
- Extensive development experience in Linux environments.
- Strong analytical and problem-solving skills with a passion for performance optimization.
- Excellent communication and collaboration skills, with the ability to articulate complex technical concepts to both technical and non-technical audiences.
- Technical Leadership: Take a leadership role in shaping the architecture and development of how our core engine works with open source data processing platforms
- Cutting-Edge Innovation: Work on challenging problems at the forefront of accelerated computing and data processing.
- Significant Impact: Your contributions will directly impact the performance and scalability of our mission-critical platform.
- Mentorship and Growth: Mentor and guide other talented engineers while expanding your own technical expertise.
- Competitive compensation, stock options, comprehensive benefits package, and leadership development opportunities