Salesforce is the leading AI CRM company, seeking a Staff Software Engineer to join the Data ... Kafka/Kafka Connect, Flink/Spark Streaming, Iceberg/Hudi/Delta or equivalent • You have hands on ...
Salesforce is the leading AI CRM company, seeking a Staff Software Engineer to join the Data ... Kafka/Kafka Connect, Flink/Spark Streaming, Iceberg/Hudi/Delta or equivalent • You have hands on ...
S. with 5+ years of data architecture, data engineering, or scientific informatics' experience. • ... Kinesis, Flink) and event-driven architectures. • Familiarity with LIMS, ELN systems (e.g ...
S. with 5+ years of data architecture, data engineering, or scientific informatics' experience. • ... Kinesis, Flink) and event-driven architectures. • Familiarity with LIMS, ELN systems (e.g ...
As part of the Data Engineering organization, we build and operate the platform that ingests data ... Kafka/Kafka Connect, Flink/Spark Streaming, Iceberg/Hudi/Delta or equivalent * You have hands on ...
As part of the Data Engineering organization, we build and operate the platform that ingests data ... Kafka/Kafka Connect, Flink/Spark Streaming, Iceberg/Hudi/Delta or equivalent * You have hands on ...
As part of the Data Engineering organization, we build and operate the platform that ingests data ... Kafka/Kafka Connect, Flink/Spark Streaming, Iceberg/Hudi/Delta or equivalent * You have hands on ...
As part of the Data Engineering organization, we build and operate the platform that ingests data ... Kafka/Kafka Connect, Flink/Spark Streaming, Iceberg/Hudi/Delta or equivalent * You have hands on ...
... engineering practices to create interoperable, machine-readable scientific data. • Design and ... Kinesis, Flink) and event-driven architectures. • Familiarity with LIMS, ELN systems (e.g ...
... engineering practices to create interoperable, machine-readable scientific data. • Design and ... Kinesis, Flink) and event-driven architectures. • Familiarity with LIMS, ELN systems (e.g ...
Staff Software Engineer, Data Platform
Miami, IN · On-site +1
As a member of the Data Platform Engineering team, you own the core Data warehouse platform, data ... Experience with with streaming frameworks such as Apache Flink or Google Cloud Dataflow
Staff Software Engineer, Data Platform
Miami, IN · On-site +1
As a member of the Data Platform Engineering team, you own the core Data warehouse platform, data ... Experience with with streaming frameworks such as Apache Flink or Google Cloud Dataflow
Staff Software Engineer, Data Platform
Austin, IN · On-site +1
As a member of the Data Platform Engineering team, you own the core Data warehouse platform, data ... Experience with with streaming frameworks such as Apache Flink or Google Cloud Dataflow
Staff Software Engineer, Data Platform
Austin, IN · On-site +1
As a member of the Data Platform Engineering team, you own the core Data warehouse platform, data ... Experience with with streaming frameworks such as Apache Flink or Google Cloud Dataflow
Staff Software Engineer, Data Platform
Florence, IN · On-site +1
As a member of the Data Platform Engineering team, you own the core Data warehouse platform, data ... Experience with with streaming frameworks such as Apache Flink or Google Cloud Dataflow
Staff Software Engineer, Data Platform
Florence, IN · On-site +1
As a member of the Data Platform Engineering team, you own the core Data warehouse platform, data ... Experience with with streaming frameworks such as Apache Flink or Google Cloud Dataflow
Staff Software Engineer, Data Platform
Atlanta, IN · On-site +1
As a member of the Data Platform Engineering team, you own the core Data warehouse platform, data ... Experience with with streaming frameworks such as Apache Flink or Google Cloud Dataflow
Staff Software Engineer, Data Platform
Atlanta, IN · On-site +1
As a member of the Data Platform Engineering team, you own the core Data warehouse platform, data ... Experience with with streaming frameworks such as Apache Flink or Google Cloud Dataflow
Staff Software Engineer, Data Platform
Boston, IN · On-site +1
As a member of the Data Platform Engineering team, you own the core Data warehouse platform, data ... Experience with with streaming frameworks such as Apache Flink or Google Cloud Dataflow
Staff Software Engineer, Data Platform
Boston, IN · On-site +1
As a member of the Data Platform Engineering team, you own the core Data warehouse platform, data ... Experience with with streaming frameworks such as Apache Flink or Google Cloud Dataflow
Data Engineer Flink information
What are some common challenges Data Engineers face when working with Apache Flink in a production environment?
Data Engineers working with Apache Flink often encounter challenges such as managing stateful stream processing at scale, ensuring fault tolerance, and optimizing resource usage for real-time data pipelines. Handling late-arriving data and tuning Flink jobs for low latency and high throughput are also frequent hurdles. Collaborating closely with data scientists and application developers is key to aligning data models and ensuring smooth data flow throughout the system.
What is Flink in data engineering?
Apache Flink is an open-source stream processing framework used in data engineering to process large-scale, real-time data streams with low latency. Data engineers working with Flink develop and maintain data pipelines, often integrating it with tools like Kafka and Hadoop to enable real-time analytics and event-driven applications.
What are the key skills and qualifications needed to thrive as a Data Engineer Flink, and why are they important?
To thrive as a Data Engineer specializing in Flink, you need strong programming skills (especially in Java or Scala), a solid understanding of distributed data processing, and experience with data architecture. Familiarity with Apache Flink, stream processing frameworks, big data tools (like Kafka, Hadoop, or Spark), and cloud platforms is typically required, along with relevant certifications. Excellent problem-solving abilities, attention to detail, and effective teamwork and communication skills help you excel in complex data environments. These competencies are crucial for building reliable, scalable data pipelines that power real-time analytics and business decision-making.
Are data engineers still in demand?
Data engineers, including those skilled in Apache Flink, are in high demand due to the increasing need for real-time data processing and scalable data infrastructure. Organizations seek professionals with expertise in big data tools, cloud platforms, and programming languages like Java or Python to build and maintain data pipelines. The role remains critical across industries such as finance, healthcare, and technology.
What is a Data Engineer Flink?
A Data Engineer Flink is a data engineering professional who specializes in using Apache Flink, an open-source stream processing framework, to build, maintain, and optimize systems that process large-scale data in real time. They design and implement data pipelines, ensure data quality and consistency, and collaborate with other engineering teams to deliver reliable and scalable data solutions. Their expertise allows organizations to process, analyze, and react to data as it is generated, enabling real-time insights and decision-making.
Is Flink an ETL?
Apache Flink is a stream processing framework often used in data engineering roles to perform real-time data transformations and processing tasks, which are key components of ETL (Extract, Transform, Load) pipelines. While Flink handles the transformation and loading stages efficiently, it is not a complete ETL tool on its own but is commonly integrated with other systems for data ingestion and storage.
What engineers make 500,000?
Senior data engineers, especially those with expertise in Apache Flink, cloud platforms, and large-scale data processing, can reach or exceed a $500,000 annual salary in high-demand markets. These roles often require advanced skills, certifications, and experience managing complex data pipelines and real-time analytics environments.
What are popular job titles related to Data Engineer Flink jobs in Indiana? For Data Engineer Flink jobs in Indiana, the most frequently searched job titles are:
What job categories do people searching Data Engineer Flink jobs in Indiana look for? The top searched job categories for Data Engineer Flink jobs in Indiana are:
Full-time
Posted 26 days ago
Job description
Job Summary:
Salesforce is the leading AI CRM company, seeking a Staff Software Engineer to join the Data Ingestion Team at Slack. The role involves designing and developing scalable streaming and batch data pipelines, implementing Lakehouse solutions, and collaborating with cross-functional partners to enhance data ingestion processes.
Responsibilities:
• Design and develop highly scalable and resilient streaming and batch pipelines for ingesting and processing structured and unstructured datasets
• Design and implement Slack’s Lakehouse solutions and enable production use cases for customers
• Improve the reliability and performance of the services/pipelines through AI assisted and/or Agentic solutions and tools.
• Automate and handle the lifecycle of datasets (schema evolution, metadata management, change and backfill management, deprecation and migration).
• As one of the tech leads, comfortably collaborate with cross functional partners and lead technical initiatives independently end to end.
• Be a role model and a multiplier, coaching and mentoring other engineers across the team and the org.
• Write, review, or provide feedback on a technical design proposal from others.
Qualifications:
Required:
• U.S. citizenship and willing to undergo a background check for GovSlack authorization. We can not provide visa sponsorship for this role.
• 7+ years of software/data engineering experience, including experience with building applications using Streaming and Lakehouse technologies, e.g. Kafka/Kafka Connect, Flink/Spark Streaming, Iceberg/Hudi/Delta or equivalent
• You have hands on experience and knowledge on building and maintaining batch data pipelines using Spark, Airflow, EMR, S3 or equivalent
• You are proficient in object-oriented and/or functional programming languages: SQL, Python, Java/Scala, Go or equivalent
• You are skilled at crafting and building robust distributed microservices with tools like Kubernetes, Docker, AWS ECS/EKS, Terraform, Grafana, etc.
• You are familiar with AI-assisted software development and automation and have hands on experience of Claude Code/Codex or equivalent
• You have excellent written and verbal communication and interpersonal skills; able to effectively collaborate with cross functional partners and explaining sophisticated technical concepts to non-technical stakeholders
Company:
Slack is a cloud-based communication and collaboration platform for teams. It is a sub-organization of Salesforce. Founded in 2009, the company is headquartered in San Francisco, USA, with a team of 1001-5000 employees. The company is currently Late Stage.
Salesforce is the leading AI CRM company, seeking a Staff Software Engineer to join the Data Ingestion Team at Slack. The role involves designing and developing scalable streaming and batch data pipelines, implementing Lakehouse solutions, and collaborating with cross-functional partners to enhance data ingestion processes.
Responsibilities:
• Design and develop highly scalable and resilient streaming and batch pipelines for ingesting and processing structured and unstructured datasets
• Design and implement Slack’s Lakehouse solutions and enable production use cases for customers
• Improve the reliability and performance of the services/pipelines through AI assisted and/or Agentic solutions and tools.
• Automate and handle the lifecycle of datasets (schema evolution, metadata management, change and backfill management, deprecation and migration).
• As one of the tech leads, comfortably collaborate with cross functional partners and lead technical initiatives independently end to end.
• Be a role model and a multiplier, coaching and mentoring other engineers across the team and the org.
• Write, review, or provide feedback on a technical design proposal from others.
Qualifications:
Required:
• U.S. citizenship and willing to undergo a background check for GovSlack authorization. We can not provide visa sponsorship for this role.
• 7+ years of software/data engineering experience, including experience with building applications using Streaming and Lakehouse technologies, e.g. Kafka/Kafka Connect, Flink/Spark Streaming, Iceberg/Hudi/Delta or equivalent
• You have hands on experience and knowledge on building and maintaining batch data pipelines using Spark, Airflow, EMR, S3 or equivalent
• You are proficient in object-oriented and/or functional programming languages: SQL, Python, Java/Scala, Go or equivalent
• You are skilled at crafting and building robust distributed microservices with tools like Kubernetes, Docker, AWS ECS/EKS, Terraform, Grafana, etc.
• You are familiar with AI-assisted software development and automation and have hands on experience of Claude Code/Codex or equivalent
• You have excellent written and verbal communication and interpersonal skills; able to effectively collaborate with cross functional partners and explaining sophisticated technical concepts to non-technical stakeholders
Company:
Slack is a cloud-based communication and collaboration platform for teams. It is a sub-organization of Salesforce. Founded in 2009, the company is headquartered in San Francisco, USA, with a team of 1001-5000 employees. The company is currently Late Stage.
About Slack
Sourced by ZipRecruiter
Industry
Internet and it
Company size
1,001 - 5,000 Employees
Headquarters location
San Francisco, CA, US
Year founded
2009