Implement efficient data transformation and aggregation using PySpark and relevant big data frameworks. * Develop robust error handling and exception management mechanisms to ensure data integrity ...
Implement efficient data transformation and aggregation using PySpark and relevant big data frameworks. * Develop robust error handling and exception management mechanisms to ensure data integrity ...
Implement efficient data transformation and aggregation using PySpark and relevant big data frameworks. * Develop robust error handling and exception management mechanisms to ensure data integrity ...
Implement efficient data transformation and aggregation using PySpark and relevant big data frameworks. * Develop robust error handling and exception management mechanisms to ensure data integrity ...
Data Engineer
Grand Prairie, TX · On-site
$109K - $131K/yr
Develop data ingestion, transformation, and integration workflows using SQL and Python. * Design and implement Lakehouse and Medallion architecture solutions. * Work with cloud data platforms ...
New
Data Engineer
Grand Prairie, TX · On-site
$109K - $131K/yr
Develop data ingestion, transformation, and integration workflows using SQL and Python. * Design and implement Lakehouse and Medallion architecture solutions. * Work with cloud data platforms ...
New
Data Modeler with SAP Hana
Dallas, TX · On-site
$54.25 - $70.25/hr
Understand data and prepare data transformation logics and data quality logics * Create and document data mapping, including data dictionaries, metadata, and entity-relationship diagrams.
Quick apply
Data Modeler with SAP Hana
Dallas, TX · On-site
$54.25 - $70.25/hr
Understand data and prepare data transformation logics and data quality logics * Create and document data mapping, including data dictionaries, metadata, and entity-relationship diagrams.
Data Analyst/Data Engineer (Healthcare) (69269)
$125K - $145K/yr
Write advanced SQL queries for data extraction, transformation, and validation * Use Python for automation, data cleaning, and advanced analysis * Build recurring reports related to: * Census trends
Data Analyst/Data Engineer (Healthcare) (69269)
$125K - $145K/yr
Write advanced SQL queries for data extraction, transformation, and validation * Use Python for automation, data cleaning, and advanced analysis * Build recurring reports related to: * Census trends
Data Engineer
Plano, TX · On-site
$110K - $132K/yr
... transformation, automation, and analytics • Collaborate with cross-functional teams including Data Analysts, Data Scientists, Product teams, and Business stakeholders to deliver data-driven ...
Data Engineer
Plano, TX · On-site
$110K - $132K/yr
... transformation, automation, and analytics • Collaborate with cross-functional teams including Data Analysts, Data Scientists, Product teams, and Business stakeholders to deliver data-driven ...
Data AI Engineer with Vector Databases
Plano, TX · On-site
$109K - $131K/yr
Work with Python and SQL for data transformation and analytics * Implement GenAI data architectures, including RAG pipelines and vector indexing * Manage and optimize Vector Databases for embedding ...
Data AI Engineer with Vector Databases
Plano, TX · On-site
$109K - $131K/yr
Work with Python and SQL for data transformation and analytics * Implement GenAI data architectures, including RAG pipelines and vector indexing * Manage and optimize Vector Databases for embedding ...
Data Engineer
Dallas, TX · On-site
$105K - $120K/yr
... data transformation frameworks. • Develop and optimize Snowflake databases, schemas, and data pipelines. • Leverage Snowflake CoCo capabilities to accelerate data engineering activities. • ...
Data Engineer
Dallas, TX · On-site
$105K - $120K/yr
... data transformation frameworks. • Develop and optimize Snowflake databases, schemas, and data pipelines. • Leverage Snowflake CoCo capabilities to accelerate data engineering activities. • ...
AWS Data Engineer - Dallas, TX - Locals Only Onsite F2F interview
Dallas, TX · On-site
$113K - $136K/yr
Strong SQL skills for data transformation and analysis. Hands-on experience with Kafka for real-time data processing. Experience working with Amazon Redshift. Knowledge of Git pipelines and CI/CD ...
AWS Data Engineer - Dallas, TX - Locals Only Onsite F2F interview
Dallas, TX · On-site
$113K - $136K/yr
Strong SQL skills for data transformation and analysis. Hands-on experience with Kafka for real-time data processing. Experience working with Amazon Redshift. Knowledge of Git pipelines and CI/CD ...
Data Engineer
Plano, TX · On-site
$110K - $132K/yr
... transformation, automation, and analytics • Collaborate with cross-functional teams including Data Analysts, Data Scientists, Product teams, and Business stakeholders to deliver data-driven ...
Data Engineer
Plano, TX · On-site
$110K - $132K/yr
... transformation, automation, and analytics • Collaborate with cross-functional teams including Data Analysts, Data Scientists, Product teams, and Business stakeholders to deliver data-driven ...
Implement efficient data transformation and aggregation using PySpark and relevant big data frameworks. * Develop robust error handling and exception management mechanisms to ensure data integrity ...
Implement efficient data transformation and aggregation using PySpark and relevant big data frameworks. * Develop robust error handling and exception management mechanisms to ensure data integrity ...
Implement efficient data transformation and aggregation using PySpark and relevant big data frameworks. * Develop robust error handling and exception management mechanisms to ensure data integrity ...
Implement efficient data transformation and aggregation using PySpark and relevant big data frameworks. * Develop robust error handling and exception management mechanisms to ensure data integrity ...
Experience supporting large-scale data transformation programs. * Experience leading cross-functional teams consisting of business stakeholders, architects, engineers, and vendors. * Experience ...
Experience supporting large-scale data transformation programs. * Experience leading cross-functional teams consisting of business stakeholders, architects, engineers, and vendors. * Experience ...
Data Technical Architect
Houston, TX · On-site
$70/hr
Rysun guides and accelerates the AI & Data strategy and Digital Transformation programs for Fortune 2000 enterprises and product startups to shape remarkable customer experiences and intelligent ...
Quick apply
Data Technical Architect
Houston, TX · On-site
$70/hr
Rysun guides and accelerates the AI & Data strategy and Digital Transformation programs for Fortune 2000 enterprises and product startups to shape remarkable customer experiences and intelligent ...
Develop source-to-target data mappings and transformation rules. * Create reusable migration scripts using SQL, T-SQL, and PL/SQL. * Build validation, reconciliation, audit, and reporting scripts.
Develop source-to-target data mappings and transformation rules. * Create reusable migration scripts using SQL, T-SQL, and PL/SQL. * Build validation, reconciliation, audit, and reporting scripts.
Sr. Data Engineer (PySpark, Azure Data Factory)
Houston, TX · On-site
$109K - $131K/yr
As an Data Engineer at Calpine Inc., you will contribute to the BI development to optimize energy ... Documentdata flows, transformation logic, and architectural decisions. * Communicateeffectively ...
Sr. Data Engineer (PySpark, Azure Data Factory)
Houston, TX · On-site
$109K - $131K/yr
As an Data Engineer at Calpine Inc., you will contribute to the BI development to optimize energy ... Documentdata flows, transformation logic, and architectural decisions. * Communicateeffectively ...
Develop source-to-target data mappings and transformation rules. * Create reusable migration scripts using SQL, T-SQL, and PL/SQL. * Build validation, reconciliation, audit, and reporting scripts.
Quick apply
Develop source-to-target data mappings and transformation rules. * Create reusable migration scripts using SQL, T-SQL, and PL/SQL. * Build validation, reconciliation, audit, and reporting scripts.
Data Engineer
Plano, TX · On-site
$109K - $131K/yr
Write clean, efficient, and production-ready SQL queries and Python code for data transformation, automation, and analytics * Collaborate with cross-functional teams including Data Analysts, Data ...
Data Engineer
Plano, TX · On-site
$109K - $131K/yr
Write clean, efficient, and production-ready SQL queries and Python code for data transformation, automation, and analytics * Collaborate with cross-functional teams including Data Analysts, Data ...
Data Engineer
Dallas, TX · On-site
$105K - $120K/yr
... transformation frameworks. • Develop and optimize Snowflake databases, schemas, and data pipelines. • Implement data quality controls, reconciliation processes, and monitoring frameworks. • ...
Data Engineer
Dallas, TX · On-site
$105K - $120K/yr
... transformation frameworks. • Develop and optimize Snowflake databases, schemas, and data pipelines. • Implement data quality controls, reconciliation processes, and monitoring frameworks. • ...
Snowflake Data Engineer - Onsite
Westlake, TX · On-site
$109K - $132K/yr
Develop and implement data transformation and loading process to populate and maintain data within the created tables and views. * Collaborate with data architects and tech leads to understand data ...
Quick apply
Snowflake Data Engineer - Onsite
Westlake, TX · On-site
$109K - $132K/yr
Develop and implement data transformation and loading process to populate and maintain data within the created tables and views. * Collaborate with data architects and tech leads to understand data ...
Data Transformation information
What are the key skills and qualifications needed to thrive in the data transformation position, and why are they important?
To thrive in a Data Transformation role, a strong background in data analysis, data modeling, and database management is essential, often supported by a degree in computer science, information systems, or a related field. Proficiency with data transformation tools like SQL, ETL platforms (such as Informatica or Talend), and cloud data services is typically required, and certifications in these areas can be advantageous. Strong problem-solving skills, attention to detail, and effective communication abilities are key for collaborating with cross-functional teams and addressing complex data challenges. These skills and qualities are important because they ensure accurate, efficient data transformations that support critical business decisions.
What are the typical daily responsibilities of someone in a data transformation role?
In a Data Transformation role, your daily responsibilities often include analyzing raw data from multiple sources, designing and executing processes to clean and reformat data, and collaborating with business and IT teams to ensure data meets project requirements. You’ll typically work with ETL (Extract, Transform, Load) tools to automate and streamline these processes, as well as perform quality checks to validate data accuracy. Coordinating with data engineers, data analysts, and stakeholders is a key part of the job to ensure projects align with business goals. Staying up-to-date with new data management technologies and best practices can also be a regular part of your routine, helping to drive ongoing improvements.
What is a data transformation?
A Data Transformation job involves converting, structuring, and optimizing raw data to make it useful for analysis, reporting, or other business processes. Professionals in this role use ETL (Extract, Transform, Load) tools, coding languages like SQL or Python, and cloud platforms to clean, aggregate, and reformat data. They ensure data integrity, improve accessibility, and support data-driven decision-making. This role is essential in industries that rely on accurate and efficient data processing, such as finance, healthcare, and e-commerce.

Job description
Job Title: PySpark Data Engineer
Summary:
We are seeking a skilled PySpark Data Engineer to join our team and drive the development of robust data processing and transformation solutions within our data platform. You will be responsible for designing, implementing, and maintaining PySpark-based applications to handle complex data processing tasks, ensure data quality, and integrate with diverse data sources. The ideal candidate possesses strong PySpark development skills, experience with big data technologies, and the ability to work in a fast-paced, data-driven environment.
Key Responsibilities: Data Engineering Development:- Design, develop, and test PySpark-based applications to process, transform, and analyze large-scale datasets from various sources, including relational databases, NoSQL databases, batch files, and real-time data streams.
- Implement efficient data transformation and aggregation using PySpark and relevant big data frameworks.
- Develop robust error handling and exception management mechanisms to ensure data integrity and system resilience within Spark jobs.
- Optimize PySpark jobs for performance, including partitioning, caching, and tuning of Spark configurations.
- Collaborate with data analysts, data scientists, and data architects to understand data processing requirements and deliver high-quality data solutions.
- Analyze and interpret data structures, formats, and relationships to implement effective data transformations using PySpark.
- Work with distributed datasets in Spark, ensuring optimal performance for large-scale data processing and analytics.
- Design and implement ETL (Extract, Transform, Load) processes to ingest and integrate data from various sources, ensuring consistency, accuracy, and performance.
- Integrate PySpark applications with data sources such as SQL databases, NoSQL databases, data lakes, and streaming platforms
- Bachelor's degree in Computer Science, Information Technology, or a related field.
- 5+ years of hands-on experience in big data development, preferably with exposure to data-intensive applications.
- Strong understanding of data processing principles, techniques, and best practices in a big data environment.
- Proficiency in PySpark, Apache Spark, and related big data technologies for data processing, analysis, and integration.
- Experience with ETL development and data pipeline orchestration tools (e.g., Apache Airflow, Luigi).
- Strong analytical and problem-solving skills, with the ability to translate business requirements into technical solutions.
- Excellent communication and collaboration skills to work effectively with data analysts, data architects, and other team members.
About Photon
Sourced by ZipRecruiter
Company size
1 - 10 Employees
Headquarters location
Cambridge, MA, US
Year founded
1984