Job Summary:
Tekfortune is a fast-growing consulting firm specialized in staffing services for leading organizations. The Data Quality | Observability Engineer is responsible for monitoring and ensuring the health and compliance of data ingestion pipelines, implementing metrics collection, and collaborating with various teams to maintain data quality standards.
Responsibilities:
โข Implement pipeline-level metrics collection: records in/out, processing lag, throughput, failure counts, and data freshness indicators.
โข Build and maintain monitoring dashboards in CloudWatch and/or Grafana for all production ingestion pipelines.
โข Configure alerting thresholds tied to agreed SLAs; ensure alerts trigger appropriately for pipeline lag, failures, and data quality breaches.
โข Capture data lineage and metadata for every ingestion pipeline and publish to the Boeing Data/Developer Portal or catalog.
โข Design and implement data quality rules (completeness, schema conformance, record counts, freshness) in collaboration with Data/Cloud Engineers and data governance/stewards.
โข Produce automated weekly and monthly SLA reports showing ingestion success rates, data freshness, incident counts, and trend analysis.
โข Develop cost monitoring views for ingestion compute spend and provide optimization recommendations.
โข Collaborate with Boeing's monitoring/observability team for dashboard and alerting integration.
โข Support incident triage by providing pipeline health diagnostics and root cause data.
โข Maintain and evolve DQ and observability standards as new sources are onboarded each month.
Qualifications:
Required:
โข 5โ7 years of experience in data quality engineering, data observability, or data operations with a platform focus.
โข Hands-on experience with CloudWatch (metrics, logs, alarms, dashboards) and Grafana.
โข Strong experience implementing data quality frameworks: Great Expectations, dbt tests, Deequ, Soda, or equivalent.
โข Familiarity with data lineage and cataloging tools: AWS Glue Catalog, Apache Atlas, DataHub, OpenLineage, or similar.
โข Proficiency in SQL and Python for metrics collection, reporting automation, and DQ rule implementation.
โข Experience building SLA dashboards and automated reporting for data pipelines.
โข Understanding of data observability concepts: freshness, volume, schema change detection, distribution anomalies.
โข Knowledge of AWS cost monitoring tools (Cost Explorer, CUR, CloudWatch billing metrics).
โข Strong communication skills for producing reports and presenting SLA performance to stakeholders.
โข Experience working with data governance teams on data contracts, classification, and quality rules.
Preferred:
โข Experience with dedicated data observability platforms (Monte Carlo, Datafold, Bigeye).
โข Familiarity with OpenTelemetry or distributed tracing for data pipelines.
โข Experience with FinOps practices for data platform cost optimization.
Company:
Tekfortune Inc. Founded in 2014, the company is headquartered in Piscataway, USA, with a team of 51-200 employees. The company is currently Growth Stage.