We recommend focusing on candidates with strong hands-on experience in Google Vertex AI-based batch processing pipelines. The ideal candidate should demonstrate expertise in the following areas:
End-to-End Machine Learning Pipeline Development
- Data pre-processing
- Batch prediction workflows
- Post-processing activities
ML Pipeline Development & Orchestration
Hands-on experience building and orchestrating machine learning pipelines using:
- Kubeflow Pipelines (KFP Components)
- Google Cloud Dataflow (including Flex Templates)
- Google Vertex AI services
Technical Skills
Candidates should also be proficient in:
- Writing SQL queries for data preprocessing and post-processing tasks
- Developing Python-based workflows to integrate and orchestrate multiple pipeline components
- Deploying, monitoring, and managing Vertex AI Batch Prediction jobs
- Applying data engineering principles for large-scale data processing
Preferred Qualifications
- Strong expertise in SQL and Data Engineering
- Experience designing scalable, production-grade data and ML pipelines
- Familiarity with Google Cloud Platform ecosystem and best practices for ML workflow automation
Note: Candidates with a strong background in SQL, Data Engineering, Vertex AI, Kubeflow Pipelines, and Google Cloud Dataflow will be highly preferred.