Your feedback helps keep ZipRecruiter safe.
Learn more about how to recognize common online scams
“WMC Global is a market leader in digital threat intelligence with headquarters in Fairfax, VA and offices in London, UK and Sydney, AU. WMC”
The Lead Data Engineer reports to the Vice President of Software Development and is a core member of the data team. This lead engineer drives how data is stored, consumed, managed, and integrated within and across products. This role will design and implement databases, data pipelines, data search and visualizations across big datasets. They will need to have extensive experience in all phases of data modeling, from conceptualization to database optimization. This role requires expert knowledge in designing and implementing solutions with structured and unstructured data in a cloud-based environment.
This is a full time, exempt position, is fully remote, and is based in the United States.
ESSENTIAL JOB FUNCTIONS:
Architect and build solutions to collect, transform, and process data at scale in a cloud-based environment
Develop and follow data engineering best practices with considerations for high data availability, computational efficiency, cost, and quality
Build and maintain environments processes, functionalities, and tools to improve all stages of solution development, e.g., proof of concepts, prototypes, and production
Create data models and database designs
Ensure that requirements for data retrieval are met
Identify areas for improvement in existing data storage and delivery systems
Develop and execute on long term data strategies across products
Creating new, experimental frameworks to collect data
Building tools to automate data collection
Develop techniques to search for patterns in large data sets, correlating similar data to find actionable results
Serve as a SME for all major data environments
Drive consistency in data usage across the organization
Know at a high level how data moves into and out of all enterprise systems
Provide guidance to keep the team current on new technologies, standards, and practices
Evaluate and make recommendations on new technologies, tools, and data
ANCILLARY JOB FUNCTIONS:
Perform database administration tasks which include database updates, snapshots, replication setup etc
Troubleshoot database incidents, including deadlocks, performance tuning slow-running queries, and data inaccuracies
Serve as a resource to outward-facing staff to determine rough LOE of new initiatives
Assist in concept development for new data products
Maintain data documentation such as ER diagrams and Data Mapping Specifications, especially in the absence of other support.
Assist in building and training, deploying AI Models for the data science team
Work well in distributed development environment across time zones
Mentor resources in the data engineering team
JOB COMPETENCIES/SKILL REQUIREMENTS/QUALIFICATIONS:
5+ years of experience in a data engineering role
Deep expertise in Python, MySQL and AWS services including RDS, Dynamo DB, Redshift, Athena, S3, Lambdas, EC2, EMR, RedShift, SQS, Kafka, Data pipelines, Step Functions etc
In-depth understanding and hands on experience using Elastic Search
Strong programming skills and ability to utilize a variety of software / languages / tools, e.g.,Python, SQL, etc.
Experience with microservice development using Docker, ECS/Kubernetes
Experience with ETL tools (Eg : AWS Glue)
Experience with CICD deployment of data engineering solutions
Deep knowledge of and experience with at least one RDBMS and a Columnar database technology (e.g. MySQL, DynamoDB)
Experience working in real time-data and streaming solutions preferred
Experience in building machine learning models preferred
Ability to break down and understand complex business problems, define a solution and implement it using advanced quantitative methods.
Solid oral and written communication skills, especially around analytical concepts and methods.
Significant experience in data modeling and database design
Understanding of software development life cycle
Knowledge of data visualization tools such as Tableau and Splunk
WMC Global is a market leader in digital threat intelligence with headquarters in Fairfax, VA and offices in London, UK and Sydney, AU. WMC Global delivers world-leading technology and solutions, enhanced by an unrivaled team of experts to protect our clients’ brands by protecting their customers. As a well-respected industry player in the mobile space, WMC Global is the most trusted digital compliance firm in North America, having partnered with all Tier 1 mobile carriers for the past 13 years. The Company is currently expanding into new sectors by securing additional customers in areas including, but not limited to, financial services and government.
Posted date: 21 days ago