Skip to Main Content
← Back to Jobs

Big Data Architect

OtterBase - TN Pittsburgh ,PA
  • Posted: over a month ago
  • Full-Time
  • Benefits: 401k, dental, medical, vision
Job Description

 

What you'll be doing:

Leading the architecture and design of our client's data platform: aggregating operational data, and providing access to insights. Be responsible for the technical and security architecture of the software applications, as well as the supporting infrastructure. Help to drive quality, reliable, secure applications, using industry standard best practices.

PRIMARY RESPONSIBILTIES:

  • Lead a development team of big data designers, developers, data scientists, and DevOps
  • Implement a big data enterprise warehouse, BI and analytics system using Hive, Spark, RedShift, EMR (Hadoop), and S3
  • Develop and maintain processes to acquire, analyze, store, cleanse, and transform large datasets using tools like Spark, Kafka, Sqoop, Hive, NiFi, and MiNiFi
  • Provide recommendations, technical direction and leadership for the selection and incorporation of new technologies 
  • Participate in regular status meetings to track progress, resolve issues, mitigate risks and escalate concerns in a timely manner
  • Contribute to the development, review, and maintenance of product requirements documents, technical design documents, and functional specifications
  • Help design innovative, customer-centric solutions based on deep knowledge of large-scale, data-driven technology 
  • Help develop and maintain enterprise data standards, best practices, security policies and governance processes 

 

EDUCATION:

Bachelors- computer information technology, computer science, management
Masters Degree Preferred- computer information technology, computer science, management

EXPERIENCE & SKILLS

  • Strong communication skills, ability to learn, demonstrate intellectual curiosity, ability to work in a team environment.  Self-motivated, passionate and strong leadership skills
  • Minimum 5+ years of experience in large systems analysis and development, addressing unique issues of architecture and data management. Has the experience to work at the highest technical level of all phases of systems analysis and development activity, across the full scope of system development cycle
  • 4+ years related experience on data warehousing and business intelligence projects
  • 3+ years implementation or development experience with the Hadoop ecosystem
  • Working knowledge of the entire big data development stack
  • Experience handling very large data sets (10’s of terabytes and up preferred)
  • Experience with secure RESTful Web Services
  • Highly proficient with Java/Scala application development
  • Expert in Apache Spark infrastructure and development
  • Experience with Sqoop, Spark, Hive, Kafka, and Hadoop
  • Experience with automated testing for Big Data platforms
  • Experience with best practices for data integration, transformation, governance and data quality
  • Experience with developing, designing and coding, completing programming and documentation, and performing testing for complex ETL applications (Spark & Scala preferred)
  • Experience with Agile software development process and development best practices
  • Experience with Big Data text mining and big Data Analytics preferred
  • Understanding of Big Data Architecture along with tools being used on Hadoop ecosystem
  • Ability to lead tool suite selection and lead proofs of concepts
  • Ability to share ideas among a collaborative team and drive the team based on technical expertise and sharing best practices from past experience

OtterBase - TN

Address

Pittsburgh, PA
USA

Website

What email should the hiring manager reach you at?

By clicking the button above, I agree to the ZipRecruiter Terms of Use and acknowledge I have read the Privacy Policy, and agree to receive email job alerts.

What email should we contact you at once we get salary info from the hiring manager?

By clicking the button above, I agree to the ZipRecruiter Terms of Use and acknowledge I have read the Privacy Policy, and agree to receive email job alerts.

Our qualification feature is only available to registered members - what email address would you like for us to keep on file?

By clicking the button above, I agree to the ZipRecruiter Terms of Use and acknowledge I have read the Privacy Policy, and agree to receive email job alerts.