Skip to Main Content
← Back to Jobs

Senior Big Data Engineer

the rubicon project Playa Vista, CA
  • Expired: 1 day ago. Applications are no longer accepted.
Senior Big Data Engineer at Rubicon Project (View all jobs)
Playa Vista, Los Angeles

Are you a Mid-Level/Sr Engineer who gets excited about high performance Big Data implementation?  The Rubicon Project is looking for a Senior level Big Data software engineer who is thorough and agile, capable of breaking down and solving problems, and have a strong will to get things done.  In the Data Operations team you will work on real-world problems working on big data tech stack where reliability, accuracy and speed are paramount, take responsibility for your systems end-to-end and influence the direction of our technology that impacts customers around the world.

This is an exciting and challenging role that will require hands-on experience with big data processing development to be deployed on Linux. You will be responsible for the day to day operation and new developments. We are seeking a candidate with good skills in software development life cycle, build data services with Java (or Scala), script languages, like, Python etc. This position includes 24x7 production support.

What you will be doing:

  • Design, develop and support various big data platforms applications, using supported tech stack technologies like Hadoop MR, Spark, Kafka, Druid, ETL and data warehouse applications integration
  • Develop applications with Scala, Spark programming languages and Big Data technology; including the script languages (Python, shell etc.) to wrap applications execution.
  • Involved in the design and implementation of full cycle of data services, from data transportation, data processing, ETL to data delivery for reporting
  • Identify, troubleshoot and resolve production data integrity and performance issues

What we look for:


  • Strong experience Java 7 or above.
  • One or more of the following data processing technologies: Hadoop Map Reduce, Spark, Kafka Streams, Flink, Storm, Apache Beam.
  • Kafka (preferred) OR one of the mainstream queue systems: ZeroMQ, RabbitMQ, ActiveMQ
  • Experience with some of the following AWS tools: EMR, Kinesis, Firehose, Redshift, RDS, S3 API, Lambda, SQS
  • Experience with Presto, Hive, Impala or similar SQL based engine for Big Data
  • Experience with Redis, Cassandra, MongoDB or similar NoSQL databases.
  • Experience with any of the following message / file formats: Parquet, Avro, Protocol Buffer

Extra perks we offer:

  • Take time for yourself: Our vacation days are unlimited and you get a week off the 4th of July and the last week of the year.
  • Stay healthy: Choose from a variety very low cost medical, dental and vision plans to cover you and your loved ones.
  • Enjoy your stay: Each Rubicon Project office enjoys a variety of benefits like daily catered lunches, a fully stocked kitchen with healthy snacks.
Apply for this Job
* Required

First Name *
Last Name *
Email *
Phone *
Location (City) *
Resume/CV *
Drop files here
Attach Dropbox Paste
</div></div> <div class="field"> <label>Cover Letter</label> <div class="attach-or-paste " aria-required="false" data-allow-s3="true" data-field="cover_letter" data-file-types="["pdf","doc","docx","txt","rtf"]" data-model="job_application"> <div class="drop-zone hidden"> Drop files here </div> <div id="cover_letter_chosen" class="chosen"> <img src=""/>   <span id="cover_letter_filename"/> <a href="#" class="remove"> <img width="11" height="11" src=""/></a></div> <div class="link-container"> <a data-source="attach" href="#">Attach</a> <a data-source="dropbox" href="#">Dropbox</a> <a data-source="paste" href="#">Paste</a></div> <div data-messages="uploaders"/> <div class="validation-anchor" style="display: none"/> <div class="progress-bar"> <div class="upload"> <div class="progress"><div class="bar" style="width: 0%"/></div> </div> <a href="#" class="cancel"> <img width="11" height="11" src=""/></a></div> <textarea name="job_application[cover_letter_text]" id="cover_letter_text" class="paste"/></div> </div> </div> <hr/><div id="custom_fields"> <div class="field"> <label>LinkedIn Profile <br/><input type="hidden" name="job_application[answers_attributes][0][question_id]" id="job_application_answers_attributes_0_question_id" value="13143133"/><input type="hidden" name="job_application[answers_attributes][0][priority]" id="job_application_answers_attributes_0_priority" value="0"/><input type="text" name="job_application[answers_attributes][0][text_value]" id="job_application_answers_attributes_0_text_value" aria-required="false" maxlength="255" autocomplete="custom-question-linkedin-profile"/></label></div><div class="field"> <label>Do you have Java experiencer, and if so, how many years?What are the Big Data technologies you've worked with: <span class="asterisk"> *</span> <br/><input type="hidden" name="job_application[answers_attributes][1][question_id]" id="job_application_answers_attributes_1_question_id" value="13143134"/><input type="hidden" name="job_application[answers_attributes][1][priority]" id="job_application_answers_attributes_1_priority" value="1"/><input type="text" name="job_application[answers_attributes][1][text_value]" id="job_application_answers_attributes_1_text_value" aria-required="true" maxlength="255" autocomplete="custom-question-do-you-have-java-experiencer-and-if-so-how-many-yearswhat-are-the-big-data-technologies-youve-worked-with-"/></label></div><div class="field"> <label>We are accepting candidates with various visa statuses. Will you need a H1B transfer/H1B sponsorship or any visa assistance now or in the future?<span class="asterisk"> *</span> <br/><input type="hidden" name="job_application[answers_attributes][2][question_id]" id="job_application_answers_attributes_2_question_id" value="13311952"/><input type="hidden" name="job_application[answers_attributes][2][priority]" id="job_application_answers_attributes_2_priority" value="2"/><input type="text" name="job_application[answers_attributes][2][text_value]" id="job_application_answers_attributes_2_text_value" aria-required="true" maxlength="255" autocomplete="custom-question-we-are-accepting-candidates-with-various-visa-statuses--will-you-need-a-h1b-transferh1b-sponsorship-or-any-visa-assistance-now-or-in-the-future"/></label></div> </div> <hr/><div id="eeoc_fields"> <h3 id="eeoc_title">U.S. Equal Opportunity Employment Information (Completion is voluntary)</h3> <p> Individuals seeking employment at Rubicon Project are considered without regards to race, color, religion, national origin, age, sex, marital status, ancestry, physical or mental disability, veteran status, gender identity, or sexual orientation. You are being given the opportunity to provide the following information in order to help us comply with federal and state Equal Employment Opportunity/Affirmative Action record keeping, reporting, and other legal requirements. </p><p> Completion of the form is entirely <strong>voluntary</strong>. Whatever your decision, it will not be considered in the hiring process or thereafter. Any information that you do provide will be recorded and maintained in a confidential file. </p> <div class="field" id="gender_dropdown_container"> <label for="job_application_gender">Gender</label> <select name="job_application[gender]" id="job_application_gender"><option value="">Please select</option><option value="1">Male</option><option value="2">Female</option><option value="3">Decline To Self Identify</option></select></div> <div class="field" id="hispanic_ethnicity_dropdown_container"> <label for="job_application_hispanic_ethnicity">Are you Hispanic/Latino?</label> <select name="job_application[hispanic_ethnicity]" id="job_application_hispanic_ethnicity"><option value="">Please select</option><option value="Yes">Yes</option><option value="No">No</option><option value="Decline To Self Identify">Decline To Self Identify</option></select></div> <div class="field" id="race_dropdown_container"> <label for="job_application_race">Please identify your race</label> <select name="job_application[race]" id="job_application_race"><option value="">Please select</option><option value="1">American Indian or Alaskan Native</option><option value="2">Asian</option><option value="3">Black or African American</option><option value="4">Hispanic or Latino</option><option value="5">White</option><option value="6">Native Hawaiian or Other Pacific Islander</option><option value="7">Two or More Races</option><option value="8">Decline To Self Identify</option></select></div> <a target="_blank" href=""> Race & Ethnicity Definitions</a> <p> If you believe you belong to any of the categories of protected veterans listed below, please indicate by making the appropriate selection. As a government contractor subject to Vietnam Era Veterans Readjustment Assistance Act (VEVRAA), we request this information in order to measure the effectiveness of the outreach and positive recruitment efforts we undertake pursuant to VEVRAA. Classification of protected categories is as follows: </p><p> A "disabled veteran" is one of the following: a veteran of the U.S. military, ground, naval or air service who is entitled to compensation (or who but for the receipt of military retired pay would be entitled to compensation) under laws administered by the Secretary of Veterans Affairs; or a person who was discharged or released from active duty because of a service-connected disability. </p><p> A "recently separated veteran" means any veteran during the three-year period beginning on the date of such veteran's discharge or release from active duty in the U.S. military, ground, naval, or air service. </p><p> An "active duty wartime or campaign badge veteran" means a veteran who served on active duty in the U.S. military, ground, naval or air service during a war, or in a campaign or expedition for which a campaign badge has been authorized under the laws administered by the Department of Defense. </p><p> An "Armed forces service medal veteran" means a veteran who, while serving on active duty in the U.S. military, ground, naval or air service, participated in a United States military operation for which an Armed Forces service medal was awarded pursuant to Executive Order 12985. </p> <div class="field" id="veteran_status_dropdown_container"> <label for="job_application_veteran_status">Veteran Status</label> <select name="job_application[veteran_status]" id="job_application_veteran_status"><option value="">Please select</option><option value="1">I am not a protected veteran</option><option value="2">I identify as one or more of the classifications of a protected veteran</option><option value="3">I don't wish to answer</option></select></div> <hr/><div class="eeoc_form_data"> <p>Form CC-305</p> <p>OMB Control Number 1250-0005</p> <p>Expires 1/31/2020</p> </div><h3 id="disability_status_header">Voluntary Self-Identification of Disability</h3><strong>Why are you being asked to complete this form?</strong><p> Because we do business with the government, we must reach out to, hire, and provide equal opportunity to qualified people with disabilities<sup>1</sup>. To help us measure how well we are doing, we are asking you to tell us if you have a disability or if you ever had a disability. Completing this form is voluntary, but we hope that you will choose to fill it out. If you are applying for a job, any answer you give will be kept private and will not be used against you in any way. </p><p> If you already work for us, your answer will not be used against you in any way. Because a person may become disabled at any time, we are required to ask all of our employees to update their information every five years. You may voluntarily self-identify as having a disability on this form without fear of any punishment because you did not identify as having a disability earlier. </p><strong>How do I know if I have a disability?</strong> <p> You are considered to have a disability if you have a physical or mental impairment or medical condition that substantially limits a major life activity, or if you have a history or record of such an impairment or medical condition. </p><p>Disabilities include, but are not limited to:</p> <ul><li>Blindness</li> <li>Deafness</li> <li>Cancer</li> <li>Diabetes</li> <li>Epilepsy</li> <li>Autism</li> <li>Cerebral palsy</li> <li>HIV/AIDS</li> <li>Schizophrenia</li> <li>Muscular dystrophy</li> <li>Bipolar disorder</li> <li>Major depression</li> <li>Multiple sclerosis (MS)</li> <li>Missing limbs or partially missing limbs</li> <li>Post-traumatic stress disorder (PTSD)</li> <li>Obsessive compulsive disorder</li> <li>Impairments requiring the use of a wheelchair</li> <li>Intellectual disability (previously called mental retardation)</li> </ul><div class="field" id="disability_status_dropdown_container"> <label for="job_application_disability_status">Disability Status</label> <select name="job_application[disability_status]" id="job_application_disability_status"><option value="">Please select</option><option value="1">Yes, I have a disability (or previously had a disability)</option><option value="2">No, I don't have a disability</option><option value="3">I don't wish to answer</option></select></div> <strong>Reasonable Accommodation Notice</strong> <p> Federal law requires employers to provide reasonable accommodation to qualified individuals with disabilities. Please tell us if you require a reasonable accommodation to apply for a job or to perform your job. Examples of reasonable accommodation include making a change to the application process or work procedures, providing documents in an alternate format, using a sign language interpreter, or using specialized equipment. </p><p> <sup>1</sup>Section 503 of the Rehabilitation Act of 1973, as amended. For more information about this form or the equal employment obligations of Federal contractors, visit the U.S. Department of Labor's Office of Federal Contract Compliance Programs (OFCCP) website at <a href="" target="_blank"></a>. </p><p> PUBLIC BURDEN STATEMENT: According to the Paperwork Reduction Act of 1995 no persons are required to respond to a collection of information unless such collection displays a valid OMB control number. This survey should take about 5 minutes to complete. </p></div> <div id="captcha_container"/> <hr/><div id="error_message"/> <div id="submit_buttons"> <input type="button" value="Submit Application" id="submit_app" class="button"/></div></form> </div></div> </div>

the rubicon project

Why Work Here?

A leader in the industry with amazing benefits, qualify work life balance, and long-term career growth

Rubicon Project is a leading technology company automating the buying and selling of advertising. Relentless in its efforts for innovation, Rubicon Project has engineered one of the largest real-time cloud and Big Data computing systems, processing trillions of transactions within milliseconds each month. The company’s pioneering technology created a new model for the advertising industry – similar to what NASDAQ did for stock trading. Rubicon Project’s automated advertising platform is used by the world’s leading publishers and applications to transact with top brands around the globe. Founded in 2007, Rubicon Project’s mission is to keep the Internet free and open and fuel its growth by making it easy and safe to buy and sell advertising. Rubicon Project pioneered advertising automation technology to enable the world’s leading brands, content creators and application developers to trade and protect trillions of advertising requests each month and to improve the advertising experiences of consumers. Rubicon Project is a publicly-traded company (NYSE: RUBI) headquartered in Los Angeles, California.


Playa Vista, CA