Title : Senior software engineer - Big Data
Duration : 6+ months
Location : San Bruno,CA
Client : Retail Domain
Responsibilities and scope of this role include:
- Build scalable, high- performance, and efficient pipelines and workflows that are capable of processing billions of transactions and real-time customer activities.
- Work with big data and provide to our data scientists the right tools, data marts and rollups to build their machine learning models.
- Fluent in Pig and/or Hive with experience in building UDFs, Pig and Hadoop streaming.
- Build automated reports that can help the team to proactively identify quality and/or coverage problems in releases or new versions of our models.
- Apply knowledge of Azkaban, Oozie or Hamake for workflow management and job scheduling.
- Provide senior leadership and demonstrable, programming expertise and proficiency in Java, C/C++, or Python.
- Work on Data Warehousing architecture and data modeling best practices.
- Bachelor of Science degree or equivalent in Computer Science, Computer Engineering, Electrical Engineering or a related field plus 7-10 years of software engineering experience at a senior level; OR a Master’s degree or higher with 5-7 years of senior software engineering experience.
- Must have demonstrable, programming proficiency in one or more of the following: Java, C/C++, or Python.
- Deep understanding of Map Reduce framework & Hadoop.
- Fluent in Pig and/or Hive with experience in building UDFs, strong scripting ability.
- Proven expertise and understanding of ETL techniques.
- Knowledge of Azkaban, Oozie or Hamake for workflow management and job scheduling.
- Must be team oriented and collaborative to interact with both managers and cross functional teams.
- Ability to thrive in a fast paced environment on multiple projects in various phases and under tight deadlines