HADOOP ENGINEER ( Job Ref : 4NDICK269734 )
- Job Title:
- HADOOP ENGINEER
- Job Location:
- Pleasanton - California
- Category :
- IT Software - DBA / Datawarehousing
- Information Technology
- Work Type:
- Full Time
- Job Role:
- Software Developer
- Job Type:
- Annual Salary:
- Experience Required:
- 5-8 Years
Technical Knowledge and Skills:
Consultant resources shall possess most of the following technical knowledge and experience:
- Provide technical leadership, develop vision, gather requirements and translate client user requirements into technical architecture.
- Strong Hands-on Experience in building, deploying and productionizing ML models using software such as Spark MLLib, TensorFlow, PyTorch, Python Scikit-learn etc. is mandatory
- Ability to evaluate and choose best suited ML algorithms, perform feature engineering and optimize Machine Learning Models is mandatory
- Strong fundamentals in algorithms, data structures, statistics, predictive modeling, & distributed systems is must
- Design and implement an integrated Big Data platform and analytics solution
- Design and implement data collectors to collect and transport data to the Big Data Platform.
- 4+ years of hands-on Development, Deployment and production Support experience in Hadoop environment.
- 4-5 years of programming experience in Java, Scala, Python.
- Proficient in SQL and relational database design and methods for data retrieval.
- Knowledge of NoSQL systems like HBase or Cassandra
- Hands-on experience in Cloudera Distribution 5.x
- Hands-on experience in creating, indexing Solr collections in Solr Cloud environment.
- Hands-on experience building data pipelines using Hadoop components Sqoop, Hive, Pig, Solr, MR, Spark, Spark SQL.
- Must have experience with developing Hive QL, UDF’s for analyzing semi structured/structured datasets.
- Must have experience with Spring framework
- Hands-on experience ingesting and processing various file formats like Avro/Parquet/Sequence Files/Text Files etc.
- Hands-on experience working in Real-Time analytics like Spark/Kafka/Storm
- Experience with Graph Databases like Neo4J, Tiger Graph, Orient DB
- Must have working experience in the data warehousing and Business Intelligence systems.
- Expertise in Unix/Linux environment in writing scripts and schedule/execute jobs.
- Successful track record of building automation scripts/code using Java, Bash, Python etc. and experience in production support issue resolution process.
- Experience with R, Jupyter/Zeppelin
Strong SQL skills
Java, Spring, Scala, Cloudera Hadoop, MLLib, Spark, HBase, Neo4j, Solr, Python, Machine Learning
- Strong analytical skills with the ability to analyze information and identify and formulate solutions to problems.
- Provide more in-depth analysis with a high-level view of goals and end deliverables.
- Complete work within a reasonable time frame under the supervision of a manager or team lead.
- Plan and manage all aspects of the support function.
- Extensive knowledge of and proven experience with data processing systems, and methods of developing, testing and moving solutions to implementation.
- Strong knowledge in project management practices and ability to document processes and procedures as needed.
- Work collaboratively with other support team members and independently on assigned tasks and deliverables with minimum supervision
- Communicate effectively with users at all levels, from data entry technicians up to senior management, verbally and in writing.
- Self-motivated, working closely and actively communicating with team members to accomplish time critical tasks and deliverables
- Ask questions and share information gained with other support team members, recording and documenting this knowledge
- Elicit and gather user requirements and/or problem description information, and record this information accurately
- Listen carefully and act upon user requirements
- Convey and explain complex problems and solutions in an understandable language to both technical and non-technical persons
- Present technical solutions to management and decision makers
- Follow the lead of others on assigned projects as well as take the lead when deemed appropriate
- Think creatively and critically, analyzing complex problems, weighing multiple solutions, and carefully selecting solutions appropriate to the business needs, project scope, and available resources
- Take responsibility for the integrity of the solution
Established in :
5976 W. Las Positas Blvd., Suite 200, Pleasanton, California - 94588, United States
Website : http://www.buxtonconsulting.com/