Job Description :Our client in MISSISSAUGA,ON is looking for Big Data DeveloperMust Have Primary Skills :Big Data
Hadoop, Hive, Spark, Impala, Sqoop, Kafka
Required Qualifications
5+ years of Big Data development experience through
full lifecycle
Hands on experience in Hadoop, Hive, Spark, Impala,
Sqoop, Kafka and other technologies in Cloudera’s CDH distribution
Experience in Python and Pyspark
Prior ETL development experience in building data
warehouses and data pipelines is a plus
Prior experience on building solutions & reusable
components on Big Data platforms
Experience with Red Hat Linux and UNIX Bash Shell
Scripting preferred
Prior experience on designing and developing data
ingestion process, Data Quality
Knowledge on performance tuning in Hadoop eco systems –
Spark, Hive, Impala
Experience in defining technical design for use cases
in Hadoop eco systems
Experience in release management tools and technologies
like Bitbucket, Jenkins and others
Expereince in SQL and NOSQL
Expereince in working on different file formats like
parquet, avro, json, xml, delmited, flat files etcNice To Have Secondary Skills :BigData/Kafka Candidate must be 5+ year of experience on below skills
- HDFS
- Hive/HQL
- Shell Scripting
- Spark
-
Arcadia
- Kafka
- Good communication skills
- Good in troubleshooting
- Agile knowledge is a must
- Good team playerProven Experience In :Responsibilities:
Design, develop and implement robust and scalable data
pipelines on Big Data technology stack.
Perform defect analysis and support offshore
development teams in resolving defects/production issues.
Coordinate and work with Platform engineering team,
support teams and architects to triage technical issues and identify
resolutions.
Triage production issues and ensure BAU.
Proactively notify stakeholders of technical risks,
bottlenecks, problems, issues, and concerns.
Compliance with Citi's System Development Lifecycle and
Information Security requirements.
Utilize advanced knowledge of system flow and develop
standards for coding, testing, debugging, and implementation
Design, development and enhancement in existing
framework.
Technical brainstorming on best approaches to tackle
changes in the system
Managing time and changing priorities in a dynamic
environment
Ability to provide quick turnaround to software issues
and management requests
Ability to assimilate key issues and concepts and come
up to speed quickly
Develop prototypes and proof of concepts for the
selected solution