Data Engineer AWS/ Saprk

KAYGEN is an emerging leader in providing top talent for technology based staffing services. We specialize in providing high-volume contingent staffing, direct hire staffing and project based solutions to companies worldwide ranging from startups to Fortune 500 and Managed Service Providers (MSP) across a wide variety of industriesDescription:

• The Data Engineer will collaborate with product owners, developers, database architects, data analysts, visual developers and data scientists on data initiatives and will ensure optimal data delivery and architecture is consistent throughout ongoing projects
• Create and maintain optimal data pipeline architecture, assemble large, complex data sets that meet functional / non-functional business requirements
• Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing for greater scalability
• Work with Production Support to analyze and fix Production issues
• Analyze requirement documents and Source to target mapping

• 5+ years experience in Big Data stack environments like AWS EMR, Clourdera, Hortonworks including strong understanding of solution and technical design
• 3+ years of SPARK in batch mode
• 3+ years of experience in scripting using Python
• Strong hands on SQL Coding
• Experience with working with offshore resources