Big data Engineer

Job Description: The Big Data Engineer is responsible for engaging in the design, development and maintenance of the big data platform and solutions at Quicken Loans. This includes the platform host data sets that support various business operations and enable data-driven decisions as well as the analytical solutions that provide visibility and decision support using big data technologies. The Big Data Engineer is responsible for administering a Hadoop cluster, developing data integration solutions, resolving technical issues, and working with Data Scientists, Business Analysts, System Administrators and Data Architects to ensure the platform meets business demands. This team member also ensures that solutions are scalable, include necessary monitoring, and adhere to best practices and guidelines. The Big Data Engineer helps mentor new team members and continues to grow their knowledge of new technologies.ResponsibilitiesDevelop ELT processes from various data repositories and APIs across the enterprise, ensuring data quality and process efficiencyDevelop data processing scripts using SparkDevelop relational and NoSQL data models to help conform data to meet users’ needs using Hive and HBaseIntegrate platform into the existing enterprise data warehouse and various operational systemsDevelop administration processes to monitor cluster performance, resource usage, backup and mirroring to ensure a highly available platformAddress performance and scalability issues in a large-scale data lake environmentProvide big data platform support and issue resolutions to Data Scientists and fellow engineersRequirementsMaster's degree in computer science, software engineering or a closely related field2 years of experience with Hadoop distribution and ecosystem tools such as Hive, Spark, NiFi and Oozie2 years of experience developing batch and streaming ETL processes2 years of experience with relational and NoSQL databases, including modeling and writing complex queriesProficiency in at least one programming language, such as Python or JavaExperience with Linux system administration, scripting and basic network skillsExcellent communication, analytical and problem-solving skillsMultiple positions open on several different teams.1) Team is building trusted data products2) Another team the Big Data Engineer would work very closely with data scientists to help implement their models3) Platform and Support team could also use people to support DevOps and Performance Tuning activitiesWe are an equal opportunity employer. It is our policy to provide employment, compensation, and other benefits related to employment without regard to race, color, religion, sex, gender, national or ethnic origin, disability, veteran status, age, genetic information, citizenship, or any other basis prohibited by applicable federal, state or local law.