Senior Big Data Engineer Irving-TX
Vacancy expired!
Looking for a Senior Big Data Engineer to work with a small team responsible for building a Big Data solution that will enable operations for a large enterprise environment. Must be able to design, build and maintain Enterprise Level Data Pipe-Lines utilizing the tools available within Big Data Eco-System. As a Senior Big Data Engineer - You will work on Advanced Analytics using Big Data technologies such as Hadoop and Data Warehousing. - Build analytical solutions to enable Data Scientist by manipulating large data sets and integrating diverse data sources. - Perform ad-hoc analysis and develop reproducible analytical approaches to meet business requirements. - Perform exploratory and targeted data analyses using descriptive statistics and other methods. - Use complex algorithms to develop systems & applications that deliver business functions or architectural components.
Typical duties will include the following: - Work closely with the data scientists, and database and systems administrators to create data solutions. - Follow best practices on design and implementation to aid in company-wide data governance. - Improve existing data pipelines by simplifying and increasing performance. - Design, build, and deploy new data pipelines within Big Data Eco-Systems. - Documents new/existing pipelines, Data Sets and Data Sets. - Abides by department development standards and SOP's. - Attends all department meetings. - All other duties as assigned. - Keeps updated on latest technologies relevant to position’s duties. - Has great knowledge of commonly used software concepts and design. - Great knowledge of the development lifecycle. - Keeps management updated on projects and assigned work. You will need to have: - Bachelor’s degree or six or more years of work experience. - Six or more years of relevant work experience. Even Better If You Have: - Experience of designing, building, and deploying production-level data pipelines using tools from Hadoop stack (HDFS, Hive, Spark, HBase, Kafka, NiFi, Oozie, Splunk etc). - Experience with various NoSQL databases (Hive, MongoDB, Couchbase, Cassandra, Neo4j). - Experience with analytic or feature engineer programming (python or scala or java). - Experience implementing open source frameworks & exposure to various open source & - package software architectures (AngularJS, ReactJS, Node, Elasticsearch, Spark, Scala, - Splunk, Apigee, and Jenkins etc.). - Experience troubleshooting JVM-related issues. - Experience with SQL databases and Change Data Capture. - Experience and strategies to deal with mutable data in Hadoop. - Experience with Stream sets. - Experience of Agile and DevOps methodologies. - Experience in Data Quality Tools on Hadoop/Big Data Platform - Relies on limited experience and judgment to plan and accomplish goals. - Works under general supervision. - Experience in full development life cycle and significant experience in delivering applications and architecture services. - Experience in data visualization tools like Kibana, Grafana, Tableau and associated architectures. - Experience evaluating and implementing cutting-edge digital technologies. - Experience with Cloud technologies (AWS, Google Cloud Platform, PCF, Docker, Kubernetes and application migration.Vacancy expired!