02 Mar
BigData Consultant with Python & PySpark
Vacancy expired!
We, International Information Technology Team (i2t2) Inc. are looking for a BigData Consultant (Mid-Level & Senior Level resources) for one of our clients located in East Coast region.
Job Title: BigData Consultant – PySpark & PythonLocation: REMOTEDuration: Open for long-term Contract and Fulltime/Permanent Responsibilities:- Cleanse, manipulate and analyze large datasets (Structured and Unstructured data - XMLs, JSONs, PDFs) using Hadoop platform.
- Develop Python, PySpark, Spark scripts to filter/cleanse/map/aggregate data.
- Manage and implement data processes (Data Quality reports)
- Develop data profiling, deduping logic, matching logic for analysis
- Programming Languages experience in Python, PySpark and Spark for data ingestion
- Programming experience in BigData platform using Hadoop platform
- Present ideas and recommendations on Hadoop and other technologies best use to management
- 3+ years of programming experience in Hadoop, Spark, Python for data processing and analysis.
- 3+ years of experience - using Hadoop platform and performing analysis.
- Familiarity with Hadoop cluster environment and configurations for resource management for analysis work
- Strong SQL experience is a must
- Must be able to manage multiple priorities and meet deadlines
Vacancy expired!