05 Mar
Data Warehouse Designer/Developer
Vacancy expired!
Dedicated Tech Services, Inc. (DTS) is an award-winning IT consulting firm based in Columbus, OH. We now have an opening for a
Data Warehouse Designer/Developer with Cloudera Big Data experience in Columbus, Ohio. s and those authorized to work in the US are encouraged to apply. We are unable to sponsor at this time. Direct applicants only, no corp-to-corp candidates, third parties, or agencies, please. This is a right-to-hire position with our client. Required Skills and Experience:- Minimum of four years of experience with Informatica PowerCenter on Data Warehousing or Data Integration projects
- Analysis, design, development, support, and enhancements of ETL in a data warehouse environment with Cloudera Big Data technologies including two years of experience in all or most of the following:
- Hadoop
- MapReduce
- Sqoop
- PySpark
- Spark
- HDFS
- Hive
- Impala
- Stream Sets
- Judi
- Oozie
- Hue
- Kafka
- Yarn
- Python
- Flume
- Zookeeper
- Sentry
- Cloudera Navigator)
- Informatica
- Oracle SQL
- PL/SQL
- Unix commands and shell scripts
- HDFS commands and file formats
- Jobs scheduling
- Proven ability to write high quality code
- 7+ years of experience in implementing complex ETL logic
- 3+ years of experience Develop and enforce strong reconciliation process
- Experience creating ETL design documentation
- 5+ years of strong SQL experience (prefer Oracle)
- 5+ years of relational database, data vault and dimensional model design
- 2+ years of experience in UNIX/LINUX shell scripting
- Experience developing standards and practices for developing and following centralized metadata repository
- Literate in Excel, PowerPoint, Word, etc.
- Effective communication, presentation, and organizational skills
- Ability to establish priorities & follow through on projects, paying close attention to detail with minimal supervision
- BS/BA degree or combination of education & experience
- Familiar with Waterfall, Agile, and hybrid approaches
- Demonstrated effective leadership, analytical and problem-solving skills
- Excellent written and oral communication skills with technical and business teams.
- Ability to work independently, as well as part of a team
- Passion for staying up-to-date on current technologies
- Strong listening skills
- Ability to recognize opportunities for improvement.
- Participate in team activities, design discussions, stand up meetings and planning review with team.
- Create jobs in Informatica PowerCenter, Informatica Developer IDQ, Hadoop using SQOOP, PySpark, and Stream Sets to meet the business user needs.
- Perform data analysis, data profiling, data quality and data ingestion in various layers using database queries, Informatica PowerCenter, Informatica Analyst score cards, PySpark programs and UNIX shell scripts.
- Follow the organization coding standard document, create mappings, sessions and workflows as per the mapping specification document.
- Updating the production support Run book and the Control M schedule document as per the production release.
- Create and update design documents, provide detail description about workflows after every production release.
- Continuously monitor the production data loads, fix the issues, update the tracker document with the issues, Identify the performance issues.
- Performance tuning long-running ETL jobs
- Perform quality assurance check, reconciliation post data loads and communicate to vendor for receiving fixed data.
- Participate in ETL code review and design re-usable frameworks.
- Create SQOOP scripts to ingest historical data from EDW Oracle database to Hadoop IOP and create PySpark programs to ingest historical and incremental data.
- Create Hive tables and Impala views creation scripts for dimension tables.
- Extract unstructured and semi-structured data using data processor transformation in IDQ.
- Perform gap and impact analysis for new requirements and enhancements.
- Create mockup data, perform unit testing and capture the result sets against the jobs developed in lower environment.
- Create Remedy incidents to fix production issues and to trigger Control M jobs for FTP and ETL jobs on an ad-hoc, weekly, monthly, and quarterly basis as needed.
- Create Support Requests to deploy Database, UNIX, ETL and SAS code to UAT environment.
- Model and create data warehouse dimension tables as needed.
- Create change requests, work plans, test results, checklist documents
- Write re-usable UNIX shell scripts for file archival, file validations and informatica workflow looping.
- Develop re-usable framework for Audit Balance Control to capture reconciliation, mapping parameters and variables.
- Write stored procedures for gathering stats
- Write complex SQL queries and perform tuning
- Participate in meetings to continuously upgrade the functional and technical expertise.
Vacancy expired!