Senior Azure Data Engineer

We are seeking a Nashville based Senior Data Engineer to join the team.The key job responsibilities include, but are not limited to:

• Design and develop data ingestion pipeline framework using Azure Data Factory and Data Bricks
• Design and develop event-driven ingestion and regression framework to consume and publish data
• Design and develop event-driven ingestion and regression framework to consume and publish data
• Design and develop streaming ingestion solutions to process live data from various vendors
• Design and develop Continuous Integration and Continuous Deployment strategies
• Set up and deploy cloud-based data services such as blob services, databases, and analytics.
• Secure the platform and the stored data. Make sure only the necessary users can access the data.
• Ensure business continuity in uncommon conditions by using techniques for high availability and disaster recovery.
• Monitor to ensure that the systems run properly and are cost-effective.
• Define the data: Identify the data to be extracted. Define data by using a database query, a set of files, or an Azure Blob storage name for blob storage.
• Define the data transformation: Data transformation operations can include splitting, combining, deriving, adding, removing, or pivoting columns. Map fields between the data source and the data destination. You might also need to aggregate or merge data.
• Test the ETL / ELT / ELTL job in a development or test environment. Then migrate the job to a production environment to load the production system.
• Monitor the job: ETL operations can involve many complex processes. Set up a proactive and reactive monitoring system to provide information when things go wrong. Set up logging according to the technology that will use it.Should be able to document and articulate to the team on technology choices and approaches

• Strong, effective communication skills (oral and written), combined with the ability to engage the business in substantive discussion and resolution of issues
• Equally strong analytical skills to map data elements between systems, develop transformation logic, and resolve issues with data integrity.
• Demonstrated ability to meet deadlines
• Excellent interpersonal skills in order to interface with multiple constituent groups with potentially conflicting priorities and perspectives; the ability to support the entire software development and implementation lifecycle; work on multiple streams simultaneously; and guide users through complex application launches.Proven track-record to own the entire project lifecycle
• Excellent documentation skills to document all relevant processes, architecture and design decisionsShould be able to coach and mentor other team members