22 Sep
Senior Developer/Lead /Bigdata Architect
Vacancy expired!
Net2Source Inc. is an award-winning total workforce solutions company recognized by Staffing Industry Analysts for our accelerated growth of 300% in the last 3 years with over 5500+ employees globally, with over 30+ locations in the US and global operations in 32 countries. We believe in providing staffing solutions to address the current talent gap - Right Talent - Right Time - Right Place - Right Price and acting as a Career Coach to our consultants. If you are interested, please reply with below detail. Updated Resume. Current location Salary expectation Best time for call POSITION:1Job Title : Senior Bigdata -Senior Developer/LeadLocation : Bethesda, MD (Remote) Duration :Fulltime Job Description :- Summary The Big Data lead will be responsible for building, maintaining data pipelines and data products to ingest, process large volume of structured / unstructured data from various sources. The Big Data lead will work on analyzing the data needs, migrating the data into an Enterprise data lake, build data products and reports. The role requires experience with building real time and batch based ETL pipelines with strong understanding of big data technologies and distributed processing frameworks with. Responsibility /Expectations from the role
- Involved in the Study of the business logic and coordinate with the client to gather the requirements.
- Experience with Hive, Nifi, Airflow, Snowflake and Kafka
- Experience in Cloud environments, particularly AWS - EC2, S3, Lambda
- Experience in Unit testing, integration testing for Spark scripts
- Scripting with Bash, CSH, Shell, or Python
- Designing and implementing large-scale ingest systems in a Big Data environment
- Optimizing all stages of the data life cycle, from initial planning, to ingest, through final display and beyond
- Designing and implementing data extraction, cleansing, transformation, loading, and replication/distribution capabilities
- Developing custom solutions/code to ingest and exploit new and existing data sources
- Working with Sponsor development teams to improve application performance
- Providing support to maintain, optimize, troubleshoot, and configure the AWS/spark/Hadoop environment as needed
- Collaborating with team-mates, other service providers, vendors, and users to develop new and more efficient method
- Experience with CI/CD pipelines, unit tests, integration, and regression testing
- Significant experience with Airflow
- Strong ability to manage competing priorities and communication to multiple stakeholders
- Bachelor's Degree in Computer Science, Computer Engineering, or a related discipline with experience in software design and development.
- More than 10+ years of technical experience in data migration with design of Big Data systems
- Experience in Data processing in Batch and Real time with design and implementation using Pyspark/Spark Scala.
- Experience in event driven architecture design/solutioning using Kafka.
- Experience in Big Data technologies including SQL and NoSQL (HIVE, Hadoop, Couchbase DB, PostgreSQL) and Snowflake.
- Experience working in large complex transformation or migration programs from initiation to implementation, preferable in travel & transportation hospitality domain
- Experience in Data ingestion from on premise, cloud, structured, semi-structured and unstructured data
- Experience in modeling, design and mapping of data from different systems
- Experience in ODS, Data warehouse, staging, data lake and data mart implementation
- Experience in data architecture on AWS and Azure cloud (S3, SNS, SQS, Lambda, Azure data lake, SQL Data warehouse, DynamoDB)
- 10+ years of Bigdata technologies, handling large amounts of data, hundreds of Terabytes or more
- Experience in legacy technologies Netteza, Oracle, DB2
- Knowledge with data governance and best practices
- Propose and establish framework for necessary contributions from various development and program initiatives
- Bachelor's Degree in Computer Science, Computer Engineering, or a related discipline with experience in software design and development
- Provide detailed specifications for proposed solutions
- Advanced programming skills with Python, Java, Scala.
- Advanced understanding of software development and collaboration, including experience with tools such as Git.
- Excellent written and verbal communication skills, comfortable presenting in front of non-technical audiences.
- Experience is orchestration tools like Airflow, Jenkins
- Experience in Apache Ni-Fi process
- Excellent knowledge in hospitality domain in terms of migration
- Cloud Solution Architect Certification, preferable AWS
- Experience designing, architect Big Data solutions on cloud
- Experience in large scale migration from legacy to cloud implementation
- Experience with Spark streaming/batch, Kafka and Snowflake
- America's Most Honored Businesses (Top 10%)
- Awarded by USPAAC for Fastest Growing Business in the US
- 12th Fastest-Growing Staffing Company in the USA by Staffing Industry Analysts in the US (2020, 2019, 2020)
- Fastest 50 by NJ Biz (2020, 2019, 2020)
- INC 5000 Fastest growing for 8 consecutive years in a row (only 1.26% of companies make it to this list).
- Top 100 by Dallas Business Journal (2020 and 2019)
- Proven Supplier of the Year by Workforce Logiq (2020 and 2019)
- 2019 Spirit of Alliance Award by Agile1
- 2018 Best of the Best Platinum Award by Agile1
- 2018 TechServe Alliance Excellence Awards Winner
- 2017 Best of the Best Gold Award by Agile1(Act1 Group)
Vacancy expired!