Job Description
Data Engineer
Location: Remote
Skills: Snowflake DBT Python DWH
Required:
5 years experience with the following:
- Snowflake (Columnar MPP Cloud data warehouse)
- DBT (ETL tool)
- Python
- Experience designing and implementing Data Warehouse
Responsibility:
- Build High level technical design both for Streaming and batch processing systems
- Design and build reusable components frameworks and libraries at scale to support analytics data products
- Perform POCs on new technology architecture patterns
- Design and implement product features in collaboration with business and Technology stakeholders
- Anticipate identify and solve issues concerning data management to improve data quality
- Clean prepare and optimize data at scale for ingestion and consumption
- Drive the implementation of new data management projects and restructure of the current data architecture
- Implement complex automated workflows and routines using workflow scheduling tools
- Build continuous integration testdriven development and production deployment frameworks
- Drive collaborative reviews of design code test plans and dataset implementation performed by other data engineers in support of maintaining data engineering standards
- Analyze and profile data for the purpose of designing scalable solutions
- Troubleshoot complex data issues and perform root cause analysis to proactively resolve product and operational issues
- Lead Mentor and develop offshore Data Engineers in adopting best practices and deliver data products.
- Partner closely with product management to understand business requirements breakdown Epics
- Partner with Engineering Managers to define technology roadmaps align on design architecture and enterprise strategy
Accountability:
- Requires minimal daily supervision
- Receives detailed instruction on new assignments and determines next steps with guidance
- Regularly reviews goals and objectives with supervisor
- Demonstrates competence in relevant job responsibilities which allows for increasing level of independence
- Ability to manage and prioritize multiple tasks
- Ability to work under pressure and meet deadlines
Problem Solving:
- Makes logical suggestions of likely causes of problems and independently suggests solutions
- Excellent organizational skills are required to prioritize responsibilities thus completing work in a timely fashion
- Outstanding ability to multiplex tasks as required
- Excellent project management and/or business analysis skills.
- Attention to detail and concern for impact is essential.
Preferred:
- Azure/AWS cloud technology
- SQL objects (procedures triggers views functions) in SQL Server. SQL query optimizations
- Understanding of TSQL indexes stored procedures triggers functions views etc.
- Design and development of Azure/AWS Data Factory Pipelines preferred.
- Design and development of data marts in Snowflake preferred
- Working knowledge of Azure/AWS Architecture Data Lake Data Factory
- Business analysis experience to analyze data to write code and drive solutions
- Knowledge of: Git Azure DevOps Agile Jira and Confluence
- Working knowledge on Erwin for data modelling
Job Tags
Offshore,