Job Description

Hi,

I am Dilshadul Karim working with Nityo Infotech Solutions.

I am having this urgent requirement for our client.

Please have a look at the job description below and send your Updated resume if you are Interested.

Job Title: Lead Data Engineer/Architect
Location: Irvine (3 days hybrid)
Type : Long term Contract

Client : Infosys/Capital Group



Job Description:

We are seeking a highly skilled and experienced Lead Data Engineer/Architect to join our dynamic team. The ideal candidate will have a strong background in data engineering, with extensive experience in Python , PySpark , AWS services , PostgreSQL , Apache Airflow and Docker .

This role requires a professional with a proven track record of designing, implementing, and maintaining robust data pipelines and architectures.

Key Responsibilities:

• Design and Develop Data Pipelines: Create and maintain scalable data pipelines using Python and PySpark to process large volumes of data efficiently.
• Cloud Integration : Utilize AWS services (such as S3, CloudWatch, ECS, ECR, Lambda) to build and manage cloud-based data solutions.
• Database Management: Design, implement, and optimize PostgreSQL databases to ensure high performance and reliability.
• Workflow Orchestration : Use Apache Airflow to schedule and monitor complex data workflows.
• Containerization: Implement and manage Docker containers to ensure consistent and reproducible environments for data processing tasks.
• Data Quality and Governance : Ensure data quality, integrity, and security across all data pipelinesand storage solutions.
• Collaboration: Work closely with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions that meet business needs.
• Mentorship: Provide guidance and mentorship to junior data engineers and contribute to the continuous improvement of the team's skills and processes.

Qualifications:

Education : Bachelor's or Master's degree in Computer Science, Engineering, or a related field.

Experience: 10+ years of overall experience in data engineering or related fields.

Technical Skills:

• Proficiency in Python and PySpark.
• Extensive experience with AWS services (S3, CloudWatch, ECS, ECR, Secrets Manager, Cloud9 IDE).
• Strong knowledge of PostgreSQL and database optimization techniques.
• Hands-on experience with Apache Airflow for workflow orchestration.
• Proficiency in Docker for containerization.

Soft Skills:

• Excellent problem-solving and analytical skills.
• Strong communication and collaboration abilities.
• Ability to work in a fast-paced, dynamic environment.
• Preferred Qualifications:
• Familiarity with CI/CD pipelines and DevOps practices.
• Knowledge of data warehousing concepts and ETL processes.

Thanks & Regards

Dilshadul Karim

(Technical Recruiter)

Desk : 609-998-5927 Ext (2534)

Dilshadul.k@nityo.com

Job Tags

Similar Jobs