Location: Remote
Experience: 8+ years in Data Engineering, with significant experience in AWS Glue and a minimum of 3+ years in a lead role
***Immediate joiners only***
About the Role
We are seeking a highly skilled and experienced AWS Glue Lead to join our dynamic data team. The ideal candidate will be a hands-on technical expert with proven leadership capabilities, responsible for architecting, developing, and managing our data pipelines and ETL processes entirely within the AWS ecosystem. This is a crucial role for individuals who thrive in a fast-paced environment and can hit the ground running immediately.
Key Responsibilities
- Technical Leadership: Lead and mentor a team of data engineers, providing technical guidance, conducting code reviews, and ensuring adherence to engineering best practices and coding standards.
- Architecture & Design: Design and architect scalable, reliable, and secure data lake and data warehouse solutions on AWS, primarily using AWS Glue, S3, Athena, and other relevant AWS services.
- ETL Development & Management: Oversee the development, implementation, and maintenance of robust and automated ETL pipelines using AWS Glue, PySpark, and Python.
- Performance Optimization: Monitor, troubleshoot, and optimize existing data pipelines and ETL jobs for maximum performance and cost efficiency.
- Stakeholder Collaboration: Collaborate closely with business analysts, data scientists, and cross-functional teams to translate complex business requirements into effective technical data solutions.
- Data Governance & Quality: Implement and enforce data quality, security, and governance standards using tools like AWS Glue Data Catalog, AWS Lake Formation, and AWS Glue Data Quality.
- Project Management: Manage project timelines, resources, and deliverables, ensuring timely and successful project completion.
- Documentation: Ensure comprehensive documentation of data workflows, processes, and systems is created and maintained.
Required Qualifications & Skills
- Experience:
- Minimum of 8+ years of total experience in data engineering or a related field.
- Minimum of 3+ years of experience in a technical lead or team lead capacity.
- Extensive, hands-on experience with AWS Glue (ETL, Data Catalog, DataBrew, etc.).
- Technical Proficiency:
- Expert-level proficiency in PySpark, Python, and SQL.
- Deep understanding of the AWS ecosystem: S3, IAM, Lambda, Athena, Redshift, etc.
- Strong knowledge of ETL concepts, data warehousing principles, and data modeling techniques.
- Familiarity with data formats such as Parquet, ORC, Avro, and JSON.
- Leadership Skills:
- Excellent leadership, team management, and mentoring skills.
- Strong communication and interpersonal skills, with the ability to articulate technical concepts to non-technical stakeholders.
- Exceptional problem-solving and analytical abilities.
- Availability:
- Must be available to start immediately upon offer acceptance.
Preferred Qualifications
- AWS Certified Data Analytics – Specialty or AWS Certified Solutions Architect certification.
- Experience with Infrastructure as Code tools (e.g., Terraform or CloudFormation).
- Knowledge of CI/CD pipelines and DevOps practices.
- Prior experience working in a fully remote environment.
Posts
