About
Data Engineer with hands on experience of building scalable data pipelines and cloud-based data solutions using Azure Databricks, PySpark, and Delta Lake. Experienced in Data Warehousing, Data Modeling, and end-to-end ETL/ELT pipelines, with hands-on knowledge of Medallion Architecture and Lakehouse design. Focused on developing reliable, efficient, and analytics-ready data solutions.
Skills & Expertise (21)
Work Experience
Software Engineer Level 2
Hexaware Technologies
Dec 2024 - Present
Managed 15+ production Databricks pipelines processing 50GB+ daily data across Bronze and Silver layers in a Medallion Architecture. Built data quality validation scripts in SQL and PySpark across 1M+ records (null checks, duplicate checks, source-target row-count match) to ensure data integrity. Diagnosed and resolved 50+ Spark job failures via log analysis (out-of-memory, data skew, schema mismatch), maintaining 99% pipeline SLA. Optimized Delta Lake tables using OPTIMIZE (with Z-ORDER) and VACUUM on a 2TB+ warehouse, improving query performance and reducing storage cost. Supported schema modifications on 5+ core tables (column additions, data-type changes) under senior guidance, using Delta schema evolution. Implemented incremental data processing and transformation logic in Databricks using Auto Loader and Spark Streaming Dataframe to efficiently process large datasets and reduce unnecessary data reprocessing.
Education
B.Tech in Computer Science and Engineering - Graphic Era Deemed to be University
2020 - 2024 · Afghanistan
CBSE Class XII - Scottish International School
2018 - 2019 · Afghanistan
ICSE Class X - St. Francis School
2017 - 2018 · Afghanistan
Certifications
No certifications added yet
Interested in this developer?
Profile Score Breakdown
Profile Overview
Availability Details
Visa Status
Need Sponsorship
Relocation
Open to Relocation
Skills (21)
Click a skill to find developers with the same skill
