Qurshid Ahammed

Data Engineer

India 5+ yrs exp 90 · Outstanding

About

Result-driven Data Engineer with 4.6+ years of core experience architecture-designing and executing scalable ETL pipelines using PySpark, Python, SQL, and AWS Cloud ecosystems. Expert in performance optimization, data migration frameworks, and pipeline automation inside fast-paced Agile setups. Demonstrated competence in optimizing Spark data execution engines and managing robust cloud infrastructures. Available to start working immediately with zero notice period.

Skills & Expertise (26)

PySpark Advanced

8.5/10

4.6

Years Exp

Python Advanced

8.3/10

4.6

Years Exp

Apache Spark Advanced

8.0/10

4.6

Years Exp

AWS Glue Advanced

8.0/10

4.6

Years Exp

Amazon Redshift Advanced

8.0/10

4.6

Years Exp

Core SQL Advanced

7.5/10

4.6

Years Exp

Postgresql Advanced

7.5/10

4.6

Years Exp

DynamoDB Advanced

7.5/10

4.6

Years Exp

Git Workflow Intermediate

7.0/10

4.6

Years Exp

Jira Intermediate

7.0/10

4.6

Years Exp

Gitlab Intermediate

7.0/10

4.6

Years Exp

MySql Intermediate

7.0/10

4.6

Years Exp

Snowflake Intermediate

7.0/10

4.6

Years Exp

SQL Server Intermediate

6.5/10

4.6

Years Exp

Agile Intermediate

6.5/10

4.6

Years Exp

Oracle 11g Intermediate

6.0/10

4.6

Years Exp

IAM Redshift RDS Databricks Lambda EC2 EMR S3 Hive SQL

Work Experience

Data Engineer

Independent Consultant

Nov 2024 - Present

Architected and implemented production-grade data integration pipelines using PySpark and AWS Cloud environments to process and clean enterprise-scale analytical datasets. Built automated, serverless ETL workflows using AWS Glue and AWS Lambda, reducing daily manual ingestion work cycles by 100%. Integrated high-volume data streams across multiple corporate endpoints, streaming direct raw logs from S3 and DynamoDB targets safely into Amazon Redshift data warehouses. Designed unified automated data quality check modules and rigorous reconciliation checks, ensuring 99.9% data integrity and consistency for dashboard analytics metrics. Optimized legacy Spark jobs by refactoring active transformations and applying targeted partitioning strategies, resulting in a 30% reduction in cloud compute runtimes and operational costs. Monitored live cluster processing setups using cloud logging frameworks, actively resolving operational bottlenecks to guarantee uninterrupted processing schedules.

Data Engineer

Vtalk Solutions

Dec 2022 - Oct 2024

Designed highly scalable analytical ETL pipelines leveraging PySpark environments tightly integrated alongside core AWS cloud infrastructure endpoints. Programmed, packaged, and deployed automated data-processing application packages across AWS Glue processing jobs for massive corporate migrations. Successfully migrated structured transaction tables from relational RDS databases and S3 storage targets directly into consolidated Amazon Redshift data Warehouses. Engineered strict data validation checks and validation routines to fully eliminate data loss issues during active database server migrations. Tuned big data processing runtime steps using PySpark execution graph updates, cutting compute requirements and reducing job execution latency values. Maintained proactive operational support logs for active AWS Glue scripts, identifying cluster performance variations and resolving script blockages.

Data Engineer

Concentrix Pvt Ltd

Nov 2020 - Nov 2022

Created custom automated data validation testing matrices leveraging SQL queries and PySpark frameworks, improving pipeline data intake transparency. Executed end-to-end sandbox-to-production testing methodologies for complex data paths across dev, staging, and deployment target layers. Investigated and resolved critical production pipeline issues using Jira ticketing, meeting performance SLAs. Implemented programmatic code check routines to isolate incoming format corruptions before data hit database targets. Deployed clean, validated ETL application scripts following standard enterprise change management policies and version controls. Partnered with business analysts and product engineering stakeholders to translate reporting business requirements into performant backend tables.