nikhil puthuru
Azure Data Engineer
About
Azure Data Engineer with 4+ years of IT experience with (3.9 years relevant) specializing in ETL/ELT pipeline development, Data Lakehouse Architectures, and Cloud Data solutions using Azure Data Factory (ADF), Azure Databricks (PySpark/Spark SQL), Delta Lake, ADLS Gen2, Synapse Analytics, and Azure SQL. Proven expertise in metadata-driven frameworks, schema evolution, CI/CD automation (GitHub Actions), RBAC, & Key Vault-based security. Skilled in integrating structured, semi-structured, and streaming data from APIs, FTP, SaaS, & on-prem systems. Improved performance optimization with optimization techniques (partitioning, Z-Ordering, caching, indexing), Ensuring data quality, compliance (HIPAA, GDPR, PCI), and delivering scalable, secure, & reliable data workflows across enterprise environments. Experienced in Agile delivery, cross-functional collaboration, & enabling analytics & BI integration (Power BI, Synapse) to drive business insights. Additionally, experienced in delivering end-to-end data ingestion and transformation workflows integrating Generative AI (Azure OpenAI, Crew AI) into data engineering pipelines, and translating business requirements into scalable, high-performance data solutions while ensuring security and compliance.
Skills & Expertise (39)
Work Experience
Data Engineer
AAYS Advisory Private Limited (NexgAI)
Jan 2026 - Aug 2026
Developed an end-to-end Python-based data validation pipeline to ingest corporate hierarchy data from Excel, perform data cleansing, and transform into structured CSV for AI-powered analysis. Integrated Azure OpenAI (GPT-5.0) with Crew AI to analyse corporate parent-child relationships and generate confidence scores (Moody_HM_PERC, AIS_PERC) for hierarchy validation. Implemented Bing Search API integration to gather real-time evidence for corporate relationship verification, feeding results into AI models for automated analysis and classification. Designed SQLite caching and checkpoint mechanisms to ensure fault tolerance and resume-on-failure capability, handling large-scale datasets reliably. Built output generation logic producing enriched CSV files with validation statuses (CONFIRMED, NO_RELATIONSHIP, PARTIAL) alongside confidence scores for business reporting. Performed continuous data quality checks and resolved data ingestion, schema, and validation issues to ensure pipeline reliability. Managed RBAC and encryption policies to ensure data security and compliance for enterprise-level client data.
Data Engineer
LogIQ Labs Pvt Ltd
Dec 2024 - Sep 2025
Designed and maintained scalable Azure Data Factory (ADF) and Databricks pipelines to enable secure, high-volume data movement across business units. Built and optimized Delta Lake tables using partitioning, indexing, and caching to improve query performance and analytical workloads. Acted as module lead for customer data ingestion (ADF), metadata-driven transformations (Databricks), and CI/CD deployment components, owning end-to-end delivery and supporting junior engineers. Implemented CI/CD pipelines for ADF and Databricks using GitHub Actions/Azure DevOps, automating deployments across Dev/QA/Prod. Integrated on-prem, cloud, API, and streaming sources using ADF, Event Hubs, and Logic Apps.
Data Engineer
Softalk Technologies Limited
Nov 2021 - Nov 2024
Built end-to-end ETL pipelines using ADF, Databricks, and Azure SQL to process structured and semi-structured customer datasets. Improved SQL & Delta Lake performance using partitioning, Z-Ordering, indexing, and caching techniques. Created ETL/ELT processes to ingest retail, e-commerce, inventory, and customer-level datasets. Automated ADF & Databricks deployments using Azure DevOps CI/CD, improving operational efficiency. Supported Azure Synapse workflows and integrated data with Power BI dashboards for improved reporting.
Associate Data Engineer (Sr. Software Engineer)
Softalk Technologies Limited
Nov 2021 - Nov 2024
Built ETL pipelines in ADF to automate ingestion between Azure Blob and Azure SQL, processing CSV, JSON, and Parquet formats with optimized ingestion workflows. Designed secure workflows using Azure Logic Apps and implemented Key Vault-based data handling, adhering to healthcare compliance standards. Developed PySpark notebooks to process structured and semi-structured clinical data in Databricks. Integrated Application Insights for monitoring performance, errors, and request metrics, and created SQL stored procedures to support analytics and clinical reporting.
Education
B. Tech - Mechanical Engineering - SVCN College of Engg & Tech
- 2017 · Afghanistan
Certifications
No certifications added yet
Interested in this developer?
Profile Score Breakdown
Profile Overview
Skills (39)
Click a skill to find developers with the same skill
