Senior Data Engineer ( Microsoft Fabric & Azure)

Job Title: Senior Data Engineer – Microsoft Fabric & Azure

Location: Chicago, Illinois (Hybrid – 2 days/week onsite)

Job Summary

STAFFXPERT LLC is seeking a Senior Data Engineer – Microsoft Fabric & Azure on behalf of our client in Chicago, Illinois. We are looking for a highly skilled data engineering professional with deep expertise in Microsoft Fabric and the Azure Data Platform to design, build, and optimize scalable enterprise data solutions. This role is ideal for a candidate passionate about modern data architecture, cloud-native engineering, governance, and large-scale analytics environments.

The ideal candidate will bring hands-on experience across Microsoft Fabric workloads, Azure-based data services, large-scale data processing, and modern data governance frameworks to support enterprise-level analytics and AI initiatives.

Key Responsibilities

• Design and develop end-to-end enterprise data platforms using Microsoft Fabric, including:
• Lakehouse
• Warehouse
• OneLake
• Dataflows Gen2
• Build, optimize, and maintain scalable Spark-based workloads using PySpark and SparkSQL for high-volume data processing.
• Develop and support MLOps pipelines and feature engineering frameworks to enable advanced analytics and AI use cases.
• Design and implement scalable ETL/ELT pipelines using:
• Azure Data Factory (ADF)
• Microsoft Fabric Data Pipelines
• Dataflow Gen2
• SSIS (On-Premises, Azure-SSIS Integration Runtime, and migration scenarios)
• Implement modern enterprise data modeling strategies, including:
• Medallion Architecture (Bronze, Silver, Gold)
• Dimensional Modeling (Star and Snowflake schemas)
• Process and manage multiple structured and semi-structured data formats such as Parquet, JSON, and XML.
• Implement and manage enterprise data governance solutions using Microsoft Purview, including:
• Data cataloging and classification
• Automated lineage across Azure and Fabric ecosystems
• Business glossary management
• Enforce enterprise-grade security and governance controls, including:
• Role-Based Access Control (RBAC)
• Column-level security
• Data masking policies
• Governance alignment across Fabric and Purview
• Optimize performance, scalability, reliability, and cost efficiency across enterprise data environments.
• Implement CI/CD pipelines and Infrastructure as Code (IaC) for data engineering and governance assets.
• Collaborate cross-functionally with Security, Compliance, Analytics, and BI teams to deliver trusted and governed data solutions.

Required Skills & Experience

< data-end="2769" data-start="2735">Microsoft Fabric (Required)

• Strong hands-on experience with Microsoft Fabric Data Engineering workloads
• Expertise with:
• Lakehouse and Warehouse architectures
• OneLake ecosystem
• Fabric pipelines and notebooks
• Capacity planning and performance optimization

< data-end="3062" data-start="3031">Data Engineering & Spark

• Advanced expertise in PySpark, including:
• Joins, window functions, and UDFs
• Partitioning strategies, shuffle optimization, and caching
• Strong SparkSQL experience
• Experience processing large-scale datasets (TB )

< data-end="3335" data-start="3309">Azure Data Platform

• Azure SQL Database / SQL Server
• Azure Data Factory (ADF)
• SSIS / Azure-SSIS Integration Runtime
• Azure Data Lake Storage (ADLS Gen2)

< data-end="3515" data-start="3482">Data Governance & Security

• Strong experience with Microsoft Purview, including:
• Data cataloging and scanning
• Automated data lineage across Fabric, ADF, SQL, and ADLS
• Business glossary management
• Experience integrating governance frameworks with enterprise security policies and Azure RBAC

Preferred Qualifications

• Strong experience working within enterprise-scale Azure data ecosystems
• Expertise in modern data architecture, governance, and performance optimization
• Experience supporting scalable analytics and AI-driven initiatives