Lead Data Engineer (This Job Only for U.S. Citizens)

Job Title: Lead Data Governance Engineer / Azure Data Engineer (Microsoft Purview)

Location: Remote / Hybrid (USA)

Duration: Long-Term Contract

Note: Azure Data Factory/pipelines, REST API, Azure Data Lake, and Snowflake

Job Summary

We are seeking an experienced Data Governance Engineer with strong expertise in Microsoft Purview and Azure data platforms. The ideal candidate will be responsible for designing and implementing end-to-end data governance, cataloging, and data quality frameworks, along with building scalable data ingestion pipelines integrating multiple marketing platforms.

This role requires a strong blend of data engineering, governance, metadata management, and cloud automation experience.

Key Responsibilities

• Architect and implement end-to-end data ingestion pipelines integrating marketing data from multiple vendor APIs (Google Ads, Bing Ads, Salesforce, Taboola, Yahoo, etc.) for real-time analytics.
• Design and manage centralized data cataloging using Microsoft Purview across Azure and Snowflake environments.
• Implement data classification strategies using built-in and custom classifiers (PII, financial data) to ensure compliance.
• Build and maintain business glossary and metadata management frameworks, aligning business and technical definitions.
• Develop and enforce data quality frameworks, including profiling, anomaly detection, and validation rules.
• Implement threshold-based monitoring and alerting mechanisms for critical datasets.
• Automate governance workflows using Purview workflows and Azure services to reduce manual processes.
• Configure and manage automated data scanning pipelines using Purview scanners.
• Enable end-to-end data lineage tracking for impact analysis and data transparency.
• Integrate governance solutions with modern data platforms and automate metadata capture using pipelines and workflows.

Required Skills

Core Data Engineering & Integration

• Strong experience with Azure Data Factory/pipelines
• Experience with REST API integrations (JSON, OAuth)
• Data ingestion from third-party platforms (Google Ads, Salesforce, etc.)
• Hands-on experience with Azure Data Lake and Snowflake

Microsoft Purview (Must Have)

• Expertise in data cataloging and discovery
• Data lineage tracking and impact analysis
• Data classification (PII, sensitive data)
• Business glossary and metadata management
• Data quality implementation within Purview
• Purview scanners and automated data discovery
• Governance workflow automation

Cloud & Data Platforms

• Strong experience with the Microsoft Azure ecosystem
• Exposure to Microsoft Fabric (preferred)
• Experience with Snowflake
• Knowledge of Azure Synapse Analytics

Data Governance & Quality

• Strong understanding of data governance frameworks
• Metadata management and data stewardship practices
• Data quality monitoring and validation techniques
• Compliance standards (PII, GDPR, etc.)

Programming & Tools

• Strong SQL skills
• Python for data processing and automation
• Experience with CI/CD tools (Azure DevOps, Git)
• Automation using Logic Apps / Power Automate

Nice to Have

• Experience with Databricks
• Reporting/analytics exposure using Power BI
• Understanding of Data Mesh architecture

Experience Required

• 10+ years of overall IT experience
• 5–7 years of hands-on experience with Microsoft Purview
• Experience working with enterprise data governance initiatives

Soft Skills

• Strong communication and stakeholder management skills
• Ability to translate business requirements into technical solutions
• Experience working in Agile environments