Senior Software Architect Cloud Data Platforms & Databricks

Senior Software Architect – Cloud Data Platforms & Databricks
Location
Remote / Hybrid (depending on project requirements)
Overview
We are seeking an experienced Senior Software Architect with deep expertise in Cloud Data Platforms, Databricks, Azure, and AWS to lead the evolution of a large-scale Enterprise Data Platform.
This role will be responsible for defining architecture standards, enabling platform capabilities, and guiding engineering teams in the design and adoption of a secure, scalable, and cost-efficient Databricks Lakehouse architecture across Azure (primary cloud) and AWS (secondary cloud).
The successful candidate will partner closely with engineering, networking, security, analytics, and business teams to deliver modern data solutions supporting analytics, reporting, governance, AI/ML initiatives, and both batch and streaming workloads.Key Responsibilities
Cloud & Data Platform Architecture

• Lead architecture and feature enablement initiatives for an enterprise-scale Databricks Lakehouse platform.
• Define reference architectures, standards, and best practices for Data Lakes, Lakehouse architectures, analytics platforms, and cloud-native data solutions.
• Drive platform performance optimization, scalability improvements, and cloud cost management initiatives.
• Provide technical leadership and architectural guidance across multiple delivery teams.

Security & Networking

• Partner with security and networking teams to design secure cloud connectivity models, including:
  • Private Endpoints / Private Links
  • VNet and VPC Integration
  • Routing and Firewall Strategies
  • Network Segmentation
• Ensure solutions comply with enterprise security, governance, and regulatory requirements.

Governance & Data Management

• Define and promote best practices around:
  • Data Modeling
  • Metadata Management
  • Data Lineage
  • Data Governance
  • Data Quality
  • Master Data Management (MDM)
• Support architectural governance activities, design reviews, and technology decision processes.

Analytics & Business Intelligence

• Collaborate with analytics teams to enable scalable reporting and business intelligence solutions.
• Support integration with Power BI, Tableau, and other reporting platforms.
• Design architectures that facilitate data-driven decision making across the organization.

Technical Leadership

• Mentor Solution Architects, Data Engineers, and development teams.
• Review architectural designs and ensure alignment with enterprise standards.
• Stay current with emerging technologies in cloud computing, data engineering, governance, analytics, and AI/ML.

Required Qualifications

• Bachelor's or Master's Degree in Computer Science, Information Technology, Engineering, or a related discipline.
• 8+ years of experience in Software Engineering, Cloud Architecture, or Enterprise Data Platform Architecture.
• Proven experience delivering large-scale enterprise data platforms within highly governed environments.
• Strong stakeholder management experience across global organizations.

Required Technical Skills
Must Have

• Databricks Architecture (expert level)
• Apache Spark
• Delta Lake
• Enterprise Data Platform Design
• Data Lake / Lakehouse Architectures
• Azure Cloud Architecture
• ETL / ELT Design and Optimization
• Distributed Data Processing

Strongly Preferred

• AWS Architecture and Data Services
• Azure Networking and Security Architecture
• Private Endpoints / Private Links
• VNet Integration
• Cloud Security Patterns
• Power BI and/or Tableau
• Azure Data Factory
• AWS Glue
• Kafka
• Fivetran

Data Governance

• Data Catalogs
• Data Lineage
• Data Quality Frameworks
• Metadata Management
• Master Data Management (MDM)

Preferred Experience

• Real-time Data Streaming Architectures
• Event-Driven Architectures
• Microsoft Fabric
• Snowflake
• Amazon Redshift
• Containerization (Docker, Kubernetes)
• CI/CD Pipelines
• AI/ML Enablement Platforms

Certifications (Preferred)

• Microsoft Azure Certifications
• AWS Certifications
• Databricks Certifications