Azure Administration, ADLS and Data bricks Administration

NDIA

Detailed JD (Roles and Responsibilities)

Proficiency with Azure Portal, Azure PowerShell, and Azure CLI.

Knowledge of core Azure services, including compute, storage, and networking.

Understanding of security best practices for cloud environments.

Experience with monitoring and logging tools like Log Analytics.

Familiarity with backup and disaster recovery concepts and tools like Azure Site Recovery.

Ability to script and automate tasks.

Strong problem-solving and troubleshooting skills.

Creating, configuring, and managing ADLS Gen2 storage accounts, including hierarchical namespace enablement

Assigning roles and permissions (e.g., Storage Blob Data Contributor, Storage Blob Data Reader) to users, groups, and service principals for granular access to containers and files within ADLS.

Implementing policies for data retention, tiering (hot, cool, archive), and deletion to optimize storage costs.

Configuring features like soft delete, versioning, and replication (ZRS, GRS) for data resilience and disaster recovery.

Setting up diagnostic logging and monitoring to track storage usage, access patterns, and potential security issues.

Adding users and groups, assigning account admin roles, and managing their access to workspaces and Unity Catalog.

Creating and managing metastores, external locations, and storage credentials to govern data access in ADLS.

Creating and managing Databricks workspaces within the Azure portal, including selecting the appropriate pricing tier.

Creating, configuring, and managing Databricks clusters, including enabling Photon and Unity Catalog integration.

Managing permissions for workspace objects like notebooks, jobs, and SQL Warehouses.

Creating and managing service principals for programmatic access to Databricks and integrated Azure resources.

Creating an Access Connector for Azure Databricks, assigning it the necessary roles on the ADLS storage account, and then creating a storage credential and external location within Databricks' Unity Catalog.

Utilizing Unity Catalog to centralize data governance for ADLS data, enabling fine-grained access control and simplified data sharing across Databricks workspaces.

Extensive experience in Snowflake administration, performance tuning, and optimization.

Strong knowledge of Snowflake architecture, components, and features.

Proficiency in SQL.

Experience with cloud platforms (e.g., Azure, AWS) is often required.

Familiarity with ETL processes and tools.

Scripting skills in Python or Shell are often preferred.

Understanding of data security principles and data governance.

Experience with CI/CD tools is a plus.

Strong analytical and troubleshooting skills.

Total Experience

8-10 YEARS