Description:
Skill requirements in order of preference
1. 5 (min.) - 8 years in data engineering, of which at least 3+ years on Azure.
2. 5+ years of development experience across ADF, Databricks, Python.
3. Azure Data Factory: Pipelines, data flows (mapping/wrangling), triggers, integration runtimes, parameterization, error handling & retries.
4. Azure Databricks: Databricks Notebooks, Jobs/Workflows, PySpark/Scala, Delta Lake, cluster configuration and optimization.
5. Programming: Advanced Python or Scala (at least one), strong coding standards, modularization, and unit testing (e.g., pytest)
6. Practical experience with Scrum/Kanban; story slicing, Azure DevOps
7. ETRM skills are desirable
