Databricks Engineer.
Design, build, and maintain scalable data pipelines using Apache Spark on Databricks.
Develop ETL/ELT workflows to ingest, process, and transform large volumes of structured and unstructured data.
Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and deliver high-quality datasets.
Optimize and tune Spark jobs and data flows for performance and cost efficiency.
Implement data governance, quality checks, and error handling within pipelines.
Use Databricks notebooks for data exploration, transformation, and operational workflows.
Integrate Databricks with other data platforms like Delta Lake, Azure Data Lake, S3, Synapse, or Snowflake.
Maintain version control and CI/CD pipelines for Databricks workflows using tools like Git, Azure DevOps, or Jenkins.
Monitor and troubleshoot data pipelines, ensuring reliability and availability.
