Hiring for PySpark_Bangalore
Job Description:
1. PySpark Capable of writing efficient data transformation logic using PySpark.
2. Delta Lake Management Experienced in using Delta Lake for ACID-compliant data storage and versioning.
3. Databricks Notebooks Proficient in developing, testing, and collaborating through Databricks notebooks.
4. SQL Strong in writing complex SQL queries to analyze and manipulate large datasets.
5. Data Pipeline Orchestration Familiar with scheduling and automating workflows using Databricks Jobs.
6. Cloud Platform Integration Experienced in integrating Databricks with AWS for data ingestion and storage.
7. CICD in Databricks Knowledge of DevOps practices and implementing CICD pipelines in a Databricks environment.
8. Data Governance Security Understands data access control, encryption, and compliance within Databricks.
9. PowerBI Ability to create finance reports in powerBI
