Databricks Pyspark

Req ID- 10290307

Job Title: Developer

Work Location: ~PUNE~BANGALORE~

Skill Required: Digital : Databricks~Digital : PySpark

Experience Range : 6-8Years

Job Description:
Job Description for Data Engineer:Design and develop ETL processes based on functional and non-functional requirements in Databricks, python / PySpark within Azure platform.Understand the full end to end development activities from design to go live for ETL development in Azure platform.Recommend and execute improvements.Execute and provide support during testing cycles and post-production deployment, engage in peer code reviews.Apply automation and innovation on new and on-going data platforms that aligned to business strategies.Document component design for developers and for broader communication.Understand and adopt an Agile (SCRUM like) software development mindset.Follow established processes/standards, business technology architecture for development, release management and deployment process.Design, develop and implement reporting platforms and complex ETL frameworks that meet business requirements.Provide data analysis and requirements within enterprise platform.Develop, maintain knowledge of data available from upstream sources and data within various platforms.
Essential Skills:
Must-have skillsAzure Databricks, Azure Data FactoryPython / Spark / Pyspark / Shell ScriptSynapse5+ years of experience in ETL/Data area with minimum of 3 years of development experience in above technologies (PySPark, ADB, ADF) is mustSQL, Working experience with data modeling, relational modeling and dimensional modeling, Data Warehousing, ETLGeneral knowledge about file formats (e.g. XML, CSV, JSON), databases (e.g. MS SQL, Oracle) and different type of connectivity is also very useful.Data files movement via mailboxSource-code versioning/promotion tools, e.g. Git/JenkinsGood communication skill, team player