Data Engineering - EY001

Openings*

Data Engineering

ECMS RQ# *

Duration of contract*

6 months

Total Yrs. of Experience*

6-8 years

Relevant Yrs. of experience*

5-6 years

Detailed JD *(Roles and Responsibilities)

Job Description:

We are looking for a skilled Sr. Data Engineer to join our team. A Data Engineer is a professional who designs, constructs, tests, and maintains architectures such as databases and large-scale processing systems. They are responsible for developing and translating computer algorithms into prototype code and maintaining, organizing, and identifying trends in large datasets. Data engineers essentially Client the groundwork for data analysis and systems that Data Scientists use for their analytics and business intelligence work. They manage and optimize data pipelines, create secure databases, and handle data cleansing.

Responsibilities:

• 1. Design, construct, install, test, and maintain highly scalable data management systems.
• 2. Develop and implement data architecture and database designs.
• 3. Build large-scale data processing systems.
• 4. Ensure systems meet business requirements and industry practices.
• 5. Integrate new data management technologies and software engineering tools into existing structures.
• 6. Collaborate with data scientists and architects on several projects.
• Data Engineers must have strong technical skills including a deep understanding of database structure, data modeling, and computer science fundamentals. Common knowledge and skills often associate with languages like Python, Java, Scala; and technologies like Hadoop, SQL, and MongoDB.

Requirements:

• Experience in Databricks, SQL, PySpark, Spark, Python, and Azure Data Factory (ADF).
• The ideal candidate should have experience in Fabric Data Layer Engineering Lead ,PowerBI, ETL, SSMS, AWS, Power BI Report Server, Azure Synapse, Spotfire, Azure Data Bricks, Azure Data Factory, HIVE, Hadoop, NIFI, Kafka, Python 3.x, API Development, Embedded Systems, Tokenization, Search Index Configuration, Open AI Integration.
• SQL DBA, Stored procedure, table trigger statements, bulk update/archival jobs.
• Must have strong Python, Databricks and Pyspark.
• In depth knowledge in SQL development including writing complex stored procedure, Performance tuning.
• Require SQL expertise to analyze PBI code structure- removal of garbage and optimizing query.
• Good Knowledge on SSIS

Experience in developing and maintaining data security policies and procedures

Mandatory skills*

Python, PySpark, Apache Airflow, Databricks, Azure Data Factory, SQL

Domain*

Services

Vendor billing rate*

8000 INR / Day (negotiable)

Work Location*

Bangalore, Pune, Hyderabad preferred (All offshore locations) // WFO

Background check process to be followed: *

After onboarding

Mode of Interview: Telephonic/Face to Face/Skype Interview*

Telephonic /Skype Interview