IND_Developer

Job Title: Data Engineer – PySpark & Palantir Foundry

Location: Hyderabad, India
Experience: 5+ Years

About the Role

We are looking for a skilled and motivated Data Engineer with strong expertise in PySpark, Python, and Palantir Foundry to join our growing data engineering team. The ideal candidate will have hands-on experience building scalable data pipelines, integrating diverse data sources, and delivering high-quality data solutions to support analytics and business decision-making.

Key Responsibilities

• Design, develop, and maintain scalable and reliable data pipelines using PySpark.
• Build and manage data ingestion, transformation, and integration processes within Palantir Foundry.
• Perform data cleansing, validation, transformation, and aggregation to ensure high-quality datasets.
• Collaborate with data scientists, analysts, and business stakeholders to understand requirements and deliver data solutions.
• Optimize data processing workflows for performance, scalability, and cost efficiency.
• Develop and maintain reusable data engineering frameworks and best practices.
• Monitor, troubleshoot, and resolve data pipeline issues in a timely manner.
• Create and maintain technical documentation for data pipelines, workflows, and processes.
• Ensure adherence to data governance, security, and quality standards.

Required Skills & Qualifications

• 5+ years of experience in Data Engineering or related roles.
• Strong hands-on experience with PySpark and Python.
• Experience working with Palantir Foundry (Transforms, Code Repositories, Pipelines, Data Lineage, etc.).
• Strong proficiency in SQL and relational databases.
• Experience designing and implementing ETL/ELT pipelines.
• Good understanding of data warehousing concepts and data modeling.
• Familiarity with cloud platforms such as AWS, Azure, or GCP and their data services.
• Experience with large-scale data processing and optimization techniques.
• Strong problem-solving and analytical skills.
• Excellent communication and stakeholder management abilities.

Preferred Qualifications

• Experience working in Agile development environments.
• Knowledge of data governance and data quality frameworks.
• Exposure to CI/CD practices for data engineering workflows.
• Experience supporting analytics, reporting, or machine learning use cases.