Palantir Data Engineer

Job Description\nJob Title: L2 Palantir Data Engineer\nTechnical Level: L3 Experience: 5 + years\n\nRole Overview\nWe are seeking a highly capable L2 Palantir Data Engineer to design, build, optimize, and support enterprise-grade data pipelines and governed data products on the Palantir Foundry platform.\n\nThe ideal candidate will bring strong hands-on expertise in Palantir Foundry, PySpark, Python, SQL, distributed data processing, and enterprise data engineering practices, with the ability to translate complex business requirements into scalable and reliable data solutions.\n\nThis role requires ownership of end-to-end data engineering deliverables including ingestion, transformation, validation, deployment, monitoring, and production support. The candidate should be comfortable working independently on complex engineering tasks, collaborating with architects, analysts, data scientists, and business stakeholders, and contributing to high-quality, secure, and governed enterprise data platforms.\n\nRoles and Responsibilities\nDesign, develop, and maintain scalable, reusable, and production-grade data pipelines using Palantir Foundry, PySpark, Python, and SQL.\nBuild and support ingestion frameworks for structured, semi-structured, and external enterprise data from databases, APIs, files, cloud storage, and business applications.\nPerform data transformation, cleansing, enrichment, standardization, and aggregation to create analytics-ready datasets for operational and strategic use cases.\nWork with cross-functional stakeholders to understand business requirements and convert them into robust data engineering solutions aligned with enterprise standards.\nOptimize data processing jobs for performance, scalability, reliability, and cost efficiency using partitioning, tuning, incremental processing, and efficient compute usage.\nImplement data quality checks, reconciliation logic, exception handling, health monitoring, and alerting to improve trust and stability of downstream data products.\nManage code repositories, version control, release processes, and deployment practices in line with SDLC and engineering governance standards.\nTroubleshoot production incidents, perform root cause analysis, and implement corrective and preventive actions for data pipeline issues.\nSupport metadata, lineage, governance, and documentation requirements across datasets, transformations, and operational workflows.\nContribute to continuous improvement of engineering patterns, reusable components, and platform best practices within the Palantir ecosystem.\nSkills Matrix\nMust-Have Skills\nStrong hands-on experience with Palantir Foundry in enterprise data engineering environments.\nAdvanced proficiency in PySpark, Spark SQL, Python, and SQL for large-scale data processing and transformation.\nExperience in designing and supporting ETL/ELT pipelines and modern data integration workflows.\nSolid understanding of data modeling, data warehousing, metadata, lineage, and governance concepts.\nExperience working with distributed data processing and large-volume datasets in production environments.\nStrong problem-solving capability with experience in debugging, incident handling, and production support.\nFamiliarity with Agile delivery models, version control, and structured code management practices.\nGood-to-Have Skills\nExperience with Palantir Ontology, operational workflows, and advanced Foundry components.\nExposure to API integrations, application interfaces, and external system connectivity.\nKnowledge of CI/CD, DevOps practices, and automation in data platform delivery.\nExperience with AWS, Azure, or Google Cloud data services.\nFamiliarity with data formats such as Parquet, Avro, and JSON.\nExposure to analytics, AI/ML enablement, dashboards, or operational reporting within Foundry-led environments.\nDomain experience in insurance, reinsurance, banking, or other highly regulated data-intensive industries is preferred.\nTechnical Competencies\nPalantir Foundry pipeline development, dataset management, transformation workflows, and governed data operations.\nPySpark, Spark DataFrames, Python programming, and SQL-based data engineering.\nData ingestion, ETL/ELT architecture, batch and incremental processing design.\nRelational databases, dimensional modeling, query optimization, and performance tuning.\nData quality controls, observability, logging, monitoring, and operational support processes.\nCloud platform familiarity across storage, compute, and integration services.\nVersion control systems such as Git and engineering lifecycle management practices.\nSecurity, compliance, and governance awareness for enterprise data platforms.\nProfessional Certifications\nPalantir Foundry Data Engineer training or certification.\nApache Spark or PySpark-related certification.\nPython or SQL certification relevant to data engineering practice.\nPreferred Certifications\nAWS Certified Data Engineer or relevant AWS certification.\nMicrosoft Azure Data Engineer Associate or equivalent Azure certification.\nGoogle Cloud Professional Data Engineer or equivalent GCP certification.\nCertifications in modern data engineering, DevOps, or cloud data platforms.\nPreferred Qualifications\nBachelor’s or Master’s degree in Computer Science, Information Technology, Data Engineering, Software Engineering, or a related discipline.\n10+ years of overall experience in data engineering, big data, or enterprise data platform implementation, with strong recent hands-on work in Palantir Foundry.\nProven experience delivering production-grade data pipelines and governed data solutions in large enterprise environments.\nAbility to work independently on complex assignments while collaborating effectively across technical and business teams.\nStrong written and verbal communication skills, with the ability to document technical solutions clearly and engage stakeholders confidently.\n\nIdeal Candidate Profile\nThe ideal candidate is a senior data engineering professional with deep Palantir Foundry expertise and strong command over PySpark, Python, SQL, and enterprise data engineering practices. This person demonstrates ownership, production support maturity, sound engineering discipline, and the ability to build scalable, secure, and analytics-ready data products in complex business environments.\nThe candidate should be comfortable handling end-to-end data engineering responsibilities, from requirements analysis through deployment and support, while maintaining quality, governance, and operational excellence.