Algotale- GCP Data engineer
Job Description
We are seeking a highly skilled Data Engineer / Integration Engineer to join our team. The ideal candidate will have expertise in Python, workflow orchestration, cloud platforms (GCP/Google BigQuery), big data frameworks (Apache Spark or similar), API integration, and Oracle EBS. The role involves designing, developing, and maintaining scalable data pipelines, integrating various systems, and ensuring data quality and consistency across platforms. Knowledge of Ascend.io is a plus.
\nKey Responsibilities:
\nDesign, build, and maintain scalable data pipelines and workflows.
\nDevelop and optimize ETL/ELT processes using Python and workflow automation tools.
\nImplement and manage data integration between various systems, including APIs and Oracle EBS.
\nWork with Google Cloud Platform (GCP) or Google BigQuery (GBQ) for data storage, processing, and analytics.
\nUtilize Apache Spark or similar big data frameworks for efficient data processing.
\nDevelop robust API integrations for seamless data exchange between applications.
\nEnsure data accuracy, consistency, and security across all systems.
\nMonitor and troubleshoot data pipelines, identifying and resolving performance issues.
\nCollaborate with data analysts, engineers, and business teams to align data solutions with business goals.
\nDocument data workflows, processes, and best practices for future reference.
\nRequired Skills & Qualifications:
\nStrong proficiency in Python for data engineering and workflow automation.
\nExperience with workflow orchestration tools (e.g., Apache Airflow, Prefect, or similar).
\nHands-on experience with Google Cloud Platform (GCP) or Google BigQuery (GBQ).
\nExpertise in big data processing frameworks, such as Apache Spark.
\nExperience with API integrations (REST, SOAP, GraphQL) and handling
\nstructured/unstructured data.
\nStrong problem-solving skills and ability to optimize data pipelines for performance.
\nExperience working in an agile environment with CI/CD processes.
\nStrong communication and collaboration skills.
\nPreferred Skills & Nice-to-Have:
\nExperience with Ascend.io platform for data pipeline automation.
\nKnowledge of SQL and NoSQL databases.
\nFamiliarity with Docker and Kubernetes for containerized workloads.
\nExposure to machine learning workflows is a plus.
\nWhy Join Us?
\nOpportunity to work on cutting-edge data engineering projects.
\nCollaborative and dynamic work environment.
\nCompetitive compensation and benefits.
\nProfessional growth opportunities with exposure to the latest technologies.
\nHow to Apply:
\nInterested candidates can apply by sending their resume to (your email/contact).