| Roles and Responsibilities |
Design, build, and maintain robust and scalable ETL/ELT data pipelines on GCP. Work with tools like Cloud Dataflow, BigQuery, Pub/Sub, Dataform, Cloud Composer (Airflow), and Cloud Storage. Develop and optimize data ingestion workflows from various structured and unstructured sources. Implement data transformation logic using SQL, Python, or Apache Beam. Monitor, troubleshoot, and enhance data pipelines to ensure high availability and performance. Collaborate with data analysts, scientists, and business stakeholders to understand data needs and deliver effective solutions. Ensure data quality, integrity, and governance practices are followed across pipelines. Participate in code reviews and promote best practices in data engineering and cloud architecture. Contribute to the design and development of a modern, scalable, and secure cloud data platform. |
