Design, develop, and maintain robust, scalable, and efficient data pipelines to collect, process, and store structured and unstructured data.
Build and optimize data warehouses, data lakes, and ETL/ELT workflows.
Integrate data from multiple sources including databases, APIs, and streaming platforms.
Collaborate with data scientists and analysts to understand data requirements and deliver high-quality datasets.
Ensure data quality, integrity, and security throughout the data lifecycle.
Monitor and troubleshoot data pipeline performance and failures.
Implement data governance and compliance policies.
Automate data workflows and implement data orchestration tools (e.g., Apache Airflow).
Optimize storage and query performance in cloud and on-premises environments.
Keep up to date with emerging data engineering tools, techniques, and best practices.
