Development environment (Python/PySpark, AWS & amp; Databricks)
experience in a DevOps CI/CD environment
experience in data integration and pipeline development.
Experience with AWS Cloud on data integration with a mix of Apache Spark, Glue, Kafka, Kinesis, and Lambda in S3 Redshift, RDS, MongoDB/DynamoDB ecosystems Databricks, Redshift experience is a major plus.
experience using SQL in related development of data warehouse projects/applications (Oracle & amp; SQL Server)
Strong real-life experience in python development especially in PySpark in AWS Cloud environment.
Design, develop test, deploy, maintain and improve data integration pipeline.
