Application Development Technical Lead Analyst - Vice President

The Senior Big Data engineering lead will play a pivotal role in designing, implementing, and optimizing large-scale data processing and analytics solutions. This role requires a visionary leader who can drive innovation, define architecture strategy, and ensure the scalability and efficiency of our big data infrastructure.

• Key Responsibilities
\n
• Lead the design and development of a robust and scalable big data architecture handling exponential data growth while maintaining high availability and resilience.
\n
• Design complex data transformation processes using Spark and other big data technologies using Scala, Pyspark
\n
• Design and implement data pipelines that ensure data quality, integrity, and availability.
\n
• Collaborate with cross-functional teams to understand business needs and translate them into technical requirements.
\n
• Evaluate and select technologies that improve data efficiency, scalability, and performance.
\n
• Oversee the deployment and management of big data tools and frameworks such as Hadoop, Spark, Kafka, and others.
\n
• Provide technical guidance and mentorship to the development team and junior architects.
\n
• Continuously assess and integrate emerging technologies and methodologies to enhance data processing capabilities.
\n
• Optimize big data frameworks, such as Hadoop, Spark, for performance improvements and reduced processing time across distributed systems.
\n
• Innovate in designing, developing, and refining data pipeline architectures to enhance data flow and ensure data processing capabilities.
\n
• Implement data governance frameworks to ensure data accuracy, consistency, and privacy across the organization, leveraging metadata management and data lineage tracking.
\n
• Conduct benchmarking and stress testing of big data solutions to validate performance standards and operational capacity.
\n
• Ensure compliance with data security best practices and regulations.
\n

Qualifications:

\n
• Bachelor's or Master’s degree in Computer Science, Information Technology, or related field.
\n
• Atleast 8-10 years overall software development experience on majorly working with handling application with large scale data volumes from ingestion, persistence and retrieval.
\n
• Deep understanding of big data technologies, including Hadoop, Spark, Kafka, Flink, NoSQL databases, etc.
\n
• Experience with Bigdata technologies Developer Hadoop , Apache Spark ,Python, PySpark
\n
• Strong programming skills in languages such as Java, Scala, or Python.
\n
• Excellent problem-solving skills with a knack for innovative solutions.
\n
• Strong communication and leadership abilities.
\n
• Proven ability to manage multiple projects simultaneously and deliver results.
\n

Preferred Skills:

\n
• Experience with data modeling and ETL/ELT processes.
\n
• Experience in moving ETL frameworks from proprietary ETL technologies like Abinitio to Apache Spark
\n
• Familiarity with machine learning and data analytics tools.
\n
• Knowledge of core banking/financial services systems
\n

------------------------------------------------------

\n
\n

Job Family Group:

\n
Technology\n

------------------------------------------------------

\n
\n

Job Family:

\n
Applications Development\n

------------------------------------------------------

\n
\n

Time Type:

\n
Full time\n

------------------------------------------------------

\n
\n

Most Relevant Skills

\n
Please see the requirements listed above.\n

------------------------------------------------------

\n
\n

Other Relevant Skills

\n
For complementary skills, please see above and/or contact the recruiter.\n

------------------------------------------------------