We are seeking a competent Big Data Developer to join data engineering team of a large Financial institution. The successful candidate will be responsible for designing, building, and maintaining big data pipelines and platforms that support critical business and analytics initiatives. This role requires a solid foundation in big data technologies, programming, and data management principles-along with the discipline to write clean, testable, and secure code within a CI/CD environment. Key Responsibilities Design, develop, and optimise big data processing workflows using HDFS, Trino, and S3 bucket storage.
Write maintainable and scalable Python code for data ingestion, transformation, and validation.
Collaborate with data architects and data scientists to implement robust data models and data designs.
Develop and optimise SQL queries across multiple database platforms including MSSQL, Oracle, and PostgreSQL.
Build and maintain CI/CD pipelines, incorporating code vulnerabili...