We are seeking a senior data engineer to design, build, and maintain production-grade data pipelines and lakehouse solutions in an on-premise environment, while driving data platform initiatives, performance, security and technical best practices.

Mandatory Skill(s)

• 8+ years of IT experience with 5+ years in data engineering/data pipelines;
• Expert SQL/SQL Server – query optimization, indexing & performance tuning;
• Advanced Python & PySpark;
• Apache Spark and distributed data processing;
• Kubernetes & Docker for on-premise deployments;
• Data modelling – relational & NoSQL; hands-on MongoDB/Cassandra;
• Lakehouse/Data Lake architecture, including iceberg, partitioning & metadata;
• ETL/ELT orchestration – Apache Airflow or similar;
• CI/CD, Git, automated testing & IaC;
• Data security, governance, RBAC, masking & compliance.

Desirable Skill(s)

• Experience in data virtualization, logical data warehouses, Data Mesh/Fabric architectures, metadata management, and data lineage;
• Familiarity with real-time streaming technologies such as Kafka/Flink, along with experience in technical leadership, mentoring, and Agile data architecture. 

Responsibilities

• Design and develop production-grade ETL/ELT pipelines using Python/PySpark;
• Build and manage multi-layer lakehouse architecture across raw, curated, and consumption layers;
• Deploy scalable data pipelines using Kubernetes/Docker in on-premise environments;
• Develop data models and optimize complex SQL Server queries and performance;
• Establish CI/CD, automated testing, monitoring, and version-control practices;
• Implement data security, governance, and access controls;
• Mentor engineers, conduct code reviews, and drive engineering best practices;
• Collaborate with data scientists, BAs, and stakeholders to deliver business-ready datasets;
• Provide L3 support and lead resolution of complex data engineering challenges.

If you are interested in this role, click on the “Apply to this job” button below or you could also write in with your CV to Roshni Mishra at roshni.m@sciente.com quoting the job title.