Senior Platform Engineer (Azure):
• Proficient in distributed system architectures and Azure Data Engineering services like Event Hub, Data Factory, ADLS Gen2, Cosmos DB, Synapse, Databricks, APIM, Function App , Logic App, and App Services
• Implement, and manage infrastructure using IaC tools such as Azure Resource Manager (ARM) templates and Terraform.
• Manage containerized applications using Docker and orchestrate them with Azure Kubernetes Service (AKS).
• Set up and manage monitoring, logging, and alerting systems using Azure Monitor, Log Analytics, and Application Insights.
• Implement disaster recovery (DR) strategies, backups, and failover mechanisms for critical workloads.
• Automate infrastructure provisioning, scaling, and management for high availability and efficiency.
• Experienced in managing and maintaining clusters across Development, Test, Preproduction, and Production environments on Azure.
• Skilled in defining, scheduling, and monitoring job flows, with proactive alert setup.
• Adept at troubleshooting failed jobs in azure tools like Databricks and Data Factory, performing root cause analysis, and applying corrective measures.
• Hands-on experience with distributed streaming tools like Event Hub.
• Expertise in designing and managing backup and disaster recovery solutions using Infrastructure as Code (IaC) with Terraform.
• Strong experience in automating processes using Python, shell scripting, and working with Jenkins and Azure DevOps.
• Proficient in designing and maintaining Azure CI/CD pipelines for seamless code integration, testing, and deployment.
• Experienced in monitoring and troubleshooting VM resources such as memory, CPU, OS, storage, and network.
• Skilled at monitoring applications and advising developers on improving job and workflow performance.
• Capable of reviewing and resolving log file issues for system and application components.
• Adaptable to evolving technologies, with a strong sense of responsibility and accomplishment.
• Knowledgeable in agile methodologies for software delivery.
• 3-6 years of experience with Azure and cloud platforms, leveraging cloud-native tools to build, manage, and optimize secure, scalable solutions.
We are looking for Data Platform Engineer who has strong understanding of Azure administration and Azure DevOps, which are crucial for the role.
They need to have solid experience in below topics:
o Storage Account:
o Data Lake (ADLS):
o Databricks:
Unity Catalog:
Types of Tables:
Parquet vs Delta:
Optimize & Vacuum:
Security Segregation:
Job Slowness Issues:
Version Control of Databricks:
o Azure Data Factory (ADF):
o Event Hub:
o Messages not flowing:
o Purpose of Event Hub and Event Grid:
o Terraform Setup for Event Hub:
o Key Vault:
o Cosmos DB:
o Synapse: