Key Responsibilities:1. Design, develop, and deploy scalable data pipelines on the Azure cloud platform using AzureData Factory (ADF).2. Architect end-to-end data solutions to support business requirements and analyticalinsights, with a focus on Databricks (PySpark) for data processing.3. Implement and optimize data processing workflows using technologies such as Databricks,Spark, and PySpark.4. Integrate diverse data sources including structured and unstructured data for analysis andreporting purposes.5. Collaborate with cross-functional teams to understand data requirements and deliveractionable insights.6. Ensure data quality, reliability, and security across all data processes and systems.7. Provide technical guidance and mentorship to junior team members, particularly in ADF andDatabricks (PySpark).8. Stay abreast of emerging technologies and best practices in the field of Big Data and cloudcomputing, with a focus on ADF and Databricks.Preferred Qualifications:1. Azure/AWS certification (e.g., Azure Data Engineer, AWS Certified Big Data - Specialty).2. Experience with DevOps practices and tools for CI/CD pipelines.3. Familiarity with containerization technologies such as Docker and Kubernetes.4. Knowledge of data visualization tools such as Power BI or Tableau. Qualifications:1. Bachelor's or Master's degree in Computer Science, Engineering, or related field.2. Minimum of 3 years of experience in designing and implementing data solutions on theAzure cloud platform, with a focus on ADF and Databricks (PySpark).3. Proficiency in Big Data technologies such as Hadoop, Spark, and other relevant tools.4. Strong programming skills in languages such as Python, Scala, or Java.5. Extensive experience with SQL and relational databases.6. Hands-on experience with cloud services such as Azure or AWS.7. Proven track record of delivering complex data projects on time and within budget.8. Excellent problem-solving skills and attention to detail.9. Strong communication and interpersonal skills, with the ability to collaborate effectivelywith cross-functional teams.
