Lead Data Engineer

u&u · Sydney CBD NSW · Full time
Posted 4d ago

Lead Data Engineer

KEY POINTS WE FOUND
  • Lead the design, development, testing and support of enterprise data pipelines and data products on Azure Databricks.
  • Set engineering standards and provide technical leadership across data architecture, CI/CD, testing, observability, lineage and documentation.
  • Partner with BI, software engineering and business stakeholders to deliver scalable, reusable data solutions.

Job Details A large property company are looking for an Azure Databricks Expert to design and built pipelines, integrations and own their data platform. The role Act as the Azure Databricks SME, providing deep technical expertise across the data environment; Flexible Hybrid model, Sydney CBD; $190,000 - $240,000 Package including bonus; and Set engineering standards and drive best practice across data design, development and deployment. Responsibilities Design, build, test and support enterprise data pipelines and data products; Provide specialist expertise across Azure Databricks, Spark/PySpark, SQL, Delta Lake and Unity Catalogue; Establish strong engineering practices across CI/CD, testing, observability, lineage and documentation; Lead solution design, code reviews and technical problem-solving across complex data initiatives; and Partner with BI, Software Engineering and business stakeholders to deliver practical, reusable data solutions. About you An experienced Senior Data Engineer with deep Azure Databricks expertise; Strong hands-on capability across Spark/PySpark, SQL, Delta Lake and Unity Catalogue; Strong Azure engineering foundations across cloud, data integration, APIs and DevOps; Comfortable operating as the Databricks SME, setting technical direction and supporting other engineers; and Pragmatic and collaborative, with the ability to turn complex requirements into scalable, production-ready solutions. How to Apply For more information or a confidential discussion please click to reveal the contact details for Chris Allum at u&u chris.allum@uandu.com or 02 8245 7927 quoting reference number 46892. At u&u Recruitment Partners, we value diversity, equity and inclusion. We welcome applications from Aboriginal and Torres Strait Islander people, people with diverse cultural and linguistic backgrounds and people with disability. Should you require reasonable adjustments throughout the recruitment process (including alternate formats to apply), or have a preferred method of communication, we encourage you to make a request via adjustments@uandu.com or contact u&u on 02 8245 7900 to discuss. In response to these requests, we will collaborate closely with you to implement the appropriate adjustments. Additionally, for a barrier-free and inclusive online experience, you can access u&u’s opportunities using accessibility software Recite Me at https://www.uandu.com/jobs. Please  submit your resume in Word format only.

Stay Safe While Job Hunting

We vet all employer accounts and do our best to keep job ads safe, but scams can still occur. Be cautious when sharing personal information — never provide financial details or make payments during the application process. For extra security, use the Apply button on our site when proceeding.

Skills

0 of 27 matched
Apache sparkApi integrationAzure databricksBusiness intelligenceCi/cdCloud engineeringCode reviewData integrationData lineageData pipeline developmentData platform engineeringData product developmentDelta lakeDev opsExcellent communication skillsMicrosoft azureObservabilityPragmatismProblem solvingPythonSoftware testingSolution designSqlStakeholder collaborationTechnical documentationTechnical leadershipUnity catalog