Data Engineer
- Develop and maintain data pipelines using Azure and Pyspark.
- Transform and load data using big data technologies like Spark and Hive.
- Collaborate with teams to ensure data quality and accessibility.
About the job Data Engineer
Role Data Engineer
Required Technical Skill
Set
1) Strong Data Engineering skills using Azure and Pyspark
2) Good Knowledge on SQL
3) Preferred experience in Big data/Hadoop technologies like
Spark, Hive, Hbase and Kafka.
4) Preferred experience in ETL process.
5) Good Communication Skills
Desired Experience Range 5 – 10 years
Location of Requirement Perth/Australia
Desired Competencies (Technical/Behavioral Competency)
Must-Have
1. Strong Data Engineering skills using Azure and Pyspark (or
Databricks, Hadoop/Spark using Java/Scala)
2. Experience in Azure Data Factory and other Azure services
(ADLS, Azure SQL, Blob Storage)
3. Experience in loading and transforming the data using spark or
any big data technologies( Hive, Kafka, Hbase ,Spark or Storm).
4. Very good SQL knowledge
5. Knowledge in Azure Devops
6. Good Communication skills
Good-to-Have
1. ETL Process experience in any Cloud or any on-premise big data
platform.
2. Azure Solution Architecture Data Engineer Certification
3.Good data warehousing concepts
We vet all employer accounts and do our best to keep job ads safe, but scams can still occur. Be cautious when sharing personal information — never provide financial details or make payments during the application process. For extra security, use the Apply button on our site when proceeding.
Skills
0 of 13 matchedPyramid Global Technologies