Spark Scala Developer

Job Title: Spark Scala Developer
Contract Type: Permanent
Location: Melbourne CBD, Victoria
Salary: Negotiable
Start Date: ASAP
Reference: 237869_1628136436
Contact Name: Sue Templeton
Contact Email:
Job Published: August 05, 2021 14:07

Job Description

Acting as the software engineer of the group to perform design and implementation tasks for our various platforms. Main accountabilities are

  • Designing and implementing ingest, processing and curation solution using Big data technologies
  • Identify, resolve, and/or escalate all issues that threaten the successful delivery of assigned initiatives
  • Utilizing open source tool sets related to the Hadoop ecosystem.
  • Create high quality API's for data accessibility on the Hadoop platform that ultimately integrate with the broader environment.
  • Using big data analytics techniques provide data and analytics output as required to the broader community including business partners in a manner they can understand to ensure communication and buy in.
  • Producing high quality code with excellent test coverage
  • Code reviews and Efficient and effective code maintenance
  • Analysing product requirements and breaking down into user stories and tasks
  • Estimating and planning to assist with the development of the product roadmap
  • Mentoring junior data engineers
  • Developing build, test and deployment automation
  • Research and development of potential new technologies
  • Defining processes and standards within the team and contributing to the continuous improvement of the product development process
  • Producing a high standard of technical documentation
  • Working with customers to understand requirements and technical issues
  • Ability to provide technical expertise for new and ongoing initiatives to work with project stakeholder to come up most feasible technical solution.

Technology skills

  • Proficiency in Scala and Java is essential.
  • Must have experience working with real-time ingestion, processing, and curation of data at big data scale.
  • Big Data developer experience with hortonworks or Cloudera would be beneficial.
  • Fundamental knowledge of Hadoop components such as Kafka, Spark, Hive, HBase etc. is mandatory.
  • Previous experience developing data pipelines using Spark.
  • Past experience with NoSQL data stores such as ElasticSearch, MongoDB, Couchbase etc would be beneficial.
  • Experience on data analytics using SparkML or other ML tools such as R is a good to have.


  • Demonstrated technology subject matter expertise.
  • Familiarity with technology hands on configuration.
  • Able to communicate effectively with stakeholders.
  • Able to comprehend and deliver to a schedule.
  • Ability to work under pressure to deliver to near-term deadlines.
  • Ability to work in a matrix organisation, taking direction from both line and project management.
  • Self-starter, able to get up to speed quickly and work independently.
  • Solid experience working in software development
  • Experience working on Big data technologies