Our client is a major industry player within the Telecommunications sector and has been operating within Australia for decades. The team you will join is responsible for the Design and build of a reusable framework for batch and real-time ingestion of data into Big Data platform
The role:
- High level and detailed system design
- Data collection, storage and processing including data curation
- Data ingestion (real-time and batch code) and stitching
- Design and build ETL pipelines
- Visualisation and reporting skills (nice to have)
- Have an end-to-end vision and see how an actual design will translate into process & data solutions through the layered architecture within the EIM landscape as well as through the MDM and Big Data architecture.
- Accountable for the correctness of the high-level and detailed design and the mapping to the business requirements.
- Code development, implementation, hand over documentation in a DevOps model
- Unit testing and regression testing skills (Junit etc.)
- Responsible to provide awareness of other in-flight projects that may impact the solution on hand and provide mitigation paths.
- Responsible to provide direction for investigations & impact analysis of enhancements as well as change requests.
- Works collaboratively with data scientists and data engineers to gain an understanding of business needs related to the quality of information, data preparation for analytical needs or performance and ensures that best practices are followed.
The candidate:
- Degree qualified in Business management, IT, Computer Systems, software or computer engineering fields or equivalent.
- 3+ years of defining solutions, designs and implementations in Big Data platforms.
- 7+ years of defining solutions & designs for BI or data integration / analytical initiatives
- Strong exposure to Cloudera, AWS eco-systems
- Strong exposure to CI / CD environment and toolset
- Strong technical knowledge of data integration, metadata and Big data analytics tools.
- Solid understanding of big data management best practices with experience in migrating from a data warehouse to big data eco-system.
- Good understanding of Telco DW / Big Data solutions.
Tools (not limited to)
- Spark
- Python
- Informatica BDM
- StreamSets / Apache Nifi
- Cloudera toolset
- Kafka
Please submit CV on WORD format ONLY
