23 February 2016 · Absolute IT

Chartered Accountants Australia and New Zealand’s latest publication, Future proofing the profession: preparing business leaders and finance professionals for 2025, claims New Zealand is suffering from an emerging skills gap.

The report identifies STEM skills as a significant asset for the jobs of the future, but the corresponding tertiary enrolments in courses of this nature are in decline. CAANZ’s Professor James Guthrie says that “This in conjunction with technology disruptions, the impact of regulation and the lack of these skills in current and future workforce are critical issues that the Australian and New Zealand governments need to deal with now.” And with this in mind, it’s expected that NZ businesses will drive an over 500% boost in digital jobs over the next 10 years.

The number of vacancies for digital jobs that we have processed in New Zealand has grown by a whopping 70% over the last 12 months. There’s a widening gap between a need for digital expertise and the pool of talent. Digital technology is a huge growth area for New Zealand’s tech sector. In fact 54 NZ tech companies were just named in the 2015 Deloitte Technology Fast 500 Asia Pacific index, setting a new record for Kiwi representation.

And one of the biggest growth areas is in data – specifically big data, and data analytics.

Big Data jobs

Big data is a term that refers to large volumes of data that are byproducts of day-to-day business. The amount of data available to businesses today is huge. Challenges to big data projects include analysis, capture, data curation, search, sharing, storage, transfer, visualisation, and information privacy. But once that tricky territory has been navigated, businesses can analyse that data for insights that better inform their business goals and strategy. This need to extract trends and insights from big data puts a huge emphasis on hiring IT talent that can work within the realm of big data.

There are two categories of IT jobs that are emerging in the big data sector: big data analysts (such as data scientists and data engineers) and big data architects. Data analysts help businesses to collect manage and gather insights from their big data whereas architects help businesses to find the best way to structure, access and store their big data.

The rise of the big data engineer

In recent years, businesses have been looking to hire data scientists and data analysts who can delve into their databases and uncover hidden insights. When a business fully understand what their customers are doing, where they’re going, and why they’re leaving, they can refine their processes and build better business strategies.

A data engineer is the “all purpose” talent of any big data project, working with data analysts and programmers to ensure data pipelines are:

  1. scalable
  2. repeatable
  3. secure
  4. serve multiple stakeholders within the business

“At Absolute IT we have seen a month on month increase in the demand for big data specialists. The truth is that businesses are struggling with the challenge to extract commercial value from their data and there is a huge opportunity for IT professionals with skills in this area,” says Grant Burley, Director at Absolute IT

Big data and the demand for data scientists

The data scientist is one of the most sought after IT specialists in tech. More and more industries are turning to big data analytics for customer insights, and enterprise information management. When this IT job first emerged there were no specialist tertiary qualifications or programmes to train up a big data scientist. But that’s all changing.

Massey University recently revealed its Master of Analytics with the idea of equipping graduates with business analytical skills, data extraction skills, and ultimately, to teach students how to transform data into information that is useful for crucial business decisions. Whilst AUT also offer a Master of Analytics, with study options beginning in 2016.

The must have skills if you want to work in big data in 2016

Going back to university isn’t an option for everyone, so just what skills are needed in big data? Here’s a short list of the go to skillsets IT employers are looking for in data specialists:

  1. Machine learning and data mining – data mining has been happening for as long as it’s been collected, but contemporary data mining is a whole other level. Data analysts who can adopt machine learning technology have better prospects when it comes to predictive analysis, which is in high demand in the IT job market.
  2. Statistical and quantitative analysis – this is the backbone of big data analysis. Big data analysis isn’t just for big data companies anymore, industries all over the world and in areas outside of the IT industry, is leveraging big data insights. If you have an understanding of the wider aggregation and application of data and statistical insights, you’re going to be extremely valuable.
  3. Open source expertise – in a lot of industries the open source software framework Apache Hadoop is part and parcel of working in big data. An understanding of how Hadoop works and how other open source processing engines such as Apache Spark and NoSQL plug into this framework are critical skills if you want to work in open source.
  4. SQL – This tried and true data language has a solid history, and that’s what makes it even more relevant today.
  5. Data visualisation – big data can be hard to comprehend, and even harder to relay to people who aren’t data or statistical experts.Knowing how to best present the shape and trends within data is extremely important when communicating complex ideas to people at all levels of the organisation.
  6. General purpose programming languages – Experience with general programming applications like Java, C, Python, or Scala could give you the edge over other candidates, when combined with analytics skills the insights are invaluable.
  7. Creativity and problem solving – As with most IT jobs, creativity and problem solving are key to big data projects.

Data Analyst Jobs

There’s encouraging growth in the demand for data analysts, with organisations seeing the value in deciphering their unique data sets. It can be social media analytics, website analytics or the more sophisticated data insights of a CRM or custom database. If you’re thinking about a career change or you love the idea of a job in big data, then get in touch with one of our team today.

Further reading