Skip to Content
 Close search

Shifting the Dial: 5 year productivity review

Chapter 5: Improving the efficiency of markets

Benefits assessment:  conservatively estimated at
over $3.4 billion annually
Improving the efficiency of markets: What matters?




Energy is an appalling mess

  • Investment in electricity generation is now being seriously damaged by absence of clarity over future emission reduction costs
  • Governance arrangements fail to deliver clear guidance and allow pricing to be used against consumer interests
  • The renewable energy certificate system allows some generators to impose additional costs for frequency management and intermittency on consumers. This cost should be borne by generators as part of their bid costs
  • Adopt a national emission reduction policy supported by all governments. The carbon price should be reflected in electricity prices
  • Governments set specific outcome-oriented strategic direction for expert bodies, and leave them to be fully accountable for implementation
  • The pricing rules should be structured so that the prices that renewables generators receive reflect the additional costs they impose on the system
  • Investment proceeds on a sustainable basis, with choice of technology determined by rational pricing
  • Major savings relative to business as usual and reduction in the impacts that come with uncertainty, ad hoc interventions, and slow response times
  • Along with an efficient policy on carbon (so the price reflects the associated emissions of the electricity delivered), this will achieve a more level playing field for all generators that should lower long-term prices

Creating an innovation ecosystem

  • Data are an underutilised resource, and consumers have no ability to use it to gain competitive offers from alternative suppliers
  • Copyright is poorly balanced between creator and user. As data become more powerful, the current system will cost the economy more and more
  • Establish legal rights of consumers over their own data and facilitate greater access to public data
  • Revitalise competition policy in a digital world
  • Remove competition law exemption for intellectual property
  • Restore the balance — a time limited monopoly in return for sharing ideas. Offer copyright based on ‘fair use’
  • Allows consumers to better match their preferences, gain better leverage and supports the creation of new services and other opportunities to innovate
  • A more open system for research, education and innovation

Don’t abandon reforms to regulation to the too-hard basket

  • Regulator behaviour continues to be a source of unnecessary costs for business
  • Consumers can lack credible information, complaint avenues and rights of redress
  • There is considerable unfinished business in regard to reform of regulation. Reviews have persistently identified sensible recommendations
  • New anomalies in policy are emerging. These include: lagging firms and lack of innovation, whether employment in the gig economy is different, better ways to manage environmental risks, and regulation to make privatisation work well
  • Digital technologies offer new ways for regulators to engage, including encouraging RegTech
  • Digital solutions can greatly improve consumer information, but to ensure credence government may need to act, as well as enforcing consumer rights
  • Draw upon regulation reforms cited in existing studies and reviews (appendix B)
  • Use evaluation data to reduce industry assistance to highly probable net benefit levels. Cease using procurement as a tool for creating unsustainable industries
  • Some deserve further inquiry
  • Plus flexibility for regulators to respond with ‘sandbox’ type trial solutions
  • Potential for substantially lower regulatory costs and faster responses for business
  • Less need for heavy handed regulation as consumers impose greater market discipline
  • Existing unfinished business alone could add over $3 billion a year to GDP. Welfare gains on top of that are highly likely
  • Early development and testing of solutions, learning from other countries. Agile, anticipatory action or rapid response to risks that emerge

Australia is a substantially market-based economy. The more efficiently markets function as an allocation mechanism in the economy — by setting prices that align demand and supply and facilitate exchange — the closer Australia can get to peak productivity.

Market rules are not set and forget

Getting the incentives right for, and removing barriers to, innovation helps the private sector (and increasingly the services delivered by the public sector) move this peak higher.

Governments play a major role in making the rules that are essential to establish confidence and thus make markets work well. But regulation can also be abused, act against efficiency or persist well beyond its use-by date.

These market rules — providing consumers and workers with necessary protections, managing market power (particularly that inherent in public infrastructure), determining the incentives for innovation, and for skill formation and factor mobility — are not set and forget. They need regular attention to ensure that the system remains competitive, while at the same time providing the coordination needed where collective action is required to address common problems such as setting standards, developing data sharing, and cybersecurity (figure 5.1). This attention involves fine tuning of the regulatory system and processes as well as major reforms.

Despite the importance of an ongoing process of microeconomic reform, there is a clear view from participants in this report that governments at all levels have dropped the ball (reflected in, for example, AiG sub 36, Australian Chamber of Commerce and Industry (ACCI) sub. 37, Minerals Council of Australia (MCA) sub. 30). But this is not just about making things easier for business, ultimately the aim is to benefit consumers and offer employees the ability to argue for wage increases supported by higher productivity.

And the absence of a direct assault on slowing productivity (via reviews of market regulation) is not because nothing has been identified.

There is a surprising amount of agreement on what we should do. I really don’t think the problem of this room or this nation is the economics. We kind of know what to do. What we aren’t good at doing, and arguably are increasingly worse at doing, is achieving it. (Richardson 2016)

This chapter brings together the findings from a number of reviews and proposes some priority areas for reform — notably energy and data related reforms. The costs of getting the energy system wrong are just too large to contemplate, and even muddling through will impose billions in additional costs.  Greater availability of public data could deliver annual benefits of over $4 billion. Moreover, reforms to intellectual property could deliver significant additional benefits from higher rates of innovation and human capital formation. This is not to forget the ongoing need for ‘good housekeeping’ to keep the regulatory system fit for purpose. Indeed, there is a good story here, as a set of such reforms identified by the Commission would likely deliver benefits of over $3 billion annually, with further gains from reforms whose benefits are less easily enumerated.

Figure 5.1 Key elements in efficient markets

  • This graphic portrays key elements in efficient markets. At the outer ring are aspects that involve infrastructure that delivers reliable services at least cost; incentive and scope for innovation and adoption; confidence in protections for consumers, workers and the environment; systems and processes for ongoing regulatory reform; and incentives for skill formation and worker and capital mobility. The inner ring comprises factors that support well-functioning markets. These are macroeconomic stability; regulation that gets the balance right; regulator behaviour that facilitates institutions fair administration of rule of law. This inner ring is supported by competition and coordination features.
Greater availability of public data could deliver annual benefits of over $4 billion

This chapter begins with the area of most pressing need for attention — energy markets (section 5.1 and Supporting Paper 11 (SP 11)).9 There is no business, no farm, and no household that is immune from the effects of energy market problems, so their resolution will deliver the greatest market regulation reform benefits in the future time-frame of this Review.

Innovation is the next highest priority. There is a serious need to translate ideas into investment (SP 12). Many developed nations are seeing a slow-down in take-up of innovation and Australia is no exception. The OECD has dubbed this a ‘breakdown in the diffusion machine’ (McGowan, Andrews and Millot 2017).

While governments can help to create an environment more conducive to innovation, it is really up to business to make the running. Where governments can make a major difference is by freeing up data access and availability and removing obstacles imposed by the current prescriptive system of copyright exceptions (section 5.2 and SP 13).

How regulators behave affects innovation and, more generally, the regulatory burdens on business. Digital technologies offer regulators better ways to engage, including helping consumers better discipline markets (section 5.3 and SP 13).

Reforms to existing regulations to promote competition and to reduce unnecessary regulatory burdens is a constant subject of review, but is too often disappointing in delivery. Opposition from the beneficiaries of unproductive regulation is always with us, but the ability to overcome this via evidence and analysis has been dulled over time. Communication and explanation needs a reset.

Appendix B provides a broader group of reform options determined following a systematic examination of past reviews. Despite their familiarity these should not be ignored, in the search for productivity enhancement. They may be familiar, but many are still unaddressed.

Governments could add some or all of these to a core reform agenda drawn from the other chapters of this Review.

5.1 Fixing the energy mess

Australian energy markets, particularly those in eastern Australia, are in a fragile state. While the statewide blackout in South Australia, on 28 September 2016, was initially caused by nature, the duration of the blackout brought issues about the security and stability of the national electricity grid to the fore. The estimated cost of the statewide blackout in South Australia was $367 million. With electricity and gas making up 2.5 per cent of GDP, and being an essential input for all industries, the cost of failure to resolve the problems will only rise in the future. It is too difficult to put a price tag on getting energy policy right, in large part as the counterfactual — what would happen if we continue to try to muddle through without clarity on carbon pricing — is impossible to define (SP 11). Nonetheless, the returns from the reforms outlined here would be worth many billions.

Lack of clarity on emission reduction policies, increasing reliance on intermittent and variable renewable energy, moratoria on gas exploration and development, and the commencement of gas exports from the east coast, have all contributed to a system under pressure.

The challenge is how to resolve these issues while maintaining an affordable, reliable and sustainable supply of energy going forward — dubbed the ‘energy trilemma’ by the recent Finkel Review (2017).

Significant policy and technical work has been undertaken in response to these challenges, of which the Finkel Review is the most recent.10 Finkel set out an approach that seeks to find an option, amongst many, that is politically palatable. The issues affecting the sector are complex, and their solution requires considerable technical expertise as well as good regulatory and institutional design. This requires solving the immediate problems, but also needs to be mindful of transforming the energy system so that it will deliver for consumers in the long as well as the short run.

This Review focuses on setting out what the reform process must achieve to resolve the problems that bedevil the sector. These are framed within a structure that emphasises the basics, things that have been lost along the way in the energy wars. Governments must:

  • restore national agreement on simple, clear objectives — that recognise the inherent tensions between prices (costs), reliability and emissions, and provide guidance on acceptable trade-offs — then leave the field to expert implementation
  • determine which institutions do what — then let them get on with their work, holding them to account for their stated responsibilities
  • ensure that the energy sector can access the full set of instruments in doing their work — not locking in or out technologies, or excluding other solutions by design
  • set out a clear roadmap for reforms — ideally with cross-jurisdictional commitment.

Clear objectives

Energy is an input into all industries and households, and so even minor deficiencies in efficiency have cumulatively large impacts. Cost matters for all electricity users, but the need for reliability varies. For large energy intensive export-orientated manufacturers, such as metals processing industries (which are often regionally based), unplanned or long disruptions to supply can be particularly costly. Hospitals and other essential services, and computer banks need reliable power. But many households can cope well with short interruptions, or are happy to reduce consumption for short periods, particularly if scheduled.

The costs of getting the energy system wrong
are just too large to contemplate

Building in redundancy to improve reliability (whether in interconnectors, transmission, distribution, or generation) comes with higher costs that are passed onto consumers. ‘Gold plating’ the poles and wires was responsible for a substantial share of the rise in the cost of electricity over the past decade.

In part this was due to poor regulatory arrangements that rewarded over-investment, but in part it reflected an over-emphasis by governments on reliability. System reliability in power is not a yes/no question. There are known and effective mechanisms to manage reliability according to different user’s preferences. This recognition, after a damaging period of price rises, may now be in prospect. But the temptation to fiddle will remain present while the regulatory system is under reconstruction.

There is also a trade-off between reliability, cost and emissions (sustainability in Finkel’s trilemma). Currently, the regulated price is not designed to deal with this.

It is a principle of every properly-designed pricing system that the charge should reflect its harms. Thus carbon emission intensity is necessarily a matter to be reflected in the regulated pricing system. The compromises necessary to do this are much-debated, but what should be accepted is that low carbon technologies (such as solar and wind generation) are inherently part of the properly-priced future.

Some renewable technologies also impose systemic costs. They can be intermittent — with limited predictability of supply, and require frequency management services. These issues can be managed, such as by battery storage, greater linking across the network (the wind is blowing somewhere), and other ancillary services, but these raise costs, which is not well-reflected in the current system of regulated pricing.

Governments need an emissions target11 to provide certainty for the sector about the trade-offs allowed between emission reduction and cost. From these judgments better decision-making will flow on how and where to invest in future system reliability. From the user’s perspective, reliability can be as much about predictability as the ability of the system to always deliver on demand — the system must be able to deliver a reliable supply to users who face high costs of disruption.

A broadly accepted commitment on emission reduction targets over the investment profile for electricity assets is essential to give firms sufficient confidence to make the investments needed to deliver electricity at the lowest possible cost and the right levels of reliability for users over the next few decades.12

In the case of Finkel, 49 out of 50 is not a pass mark.

Governments need an emissions target
to provide certainty for the sector

The right institutional structures

The existing regulatory arrangements governing the industry emerged out of the wider energy reform process that commenced in the 1990s. A ‘national’ market (really an east coast and central market) was created, the generators/producers and networks became better managed and many passed into private hands.

The regulatory system to address market power became more sophisticated. There were failings along the way, and at some times the glacial pace of change in vital areas was frustrating (as in transmission reliability frameworks, retail competition and price controls), but the direction was known, and things largely worked well.

Over the past decade, however, the growth of renewable generation, much of which is inherently non-synchronous and non-dispatchable,13 has posed new challenges.

This, coupled with more geographic dispersion in the location of renewable generation, makes it technically harder for the system operator to maintain system reliability and security. These problems can be addressed by ancillary services that can modulate frequencies, and provide complementary generation, batteries, or other storage that can ramp up quickly as needed. But governance arrangements have not proved capable at adapting to these changing realities and dealing with emerging issues as they occur, or at least not quickly enough.

Governments need to set policy directions and let the expert bodies deliver

The core of the regulatory architecture is three expert bodies — the Australian Energy Market Commission (AEMC), the Australian Energy Regulator (AER) and the Australian Energy Market Operator (AEMO). They respectively develop policy with governments, regulate and run the system. In theory, such institutions have the advantage for governments that they can moderate the inevitable political pressures governments face to act spontaneously given popular concerns. But in practice, insufficient use has been made of the independent expertise and public explanations of those three bodies when forming energy policy.

Resolving the issues currently confronting Australian energy markets will require the cooperation of all Australian governments. A sound system would have:

  • Australian governments jointly setting a clear long-term strategic vision using outcome-focused language that integrates energy and environment policy
  • the COAG Energy Council restricting itself to developing policy to achieve this vision, with subsequent implementation clearly the role of the AEMC and AER
  • the same regulatory roles and responsibilities for the AER in each state over which it has jurisdiction
  • a nationally consistent approach to regulation for all companies operating within particular market sectors and consistency in its regulatory determinations across companies and states
  • economically efficient network pricing for the use of electricity and gas transmission and distribution networks.

Gas markets

The development of export facilities at Gladstone in Queensland, which allow liquefied natural gas (LNG) to be exported, have linked the east coast gas market into global markets and supply chains. This, inevitably, means that domestic prices for gas will be exposed to the price of exports (net of additional costs of processing and delivery). But what seem to be over-optimistic forecasts about production (or costs) from Queensland gas fields has seen the LNG exporters look to the southern states for supply which has pushed up domestic prices beyond the ‘net back’ export price. Pipelines which once carried gas to the markets in the south reversed direction as gas was diverted north.

More fundamentally, the difficulty for generators in accessing long-term supply contracts at acceptable prices, coupled with uncertainty in carbon pricing, reduced the viability of investment in gas-fired generation — the natural complement to renewables. Adding to the uncertain investment climate, the Australian Competition and Consumer Commission (ACCC) has raised concerns about constrained pipeline capacity, which benefits the capacity rights holders, and market of some pipeline operators. It has looked at whether price regulation is needed, and is currently pursuing a much wider ranging inquiry into the supply and demand for wholesale gas. An additional problem being addressed is the lack of a secondary market for trading of unused pipeline capacity.

The significant increases in the domestic price of gas have been a key factor in the rising wholesale electricity prices. These factors have also affected manufacturers that rely on gas as a feedstock. The Australian Government has introduced legislation that will allow it to use export controls to divert gas to the domestic market to ensure adequate domestic supply. Such action could impose sovereign risk, and is undesirable.

Removing the moratoria on gas exploration and development in Victoria, Tasmania and the Northern Territory, which have slowed the growth in supply, are one place to start.

There are more effective models of community engagement which exploration firms can, and should, seek to apply if given the opportunity. Local employment and investment should be upfront considerations, not left to others to guess at. Royalty regimes may need review. Bans are unlikely to be lifted simply because of pricing concerns. The decision to intervene in exports may actually relieve a pressure on States with bans.

A voluntary industry-wide code of practice might help the gas industry improve their relationship with the community, but must be accompanied by moves of substance. None of this is intended to question the science and the efforts of Chief Scientists to establish safer practice. But as is often the case, the science is not enough to carry the policy debate.

To build community confidence in gas exploration and production a code must go beyond other desirable aspects of gas exploration — safety regulation, sound scientific evidence, and monitoring and enforcement of compliance — and include clear guidelines and arrangements to manage community impacts and support landholders in negotiating land access agreements.

Solutions should be technologically neutral and efficiently priced

Prices should reflect the additional costs imposed by specific technologies

There has been considerable change in the electricity system over the past decade. On the demand side, ‘peakiness’ has risen with the widespread growth in the use of air conditioners, and roof-top solar generation servicing the household during the day. On the supply side is renewable technologies, and a shift to two-way transmission from households and some firms, that both draw on, and sell into, the system from their own generation. Technologies, such as battery storage, continue to develop which, with falling costs, will change the calculus in the future. So the institutional structures and regulatory regime must be able to accommodate these trends.

The Commission has previously criticised the use of renewable energy targets (RET) on the basis that they are not technology neutral, and so are distortionary compared with the policy of putting a price on carbon. But we have also acknowledged they have been an important tool in delivering on emissions reduction commitments. Finkel’s recommendation for a low emission energy target (LET) would be an improvement on the RET. It would better reward lower emitters, and depending on its detailed design, would move Australia’s energy market closer to the outcomes that would arise from full carbon pricing. The critical point is that while there are various ways of placing prices on carbon, establishing an agreement on one of them, even if not perfect, provides a guide for investment toward the lowest cost emission reduction options. These costs must include any additional cost imposed on the network by choice of any particular technology.

The RET is specified in terms of gigawatt hours of renewable electricity and not in terms of emissions. It was formulated and operates independently of wholesale electricity markets and is not explicitly incorporated into the spot price. This has created problems as it effectively pushes the cost of intermittency and frequency management onto the purchasers of electricity rather than onto the producers of renewable energy.

While the low marginal cost of renewable generation means that these producers can sell into the grid when they were producing (as they would have the lowest priced bid), they do not have to pay for the reserve capacity of other generators that are needed when renewable production falls relative to demand. This may be a problem with any certificate-based scheme based on the production rather than the delivery of electricity. Prices need to be able to be based on not just when the electricity is delivered (by bidding into the spot market for five minute contracts), but also its availability. Ideally prices would also vary by carbon content, rather than making the carbon related payment separate from the market for wholesale electricity. This outcome can be achieved in a number of ways. One way is for the generator to be made responsible for any additional costs, such as ancillary services to manage frequency variability, associated with the electricity it delivers into the wholesale market.

The option to access the grid must be priced

The uptake of distributed generation (in particular roof-top solar) is changing how transmission and distribution networks are used, and what for. This poses risks for the owners of these assets and challenges for the regulators in regard to maintaining system stability and reliability and in setting prices. For the system operator, these small producers are invisible, but affect the flow of electricity that must be managed.

For example, if the regulated cost of connection to the grid rises above the cost of self-reliance for more than a few customers, a dangerous spiral of users moving off grid, pushing up charges for the remaining customers, leading to more going off grid, could arise. For transmission lines, the risks are likely to be greater, and regulators need to be concerned about the long-term viability of these assets. The regulator and regulated providers need to be alive to the implications of cost recovery, pursued to its illogical conclusion.

There are various ways that the owners of transmission and distribution systems can go about pricing their services, but these will need the approval of the regulator. One option is to have an ‘insurance’ charge applied to premises that can access the grid, even if they do not draw power from it (other than in an emergency). Different charges might apply to those premises that both buy from, and sell into, the grid reflecting the different costs imposed by this use of the asset. Regulators will need to ensure that, whatever pricing regime is developed, the owners of the networks bear the costs of their decisions to under or over invest and that they cannot just pass this onto consumers (or potential consumers) in regulated prices.

Demand management can be a lower cost technology for improving reliability

The system savings from demand management are considerable. Smoothing the transmission load by shifting energy demand from peak to off-peak periods reduces the carrying capacity required and would have reduced the past investment in transmission augmentation by billions of dollars. This horse has bolted, but the same mistake must be avoided in the future. Long-term contracts that allow wholesalers to actively manage demand reduce the need to maintain higher levels of spare generating capacity to meet spikes in demand — the ‘reserve plant margin’.

Progress has been made in this area, but the peaks could be further attenuated through other demand-side measures. Forced load shedding or requesting people to reduce consumption at times when the system is expected to be under stress, such as during heatwaves, are not an efficient way of managing demand. Residential and other small customers need to have price incentives to reduce demand at such times. Progress is being made on the information and technology to support demand management, but most small consumers still do not face demand reflective pricing, including time of day pricing.

Looking to the future

While immediate action is needed to manage the looming risks of blackouts next summer, there is a real danger that the electricity system Australia needs in the future will not be delivered if the regulatory mess persists.

Short-term certainty over a carbon target is not likely to alter any investors’ preference for sitting on the fence.

The financial viability of refurbishing existing coal-fired power today appears very doubtful, as uncertainty is actually priced into current investment models. Advocates of older generation technologies who oppose (regulated) prices that reflect emissions are actually doing themselves a major disservice. Lacking information on risk, investors most likely will not commit. Thus doing nothing on emissions intensity (via price, or via some other proxy) is most likely to ensure that the fear of redundancy for coal will become a reality. A future without a coal-fired baseload has very different implications for the transmission and distribution system, and for the operability of the National Electricity Market (NEM).

Recommendation 5.1 Urgent action is needed to fix Australia’s energy markets

Australian governments must cooperate to reform the national electricity market as a priority.


Australian governments must work cooperatively to resolve the issues currently confronting Australian energy markets. They must:

  • stop the piecemeal and stop-start approach to emission reduction, and adopt a proper vehicle for reducing carbon emissions that puts a single effective price on carbon
  • clearly articulate the acceptable trade-off between reliability and cost
  • achieve more efficient pricing, by ensuring that:
    • prices paid to producers reflect any additional costs they impose on the  system (such as frequency management)
    • access to the grid, rather than just use, can be priced (so people using the grid as a back-up pay for this service)
    • prices to consumers reflect the nature of the demand that they require from the system
  • provide clear strategic direction to the expert bodies, and a clearer accountability for outcomes
  • let the market regulators and participants get on with their work, holding them to account for the outcomes
  • ensure that short-term fixes are technologically neutral and move the system toward a sustainable long-run outcome.

5.2 Enabling an innovation culture

The creation and adoption of knowledge, ideas, products, processes and ways of doing business — in short innovation — are critical for maintaining Australia’s high standard of living, ensuring its ongoing international competitiveness, creating jobs and delivering future economic prosperity (SP 12).

While innovation can only be driven by firms willing to make investments and take risks, governments can assist by providing a business environment where innovation is welcomed. This requires a regulatory regime that allows experiments and does not punish failure that brings no harm to others. It requires government to invest in the fundamental elements of a high tech society — a well-educated workforce, good communications and other infrastructure, and a strong research base. And at any point in time, there are a variety of improvements in support and other programs that will deliver better results from public investment in R&D and commercialisation (box 5.1). But by themselves these are not game changers.

What is a game changer is improving access to, and the availability of, data, and making sure that firms can operate in an environment of intellectual license.

Box 5.1 Governments and the innovation ecosystem

  • Governments can help or hinder the development of the innovation ecosystem. Too much support can potentially hinder development if firms become focused on extracting subsidies or getting other special treatment by government. While most innovators say they want government to get out of the way, some have commented on a ‘waiting for government’ mindset in Australian firms.

    Too often “solutions” are about asking state and federal governments for something rather than about fostering a new kind of private-sector driven entrepreneurship and risk.
    (Plunkett 2017, p. 5)

    Governments need to assess whether their interventions are helping or hindering, including by considering the incentives they are creating for firms to wait rather than innovate. Some recent reviews have made recommendations that aim to better align the incentives so that public funding is less likely to crowd out private investment (SP 12). These include:

    • reforming the Research and Development (R&D) tax incentive, which accounts for a third of the almost $10 billion public investment in innovation related activities (including basic research funding), as recommended by the Ferris, Finkel and Fraser review
    • consolidating small grant programs to reduce duplication and overlapping programs within and across jurisdictions, and evaluating the impacts of these programs.

    More comprehensive evaluation of the effectiveness of industry support programs, including those aimed at stimulating innovation, will be easier with the development of the Business Longitudinal Analysis Data Environment (BLADE).

    As governments set the regulatory environment, they can facilitate the innovative activity of firms (SP 13) through:

    • ensuring that new products can meet regulatory requirements quickly, which can be critical in bringing them to market. This requires a responsive regulatory regime, that also gives consumers confidence that the risks are managed
    • adopting open and common standards across jurisdictions, which can reduce costs for users and assist firms to sell into other markets
    • providing leadership in setting up strategies to build cyber security systems and capabilities and coordinating the adoption of standards and infrastructure needed to support the internet of things.

    Finally, as governments are major purchasers of goods and services, government’s willingness to offer opportunities to more innovative firms can play a role in providing space for firms to experiment. Successful sales to government can provide a platform for penetration into other markets.

Data and its analytics is the most significant
renewable resource discovery this century

Improving access to data

Data (and its analytics) is the most significant renewable resource discovered this century.

Extraordinary growth in data generation and usability has enabled a plethora of new business models, products, services and insights. These in turn are transforming everyday life — driving safety and efficiency, improving the use and allocation of resources, and enabling better decision making.

Data frameworks and protections developed prior to sweeping digitisation need reform, and were the focus of the Commission’s 2017 Inquiry on Data Availability and Use. Governments could stimulate innovation and new opportunities by:

  • making the substantial data that they collect and curate more readily available in forms that still protect the security of data and the privacy of data sources
  • empowering consumers to use and benefit from their own data.

Data that allows performance monitoring and comparison of government activities is a fundamental starting point. Governments themselves can make better use of data to improve delivery of services and enhance their own functioning and efficiency.

Access to data more broadly would enable capable and trusted researchers to play a more active role in developing solutions to seemingly intractable problems. This can be achieved through early and routine release by governments of non-sensitive datasets, and the adoption of robust processes for assessing and managing risks associated with other datasets to better allow sharing. This theme is also taken up in chapter 2, as data are at the core of delivering higher quality, integrated health services.

In addition to these benefits from improved access to data, the Commission’s Inquiry on data highlighted the role of data as a potential barrier to competition, but also an important enabler of consumer control and choice.

The central plank of the Commission’s report and recommendations was an overarching data access law (a Data Sharing and Release Act) that would give consumers — individuals and small businesses — a comprehensive right to access their data and direct that it be provided to third parties. This would enhance competition by enabling consumers to have their data, for example that accumulated over years by their bank or telecommunications company, transferred to potential alternative suppliers.

The ability to drive competition in this way will likely significantly increase in value as data collection continues to grow. But the benefits of the comprehensive right could extend beyond competition between existing providers by enabling further innovation in products and services. The Commission’s report recommended allowing each sector to develop its own rules about what data the comprehensive right will apply to, and how they will release that data.

Some speculative assessments suggest that improvements in the availability of public data could generate value of over $4 billion annually (Lateral Economics 2014, p. 28). Moreover, the reforms that allow consumers greater control over sharing their data could be worth over $1.5 billion a year through greater, and better informed, choice in banking, insurance and utilities (SP 13).

Even shifting the dial by 0.1 percentage point is
worth almost $1.9 billion to our economy

Creating an environment of intellectual license

Intellectual property (IP) arrangements, including copyright, play an important role in ensuring creators and inventors are rewarded for their efforts. However, as noted in the Commission’s Inquiry into Intellectual Property Arrangements (IP Inquiry), while copyright encourages investment in creative works by allowing creators and rights holders to exploit their value, it is poorly targeted and broader in scope than needed. It provides the same levels of protection to: commercial and non-commercial works; to those no longer being supplied to market; and to those where ownership can no longer be identified.

Conditions in licences and assignments of patents, registered designs, copyright, and eligible circuit layout rights are currently exempted from most of the laws dealing with anticompetitive business practices by subsection 51(3) of the Competition and Consumer Act 2010 (CCA). Both the Competition Policy (Harper) Review and the IP Inquiry recommended repealing the section, noting that commercial transactions involving IP rights should be subject to competition law in the same manner as transactions involving other property and assets (Harper et al. 2015; PC 2016c).

A consumer is estimated to infringe
current copyright laws over 80 times a day

A system of exceptions to copyright enables limited use of copyright material without the authorisation of rights holders. However, the IP Inquiry found Australia’s current exceptions for fair dealing are too narrow, inflexible and prescriptive. They do not reflect the way people consume and use content in today’s digital world, nor do they accommodate new legitimate uses of copyright material. For example, the existing law only introduced limited permission to make a personal-use copy of a videotape in 2006, which was 26 years after VCRs were introduced, and 8 years after the arrival of DVDs, which superseded VCRs. As a result of the existing prescriptive exceptions, a representative consumer is estimated to infringe the copyright of non–commercial and commercial works over 80 times a day.

Problems caused by the current prescriptive system include frustrating the efforts of online businesses seeking to provide cloud computing solutions, preventing medical and scientific researchers from taking full advantage of text and data mining, and limiting universities from offering flexible Massive Open Online Courses (MOOCs).

Moving from the current legislated mechanisms that only enable use of copyright material in tightly defined situations (‘fair dealing’ exceptions) to a principles-based system, which considers whether use of copyright material would harm the right holder’s interests (‘fair use’), would allow Australia’s copyright arrangements to adapt to new circumstances, technologies and uses over time. This is the approach taken in the United States and many other countries.

Moving to fair use would unlock many opportunities and avoid unnecessary payments. For example, moving to fair use would avoid the current situation where education and government users pay $18 million dollars per year for materials that would be accessible under fair use provisions (chapter 3).

Draft legislation released in December 2015 proposed expanding the safe harbour protection from copyright infringement to include search engines, universities and libraries. However, the provisions relating to safe harbour were removed from the bill as introduced with the government noting it would further consider feedback received on the proposal. The removal raised concerns from a number of tech firms that offer two-way platforms (that allow users to upload their own content that is then bought by other users) such as Redbubble and Envato. Fair use provisions recommended by the Commission could address this problem by allowing use of copyright material in a way that does not impact on the revenue stream from the intellectual property rights of the creators (SP 13).

It is very hard to say what impact this and the other changes recommended in the IP Inquiry would have on productivity. Some benefits simply comprise lower costs in the IP system, but much bigger gains will arise from greater innovation and diffusion (appendix B).

Recommendation 5.2 Creating an environment more conducive to innovation

Australian governments must be more responsive and willing to experiment to create a more innovative ecosystem for Australian business.


There are a number of things Australian governments can do to create an environment more conducive to innovation without giving firms an incentive to seek support. Such action will help, but four other areas where governments can make a material difference is in:

  • establishing consumer rights over their own data, including the right to transfer their data
  • removing the barriers to greater use of public data, including developing secure access that still respects privacy
  • adopting a copyright law with fair use exceptions
  • removing the competition law exemption for intellectual property.

5.3 Improving the performance of the regulatory system

Regulator behaviour and the regulations they enforce can raise the cost of doing business, but they are often essential in making the market work. We highlight some areas where new technologies are changing how the regulatory system can interact with markets to the benefit of consumers and firm efficiency (SP 13).

Consumers lack the power to provide market discipline

A regulatory system that empowers consumers, through information and effective complaint and redress systems, helps bring market discipline to bear on providers. Digital technologies (such as social media and comparison platforms) are providing new ways of collecting, analysing and supplying information on the consumer experience of goods and services. But they need credible quality assurance mechanisms to be effective.

Governments can help empower consumers through improving access to reliable and timely information and ensuring that consumers have effective access to complaints and redress mechanisms.

While improving the information available to consumers enhances their ability to impose market discipline, these mechanisms work best where there is a strong consumer safety net in place. That is, market participants that seek to exploit consumers, workers and other organisations face real sanctions, and consumers have access to redress. This requires regulators with teeth.

Regulator behaviour results in higher compliance costs than required

Reviews that have focused on the costs of red tape have found that much of the unnecessary compliance costs can be traced back to how the regulators implement the regulations rather than requirements in the regulations themselves. The Australian Government’s Public Governance, Performance and Accountability Act 2013 requires greater monitoring of the performance of regulators with the aim of improving their performance.

While this is an improvement on past governance arrangements, it may fail to properly engage the regulated entities, which are best placed to identify what it is that needs to be done to reduce unnecessary costs, and to report back if this is being achieved. After several years of operation the Regulator Performance Framework should be reviewed to assess if there are better, and less onerous, ways to achieve the desired improvements in regulator performance. Improvements in performance can also be hampered by lack of capabilities in the regulator. Capability reviews, such as the review of Australian Securities and Investments Commission (ASIC), can identify what is needed to bring about change in a regulator, including whether the governance arrangements are sufficient.

Digital technologies also offer regulators new ways of engaging with their clients:

  • Digital interfaces offer a way of providing a much more seamless and integrated process for business seeking information, approvals, notifications and other compliance requirements. Governments have moved to introduce single entry portals, some more successfully than others, and latecomers should look to the leaders for guidance on what works. As much, or more, attention is needed to ensuring agency cooperation as to delivering technical solutions.
  • Technologies allow greater sharing of data that can support real time risk identification and management. More systematic learning from regulatory actions can be used to inform firms how best to comply at least cost. Machine learning offers the opportunity to distil information on regulatory requirements in a way that can be tailored to the needs of individual firms.
  • RegTech — digital solutions that enable firms to meet regulatory requirements at considerably lower cost — can embed ‘compliance by design’. ASIC’s Innovation Hub, established in 2015 to assist FinTech start-ups developing innovative financial products or services to navigate the regulatory system, reported engagement with 30 RegTech companies. The NSW Government has recently released a new digital strategy, with commitments to make the client the centre of service delivery. An important aspect of this strategy is to ensure that all legislation enables digital by design — that is, it will not prevent new digital technologies and business models from being adopted.
  • As they lower the cost of information exchange and disrupt current practice, digital technologies can provide the catalyst for better coordination across regulators in some areas. Priority areas include chemical regulation, which remains fragmented.
  • Better information can also be used to design and deliver more efficient cost sharing arrangements.

5.4 An agenda for regulation reform

Making markets function efficiently requires action on at least four fronts:

  • ensuring macroeconomic stability
  • providing the legal basis for exchange
  • ensuring that markets treat players equally
  • providing coordination where cooperative approaches fail to emerge organically (North 1991).

Market regulation reform focuses on the last two of these areas. The main tool at the disposal of governments is better-judged regulation, to instil competition, manage externalities (where the behaviour of one firm impacts on others) and to encourage collective action where it is needed to develop markets. There is no shortage of guidance to governments on what should be included in the traditional regulation reform agenda, and a brief outline is provided here with more details in appendix B. Looking to the next five year review, some areas where the policy solutions are not obvious, and further research is needed, are raised.

The traditional regulatory reform agenda

Many sound recommendations have been made in a variety of reviews that would address impediments to markets operating more efficiently, but these have yet to be implemented or are progressing more slowly than desired.

After trawling through hundreds of recommendations from multiple reviews and assessing them on their merit and materiality, the Commission has harvested a rich repertoire of opportunities for an orthodox microeconomic reform agenda — that are set out systematically in appendix B, where references and benefit estimates can be found. The appendix arranges reforms in two overarching themes:

  • improving competition
  • reducing unnecessary regulatory burden.
Reforms quantified in appendix B
come to more than $3 billion a year

Realistically, it is beyond the capacity of government to do all of the reforms outlined in appendix B immediately. The Commission recognises that political capital and bureaucratic resources are finite, and there is a need for an agenda led by initiatives that will indicate to a doubting public how reforms will improve their lives. But just selecting judiciously from these reforms as part of an agenda will provide a significant boost to national productivity. The net gains to the economy from the listed reforms quantified in appendix B come to more than $3 billion a year, to which can be added potentially large gains (in the many billions) from reforms to industry assistance, government procurement and reforms to IP (see above). While most of the individual estimates are uncertain, taken together they add to a considerable case for reforming regulation and industry assistance. Deloitte Access Economics has estimated that a package of microeconomic reform, combined with favourable economic conditions in Asia, would raise Australia’s GDP by 2.5 per cent (Deloitte Access Economics 2017).

Some areas that need better guidance on future directions

Digital technologies are posing new challenges for regulators and regulation. For example, control of data and networks is growing as a source of market power. But the trade-off between the benefits offered and the persistence of the ability to extract rent is less clear than in the case of natural monopolies. The Commission has advocated a wait and see approach in this situation. But there are other questions that have emerged during the process of this Review where a more proactive approach is needed to discover the right way governments should regulate. Four areas that should form part of a research agenda are discussed below.

Boosting competition given the small size of the Australian market, oligopolies, and the nature of shareholder discipline

Competition is a force for good in that it drives the owners of firms to pursue productivity and deliver goods and services that consumers want or other firms need. But it is less effective if the interests of the managers of firms do not align with those of the owners of firms. And it is harder to achieve where the market is relatively small and economies of scale and scope are large. The rising share of lagging firms suggests that owners have become less effective in driving good management.

There could be several systemic factors. Digital disruption has made it easier for firms to outsource much of their production, focusing on the design and develop and the delivery and service ends of the production value chain (PC 2016b). Notionally this should boost productivity, but it may well take more management skills and investment to do well (Berger 2014).

A number of studies, in Australia and elsewhere, have pointed to the quality of management as a major factor in the performance of firms (Bloom et al. 2007). A study that applied the London School of Economics management measures to Australian manufacturing firms (with more than 50 employees) found overall they rated only moderately above average, with considerable variation in performance (Green 2009). There are a number of elements worthy of investigation to better understand why firms are lagging, and what role, if any, policy can play in addressing this problem. The firm longitudinal data (BLADE) will provide researchers with a better resource for conducting this research, which should inform the next five year productivity review.

In questioning how well markets are functioning, various researchers have pointed to more fundamental change reducing the ability of corporate governance arrangements to focus firms on long-term growth (SP 1). For example, big pension funds are increasingly the main shareholders so have a role to play in holding company management to account. How well they are doing this needs to be assessed.

At a firm level, an incentive to manage for short-term profit will tend to result in underinvestment in innovation and a more incremental approach to research and development (R&D). Concerns have also been raised about the incentives that firms have to train staff, given the growing view that employment is no longer for life. At a system level, such trends imply less interest in collaboration in R&D investments that benefit other firms, and a greater reliance on workers to fund their own training, or for the government to fund both the training and the R&D.

Adding to these trends could be the apparent market preference for the breakup of vertically integrated firms. It has been argued that has contributed to the loss of manufacturing in the United States (Berger 2014). The firms that emerged from this were more specialised, and had less interest in and resources available for supporting training and research that would benefit other firms. This type of ‘depletion of the industries ecosystem’ leaves many firms less able to respond to changing market opportunities (Berger 2014, p. 6). It also affects on-the-job training for workers.

Another trend has been a rise in the market share of private equity firms, which have lower disclosure requirements. Publicly listed firms are required to provide considerably more information to allow shareholders to assess performance, and there are examples of poor outcomes from private equity takeovers of public companies. For example, following the collapse of retailer Dick Smith, a Senate inquiry into the causes and consequences of the collapse of listed retailers in Australia was established in February 2016. While submissions were taken and the inquiry granted an extension, it lapsed at the 2016 federal election and the Senate agreed not to re-refer the inquiry to the new Parliament (Senate Standing Committee 2016). More investigation is required to assist governments to understand if these types of trends reflect the current regulatory rules governing corporate behaviour. The extent to which the market rules can promote positive spillovers across firms must be considered in formulating any policy response.

Beyond setting rules to encourage good corporate governance, there is a question of whether government has any role to play in assisting firms to improve their performance. Those that argue that government can be proactive often cite the German system where the many medium-sized manufacturing firms are supported by an ‘ecosystem rich in research consortia, Fraunhofer Institutes, specialist suppliers, technical universities, apprenticeship training, and local and regional banks’ (Berger 2014, p. 6).

However, there is yet to be any reliable evidence that such industry programs will work in Australia (Baily 2016; PC 2007).14 In part, the relatively high concentration of firms in many industries makes it more difficult to ensure that the returns to such investments are widely shared. In part, it is because the scale available to Australian firms means efforts have to be concentrated to achieve critical mass so that investment can become self-sustaining. The alternative, which is not uncommon, is what can be termed the ‘vegemite solution’, where governments spread their funds widely in the name of fairness, but lacking any critical mass to make a difference. Research to build an evidence base is needed to better understand the causes of poor performance in the Australian market. This will include evaluation of the effectiveness (or not) of government programs, and assessment of the transferability of models that work well elsewhere.

There is a question over whether better information on relative performance can make a difference. There are some industries, notably utilities, in which benchmarking is used as a regulatory tool. This includes water and wastewater utilities in the United States, where benchmarking is used to induce the utilities to control costs and become more efficient (Berg and Holt 2002).

More general benchmarking services are less common. An exception is the Inland Revenue Department (IRD) in New Zealand, which provides industry benchmarks and tools for businesses to work out their own performance ratios from the data they provided to the IRD (the Australian Taxation Office (ATO) do this for small businesses). The IRD also offer advice on why your business may perform outside the industry benchmark (IRNZ 2017). Such information can prompt businesses to ensure that they have not made a mistake on their tax returns, which is of benefit to the IRD, as well as prompting a firm to seek external advice. The ATO and ASIC, could consider providing such a service.

Observation 5.1

There is a fundamental lack of adequate assessment in Australia on why such a high share of firms are lagging behind in productivity growth. An evidence base is needed to assess why, including the role that corporate governance, firm ownership, digital technologies, and measurement problems might be playing in this result, to guide whether government policies could make a difference.

Emerging trends in the labour market and perhaps a third category of worker

The distinction between contractors and employees in new digitally-driven services is an emerging issue that was not covered in depth in the Commission’s 2015 Workplace Relations Framework Inquiry. Platforms like Uber, Airtasker and TaskRabbit rely on contractors to supply labour in a market where software brings the customer to the supplier (the ‘gig’ economy). Such platforms can be beneficial for workers (flexibility about when and how many hours to work) and consumers (lower price and greater quality). But there are also risks for both parties, and most particularly a concern that such employment forms might constitute sham contracting. This occurs when a worker is subject to a degree of direction that is more typical for an employee, while not being given the protections of the Fair Work Act or other laws that relate to employees.

The legality of the employment arrangements underpinning Uber and similar platforms is increasingly under challenge in countries that share the Australian common law definition of a contractor (Foulsham and Geddes 2015). Were all platform-mediated employment to be re-defined as involving contracts of employment (an employee) rather than contracts for employment (a contractor) it could thwart the adoption of improved services (including in disability care). It could also deny people who want flexible jobs the chance of a job at all.

A shift in the nature of employment also has implications for the nature of the retirement income system, which assumes a high level of ‘employee-ship’. For the present, the issue is small. For the future, should this become a significant category of worker, some reconsideration of how workplace standards are applied may be necessary.

Observation 5.2

So-called ‘gig’ employment falls somewhere between an independent contractor and casual employment. The current system does not envisage this, like many other regulatory structures facing digital disruption. If growth persists in the gig economy, challenges may arise for the workplace relations and other systems.

Exploration of greater use of market instruments to get the regulatory balance right for environmental regulations

Regulation that aims to protect the environment can impose requirements on firms to demonstrate that any harm will be adequately mitigated and that the benefits from the activity outweigh the costs of any long-term harm. Being exposed to the actual costs of redress or make good can internalise this decision for the firm. As long as they are aware of the consequences of their action and any failure to manage risks, and cannot avoid the financial liability, their decision about whether to go ahead with a project should get the balance between benefits and costs right. The difficulty has been in constructing a mechanism that can impose this liability.

For example, it has been argued that the current laws allow firms to evergreen or put mines into ‘care and protection’ to avoid remediation responsibilities. And there are examples of firms splitting off assets to reduce the firm’s exposure to remediation obligations (for example, Peabody Energy (Environmental Justice Australia 2016)).

More work is needed to explore if market-based approaches could better hold firms to account for ensuring that agreed environmental protection is achieved. That is, establishing property rights over the long-run health of the environmental assets, so firms have to pay for any damage. Hence they have to balance the risk that they will fail to manage the environmental impacts against the profits that they expect to make.

Observation 5.3

Delay may be an inevitable outcome for projects that pose major environmental risk, as governments seek to get the requirements right to protect the environment. It would be worthwhile exploring whether governments (and the community) could make greater use of market mechanisms to hold firms to account for remediation of the area directly affected by the project and redress for any damage imposed outside this area.

The correct regulatory settings are essential for privatisation to deliver net benefits

Infrastructure Australia (2016) has made a number of recommendations in relation to privatisation, a topic also considered in the Commission’s report on Public Infrastructure (2014d).

The first of these reports recommended that governments should exit direct service delivery where a competitive market for supply of infrastructure services exists. What this constitutes needs to be established. Action is also needed to embed sound principles into planning and tendering for public infrastructure projects and to make sure they are followed.

Caution has also been flagged in privatising infrastructure assets. For example, the Senate Inquiry into Privatisation of State and Territory assets and New Infrastructure advised caution, rigour, consultation, and regulation when privatising (2015).

The Chair of the ACCC, Rod Sims, in a recent speech, argued that unless privatised assets face competition they need effective regulation:

The lack of effective regulation will see higher prices for users and so can see reduced investment by them, thus causing inefficiencies. In addition, the higher price received for the sale of an unregulated asset can effectively be a tax on users or consumers, now and into the future. And it can be a poorly targeted tax on consumers. (ACCC 2016)

As with defining sufficient competition, more attention needs to be paid to determining the best approach to regulation. Price monitoring may be insufficient in some contexts, and commitments to service quality, which could include operating hours and frequency of service, may be required to ensure consumers continue to be able to access what is often an essential service. Sims suggested a ‘negotiate/arbitrate’ approach, which gives the users the threat of referring a dispute over pricing, and possibly other issues, to independent arbitration as a possible solution.

In other work (on negotiations in Workplace Relations), the Commission has identified more sophisticated variants that could be employed. The best approach may well vary across the different types of infrastructure, and more attention should be given to the best regulatory design to manage investment, service and price risks in each case.

Observation 5.4

Privatisation has the potential to deliver considerable improvements in the quality of service delivery and lower prices for consumers. However, the regulatory arrangements that govern the market are critical for a privatised entity to deliver on these benefits. Nowhere is this more important than in infrastructure and considerably more effort is needed in developing guidance on how to regulate in anticipation of privatisation.


  1. References for the energy section of this paper and greater detail is available in SP 11. References for the other sections, and estimates of benefits, can be found in SP 12, SP 13 and appendix B. Locate Footnote 9 above
  2. The Finkel Review’s issues paper identified 20 other reviews that were underway or had been recently completed. Following the recent problems in South Australia and with the National Electricity Market, a number of additional reviews have been undertaken or commissioned. Locate Footnote 10 above
  3. This is the allowable level of emissions across sectors and/or the economy, and differs from the Renewable Energy Target (RET), which sets the share of electricity that must come from renewable sources. Locate Footnote 11 above
  4. By some calculations, achieving the Paris Accord commitments of a reduction of 26 to 28 per cent on 2005 CO2 emissions will require 50 per cent of electricity to be generated by emissions-free sources (it requires a 65 per cent reduction in the emission intensity of the economy between 2005 and 2030). Locate Footnote 12 above
  5. Non-synchronous refers to electricity having variable frequency and voltage and lacking inertia (which assists in dealing with changes in frequency). Non-dispatchable reflects the intermittent nature of solar and wind technologies. Unlike most renewables, hydroelectricity is synchronous and dispatchable, as are the traditional sources of coal fired and gas generation. Locate Footnote 13 above
  6. A possible exception is the Rural Industries Research and Development matching grant system, which is supported by a sound governance regime to ensure that R&D projects are directed by industry. The Commission’s review of the system (PC 2011c) found that it had been effective in raising productivity in agricultural industries but that, as the gains were mainly captured by industry, the public contribution should be reduced. Locate Footnote 14 above

Report homeContentsNext chapter