Monday, 21 November 2016

Sour grapes and self-delusion: ACCC Regulator Performance Framework self-assessment report 2015-16

Introduction
I recently had a look at the ACCC's Regulator Performance Framework self-assessment report 2015-16 which was released on 22 November 2016.  Unfortunately, while the headline seemed to suggest that the ACCC was performing well against its KPI’s, the Report also showed that the ACCC’s Enforcement area is performing poorly in relation to almost every one of its KPIs.  The ACCC needs to be genuine about the feedback it has received and take a good hard look at the way in which its Enforcement area is operating. It is simply too easy to discount feedback from business and legal practitioners as “sour grapes” and to then massage the numbers to hide the real and pressing problems.

Purpose of the Report
The purpose of this report as follows:

The framework is concerned with how Commonwealth regulators administer regulation, with the aim of encouraging regulators to undertake their functions with the minimum impact necessary to achieve regulatory objectives. It is therefore important to note that the framework does not seek to measure the performance of the ACCC in relation to the outcomes we achieve for Australian consumers and the economy. This said, there is some interaction between the ACCC’s outcomes and the way in which we seek to minimise necessary impact and these emerge in this assessment.

Glowing self-assessment with a few sour grapes
It would seem that the ACCC also performed quite well against its benchmarks, based on Chairman Ros Sims’ self-assessment of the Report:

The ACCC considers we have established a solid benchmark with the 2015-16 self‑assessment report on which to measure and compare our performance in future reporting periods.

A majority of the ACCC’s business stakeholders have a positive view of our performance. However, businesses which have been the subject of recent ACCC enforcement activity gave less positive assessments of the ACCC’s enforcement functions.

Some key areas for attention and improvement have been identified across the ACCC, particularly in relation to fostering better communication with businesses that are subject to investigations and regulatory actions.

In other words, in the ACCC’s view, its performance across most business areas was quite positive with the exception of the Enforcement area.  However, Sims appears to explain these criticisms away as nothing more than “sour grapes” – namely, the businesses which had been the subject of recent ACCC enforcement action have provided negative feedback. Indeed, the implication is that these businesses and their legal advisors have provided false information to the ACCC about the way in which the ACCC Enforcement area is operating.

Enforcement area – the real deal
Unfortunately, when you drill down in more detail to the ACCC’s Report you realise that the responses received in relation to the ACCC Enforcement area bear little resemblance to the way in which the ACCC has presented those results.

The following table shows the various responses received from survey participants about each of the ACCC’s major areas.  The total percentage of respondents that agreed or strongly agreed with a statement regarding our performance against the KPI is represented in green, while the percentage that disagreed or strongly disagreed is in red for each function area.

M&AR
%
S Bus
%
P Safe
%
InfReg
%
Enforce
%
KPI 1:  The ACCC does not unnecessarily impede the efficient operation of regulated entities
47 / 21
(+26)
53 / 13
(+40)
65 / 11
(+54)
48 / 16
(+32)
29 / 43
(-14)
KPI 2:  The ACCC’s communication with regulated entities is clear, targeted and effective
52 / 16
(+36)
59 / 13
(+46)
64 / 13
(+51)
68 / 13
(+55)
40 / 33
(+7)
KPI 3:  The actions undertaken by the ACCC are proportionate to the regulatory risk being managed
42 / 21
(+21)
44 / 21
(+23)
57 / 14
(+44)
32 / 19
(+13)
38 / 38
(+0)
KPI 4:  The ACCC’s compliance and monitoring approaches are streamlined and coordinated
33 / 14
(+19)
36 / 18
(+18)
51 / 15
(+43)
52 / 26
(+26)
29 / 50
(-21)
KPI 5: The ACCC is open and transparent in its dealings with regulated entities
59 / 11
(+48)
50 / 20
(+30)
60 / 8
(+52)
71 / 6
(+65)
25 / 42
(-17)
KPI 6:  The ACCC actively contributes to the continuous improvement of regulatory frameworks
43 / 11
(+32)
53 / 16
(+37)
45 / 12
(+27)
37 / 27
(+10)
30 / 40
(-10)

As is apparent, responses in relation to the Enforcement area were negative in relation to four of the ACCC’s 6 KPI’s, the fifth KPI came out even and KPI 2 was positive.

However, when one examines the ACCC’s self-assessment ratings an entirely different picture appears:

M&AR

CSBPS
Inf Reg
Enforce

KPI 1:  The ACCC does not unnecessarily impede the efficient operation of regulated entities
Satisfactory
Satisfactory
Good
Satisfactory
KPI 2:  The ACCC’s communication with regulated entities is clear, targeted and effective
Good
Very good
Very good
Satisfactory
KPI 3:  The actions undertaken by the ACCC are proportionate to the regulatory risk being managed
Good
Good
Satisfactory
Good
KPI 4:  The ACCC’s compliance and monitoring approaches are streamlined and coordinated
Satisfactory
Satisfactory
Satisfactory
Poor
KPI 5: The ACCC is open and transparent in its dealings with regulated entities
Very good
Good
Very good
Satisfactory
KPI 6:  The ACCC actively contributes to the continuous improvement of regulatory frameworks
Good
Satisfactory
Satisfactory
Good

The ACCC has self-assessed negative feedback from survey participants in relation to KPI 1 - The ACCC does not unnecessarily impede the efficient operation of regulated entities - as satisfactory despite 29% of respondents stating that the ACCC was doing a good job in relation to this KPI and 43% of respondents disagreeing.  How the ACCC could possibly self-assess its performance against KPI 1 as “Satisfactory” is beyond me.

The self-assessment in relation to KPI 2 - The ACCC’s communication with regulated entities is clear, targeted and effective - is more defensible. 40% of respondents believed that the ACCC was doing a good job in relation to this KPI while 33% disagreed.

KPI 3 which relates to - Whether the actions undertaken by the ACCC are proportionate to the regulatory risk being managed - comes out at a commendable “Good” rating, which seems somewhat surprising given that 38% of respondents thought that the ACCC was doing a good job and 38% of respondents disagreed. I think that a more accurate conclusion in relation to KPI 3 would have been a “Poor” rating, given half of survey respondents thought that the ACCC was doing a poor job.

KPI 4 - The ACCC’s compliance and monitoring approaches are streamlined and coordinated - comes in rightly at a “Poor” rating, with 29% of respondent’s believing the ACCC was doing a good job and a massive 50% disagreeing with that statement.  This result should be of great concern to the ACCC, even allowing for the “sour grapes” effect.

KPI 5 namely that - The ACCC is open and transparent in its dealings with regulated entities - is again very difficult to understand given that the ACCC has self-assessed itself as “Satisfactory” with 25% of respondents believing that the ACCC was doing a good job in being open and transparent, with 42% disagreeing.  In reality, the ACCC’s self-assessment for KPI 5 should have been yet another “Poor”.

Finally, in what must be the considered the most puzzling example of the ACCC’s questionable self-assessment, the ACCC concluded that in relation to KPI 5 - The ACCC actively contributes to the continuous improvement of regulatory frameworks - it merited a “Good” grading. This is despite 30% of respondents believing that the ACCC was doing a good job and 42% disagreeing.

It is quite remarkable that the ACCC has firstly sought to discount the responses which it received from businesses and their legal advisers about the ACCC’s Enforcement performance as “sour grapes”.  What is more remarkable is that it has then sought to present the statistics in a way which masks the reality and depth of the problems which exist within the ACCC Enforcement area.   

 In my view, the ACCC’s Enforcement area’s report card against its KPIs should look more like this:

Enforce

KPI 1:  The ACCC does not unnecessarily impede the efficient operation of regulated entities
Satisfactory
Poor
KPI 2:  The ACCC’s communication with regulated entities is clear, targeted and effective
Satisfactory
KPI 3:  The actions undertaken by the ACCC are proportionate to the regulatory risk being managed
Good
Poor
KPI 4:  The ACCC’s compliance and monitoring approaches are streamlined and coordinated
Poor
KPI 5: The ACCC is open and transparent in its dealings with regulated entities
Satisfactory
Poor
KPI 6:  The ACCC actively contributes to the continuous improvement of regulatory frameworks
Good
Poor


Conclusions
The ACCC has to “get real” about the performance of its Enforcement area.  As stated above, it not acceptable for the ACCC to attribute poor feedback from businesses and legal practitioners to the “sour grapes” factor and then to massage the statistics to hide the reality and depth of the problems within the ACCC Enforcement area.  A self-assessment which does not accept the criticisms made is not a self-assessment at all, but rather an exercise in self-delusion.



http://www.accc.gov.au/media-release/accc-completes-first-regulator-performance-framework-review

No comments: