Caterina GiorgiStatistics are very helpful for researchers and policy makers. When looking at alcohol-related harms, they help us to understand the extent of the problem, the nature of the problem the impact of various interventions to address the problem now and overtime.
But when statistics are misused they can cause more harm than good.
Last week the Premier of New South Wales Barry O’Farrell, again referred to a 33% reduction in alcohol-related violence in Kings Cross venues to demonstrate that the interventions that his Government had introduced were working to reduce assaults. This is a figure that he had been criticised for using on the front page of the Daily Telegraph just the day before.
When this misuse of data became front page news, the Minister responsible for liquor, George Souris, released a statement outlining how the 33% figure has been calculated. Souris specified that when looking at the period 7 December 2012 to 31 March 2013 and comparing it to the same period the year before, there had been a 33% decline in alcohol-related assaults in licensed venues. His data referred to the ‘Kings Cross precinct’. This is the area where the intervention was introduced. Souris then tried to fix the Government’s statistics problem by throwing more numbers at it by describing trends over a six month period, which showed a 25.5% reduction.
Following this, it was then time for the NSW Bureau of Crime Statistics and Research or BOCSAR to have their turn at throwing even more statistics at the problem. They described what had been happening in Kings Cross over two years and five years – looking at data for the Police’s Kings Cross Local Area Command (LAC) and the postcode 2011 – two different boundaries to describe Kings Cross (and different to that used by Souris).
So what does this all mean?
While it all sounds quite confusing, the use of data by Souris was misleading and you don’t need a degree in Mathematics and Statistics to understand why. What you do need is to understand the facts.
Fact one – The Premier and Minister have both said that there has been a decline in assaults on licensed premises of 33%, since the Kings Cross Plan of Management was introduced in December 2012. However the media release put out by Souris yesterday says something quite different. It says that there had been a greater decline in assaults from December 2012 to March 2013 than in the same period the year before, that is from December 2011 to March 2012. This approach to looking at statistics is not an approach that researchers would have taken. This is because they would have stated that this isn’t a long enough time period to identify a trend and that statistical testing should be applied to control for general variations that may occur because of chance or seasonal variations (eg. Summer versus Autumn).
Fact two – In response to the front page story, BOCSAR released data that examined trends over five and two years in rates of assaults and had statistical testing applied to it to control for monthly variations. The BOCSAR data did not comment on the Premier’s data, nor did it make a comment on whether the Kings Cross Plan of Management had been effective. This is because a statistical Agency like BOCSAR would not simply subtract the assaults from three months in one year and three months in the next to demonstrate a trend – because that approach does not tell us whether a trend has been observed, as it is not a long enough period to determine this.
Fact three – Souris indicated that the data came from BOCSAR. But upon requesting the data from BOCSAR, the Foundation for Alcohol Research and Education (FARE) was told that they were unable to provide data for the ‘Kings Cross precinct’ because they only had data for the postcode 2011 and the Kings Cross Local Area Command at this point in time. This suggests that the data is not readily available to them and it is unlikely that they prepared the data for Souris.
So was the Premier’s calculation of the 33% figure correct?
The jury is still out. We don’t yet have access to the figures that the Premier is using, but we do know that that data was not easily produced by BOCSAR.
But does that even matter?
Not really. The issue is not whether the numbers match but whether the approach used is the correct approach in looking at changes in alcohol-related assaults. Statistical tests exist so that the significance of differences can be examined to determine whether they occurred because there was an actual change or whether they occurred because of seasonal variance, chance or other general variations.
A further issue is that the Premier is using figures that only look at what is happening on Kings Cross licensed venues. These figures only represent a small proportion of harms occurring in Kings Cross because assaults mostly don’t occur a pub, club or bar. For example, in December 2012 there were 14 alcohol related assaults reported to have occurred on licensed premises while 77 occurred outside licensed premises but in the Kings Cross Local Area Command. Those that are classified as having occurred not on a licensed premise include those that could have happened right outside a licensed venue – these are not included in the Premier’s figures.
So how will we know if the Kings Cross Plan of Management has been successful in reducing alcohol-related harms?
We will know when the Government calls on researchers or the statisticians it has available to it, to evaluate the Plan using data up to December 2013, one year after the intervention. At this point the researcher will also point out to the Premier that it would be worthwhile examining the risk registers of licensees, emergency department presentations at neighbouring hospitals for injuries due to alcohol-related assaults, the numbers of police deployed at Kings Cross and speaking to key informants including police and health professionals about what is going on, on the ground.
We should evaluate the Kings Cross Plan of Management when it is appropriate to do so and when we have these findings available they should be used to inform future directions.
Until then, using any statistics to suggest that the Kings Cross Plan of Management is successful is simply misleading and unhelpful in informing public policy formation.
1 comment