Blueprint for what?

I have just completed reading the long awaited “Impact” report of the Home Office’s Blueprint Drug Education project. As then Drug Education Forum Chair and one of the Drug Education Forum representatives on the project’s Advisory Group, I am disappointed, frustrated…

The project was promoted from the outset as the most important drugs education research project ever  to be undertaken in the UK. For those, like myself, who continue to believe in the importance of early intervention at school, family and community levels with young people, Blueprint was supposed to show us whether providing drugs education by Blueprint’s methods, based on effectiveness research from elsewhere, could have significant impact in a UK context.

It is difficult to exaggerate how important this project was promoted as being within the UK drugs education field.

I am not a statistician and I had always believed that the design had been properly validated by experts. In 23 schools we expected to be able to assess the differences in young people’s knowledge, skills, attitudes and behaviour when they benefited from the multi-component Blueprint programme, compared with students in 6 schools who didn’t get the programme.

Unfortunately I’m sure that the authors’ conclusions in this final report are absolutely valid. They state that, because of a flaw in the research design, comparisons cannot be drawn between the two groups. The report states:

“The original design of the Blueprint evaluation was not sufficiently robust to allow an evaluation of impact and outcomes and consequently the report cannot draw any conclusions on the efficacy of Blueprint in comparison to existing drug education programmes.”

Some important questions need to be answered. These include:

How could this happen with such an important, high profile and costly project?

At what stage were the flaws in the research design detected so that it became clear that comparisons between the 2 groups would not be valid? What decisions were then made, why and by whom?

What, if any, reasonable conclusions about impact on knowledge, skills, attitudes and behaviour can we draw from the study of the students who received the Blueprint intervention?

What implications, if any, do the findings have for future drug education practice and policy guidance?

What further research needs to take place?

About these ads

15 responses to “Blueprint for what?

  1. I suspect that the flaws in the research design were discovered by the Home Office at some point after the researchers reported that drug use was *higher* in the Blueprint schools than in the comparison group.

    By the way, good luck with the PhD.

    • Thanks Alex. I am very annoyed about this, as I think we should be. Thanks for your good wishes – it’ll be fun but a lot of work.

  2. Totally agree with this Eric. it truly beggars belief that the quality control at the outset could have been so scandalously poor. Any decent academic stats/survey expert would have seen the obvious problems with the methodology. These problems were entirely forseeable.

    These were not new concerns – I raised the issue of inadequate controls on several occasions at various forums and events – like you I am no stats nerd, but it seemed obvious to me that there was something very wrong . The over-emphasis on process indicators was worrying from the outset, but even those – the only lessons they seem to be able to put forward, are utterly irrelevant with ANY indication of health outcomes.

    It is a staggering failure. I can only assume a combination of gross incompetence bourne of a particular political environment which demands success and is naturally phobic of genuine evaluation.

    If there is any positive in this it is that at least they have been honest enough to admit the whole thing was, in effect, a complete waste of time and money. Which is more than can be said of the drug strategy review of 2007.

    I also think this is in many respects a tragic missed opportunity. Even if the study had been done properly and the results had been rubbish, at least we would have known and could start looking at other approaches. This non-study has merely wasted 3 million pounds of precious research money and held up the research agenda for 4 years.

    • Steve, thanks for these comments.

      I find it hard to be as generous as you though in your penultimate paragraph. The whole point of undertaking research is to learn. The UK evidence base for drugs education is woefully poor and this was an incredibly important and long-awaited programme. It cannot possibly be described in any way as positive that there is now an admission of such a cock-up.

      First, the research design should have been validated and validated again before embarking. As a non-statistician but someone who has been involved with the Blueprint project in an advisory capacity, I understood this to have happened.

      Secondly, we should always be open to learn (from such valid research in the UK, also drawing on experience elsewhere such as the EUDAP programme in Europe) and to change direction, where we learn new things. Let’s face it, no one is saying we’ve got the drugs “problem” solved so we’re doing something wrong. Even as a champion of drugs education for a long time, I have never argued that it’s provided either consistently or effectively in all our schools and communities. And the purpose of a well-conducted Blueprint research programme should have been to learn – and change practice, if appropriate.

  3. i was very shocked when i read the report too. it seems to have been a total waste of time, when the money could have been better spent elsewhere! very frustrating indeed…

  4. If the results were considered to be robust – then they appear to show that the project was completely ineffective. This would probably have made a worse ‘headline’.

    It seems like a good plan for a politician – if your initial results are not robust, but don’t say what you want, then publish them as they are; if they do show what you want, then extend the study until the results are sufficiently validated.

  5. I have the disquieting thought that this worthy effort was doomed from the start.

    Politicians already “know” that the only acceptable national initiatives in this field have to be in line with the orthodoxies: all drug taking is inherently evil (except fags and middle class wine tippling); the religious right and the tabloids are entitled to insist on continued criminalisation of (some) drug use and we certainly cannot question the global military-industrial enthusiasm for a War on Drugs.

    A frisson of embarrassment over a wasted opportunity was always going to be politically easier to take than any statistically valid
    implication that the broad thrust of government policies over many years has been worse than useless.

  6. EvidenceMatters

    It’s not been a good year for the outcome of costly, large-scale interventions that target young people. See Health outcomes of youth development programme in England: prospective matched comparison study.

    No evidence was found that the intervention was effective in delaying heterosexual experience or reducing pregnancies, drunkenness, or cannabis use. Some results suggested an adverse effect. [M]ethodological limitations may at least partly explain these findings…

    Mission: Assess impact of intensive, multicomponent youth development programme including sex and drugs education (Young People’s Development Programme) versus standard youth provision.

    Main outcomes: levels of pregnancy, weekly cannabis use, and monthly drunkenness at 18 months.

    Results: as above. Alex Stevens may note that the intervention seems to be associated (without prejudice as to causation) with worse outcomes.

    Lessons learned: Eh, systemic problems with implementation and project design.

    Cost of these lessons: circa £6 million (ignoring human costs, diverted funds etc.).

  7. It is very difficult not to be cynical about this. Clearly the research was flawed from the beginning and I seem to recall plenty of people mentioning this – both at the time and subsequently. It is a notoriously difficult, some would contend impossible, task – not least because outcomes are influenced by so many different factors, many of which are outside the scope and influence of the drug education intervention. Knowing this should have informed the design of the research, crucially in dictating the (inevitably large) size of the sample necessary for the research to be meaningful. In fact it seems evident that the HO knew this, or had it brought to their attention some way into the project, but declined to provide the, admittedly considerable, extra funding. The management of the research team also bear a lot of responsibility for this farrago. It is difficult to believe that these research tools were not independently validated. Clearly there will not be a lot more multi-million pound research projects into this subject in the near future so all in all it has been a missed opportunity for the drugs field and obviously for society at large.

  8. Alex Sutherland

    I wonder if the project could have been salvaged by comparisons with data from other schools not in the catchement area that didn’t receive the “Blueprint” treatment? I mean, if all they’re doing is looking at prevalence rates as the outcome measure, there must be other data out there that could serve as useful comparisons?

    Also, the original analysis doesn’t sound inspiring – predicting future use from previous use is the veritable chocolate teacup of drug research. I wonder if there’s any point asking the HO to release the anonymised data? It sounds like it could be interesting if someone can get their hands on it…

  9. David Uffindall

    Several years ago I was party to some initial discussions about Blueprint. It became clear from the outset that the programme was to be designed to the research model. When I learned that selective schools did not fit the model, schools in one authority which took pupils from another authority did not fit the model and that even denominational schools were not very welcome I feared then that this was flawed from the outset. I advised my then employer to have no part in it. Subsequently my professional judgement regarding Blueprint and my right to assert this was contested by my then line manager. It now appears that my judgement was right all along. I take no pride in this, but I am angry that millions of pounds have been spent on such an obviously flawed project when this money could have gone to schools and even charitable bodies with proven track records in producing high quality drug education materials.

  10. As a user of cannabis since the sixties i have watched successive governments lie and obfuscate on the drugs issue over the years.
    My conclusion is simple – there is no incentive to change the status quo without permission from the USA and any attempt to make the laws less draconian will be met with more bullpoo and lies.
    The recreational drugs issue is a purely political one and has no connection with the health of the nation.
    It has everything to do with the big pharmaceutical manufacturers, petrochem giants and food producers and more particularly their shareholders.
    To pretend otherwise is ostrich territory.

    The only real answer would be to copy Portugal and the Netherlands and see how it goes.

  11. Dave Parsons

    Who cares how or why mehpedrone was criminalised as long as it is…You should try tackling the effects of this filthy drug firsthand as we are, before you sprout such puerile rubbish. So you feel that you have not had time to consider the problem – hard luck.

    Mephadrone and all other chemical concoctions should be automatically banned from sale in the UK until such times as they are tested in the UK and found to be harmless. This would help stop foreign companies flooding the country with such dangerous products.

  12. Franklin Percival

    Calm down Mr. Parsons, do please. People have taken recreational drugs since the dawn of history.

    Following your logic we should immediately ban C2H5OH, which can easily be made at home, but the ban on this compound, widely found to be harmful in many peer-reviewed papers in UK and elsewhere, will cause social uproar. The Daily Mail does not qualify as a peer-reviewed paper, by the way.

    Why do you blame foreigners though? They are only supplying a demand?

    The situation may just be a little more complicated than first appears.

    Why, for example, in a wealthy society such as ours do somany young people find real life so unbearable that they constantly want to get off their faces?

  13. Dave Parsons

    Frank

    I am sure that young people do not find life unbearable they just find that they naturally need to rebel in various ways…..I can remember when I felt the same….. However I did not take drugs because they were not so cheaply available.

    Supplying a demand is not excuse enough for shipping misery from abroad. Why can’t all drugs and chemicals be made unlawful until tested and passed for human consumption? I realise the problem is MUCH more complicated than it appears, I am not stupid.

    However, just because the problem is fraught with problems it should not be ignored by ordinary people and they should make every attempt to influence the so called experts.

    Steps need to be taken to control designer drugs and the loophole that allows these drugs to LEGALLY flood countries should be closed.

    Make chemicals and drugs ILLEGAL UNTIL TESTED.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s