Although only 15 months old, the EU’s Horizon 2020 science and research programme has already received over 45,000 applications, of which just over 14 per cent were approved for funding.
The Commission says these are signs of success, but representatives of leading science institutions and universities questioned what they see as an excessive focus on measuring the immediate economic impact of research projects at the Science|Business conference on Horizon 2020, held in Brussels last month.
Impact is not always immediate and not necessarily easy to measure. But even though it can take decades - if not centuries - for big science projects to have a tangible economic impact, everyone knows about the success stories and why it is worth investing in blue sky research.
Smartphones depend on two big discoveries of the last century: relativity and quantum mechanics. Google maps uses relativity to correct for space-time distortion around the Earth, while the entire semiconductor industry would not exist without the understanding of quantum mechanics.
“Imagine if Einstein [had] submitted a proposal to the European Research Council for his theory of relativity,” said Sergio Betolucci, the director for research and computing at CERN. “He has no record of previous achievements, he’s clearly crazy, and nobody understands the impact; His proposal would be discarded,” said Bertolucci.
Big science and the ‘real’ economy
The new Commission is cooking up an investment plan for more jobs & growth, which will likely lead to cutting €2.7 billion off the Horizon 2020 budget. Although the cuts are in the context of an almost 40 per cent rise in research funding from 2007 to 2013, researchers across Europe are concerned.
There are features of the jobs and growth plan which are commendable, such as tackling youth unemployment and the harmonisation of financial markets, but universities do not like the cuts the Commission wants to make to the Horizon 2020 programme. They also resent the premise that big science has little impact on the economy. “We don’t like this movement to suggest that basic research has less social and economic value than what they like to call investments in the real economy,” said Jonathan Wareham, ESADE’s dean of faculty and research.
Since Jean-Claude Juncker took over the reins of the European Commission, “Impact has become an absolute political necessity,” said Rosalinde van der Vlies, head of the Evaluation unit at the European Commission’s research and innovation directorate, DG RTD. Juncker wants to “deliver concrete results for citizens,” she said.
EU institutions have a responsibility towards the taxpayer. “[We] have to measure impact because we are using taxpayer’s money and we need to report on how we are spending it,” said van der Vlies.
Impact under Horizon 2020
The Commission has three criteria on which Horizon 2020 evaluators assess the quality of research proposals: excellence, impact, and the quality of implementation. In order to measure impact, three indicators are considered: the capacity to innovate, the creation of new knowledge, and contributions to the wider societal and economic impact. However, “This is extremely difficult to measure,” said van der Vlies.
The Commission relies a lot on information coming from grant beneficiaries, to offer “a more complete story about the impact,” said van der Vlies. However, it takes a long time to gather all the necessary information, with the Commission still in the thick of ex-post evaluation of Framework Programme 7. Although only 44 per cent of the FP7 projects are finalised, the Commission has to deliver the impact evaluation to the Parliament and to the Council by the end of the year, van der Vlies noted.
This makes it almost impossible to give a rounded assessment of the impact of EU funded research. “It may take a decade” for the Commission to come up with a proper evaluation of the projects, said van der Vlies. To address this, the Commission is launching studies that look back at the impact of the Framework programmes all the way to FP2, with the aim of identifying the impact of research funding over time.
As for Horizon 2020, grant holders are now required to report on the impact for four years after the project ends. This may be a great step forward, but there is little incentive to comply, van der Vlies said. Beneficiaries agree to send this information back to the Commission but there is no enforcement mechanism. “For now we are depending on the goodwill of the beneficiaries for assessing the impact of the projects,” said van der Vlies.
Under Horizon 2020, “Impact has become very important but we need to make sure we are not becoming fundamentalists in assessing it,” said Bertolucci. There is nothing wrong with assessing impact but ignoring the long term perspective is a mistake. Horizon 2020 projects should be evaluated under, “a broader definition that takes into account the value of big science projects and long-term investments in science,” said Bertolucci.
Calculating the financial returns of big science is “ambiguous,” said Wareham. Although we may not see the immediate economic value of putting a satellite on a moving comet, the technologies required to achieve such a remarkable feat “will trickle down to the economy,” he said.
These views were expressed at the Science|Business Horizon 2020 conference on 24 March in Brussels
Receive our Policy Bulletin each Tuesday & Thursday and news about bridging Europe’s east-west innovation gap twice a month in The Widening. 
