Funding agencies, are you listening? How to build an unsuccessful research center

In 2014, my colleague David Patterson had a CACM Viewpoint entitled How to Build a Bad Research Center, offering tongue-in-cheek bad advice to build a research center that's unlikely to produce breakthrough work. Dave used the recent wrap-up of the Berkeley Parallel Computing Lab (ParLab) as a case study in building a successful center, following a pattern used by many successful centers for which Dave had been a PI or co-PI.

About a year ago (I'm behind in my reading), Jesper Larsson Träff, a parallel computing researcher at TU Wien, wrote an eerily similar Viewpoint on (Mis)managing Parallel Computing Research Through EU Project Funding, which makes the point that the bureaucracy and heavyweight project management style required for multinational EU Projects (solicited regularly by the European Commission), including reliance on artificial "deliverables" that seldom correlate with actual research breakthroughs, thwart efforts to do groundbreaking work.

Reading Träff's litany of bad research-management practices, it's almost as if the EU Project commission read the "Eight Commandments for a Bad Center" in Dave's article, but failed to realize they were supposed to be satirical. Both Dave's and Jesper's short pieces are well worth reading in their entirety, and if you're pressed for time, I hope these side-by-side representative quotes from each may convince you to do so:

Patterson: "Bad Commandment #2: Expanse is measured geographically, not intellectually. For example, in the U.S. the ideal is having investigators from 50 institutions in all 50 states, as this would make a funding agency look good to the U.S. Senate."

Träff: "…the enforced geographical and thematic spread among consortium members in reality means that time and personnel per group is often too small to pursue serious and realistic research."

Patterson: "Bad Commandment 8. Thou shalt honor thy paper publishers. Researchers of research measure productivity by the number of papers and the citations to them. Thus, to ensure center success, you must write, write, write and cite, cite, cite."

Träff: "As part of the dissemination activities it is important for all consortium members to show regular presence at various associated, often self-initiated workshops and conferences, including meetings and workshops of other projects; high publication output is encouraged … The primary purpose of many of these activities is to meet the dissemination plans [deliverables], and has led to a proliferation of workshops presenting and publishing project-relevant trivialities. Apart from the time this consumes, the apparent authority of a workshop masquerading as a scientific event at a well-established conference may reinforce docility and low standards in the naive Ph.D., and appall and deter the more observant one."

Patterson: "A key to the success of our centers has been feedback from outsiders. Twice a year we hold three-day retreats with everyone in the center plus dozens of guests from other institutions [including industrial partners]. Having long discussions with outsiders often reshapes the views of the students and the faculty, as our guests bring fresh perspectives… Researchers desperately need insightful and constructive criticism, but rarely get it. During retreats, we are not allowed to argue with feedback when it is given; instead, we go over it carefully on our own afterward."  Note: In the Berkeley research center model, industrial affiliates pay a fee to participate in the project.

Träff: "…in many cases industrial involvement makes a lot of sense. However, it seems confusing at best to (ab)use scientific research projects to subsidize European (SME) industry. Can't this be done much more effectively by direct means? In any case, it would be more transparent and less ambiguous if industrial participation in EU projects was not directly funded through the projects. Strong projects would be interesting enough that industry would want to participate out of its own volition, which in particular should be the case for large businesses."  Note: In the EU Project research center model, public funds are used to pay private companies to participate in the project.

Träff closes his article with some suggestions for moving towards a "radically better" funding model for European high-performance computing research. In this section, unsurprisingly, Träff and Patterson find themselves in agreement:

Träff: "[P]roposals and projects to a larger extent [should] be driven by individual groups with a clear vision and consortium members selected by their specific project-relevant expertise. It would make long-term scientific, and perhaps even commercial sense to make it possible and easy to extend projects that produce actual, great results or valuable prototypes…more possibilities for travel grants and lightweight working groups to foster contacts between European research groups, and more EU scholarships for Ph.D. students doing their Ph.D. at different European universities would also be welcome additions."

Patterson: "After examining 62 NSF-funded centers in computer science, the researchers found that multiple disciplines increase chances of research success, while research done in multiple institutions—especially when covering a large expanse—decreases them: 'The multi-university projects we studied were less successful, on average, than projects located at a single university. ... Projects with many disciplines involved excelled when they were carried out within one university  (J. Cummings and S. Kiesler, Collaborative research across disciplinary and organizational boundaries, Social Studies of Science 35, 5 (2005), 703–722.)"

While the two pieces don't overlap 100%, there are great lessons to be learned from reading both.

The question is whether the responsible funding agencies are reading them.