19 March 2008

PVC towers (part 2)

In my previous post, I commented that we should not expect academic insiders to have much incentive to expose the deficiencies of their own subjects or of academia in general.

If anyone on the inside ever suggests even mildly that there might be defects in the way academia operates, e.g. Robin Hanson:

academia is no more about making useful intellectual progress than advertising is about informing consumers. Professors seek prestigious careers, while funders and students seek prestige by association. Academics talk and write primarily to signal their impressive mental abilities, such as their mastery of words, math, machines, or vast detail. Yes, contributing to useful intellectual progress can sometimes appear impressive, but the correlation is weak, and it is often hard to see who really contributed how much. Progress happens, but largely as a side effect.
they are liable to get a reaction like this:
your post doesn't deserve [a serious response]. It so reeks of bitterness and agenda and is so divorced from what one has to suffer through to obtain the marginal reward of an academic position that I'll leave it to others to mire their way through your mis- and preconceptions.
No, to get a more informative and unbiased assessment, you probably need to go to someone who has turned their back on the academic world. Before reading Mencius Moldbug, I would have thought that computer science, being an eminently practical subject, was immune to the full-blown mediocratic disease, but apparently this is not so.
My Navrozov moment* at Berkeley came from the one and only paper I published, which was a clever way of reducing the time it takes for an operating system to "context switch," or shift between working on different processes.

... this same problem was popular at the time — the only real way to succeed in CS is to invent a new problem which generates more employment for your peers — and other people at Berkeley were working on it. Two of these were a pair of third-year grad students ... 'Sacco' and 'Vanzetti' [pseudonyms] came up with an entirely different solution ...

At some point during this period, however, I realized that the entire problem was a complete and utter pseudo-problem ... The lily needed no gilding at all, and it certainly did not need to be nanofabricated from isotopically pure, individually selected gold atoms ...

So I am very confident that neither of these techniques, neither mine nor Sacco and Vanzetti's, has ever been used in practice. There is no need for them, there has never been any need for them, and there will never be any need for them. And this was quite obvious in 1993.

My Navrozov moment, of course, was when I approached one of the two ... and attempted to have an intellectual discussion of this realization. The story is basically the same as Navrozov's, so it would be boring to repeat, but basically I came away with the feeling that I'd told someone his Sicilian grandmother liked to get drunk and f*** her own goats. Which, in fact, I had. Because I'd essentially told him his research was fraudulent. (Unqualified Reservations)
When I first read this, I found the similarity with my own postgraduate subject, economics, uncanny. Clearly there is a syndrome at work here which has nothing to do with the specificities of a particular subject. I had conversations with economics faculty members at Oxford which were remarkably similar to his with the fellow graduate students. There was a similar apparent shock value in daring to question whether there was really a point to some of the pseudo-theory being generated.

A system of the kind described in part one may be quite good at generating work which is 'clever' (see earlier definition), and some of the time may even produce some mildly worthwhile ideas. What it certainly will not promote, except perhaps by accident, is work of the kind which Kuhn termed 'revolutionary'. Now when people still regarded academia as a non-exclusionary locus for intellectual progress, and capital-owners considered it their role to subsidise culture, that didn't matter too much. Much of the revolutionary work was done outside academia, or by people who notionally had positions in it but who were not financially dependent on it, and academia contributed by doing the follow-up work — the kind Kuhn termed 'normal'.

Nowadays, however, academia is supposed to be the alpha and omega of all research. The creed goes as follows: research outside academia does not exist, should not exist, and should not be supported. Little wonder that progress on key conceptual issues has been on the low side for the last fifty years. Even insiders are starting to complain about it, though of course their diagnosis is going to preferentially involve any explanation that isn't a fundamental indictment of the academic system per se.
[Lee Smolin's The Trouble with Physics] portrays string theorists as tending toward arrogance, insularity, and groupthink; they value technical ability over original thought, follow faddishly the ideas of a few top physicists, and look down on adherents of other theories. This culture, in Smolin's telling, eschews the philosophical bent of Einstein and quantum theory's founders, preferring the "shut up and calculate" attitude of later particle physicists ... Smolin is right that science needs both "craftspeople" and "seers," the former focused on technical problems, the latter on deeper meanings and new ideas. He makes a plausible argument that physics institutions have become too geared toward producing crafts­people rather than seers. The way for young physicists to get jobs, tenure, and grants, he notes, is to fill in the details of research lines established by their elders.
Theoretical physics has indeed, as Smolin points out, become obsessed with formalism and calculation at the expense of conceptual understanding. His explanation for this phenomenon, however, seems unconvincing.
One reason ... is that universities are no longer growing as fast as they did for decades after World War II, so there is more competition for physics posts and less room for nonconformists. Furthermore, theoretical physicists rely heavily on financial support from just a handful of federal agencies, with some private foundation money thrown into the mix. These limited funding options provide further incentives for conventional thinking. Observing that such incentives are not limited to physics, Smolin warns that intellectual sclerosis could be developing throughout the sciences. (Review of The Trouble with Physics in Reason Magazine.)
Pace Smolin, there is a simpler and more obvious reason why sclerosis develops in an academic discipline: the same reason it develops in any monopoly that has become largely self-referential.

A similar recent complaint about absence of meaningful conceptual progress, and about the bias against models that are clear and comprehensible rather than technically impressive but vacuous, can be read at Shtetl-Optimized. This particular one is about computer science (again), but the comments to the post confirm this phenomenon is not confined to CS. (Via Overcoming Bias, where there are some more comments on the same issue. Robin Hanson's criticisms of academia are also worth looking at. See also David Thompson's blog, which regularly highlights instances of il-liberal bias in North American academia.)

(*) Mencius: "A Navrozov moment is a moment when you realize that the university, which was established as a refuge whose purpose was to pursue truth without regard for the opinions of the world, has become a power center whose purpose is to impose its own opinions on the world. As such it has no more use for independent thought than a dog has for beets ... The name honors [a piece] by Andrei Navrozov ... from his Gingerbread Race"

Some comments here.