Sunday, 29 November 2009

The Swarm Intelligence

Just scanned this very interesting article about Climategate in the journal of the American Enterprise Institute's "Enterprise" blog by someone very bright called Jay Richards. The specific points he makes (extremely well) about the ongoing, forensic analysis and reconstruction of the leaked data, revealing some of the worst abuses of the scientific process imaginable, are by now quite familiar and can be found in the usual places, several of which are linked to on this blog. It's not these legitimate observations so much, explosive as they are, as his conclusions about the significance and cultural impact of the blogosphere that really intrigued me. Here's an extract:

Of course, most of the big broadcast media are still in full blackout mode on this story, choosing instead to report on breaking news about Pete the orphaned moose. They’re following the pattern of the Dan Rather Memogate controversy in 2004. With that history-making story, the legacy media mostly tried to ignore the story, and then, when it got too big, began to spin it. Rather and CBS issued increasingly bizarre denials. Even though the gig was up within a couple of days, they continued to defend the document in question, and the stories based on it, for two excruciating weeks. (Compare CRU’s Phil Jones offering similarly risible explanations.) Meanwhile, in the parallel universe called reality, unknown and often apolitical bloggers with specialized expertise in font styles, IBM Executive Series and Selectric typewriters, military protocol, and word-processing software dismantled the details for any curious person with an Internet connection. Other, politically oriented blogs consolidated, analyzed, and broadcast the findings.

MemoGate gave many of us our first taste of the swarm-intelligence of the blogosphere, and showed that it cab beat the legacy media for getting to the bottom of a story via a networked, open-source form of peer review, with a highly refined division of labor.

We may just now be seeing the potential for this new way of transferring and analyzing information. In Memogate, remember, we were talking about a single one-page Word document. With Climategate, we’re dealing with thousands of detailed, often technical documents. They may even have been compiled internally at the CRU in response to a Freedom of Information request and were then leaked instead. So the revenge of the nerds could be especially brutal and prolonged. Already, insights and analyses are proliferating on the climate blogosphere so quickly that it’s becoming impossible for even the best consolidators to keep up.

I hadn't really grasped what Guido Fawkes meant when he talked about similar things regarding, if memory serves, the expenses scandal* . I do now. The blogosphere, with its "swarm intelligence," is no longer potentially the most powerful communication medium in the world, Climategate has proved (at least to me) that it now is the most powerful communication medium in the world.

It therefore seems that it was no accident the Climategate documents weren't first leaked to a newspaper, as with the expenses scandal (the last real scoop of the dead tree press?) but to a blog instead, albeit what turned out to be the virtual dead end of a BBC weatherman's blog. I think it likely that given the technical complexity of the material, the whistleblower eventually appreciated that no MSM (what Jay Richards calls the "legacy media") provider would even want to touch it, all compromised as they more or less all are, much less spend a lot of time and money unpacking its secrets. It needed some serious processing power to do that, and, as Mr Richards asserts, the only place that that could be found was in the blogosphere.

So, complete paradigm shifts all round, then - not just in terms of the AGW belief system, but in terms of how we produce, analyse and trust our news-information supply, too.

I for one am pretty proud to contribute my modest (some would say infinitesimal) intellectual resources to The Swarm Intelligence.

(*It could have been something else, though, I'd need to check. But he does bang on about that and the fall of the dead tree press fairly regularly, it seems to me.)


  1. This comment has been removed by the author.

  2. I do agree with you. I'm not sure the "MSM" are dead yet though. Ultimately they do still set the political agenda - most political blog posts are fed by MSM stories, aren't they? The BBC News website is in the world top 10 (unfortunately)!

    Personally I think this affair is illustrating a more significant truth - that the blogosphere is undermining the power of rulers and governments. Do you think things like Climategate didn't happen before the internet, or do you think people just didn't know about it?

  3. Thanks Adam, those are fair points. There seem to be a kind of synergy between the blogosphere eroding MSM/Government control over what people know, and increasingly shrill, desperate attempts by Governments to secure their own tenure in perpetuity. Politicians who actually find the latter proposition unpalatable are, sadly, rather few in number. All the rest, I fear, secretly find the idea rather spiffing (Speaker Bercow, as an example picked at random).

    The technology that's given us the blogosphere has a wicked double edge. It's never been easier for States to monitor and analyse their populations in excruciating detail. Much of the nebulous, Old World fear of peasant mobs, armed with pitchforks and burning torches, arriving unexpectedly in the dead of night is gone. What we've gained on one hand, as demonstrated by Climategate, we've lost on the other by having our every movement, transaction and communication made push-button-easy to intercept, record and analyse.

    We know them better and they know us better, it would seem, but the apparent symmetry of the information technology revolution is dangerously deceptive.

    Unfortunately, the blogosphere would be extremely easy to shut down and/or censor. The blogosphere might operate in the style of a distributed, fault tolerant grid but the infrastructure is highly centralised and dependent on a State approval wherever it operates. Google, of "don't be evil" fame, has censored the term Climategate from their text prediction database. Yahoo, (in)famously, shopped Chinese dissidents to the authorities in order to "keep their foot in the door". When push comes to shove, corporations, with meaninglessly few exceptions, will tow *any* line rather than perish. The internet ain't no samizdat.

    That the MSM is still covering up Climategate can only be a sign of weakness and fear on their part. Most MSM outlets that fall below the profitability threshold will evaporate rather more quickly than the Berlin wall fell. They might still set the agenda but they're all facing some kind of abyss over the next decade.

  4. Deleted that because the language was a bit too harsh to expect others to carry it without reservations.

    There, see, self-censorship - I'll be in the AGW vanguard before you know it =)

  5. The BBC sat on the story. There can be no other explanation for it.

    The important thing about the MSM is that they have lost control of the agenda - primarily a left-leaning liberal one.

    I think the best indicator of this is how you and I react to a breaking story. In the old days, the first port of call was the BBC. Now it is Guido or one of the many others who make sure things are not overlooked. Dan Hannan was a case in point. The BBC would not touch the Dan Hannan rips Gordon Brown a new Aerosol story until it became part of the story itself for NOT reporting it.

    Yes, we are pulling power back to the people. How long we have before this is stamped on, I don't know, but plenty of people in the Establishment are making noises about censoring the blogosphere by making it difficult for lone bloggers to operate.

  6. Sorry I haven't replied sooner - been rather busy. I appreciate the comments: all illuminating and excellent.

    Adam, I agree with you - not sure the MSM is dead either for the reasons you give.

    BHS: I saw that comment before you pulled it. Seemed OK to me, mate ;)

    WW: We'll have to put up a fight, then!


Any thoughts?