re: "Why the surprise?"

Richard at EU Referendum ("To discuss issues related to the UK's position in Europe and the world") keys on some areas of perennial interest to those in the intelligence professions.

Money quote(s):

"Diplomats and intelligence agencies, the strap to his piece reads, often tell ministers what they want to hear – and overvalue secret sources of information.

Secret intelligence services, naturally enough, writes Norton-Taylor, want to emphasise secret intelligence – a product which only they, in their special and privileged role, can offer. As a result, they have seriously underestimated what can be gleaned from "open sources". "

Yes and no.

There's such a thing as "intelligence fusion." That's where information from many types of sources and collection methods gets "fused" into an "all-source" product. Army folks may recall the acronym "ASAS," which is the All-Source Analysis System.

But I digress.

"All Source" intelligence includes open sources or "OSINT."

So if your analysts are on the ball, this gets included into your final product.

Ideally.

"Britain's diplomats and spooks, in common with all western intelligence agencies, also spectacularly failed to foresee the fall of the Berlin wall. Thus, he says, they must in future pay much more attention to "open sources", what they can hear on the Arab street, and what they can read, notably on the internet."

As the man says, this is something we can do better.

Lastly:

"Anyone who relies merely – or even mainly – on the MSM for their "take" on what is happening in this country, or for their general news of events, would end up very seriously ill-informed. Yet it remains the case that the political "set" in this country rely for their information on such conventional sources, and are still unduly influenced by the MSM.

However, we cannot leave it there, without also referring to what could be called mindset myopia. Our "élites" very often do not realise what is happening because they already think they know, and therefore do not avail themselves of credible information sources, because they believe they know better. They don't look for information because they don't see the need for it. And that is what really catches them out.

Either way, when our revolution eventually arrives – in whatever form it finally takes – our politicians and other agencies will also be caught by surprise. If they had the ability to see it coming, they would also have the ability to head it off.

Almost by definition, therefore, revolutions must always come as a surprise to those at whom they are aimed. The happen because those people are so out of touch that they are capable of being surprised."

Richard hits very close to a recurring dilemma for intelligence professionals: how do you get your policy-maker to value your product when they are already relying on other, outside sources, the ones they've learned to value throughout their political lives?

The case of a decision-level official who already assumes they know all there is to know is just too nightmarish to comtemplate. One needs to watch World War II movies to appreciate just what that does for a nation.