The following story was originally published on Aug. 30, before North Korea claimed to have successfully tested a hydrogen bomb designed for an intercontinental ballistic missile.
WASHINGTON — When North Korea launched a ballistic missile over Japan Monday, the $80 billion American intelligence-gathering apparatus had only hours of warning, U.S. officials told NBC News.
It was the latest in a long history of apparent surprises by the rogue regime. Lately, the North Koreans have outpaced U.S. estimates in their progress on nuclear missiles. In December 2011, the U.S. was unaware for more than 50 hours that longtime leader Kim Jong Il had died — learning the news only after it was announced on North Korean TV. In 2010, the North Koreans showed an American expert a huge new uranium enrichment plant about which the world had known nothing.
Why can't the U.S. military and intelligence agencies do a better job of ferreting out secrets from North Korea?
Because, current and former intelligence officials say, North Korea is the ultimate nightmare of an intelligence target: A brutal police state with limited internet usage occupying mountainous terrain that lends itself to secret tunnels.
"It is one of the hardest, if not the hardest, collection nation that we have to collect against," said Daniel Coats, the director of national intelligence, to Congress in May.
Related: North Korea Crisis: How Events Have Unfolded Under Trump
And it’s getting harder. U.S. officials told NBC News that North Korea has taken steps in recent months to disguise their missile-related activities, including fueling rockets inside structures, outside of aerial view.
There are three basic ways the U.S. gathers most of its foreign intelligence: collecting information from human spies; intercepting electronic communications; and observing what's happening on the ground, mainly with satellites.
All of those things are hugely problematic when it comes to North Korea.
The people problem. North Korea is one of the world's most starkly authoritarian countries, a place where criticizing the Supreme Leader can land one's whole family in a concentration camp. Russia and China are "an open book" compared to North Korea, said Bruce Klingner, a former CIA analyst now at the Heritage Foundation.
There is no U.S. embassy in North Korea and no American business community. So it's not like the CIA can send one of its own case officers in to start gathering information. But even the South Koreans have trouble running agents in the North, Klingner said. The dialect is different and "if you're trying to get access to highly classified things, you're immediately suspicious."
The internet problem. The National Security Agency, which hacks computers and intercepts email, has had some success pulling bits and bytes out of North Korea, former officials say, but North Korea is much less forgiving than most of its targets. That's because most of the country is not connected to the internet and few people have cellphones. To the extent that the regime communicates electronically, it has made increasing use of encryption, experts say.
"If you look at that satellite picture [of Asia] of the lights at night from the satellite, there is one dark area with no lights on, and that is North Korea," Coats told Congress. "Their broadband is extremely limited. So using that as an access to collection — we get very limited results."
The imagery problem. Much of what the U.S. knows about North Korean military and nuclear activity it learns from satellites observing what is happening below. The official name for that is "Intelligence, Surveillance and Reconnaissance," shorthanded as ISR. North Korean officials know that, which is why they have built one of the world's most sophisticated array of tunnels to hide their stuff. They are aided by their mountainous terrain, which makes overhead observation more difficult. It is easier to spot and assess activity over an expanse of plains.
Related: North Korea Fires 'Unidentified Ballistic Missile' Through Japanese Airspace
"They can store these launchers and missiles in tunnels and move them at night, then set them up with a tent over it," said David Albright, a nuclear weapons expert and founder of the Washington-based Institute for Science and International Security.
"We do not have consistent ISR capabilities," Coats told Congress. "And so there are gaps. The North Koreans know about these."
