Web Page sponsor Oxley Developments
06 Apr 17. The US Defense Innovation Board, a group put together by former Secretary of Defense Ash Carter and headed by Google chairman Eric Schmidt, is weighing the possibility of creating a central repository for the massive amounts of data collected by the military. Changing how the Pentagon collects, maintains and uses data is going to be vital for the U.S. to maintain its technological edge going forward, the board members concluded at an April 4 hearing at the Pentagon. At the same time, the advisers acknowleded that the project comes loaded with thorny security and cultural questions. When the group held its first public meeting in October, it laid out a collection of suggestions, including the creation of a chief innovation officer, new software testing rules, and a focus on machine learning. But after six months of research and visits to military bases, the group has zeroed in on the issue of data management as one that impacts every other idea on innovation. The exploitation of data is central to any attempt by the Pentagon to employ artificial intelligence or machine learning, with Schmidt noting in his opening remarks this week that “the military, as a general statement, has data everywhere and nowhere as a result. … It’s clear that without doing that a lot of the things the military would like to achieve are not going to happen.” Right now, there is no common database for all the data collected, with hundreds of different databases stored all over the department, the majority of which do not have common coding that would allow interfacing.
Speaking in March, William Roper, the head of the Pentagon’s Strategic Capabilities Office, said the department focuses “on data in a 1990s-era way — data for us is like something that you use to go into the fight and win, and after that fight, the purpose of the data, its raison d’etre, is over.
“And that is not the way that the commercial world, especially the big companies that are trying to work analytics and deep learning machinery — to them, that data is truly gold. Probably better, it’s probably closer to oil,” Roper said. “It’s a commodity, it’s a wealth, it’s also a fuel, and you’re data keeps working for you even after you’ve used it.”
But getting to the point where the Pentagon can use data like the commercial sector won’t be easy, with a good example of the data problem facing the Pentagon laid out during an exchange with Lt. Gen. Jack Shanahan, director for defense intelligence at the Office of the Under Secretary of Defense for Intelligence, who said the Pentagon collects 22 terabytes of data every day, roughly the equivalent of 5.5 seasons worth of video for the National Football League.
“You cannot exploit 22 terabytes worth of data the way we are doing things today,” Shanahan said.
But in 2012, Facebook said it was handling more than 500 terabytes of data a day, a number which will only have expanded in the five years since – and that data is all processed, stored and used to impact how a Facebook user interacts with the website, including creating targeted content relevant to their interests.
In other words, what Shanahan called a “tsunami” of data is something the commercial sector could handle five years ago.
Schmidt himself cited that 22 terabyte figure and noted that “within the business world, this is not overwhelming. Those kind of numbers are easily dealt with, with modern computing. So there is an example of a big gap between the commercial and defense worlds.”
In addition, all involved agreed that getting all the data in one place is another thing – figuring out how to make the various databases talk to each other, and knowing how to mine them, remains a major challenge given the number of older systems involved.
Culture and Security Concerns
During the hearing, six witnesses from inside the Pe