Shows & Panels
Shows & Panels
- The 2014 Big Picture on Cyber Security
- AFCEA Answers
- American Readiness: Renewable Power and Efficiency Technologies
- Ask the CIO
- Building the Hybrid Cloud
- Connected Government: How to Build and Procure Network Services for the Future
- Continuing Diagnostics and Mitigation: Discussion of Progress and Next Steps
- Delivering the Digital Government Mission
- Federal Executive Forum
- Federal News Radio's National Cyber Security Awareness Month Special Panel Discussion
- Federal Tech Talk
- The Future of Government Data Centers
- The Future of IT: How CIOs Can Enable the Service-Oriented Enterprise
- Government Perspectives on Mobility and the Cloud
- The Intersection: Where Technology Meets Transformation
- Maximizing ROI Through Data Center Consolidation
- Mitigating Insider Threats in Virtual & Cloud Environments
- Modern Mission Critical Series
- The New Generation of Database
- Reimagining the Next Generation of Government
- Targeting Advanced Threats: Proven Methods from Detection through Remediation
- Transformative Technology: Desktop Virtualization in Government
- The Truth About IT Opex and Software Defined Networking
- Air Traffic Management Transformation Report
- Cloud First Report
- General Dynamics IT Enterprise Center
- Gov Cloud Minute
- Government in Technology Series
- Homeland Security Cybersecurity Market Report
- National Cybersecurity Awareness Month
- Technology Insights
- The Cyber Security Report
- The Next Generation Cyber Security Experts
Shows & Panels
Big Data creates big challenges for agencies
Wednesday - 5/16/2012, 8:52am EDT
The amount of individual data bytes created is so enormous, it is fast out pacing the ability to capture, search, analyze or store it. Big Data is the general term used to describe the 2.5 quintillion bytes of data produced each day.
"On average, people are collecting almost 50 percent more data a year, so it's just a lot more data coming at them every day," said Mark Weber, NetApp's president of the U.S. public sector.
"Big Data is just datasets that have gotten so large and complex that people don't have the tools or the ability to capture it, store it, search it, retrieve it, analyze it," Weber said. "They just don't have the proper equipment or technology to do that."
Mark Weber, president, U.S. public sector, NetApp
The ever growing amount of data is creating challenges for agencies who are trying to mine it.
"If you think about it, Big Data is a gold mine," Weber said. "What you have there is kind of your family jewels and you should be able to mine that data to make great decisions, to improve your efficiency … to improve forecasting. So, if you're not doing that, you're going to be totally inefficient. You're not going to be able to make good decisions for your agency, for the intelligence world, for health-care decisions. Part of seeing all the information you're collecting is allowing you to be more efficient."
Weber pointed to intelligence agencies as one community that's in the data collection business, and, therefore faces the problems surrounding Big Data on a regular basis. "Analyzing that data and the amount of that data they have to analyze is really important to protect our country and keep us free," he said.
Likewise, improved data analysis and processing could help health care agencies keep down costs.
"Doing studies and determining why people get re-admitted to hospitals, could save massive amounts of data," Weber said. "You could save tons of money by not re-admitting people. … They have all that data. The question is, are they analyzing it and figuring out why people go back?"
NetApp's study found that about 40 percent of people are using the data they've collected and are making decisions based upon it, so there is plenty of room for improvement.
"A lot of this is innovation," he said. "The amount of data that people have, the tools don't exist today. This is a great opportunity for IT companies to innovate."
In the past, data was broken down using relational databases, which relied on similar structures to analyze datasets. Now, much of the data coming in is unstructured, arriving in different formats, such as imagery and video.
"How you search and mine that kind of information is really pretty complicated," Weber said. "That doesn't fit nicely into a relational database. That's the piece of data that is exploding exponentially is the unstructured side."
According to Weber, the number one thing that agencies need to do right now is determine who owns the data and who's responsible for mining it. In addition, agencies need to tackle the unstructured side of data analysis.
"A lot of tools exist for structured data," Weber said. "They've got to figure out what investments they need to make in the unstructured side."
To that end, the Obama administration recently announced the Big Data Initiative, committing more than $200 million in research-and-development investments.