From little things, big data grows.
That might be the slogan of the Southern Ontario Water Consortium, a partnership of eight Ontario universities which is building a data platform for water testing technologies that will help protect drinking water and predict floods.
The project’s first phase, installing a wireless network of 160 weather, stream and sub-surface water sensors along an 80 sq. km stretch of the Grand River near Waterloo, Ont., and developing an integration platform for pulling in and analyzing data, has just finished. Now the data is slowly trickling in — 150 data points (like air temperature) every 15 minutes.
One unique feature is the ability of the sensors to be triggered by events like heavy rain to increase their data collection to once a second, giving more a more granular picture of what’s going on.
One point is a byte of information. But over a year there will be 5 million data points — and that’s just from one watershed. The Grand River Conservation Authority has five other projects that may be wired and generate enough data that could turn into a torrent. Once analysts start crunching the numbers the volume will at least double.
“I’ve never seen anything quite like this before,” Odum Idika, data platform manager and watershed facility manger for the SOWC said in an interview, “because we have a very, very dense network of instruments, and all this data coming in, for me is unprecedented. It gives us such a high degree of resolution of what’s going on within the watershed. It’s really exciting because now people are going to start using the data and ask questions on the platform.”
“We’re a platform for people to test and demonstrate new water technologies, and what we’ve done is instrumented this watershed on a scale that’s never been done before. The idea is to be able to take the data and link with other partners and research groups and industry to allow them to augment the tools they already have. So if they want to do more advanced modeling with the real-time data we have they can do that. This is the first phase. now we want to start working with industry.”
The second phase still being defined. One possibility is integrating with data collected from other institutions, like the province, the Grand River Conservation Authority, as well as enhancing other organizations models.
The SOWC data integration and management platform was built with the help of IBM Canada, which assisted university software developers with $28 million worth of hardware and software to build the data collection application, housed at the University of Waterloo, and donated a copy of its Intelligent Operations for Water suite, housed at the University of Toronto’s SciNet high performance computing centre, which is used for data analysis.
Other institutions in the project are Ryerson University, University of Ontario Institute of Technology, Western University and Laurier University.
Geoff Riggs, an IBM project manager for its Smarter Cities division who helped the consortium, said the effort is part of the company’s Smarter Water practice, which develops repeatable solutions for water management for governments using IBM’s DB2 database, Maximo asset management and Cognos business intelligence engines.
IBM has created a software development kit that allows SOWC researchers to create their own analytic tools.
https://www.youtube.com/watch?v=BoHPRPgwunU&feature=player_embedded
SOWC posts some of its data online so researchers and the public can see what is being collected. “Other companies are already starting to leverage that,” Riggs noted. “For example there are engineering firms in the SOWC ecosystem are looking at that data and that helps them with decisions about structural modifications or work that’s under their management.”
“The goal is to create a hub of information around the water system so this larger group of agencies can see what’s being collected and inform their decisions.”
Researchers are already creating tools to visualize the early data being collected, he said. But, he added, big data platforms thrive on more data, which gives more resolution to a problem, and make better predications. As other SOWC research projects add their data that will give that.
Like a number of watersheds around the world, the Grand River watershed — which drains into Lake Erie — is facing pressures from urbanization. Its basin is home to more than 750,000 people providing the water supply for the Region of Waterloo, Brantford and the Six Nations reserve.
Studying watersheds isn’t new. But wireless sensor, cellular and data analytics technology is giving researchers new tools to look at what is going on.
Brenda Lucas, SOWC’s executive director, says the project’s focus is helping companies connect through academic partnerships with the IT infrastructure they need the commercialize new technologies in the areas of clean drinking water, processing waste water, and the development of analytic techniques.
Created in 2011, the consortium is backed with $19.5 million in federal and $8.4 million in provincial funding. But she said it could not have gone ahead without the donation of hardware and software from IBM, which has a history of working with academic institutions.