Managing Data with High Redundancy and Low Value Density (2017-2020)

Machination data, often found in sensor networks, GPS and RFID applications, vehicle on-board devices and medical monitoring devices, is the next generation of data we need to manage and process. In addition to large volumes and streaming nature, such data typically have high level of redundancy and low value density. There is a need to develop a new breed of database management systems that can support stream query processing as well as managing historical data to support complex data analytics, data mining and data-driven decision making. In this project we advocate a novel database approach to data storage, cleaning, compression, hierarchal summarisation, indexing and query processing for machination data.