University of Florida :: Department of Computer and Information Science and Engineering (CISE)

Departmental Report : REP-1998-275

Search Departmental Reports
By Author: 
By Time: 
Keyword search:
Report ID:REP-1998-275
Title:Query-Based Change Detection
Authors:Chris Carnes

Eric Hanson

This paper presents several algorithms that can be used to efficiently isolate changes in queriable data sources. The introduction of external trigger systems and the popularization of data warehouses have served as a catalyst for identifying efficient change detection techniques. Here we propose multiple algorithms for change detection as the optimal approach is determined by the capabilities of the underlying data source. We focus on simple and legacy data sources that support querying but do not necessarily support other methods that allow for more efficient change detection, such as triggers or asynchronous replication. We argue that most reasonable sources of data will have some structure and are at least queriable through some form of SQL-enabled gateway. Utilizing this fact, our work improves upon previously developed differencing algorithms by "pushing down" restrictions (reducing I/O), by comparing only interesting attributes (reducing CPU), and by incrementally comparing subsets where necessary (reducing locking). Combining these techniques enables frequent differencing on data sets of all sizes.

Supporting Files:file icon (1,570 KB)file icon REP-1998-275.pdf (209 KB)
Posted:December 31, 1997