Software Testing with Log File Analysis
Abstract
If our test suite is thorough, and our software has performed correctly on the test suite, we have confidence in the quality of our software. But how can we tell whether our software has performed correctly? Sometimes test results cannot be checked easily by humans or by simple comparisons to regression test results. One focus of our research is checking results using test oracles (programs that check the output of other programs). We are particularly interested in software log files, and oracles that analyze log files to detect failures. Log file analysis can be used for both unit and system testing; we can check for as many or as few requirements as we want.This talk will focus on two avenues of research we have recently been exploring. First, running random function call sequences, using coverage to measure thoroughness and log file analysis to check results, shows promise as a way of testing data structures; we have been measuring its costs and benefits. Second, we have recently undertaken a study of human participants learning about and using log file analysis for the first time, and we report on our findings.
About Dr. James H Andrews
James H. (Jamie) Andrews received his BSc and MSc degrees from the University of British Columbia, and his PhD degree in 1991 from the University of Edinburgh. He worked from 1982 to 1984 at Bell-Northern Research, Ottawa, and from 1991 to 1995 at Simon Fraser University, Vancouver. From 1996 to 1997 he was a Research Associate with the FormalWare project at the University of British Columbia, Hughes Aircraft Canada and MacDonald Dettwiler. He is now an Associate Professor of Computer Science at the University of Western Ontario, in London, Ontario, where he has been since 1997. He is currently visiting the Computer Science department at the University of Victoria. His research interests include software testing and formal specification.Event Sponsors
In addition to the support from our host sponsor SchemaSoft this event was made possible by donations from:Greenlight Power Technologies
QA Labs Past Meetings
