DataMill: Rigorous Empirical Performance Evaluation Made Easy

Wednesday, December 17th 2014, 14:00
Room: S3-218 (Computer Science Building)

Abstract: Empirical systems research is facing a dilemma. Minor aspects of an experimental setup can have a significant impact on its associated performance measurements and potentially invalidate conclusions drawn from them. The growth in complexity and size of modern systems will further aggravate this dilemma, especially with the given time pressure of producing results. So how can one trust any reported empirical analysis of a new idea or concept in computer science?

This talk introduces DataMill, a community-based easy-to-use services-oriented open benchmarking infrastructure for performance evaluation of single computer experiments. DataMill facilitates producing robust, reliable, and reproducible results.

Sebastian Fischmeister is an Associate Professor in the Department of Electrical and Computer Engineering at the University of Waterloo, Canada. He received his MASc in Computer Science at the Vienna University of Technology, Austria, and his Ph.D. degree at the University of Salzburg, Austria. He was awarded the APART stipend in 2005 and worked as a research associate at the University of Pennsylvania, USA, until 2008.

He performs systems research at the intersection of software technology, distributed systems, and formal methods. His preferred application areas are distributed real-time embedded systems in the domain of automotive systems, avionics, and medical devices.