DSN 2006 - Workshop on EMPIRICAL EVALUATION of DEPENDABILITY and SECURITY (WEEDS)

(In Conjunction with the International Conference on Dependable Systems and Networks, DSN2006)
Wednesday, June 28, 2006
Sheraton Society Hill, Philadelphia, PA, USA

MOTIVATION and THEME

Empirical evaluation of dependability is a complement to modeling and analytical methods. Although empirical evaluation applies to real systems and is more realistic, more accurate, and provides a higher level of confidence, it is not extensively used, for multiple reasons: it is time and effort consuming; the results have limited portability; there are no benchmarks to define what measures to be collected, how to measure the data, how to report it, what are the models that indicate the ranges of "good" and "bad" values, and how to compare the values for different systems. Due to these challenges, very few results and empirical data are available for dependability evaluation, comparison, and benchmarking. Acknowledging these challenges, but also the need for empirical dependability evaluation, this workshop will bring together researchers and practitioners for sharing their solutions and results, as well as for documenting their needs, problems, and ideas with respect to this topic. The workshop will include presentations and a hands-on session where participants will work on finding solutions to pre-selected questions.

SCOPE

This workshop focuses on the empirical aspect of systems dependability evaluation. "Systems" include computers, networks and software, both in operation and under development. Outside the scope are process assessment and evaluation (e.g., CMM/CMMI type assessments). Dependability is considered a set of one or more properties of computing systems, including reliability, availability, safety, confidentiality, integrity, and maintainability. Our use of the term "dependability" includes security as well.

OBJECTIVES

The objectives of this workshop are to:

Identify and describe the state of the art and state of the practice of empirical dependability evaluation
Identify the needs for empirical dependability evaluation (mainly for organizations, both industry and government)
Identify the gaps between the needs and current solutions
Develop a roadmap for getting from where we are to where we want and need to be

TOPICS

Topics include, but are not limited to the following:

Need for empirical dependability evaluation
- Why is dependability evaluated? What are the questions that the dependability evaluation helps answer?
Purpose of evaluation
- What are the evaluation goals?
Objects of evaluation
- What properties (reliability, safety, security, etc.) and of which entities (e.g., computer, network, system, information, services) are evaluated?
Approaches for evaluation
- By whom, for whom, and when is dependability evaluated?
- What are the methods, techniques, tools, and infrastructure used for empirical evaluation of dependability (including data collection, analysis, and reporting)?
- What are the models and benchmarks for interpreting these results?
- What are the indicators and measures (basic and derived) that are expressive and effective enough for dependability evaluation?
- What is the context in which evaluation is performed (e.g., laboratory experiment, academic, industrial setting, small or large scale system) and how do results transfer and scale up?
Cost and benefits of empirical evaluation

There has been some work for addressing these questions for different dependability properties (e.g., reliability, safety, availability, security) and for different systems (i.e., for specific domain, applications, configurations, or platforms). Our goal is to identify what these efforts and results have in common and what can be transferred, applied, and learned from one to another.

PARTICIPATION and SELECTION PROCESS

The workshop is open to all researchers, system and software developers and acquirers, as well as to users who are involved with or have an interest in empirical evaluation of dependability. All prospective participants are required to submit an extended abstract presenting their current results, a work-in-progress report, or a position paper.

Submitted papers must be original work with no substantially overlap with papers that have been published or that are simultaneously submitted to a journal or conference with proceedings. Papers should be at most 4 pages in IEEE proceedings style (two-column pages, single space, using 11-point font and 1-inch margins) including all figures and references. Each submission should start with a title and names and contact information of the authors. Submissions must be made electronically in PDF or Postscript format, by sending an email to [email protected] and [email protected].

Submitted contributions will be fully refereed by PC members. Accepted papers will be published in the supplement volume of DSN 2006 proceedings, conditioned by the attendance of the author to the workshop.

Important Dates

Paper submission due	March 7, 2006 (Tuesday)
Acceptance notification	April 4, 2006 (Tuesday)
Camera-ready version of papers due	April 18, 2006 (Tuesday)

WORKSHOP ORGANIZERS

Michel Cukier, University of Maryland
Ioana Rus, Fraunhofer Center Maryland

FURTHER INFORMATION

For more information on the location and on DSN06 please visit http://2006.dsn.org. For workshop information please send email to the organizers, Michel Cukier at [email protected] or Ioana Rus at [email protected].

PROGRAM COMMITTEE

Matti Hiltunen, AT&T Labs, USA
Mohamed Kaaniche, LAAS-CNRS, France
Peter Lakey, Cognitive Concepts LLC, USA
Henrique Madeira, University of Coimbra, Portugal
Frank Marotta, U.S. Army Aberdeen Test Center, USA
John Murdoch, University of York, UK
Holger Peine, Fraunhofer IESE, Germany

The International Conference on Dependable Systems and Networks
(DSN-2006)

Sheraton Society Hill, Philadelphia, PA, USA
Sunday June 25 - Wednesday June 28, 2006

DSN Invitation

Call for Contributions

Previous DSN Conferences

Related Conferences