December 18, 2008

Best Paper award for RIACS researchers

The paper "A Survey of Metrics for Performance Evaluation of Prognostics" by A. Saxena (USRA/RIACS), J. Celaya (USRA/RIACS), E. Balaban, K. Goebel (USRA IPA), B. Saha, S. Saha (USRA/RIACS), and M. Schwabacher won the Graduate of the Last Decade Best Paper award at the IEEE International Conference on Prognostics and Health Management.

The paper addresses the lack standard definitions in prognostic concepts which results in ambiguous and inconsistent interpretations. This lack of standards is in part due to the varied end-user requirements for different applications, time scales, available information, domain dynamics, etc. to name a few issues. Instead, the research community has used a variety of metrics based largely on convenience with respect to their respective requirements. Very little attention has been focused on establishing a common ground to compare different efforts. This paper surveys the metrics that are already used for prognostics in a variety of domains including medicine, nuclear, automotive, aerospace, and electronics. It also considers other domains that involve prediction-related tasks, such as weather and finance. Differences and similarities between these domains and health maintenance have been analyzed to help understand what performance evaluation methods may or may not be borrowed. Further, these metrics have been categorized in several ways that may be useful in deciding upon a suitable subset for a specific application. Some important prognostic concepts have been defined using a notational framework that enables interpretation of different metrics coherently. Last, but not the least, a list of metrics has been suggested to assess critical aspects of RUL predictions before they are fielded in real applications.