April 15, 1999

Dr. Ashok Thadani, Director
Office of Nuclear Regulatory Research, MS TWFN - 10 F12
U.S. Nuclear Regulatory Commission
Washington, DC 20555-0001

I attended the stakeholder meeting on April 14, 1999 and have several comments that I would like to share with you regarding the issues discussed. It did not really appear that we had the time during the meeting to go into much detail, so I took and made notes in an effort to be able to contribute a few useful thoughts in writing. I have 33 years experience in nuclear safety, operations, and engineering, so I hope some of this is useful.

Overall, it appeared that everyone on the panel was in violent agreement on most topics, with only mildly different perspectives in spite of the different roles and organizations represented. This agreement was a big surprise to me, but it indicates some maturing and convergence on key issues. I now believe that your task is within your grasp.

First, I would like to comment that the NUREG/CR-5392 technical presentation is really impressive. As I mentioned to Bob Youngblood during the break, I would like to see the main points developed into recommendations that would contribute to the implementation plan, which would require a significant effort toward focusing and converting his complex notions into practical summary documents and associated actions.

Now for my "new thoughts" that might be of interest.

Composite Design Verification and Action (CDVA) index

In the PRA-not-bounding-design discussion, I agree that there remain some safety system design vulnerabilities. I see them emerge frequently from the SSFI/SSEIs in which I frequently participate as the mechanical engineering contractor. I believe that you would be prudent to consider the possibility of taking advantage of all the work that has been done in the past few years in these types of design studies and in the more general design basis reconstitution efforts undertaken in the industry.

For example, I claim that one "performance based" parameter in the design area is simply the percentage of safety systems that have been "scrubbed" in design terms (as well as in the other functional areas). The collateral index that should accompany that percentage is the number of outstanding actions or condition reports impacting the design basis. For example, a plant with a composite index of 75%(30) would be in better shape than one with 75%(80), which would allow the NRC to prioritize its inspection resources. Also, such a composite index would be very easy to generate and verify.

While the better plants already have good design programs (e.g., Turkey Point and Byron in my experience), a very credible baseline design validation is possible in terms of safety system design validation percentage even for the best plants. The Performance Indicator goal is 100%(0). I believe that such a parameter would be meaningful to everyone and would be relatively easy to generate and to expand on for problem plants without adversely impacting the rest of the industry.

For example, for problem plants the number of remaining design deficiency corrective actions could be binned according to risk, using as much detail as needed to clarify the associated (degraded) level of safety. The NRC might determine that a composite design verification and action index of 90%(10) would be a threshold where the NRC might stop looking at design issues.

Based on the above, I suggest that the NRC performance based regulation action plan should specify early that the baseline inspection pilot plants should be prepared to present that composite design verification and action (CDVA) index to the NRC inspectors on their arrival at the plant. This should be accompanied by the associated systems lists and action item summaries. It should only require a few days for a plant to generate such an index and list, most of which they already have (or should have) available for their day-to-day work. These data are similar to backlog data, but they are consistent with the current design basis validation environment.

Caveat: I note that Salem, Crystal River, and now DC Cook (in which I am currently participating in an oversight role under Region III) have undertaken rather comprehensive design upgrades to reestablish credibility. These plants and perhaps others should be considered as special cases since these are recent efforts and rather comprehensive, especially DC Cook (85 systems are being looked at). Thus, they might be indexed as "100%(540) for 85 systems" or the NRC could choose to limit the number of systems to what the plants have classified as level one safety systems, which would result in higher percentages and lower open item numbers.

In either case, these Performance Indicators would change each month and allow the NRC to establish performance-based thresholds regarding progress on safety system design issue corrective actions. A more sophisticated parenthetical number could also be generated based on the associated risk rather than the number of open items, but I do not believe the NRC would need that number unless it exceeded a safety threshold in the first place. A utility might want to generate it for themselves, perhaps defending against criticism for a large number of low-risk open items.

Backlog Performance Index

Similar to the composite design verification and action index described above, it is possible to develop a performance indicator and associated thresholds in the maintenance area. For example, the index could consist of the overall backlog, the safety system backlog, and a risk based number. Again, one would not have to generate the risk component except where the other numbers indicate a possible problem or if the utility chooses to defend its large backlog in terms of risk. The NRC could establish performance based thresholds in each area, including a threshold at which the index requires a risk estimate.

For example, a good plant index might be 325(5), indicating 325 separate backlog actions with only 5 discreet upgrades having potential safety implications. A not-so-good plant might have an index such as 743(34 for 2.5-3). A reasonable threshold index for NRC interest might be 500(10), with a total risk estimate component required above this index. All plants would be motivated to stay below this index just to avoid have to generate and defend the risk component of the index. It would also give the informed public a way to appreciate the level of plant safety (or risk) associated with the backlog. Also, of course, this approach would afford the NRC and the industry a chance to develop risk insights further and cast them in day-to-day priorities and progress.

Work Cycle Trending

One of the other issues raised on April 14, 1999, but at the Millstone restart meeting was the issue of outage and non-outage work. The plants continually reschedule outage work for completion during operations and vice versa, creating some confusion for those who are trying to evaluate whether progress is being made. Using the same approach as above for the backlog performance based index, it is possible to have work cycle versions that separate what is strictly outage work from what is not. Moreover, non-outage work could be further binned as required during operation or operationally independent. All of the work cycle performance indexes could be rolled up in the work performance backlog index.

A good plant might have an outage work performance index such as 150(7) and an operational work performance index of 100(2). A not-so-good plant might exceed the total work performance threshold of 500(10) described above by having higher numbers. This would also require a risk-based explanation and calculation that would enlighten all the stakeholders as to the plant safety or risk status. It is easy to perceive that such performance based indexes would be risk informed without being burdensome on the better plants. Again, the thresholds for risk indexing would encourage plants to perform well in prioritizing and accomplishing backlogged work both for outages and for operations.

Also importantly, these work performance numbers could exist contemporaneously, without regard to plant status. One would, for example, simply expect the outage work cycle index to improve faster than the non-outage work index when an outage was in progress. The indexes would both need to be below the regulatory threshold before the plant could start up without specifically obtaining NRC concurrence in this area.

Again, these work performance numbers would be compatible and consistent with the total backlog index, but they would also allow trending between outage cycles and would force a plant to commit a job to one category or the other up front rather than make excuses for deferrals late in an outage. Moreover, such indexes are entirely performance based and risk informed, and they include no non-performance trivia. This would effectively point inspection teams toward meaningful evaluations and results.

Training Performance Indicators

Other functional areas such as training are also amenable to performance indicators and thresholds. After 33 years, I have come to the conclusion that training is probably the most neglected performance area when it comes to NRC oversight relative to its importance to plant operational efficiency and safety. When equipment does not work, it is usually due to a problem in the maintenance training program. When design modifications go astray, it is almost always due to engineering support training problems. When control room operators make a mistake, training or culture is a root cause.

Thus, I encourage the NRC to specify programmatic performance indicators wherever it makes sense to do so. If I were a plant manager, I would want to know on a weekly basis how training is going. Each individual associated with a nuclear plant should have a training program, and that training should be reinforced at least on a two year cycle or on a basic refueling cycle of 18 months. I would like to know what percentage of the plant workers, supervisors, and managers are up to date (it should always be reported as 100%) and I would like to know the number of formal classroom man-hours expended each week relative to the schedule (also something that should be 100%). In instances where we are less than up to date, I would like to know about the backlog in hours. Thus, the training index should either be 100% or it should be something like 95%(250 hours).

As for all other performance based indicators, the training index can be broken down into its various parts and rolled up to whatever level is needed. The NRC could establish a training indicator threshold of 90%(500 hours), below which training would not be an issue. For those plants exceeding the threshold, the index should be further reported in terms of safety significant training (operators and safety system design and maintenance personnel) and non-safety significant (administrative support and janitorial).

The importance of such performance indicators as this is increased in that they are "leading indicators" that would allow all concerned to avoid dealing with the unavoidable consequences. If a plant or unit was at 80%, I would seriously consider declaring the plant to be unsafe. The question is, are you training your people or not? What is the training backlog?

Other programmatic areas should also be considered for similar performance data collection and application. Moreover, all of these indices should be connected in some way, indicating interactions and results. How many plant personnel are there for the identified jobs? How many are safety related? How many are filled? What is being done about the hiring backlog? What are the projected impacts on the training backlog, the outage backlog, and safety issue resolution? With answers to such questions in hand, then we can more easily identify and pursue policy and management issues that need to be raised with senior utility officials, avoiding impacts on the good plants and focusing on the not-so-good or badly trending plants.

Implementation Suggestions

All of the performance based indicators described above can easily be implemented at plants that are being well managed. Thus, I believe that it is reasonable to require each baseline inspection pilot plant to present them to the inspection teams as part of the baseline effort. All of these performance data points should be consolidated on one piece of paper. If some plants can not provide all of these data, deficiencies in specific areas will allow the inspection teams to focus their efforts accordingly.

I believe that Regulatory Research should be focused on developing and refining these performance areas, developing logical and reasonable thresholds and laying out future expectations for improvement. This will likely accomplish the most significant inspection and regulatory oversight upgrades and efficiencies in the new regulatory environment simply because these are programmatic, time-early parameters that can easily be implemented and effectively applied on a uniform basis across the industry.
 
 

Thank you for the opportunity to contribute substantive ideas to this development program. Overall, I believe you are going to achieve significant improvements in the reactor safety regulatory process.

Sincerely,

 /s/

Charles R. Jones
Nuclear Safety Engineer