February 19, 1999
 
 

Chief, Rules and Directives Branch
Division of Administrative Services
Office of Administration, MS T-6 D59
U.S. Nuclear Regulatory Commission
Washington, DC 20555-0001

As an independent nuclear safety consultant, I welcome the opportunity to provide my perspectives on the proposed improvements to the NRC's oversight processes for nuclear power reactors. I have significant experience in nuclear safety achievement and oversight for commercial nuclear plants, for government nuclear reactors and facilities, and for Navy nuclear propulsion reactor and propulsion plants. In addition to my experience in these three areas, I also participated intensively in the major DOE undertaking to convert commercial plant nuclear safety oversight processes for use at government nuclear facilities.

The NRC Staff efforts to develop improvements in the oversight process have been impressive. Nevertheless, the plans and obstacles to be faced over the next year are at least daunting if not insurmountable. Moreover, the task of dealing with a range of comments such as those provided below adds to the task although some of the comments might be useful in avoiding future problems. I have attempted to limit my comments to those that seem most important for near-term consideration.

First, I should point out that my use of the words daunting and insurmountable has some basis. Nuclear engineers and managers sometimes make things more difficult than they need to be, getting at each detail from multiple directions. While you have laid out about 40 inspectable areas, you have listed a number of them under different cornerstones, which produces almost 70 annual evaluations of some kind per plant (or per unit in some cases) each year. Moreover, you have made them largely reactive to the extent that performance indicators are used to keep the NRC focused on apparent problem plants, and are apparently committing to undertake a significant baseline inspection program that is also threshold based.

Besides the task of looking at about 7,000 inspection area thresholds (assuming one threshold per cornerstone inspection area) and about 5,000 PI thresholds each year in the industry, the reactive nature of the proposed approach can make it difficult to keep up with those thresholds that are crossed. While some thresholds might be viewed as passive or optional, the NRC will still have to evaluate and document them. Given the significance of nuclear safety and the uncertainty of the new process, the initial program will probably have to include more NRC reaction than will eventually be the case.

Perhaps more importantly, the NRC is creating thresholds at which it must react in an objective manner, without as much subjective input as in the past. Making the inspection and assessment process scrutable not only means that NRC actions will be questioned, the inactions will also be questioned. It remains to be seen whether this flip side of the threshold concept becomes a problem. Nevertheless, it strikes me as being something that the NRC has not fully considered at this point.

With regard to the 7,000 inspectable area actions, perhaps one way to bring this daunting challenge within reach is to apply some of the concepts that I suggested in October 1998 as part of my submittal that I titled "Delta Scores." In that submittal (see Internet URL http://ruleforum.llnl.gov/cgi-bin/downloader/AP_RPC_public/329-0004.htm), I suggested that the licensees should be expected to monitor and report their own performance, making it evident when they believed they were having trouble in a particular area (a self score of less than zero). The NRC would be free to agree or disagree to the extent that NRC resources allowed during its routine inspections (which you call baseline inspections).

My simpler approach has only 5 key inspectable areas for each functional area and is meant to be proactive in support of the continuous improvement thinking used to date since the accident at TMI 2. Allowing the plants to state their own perceived status under my "Delta Scores" method would give them credit for their current level of expertise. I think this is also what is intended under the SECY-99-007 approach.

Based on all of the above, I recommend that the NRC and the industry use a scoring system along the lines of "Delta Scores" for the inspectable areas. While this would not have all of the certainty advantages currently found in the proposed reactive approach, it would be a way of placing more of the burden where it belongs, on the licensee. The licensees are also in a much better position than is the NRC to evaluate the inspectable areas in a continuing and comprehensive manner. The better plants will do this anyway.

I also note that many of the thresholds that provide that certainty for inspectable areas have not been established. Allowing the licensees to be responsible for determining and scoring their own status for each unit from the almost 70 proposed perspectives might make the proposed inspection and assessment process more manageable within the NRC's limited resources.

The following pages address the specific questions in Federal Register of January 22, 1999 (Volume 64, Number 141, pages 3576-3578).
 
 

Framework Question A.1: Are there any other areas that need to be addressed…?

Response: Yes. Poor management, poor training, and poor safety culture (as well as organizational culture) have always been central characteristics of poorly performing commercial nuclear plants. Management and training are closely related to culture, and they are all cornerstones of nuclear safety. They have been short changed historically by the NRC (QED?), and this approach continues in the current development program.

To put things into better perspective, the Navy nuclear program considers good management, training, and safety culture to be essential prerequisites, and they are all taken for granted. It is very unusual for the Navy to have a problem in one of these areas since they come first.
 

Framework Question A.2: Are there alternative means of setting thresholds between bands that should be considered?

Response: Yes. The proposed thresholds are symptom based rather than root cause based. Since the NRC is attempting to use thresholds as a means of differentiating between plants and planning NRC inspection efforts, it would seem logical that one way would be to consider the root causes in conjunction with threshold crossings. If a root cause turned out to be in one of the crosscutting areas such as management, training, or safety culture, this should probably be of more interest.

Other parameters (of less importance) could also be used, such as the apparent effort and time required to implement corrective actions, whether the plant is operating or shutdown, whether a potential generic issue is involved, and whether the problem is repetitive. In any case, the NRC should take care to document fully what it considers and does regarding threshold crossings.
 

Framework Question A.3: Will these PIs, along with inspection findings, be effective in determining varying levels of licensee performance?

Response: No. They will get you into the "ballpark," but you will have to look closely at root causes if you are to determine the "varying levels of licensee performance." Also, once in the "ballpark" you will have an opportunity to expand the scope of inspections for obviously poorly performing plants. Your effectiveness will still be reduced (even after expanding inspection scope) because of the time-late, reactive nature of your approach, an issue raised by the Commissioners last year.
 

Framework Question A.4.: Are there any other comments related to the oversight framework, PIs, or thresholds?

Response: I have five comments on specific framework topics discussed in SECY-99-007.
 

Framework Topic: Cornerstone -- Ensure the availability, reliability, and capability of mitigating systems.

Comment: Inspections of safety or mitigating systems have been the primary basis for SSFIs in the past. In the future, the inspectable area "Safety System Design and Performance Capability" will address a selected risk significant system. The associated safety system PIs will probably be used to make a system selection, so the PI information will have to be evaluated periodically to ensure each plant is being accurate in its PI reporting. To be effective, this PI review should include a review of each system that has had an SSFI (or similar design-level validation) in previous years. The associated problems or findings should have been corrected or dispositioned in an acceptable manner. If the earlier SSFI items have been put to rest and the current inspection indicates that the design makes sense and the periodic tests are adequate, then the PI and area inspection basis for assessing this cornerstone can be gradually established.

In a recent SSFI on an Auxiliary Feedwater System (in which I participated) provided an example of the need for such a basis. The system appeared to be functional but had critical design and operational oversights that would have limited its effectiveness in an actual emergency. Thus, the first task if the NRC were to inspect this plant today would be to look at the auxiliary feed system and any other risk significant systems having major operational shortcomings (e.g., the ice condensers).

Thus, I do not think that this aspect of the new process can initially be limited to a single selected system at some plants. I suggest the NRC consider substituting a look at all outstanding design and operating issues and the resolution of previous ones in its first pass through this inspectable area, looking at each such system rather than picking only one system. This would be a sufficient level of effort and would be a much more effective use of the inspection team's skills and time at such plants.
 

Framework Topic: Cornerstone -- Ensure the integrity of the fuel cladding, reactor coolant system, and containment boundaries.

Comment: Barrier integrity has historically been monitored by radiological and leakage parameters and is difficult to inspect for directly. Some ISI results are also be important in this area. For these physical barriers, it would be appropriate for the reactor coolant system to have been the subject of an SSFI. RCS SSFIs are seldom undertaken since it is not generally considered to be a safety system. Nevertheless, during a relatively unusual RCS SSFI 1998, I noted significant shortcomings in how the plant was managing its reactor vessel embrittlement monitoring program. I subsequently attended a NRC Research symposium in which it was reported that important issues remain in the reactor vessel embrittlement area. Thus, the proper incorporation of this barrier cornerstone requires further design, operational, and ISI program research and validation of the reactor coolant system, a system that may not have been a high priority in the past.
 

Framework Topic: Cornerstone -- Protect nuclear plant workers from exposure to radiation.

Comment: Some plants currently issue contaminated protective clothing to workers rather than replacing it when fixed contamination can not be removed in the laundry. Even as an inspector I have been issued "clean" but fixed-contaminated protective clothing. A NRC HP expert at the regional office defended this practice based on economics. To me, this practice will always be unacceptable, and it reflects a NRC and commercial industry culture that does not adequately support this worker exposure protection cornerstone. Thus, this cornerstone needs to be validated programmatically, and the Commissioners should be asked to concur that such practices, if they are truly part of the NRC ALARA culture, are acceptable.
 

Framework Topic: PIs were found to be able to differentiate between good and bad plants but not average plants. Some poorly rated plants would not have been identified by PIs because the key issues were design related. It is suggested that this type of issue will be addressed in the baseline inspection program (complementary inspection efforts).

Comment: Ideally, each plant must be able to show that all of the safety significant systems have been subjected to a design review that validates its design intent functionality both in design documentation and in ongoing (periodic and post-maintenance) operational test results. Thus, I suggest that the NRC treat safety systems as falling into two categories, those with a properly validated design and operational basis and those without such a basis, initially paying less attention to whether or not a system has any red flags (PIs that are probably already being addressed).
 

Framework Topic: Crosscutting areas manifest themselves as root causes of performance problems. The intent is to rely on findings (PI and inspection) to point to weaknesses in crosscutting areas.

Comment: Using a reactive approach to "crosscutting" areas is based on the notion that areas such as training, corrective action programs, and safety culture are best monitored in a reactive manner. This notion may be appropriate if it could be assured that all paths to failure provide an accommodating delay period and sets of precursors such that it is possible to react to degradations in time. It is interesting to compare this approach with the Navy approach in which the crosscutting areas are emphasized rather than relegated. Based on my 32 years of diverse nuclear experience, this strikes me as the most disappointing aspect of the new approach, although it is not very different from the previous methods actually used. The NRC should at least include the plant's administrative infrastructure (management) as one of the listed crosscutting areas in which root causes and common cause failures are to be pursued. Given that the industry believes it is managing its plants well, the NRC should take this opportunity to state that the industry will be held accountable in this performance-based area.
 

Inspection Question B.1.: The proposed baseline inspection program is based on a set of inspectable areas that, in conjunction with the PIs, provides enough information to determine whether the objectives of each cornerstone of safety are being met. Are there any other areas not encompassed by the inspectable areas that need to be reviewed to achieve the same goal?

Response: Yes. While the 41 inspectable areas listed are topically comprehensive for the intended reactive approach, this approach is a significant departure from the more crosscutting functional area approach. Moreover, the list is not internally consistent. For example, licensed operator and emergency preparedness training programs are looked at, but the maintenance personnel, engineering technical support, and non-licensed operator training programs are not addressed.

There is nothing in recent history that suggests that these and other areas should not be scrutinized. For example, at one plant in 1998 I discovered that the maintenance training program had accidentally deleted training on proper bolting techniques about 18 months before our inspection. My discovery was based on a general plant walkdown during which I was able to detect poor bolting practices and other generic issues such as plant cleanliness and equipment labeling. The maintenance training organization was embarrassed to admit that they had inadvertently deleted this fundamental course from the curriculum. As currently documented, the proposed process sits astride training and is inviting a range of failures that are difficult to identify and correct.

Such areas may be viewed as "requirements" based but they also have significant value in providing early indication of likely plant material and operational degradations. Do the inspectable areas listed in the SECY include walkdowns? I did not see any, although this is generally part of an indepth look at a system and is indirectly suggested by using the term "as-built condition." The baseline inspection program should include a general plant walkdown.

Also, the inspection program "basis documents" should include additional entries regarding the specific bases for not inspecting in the traditional functional areas. Justification works in both directions, not just in the direction you intend to go, but also in the direction you do not intend to go. So far, the justification for not looking at such essential areas is that they will be uncovered when problems arise and root causes are addressed. Will the NRC need to establish thresholds for such things as numbers of mislabeled components and poorly assembled pumps, valves, and pipe flanges? If not, why would the licensees maintain good practices in these areas?
 

Inspection Question B.2.: Are there any other comments related to the proposed baseline inspection program?

I have three comments on inspection program topics included in SECY-99-007.
 

Inspection Topic: Based on a review of inspection programs in other industries, external insights will be used in the baseline inspection program. These include (1) indirect measures of compliance and "early warning" of potential safety and security problems, (2) less frequent but more intensive inspections are better than individual inspections, (3) checklists/protocols make inspections more systematic, and (4) generic findings need to be published to the industry.

Comment: The external examples are more consistent with the previous NRC inspection program than the new baseline program. The "insights" on checklists and feedback to the industry (items 3 and 4) are obvious parts of any inspection program and obscure or dilute the needed focus on timeliness and depth considerations (items 1 and 2). Crosscutting areas provide depth. The NRC has historically failed to address crosscutting areas directly, and the new process will do even less.

Interestingly, the greatest assessment success story to date is in the area of maintenance, a crosscutting area now likely to be used as a significant source of PIs through safety system unavailability considerations. If you want to take credit for comparing the new process with other industries, you should more clearly examine and justify the differences, including your approach to achieving timeliness while avoiding direct looks at generic functional area requirements or fundamentals. As currently presented, we seem to be having our cake and eating it too.
 

Inspection Area Topic: In addition to the PI-related (supplementary, complementary, and validation) inspection areas, it is intended that the baseline program will include a "comprehensive review of licensee effectiveness in identifying and resolving problems."

Comment: It is apparently intended that the licensees would be inspected for their ability to deal with ongoing lists of problems in a risk-aware manner. Since problems can not be effectively resolved without identifying and correcting root causes, does this mean that the NRC will for the first time look at management root causes or is this still out of bounds? The NRC and the utilities have both neglected the management root-cause area except after a plant was placed on the NRC Watch List. As presented here, the NRC is seeking to take credit for something (a comprehensive review) that it really does not intend to do.
 

Inspection Area Topic: Inspection procedures will be a brief methods checklist for each inspectable area.

Comment: The inspection procedures will probably have to include much more than brief methods checklists if they are to be predictable and avoid returning to the more subjective inspection mode. It is apparent that the intent is to also develop criteria or thresholds for the various inspection areas, allowing some consistency between inspection teams and between peer plants, so more structure seems to be needed, not less.

While I think a brief method can actually be a viable approach within a topical area, to make this realistic each inspection procedure should be supplemented by a "living" explanatory document. This would be a document that captures (at least anecdotally) the key issues and considerations actually being applied for the respective inspection procedure at different plants. This should include root causes and, thus, the bases for expanded inspections. Such documentation would form the basis for coordination, inspection area consistency, and inspector training. It should also be subject to periodic adjustments based on industry feedback.
 

Question C.1.: Are the proposed assessment periods sufficient to maintain a current understanding of licensee performance?

Response: Yes, if they can be implemented without overloading the NRC organization at key points during the year.
 

Question C.2.: Will the use of the action matrix and underlying decision logic reasonably result in timely and effective action?

Response: No. The revised approach is intentionally reactive and is therefore inherently less timely and, thus, less effective. There will be potential benefits in the areas of efficient use of resources (both industry and NRC), but timeliness and effectiveness are not logically supported in the absence of proactive looks at basic functional area requirements.

Perhaps it would help to define what is meant by timeliness and effectiveness. If I have a heart attack, I hope to get to the hospital in a timely manner and to be revived effectively. Nevertheless, my "hopes" would have been significantly improved if I had proactively attended to the fundamental requirements of exercise and proper diet in the first place.
 

Question C.3.: Communicating Assessment results -- Do these reports and meetings provide sufficient opportunity for licensees and the general public to gain an understanding of performance and to interact with the NRC?

Response: Yes. Even more effective is the approach periodically taken by DOE. They sometimes invite public interest groups and State agencies to have representatives serve as members of the inspection teams. The communication effects are dramatic, increasing credibility and reducing allegations. Even if they do not choose to participate due to funding or time issues, it is still much less likely that they will then complain about the process or its results.
 

Question C.4.: Are there any other comments related to the proposed assessment process?

I have three comments related to assessment process topics.

Assessment Topic: The assessment process includes consideration of PI and inspection area thresholds.

Comment: It is not clear as to where in the proposed process the inspection area thresholds will be developed and applied. Do the inspectors filter their reports based on inspection area thresholds (no NRC interest) or does the assessment process get all the raw data from different inspections and then screen against the thresholds? If the inspectors do the filtering, something could be lost among multiple inspections, especially for any problem plants requiring reactive inspections. A strategy seems to be needed to allow some of the information to be filtered out without compromising the NRC's ability to be circumspect in subsequent assessment deliberations. The inspection and assessment processes may need to be better coordinated than is presented here.
 

Assessment Topic: Assessment reviews are continuous, quarterly, mid-cycle, and end-of-cycle (annual).

Comment: The assessment process seems to be artificially tied to the budget process since both are annual. The more likely assessment cycle would be per refueling cycle or on a schedule consistent with technical specification requirements, both of which are often about 18 months. Such a cycle was also a SALP objective. Providing this "common denominator" would contribute to meeting the objective of being repeatable among plants and would probably reduce some of the intensity imposed by an arbitrary annual cycle.

Assessing all of the plants at the same calendar time seems to be a bit unmanageable. Quarterly and semi-annual reviews would still be appropriate, but the NRC may as a general practice want to wait for the end of a refueling cycle before issuing a comprehensive assessment on a plant. This would level the assessment effort without significantly impacting plans and budgets. Moreover, I think the industry's desire for predictability would be better served as well.

Also, consideration should be given to the use of refueling cycles or 18-month cycles so that the Commission can have a more consistent body of information to compare. This would make the assessment process more balanced in terms of time and resources throughout the agency, while making the total industry data available and applicable on a continuing basis, while providing a realistic framework within which to view the data. NRC regulatory actions would be more timely if data are reviewed and actions taken before each plant comes back on line after an outage. This would not only be timely from a safety perspective, the licensees would be much more motivated and able to fix their risk informed and performance based problems. The expectation that all such problems and their root causes would be dealt with before startup is something that the NRC should "get" in return for going down this path with the industry.
 

Assessment Topic: Certain random, unavoidable events are expected and will have to be considered as part of the overall plant performance.

Comment: Unless the industry and the NRC admit to the possibility of management root causes, the population of unavoidable events will probably remain high and may even increase. The notion of "unavoidable events" needs to be clarified beyond random failures of components and people. Perhaps a quantitative means of tracking the level of unavoidable events is needed, with continuous scrutiny and verification at the industry and individual plant levels.
 
 

Thank you for the opportunity to contribute to this development program.  Overall, I believe you are going to achieve significant improvements in the reactor safety assessment process.
 
 

Sincerely,

/signed/

Charles R. Jones