Management Process and Programs Assessment (MPPA):
TRAINING QUESTIONS
Related Assessment Activities: TA1, TA2, EA4
Related MPPA Hierarchy Sections: 2B.2, 2B.3.2, 2B.3.4, 2B.3.7, 2E.1.1, 2E.1.3, 2F.1.3
Basis
The Training Organization at a nuclear facility requires enough high-quality personnel to enable the training program to effectively ensure that the work force is extremely well qualified and well trained. Resources should be available to readily accommodate every individual and group within the facility (operations, engineering, maintenance, fire protection, training, radiation protection) in achieving initial and periodic training objectives, including sustaining a well-defined and effective nuclear safety culture. With inadequate numbers or quality of personnel in the Training Organization, training goals will not be met, compromising facility safety.
Additionally, training facilities must be available and include appropriate support equipment needed to train each individual to work in a safe, productive, and efficient manner. Sometimes vendor facilities should be used to supplement the facilities of the Training Organization. Management must be willing to support obtaining whatever training resources are needed to develop the expertise of the work force and to ensure continued ability to understand and control plant systems and equipment. This expertise should normally exceed the level exhibited by equipment vendors for the equipment application at the nuclear facility.
Symptoms
• Instructors are not experienced in relevant plant activities and have deficiencies relative to documented job performance measures as evidenced by review of their work experience.
• Plant personnel must rely on outside experts to address system or equipment operational and maintenance problems which are considered routine tasks in the job task analysis.
• Requalifications or recertifications have lapsed or were waived in excess of established limits because required formal training was not performed on time.
• Significant numbers of incidents have occurred that could be attributed to lack of training, either on technical subjects or on nuclear safety standards and practices.
• Many of the important divisions' internal training programs are not established, as evidenced by records reviews.
• Training is not conducted or is infrequently conducted (e.g., less than annually) based on established training objectives for all plant personnel regarding nuclear safety culture principles such as communications formality.
• The support equipment in the training facilities is inoperable or otherwise unused in the conduct of training.
• The trainee-to-teacher ratio is too high (e.g., 30 to 1), and classrooms at training facilities are overly crowded (e.g., standing room only, inaccessibility around the classroom because of the high density of students).
• The facility is not operating because of a lack of qualified operators to comprise a sufficient number of shifts, or qualified operators are working extended hours on a routine basis at or above the facility's established overtime limits.
• Managers and supervisors do not participate directly in the training process on a routine basis (e.g., once per quarter).
• The actual time to perform routine technical tasks (e.g., repacking a valve) is substantially longer than expected based on approved, documented job performance measures.
• There is a large backlog of work to be performed in certain skill areas or on complex equipment due to a lack of skilled or trained workers as evidenced by a review and comparison of work planning documents and actual work packages.
• Training has little or no connection to current job and task requirements or is outdated as evidenced by a comparison of documented job analysis and learning objectives with actual work performed.
• The reference material in support of the training is inaccurate, outdated, and not readily accessible.
• There is no systematic process to improve and update training programs (e.g., post training evaluation feedback )
Functional Area Coordination
TA1: An inadequately staffed Training Organization may not have enough people to fill job positions documented by training procedures. Procedures provide the base line quantity of staff to establish good working relationships with other organizations, including providing support for special training, and collection and application of lessons learned from Operating and Maintenance Organizations. The individual assessing the Training Organization interactions with other organizations should consider whether these interactions are affected by inadequate manning of the Training Organization. For example, other organizations might have an as-built training schedule that is completely different from the originally planned schedule because of canceled training or unavailable training resources (facilities and instructors).
TA2: If facility training records are being maintained in a systematic manner, the assessor evaluating the records should consider whether the records problems are due to inadequate staffing. This type of problem could manifest itself if one observed a disheveled training records area or a large backlog of records that are not filed.
EA4: If Training Organization staffing or facilities are numerically deficient, disorganized or lack certain qualifications as established by training procedures, system engineers may not be receiving adequate training. Although not all organizations need a system engineer program, those having that do should have training support to enable the system engineers to be fully knowledgeable of their systems and equipment. Provision should also be analyzed to determine the need for system engineers to receive training at vendor sites. This type of support can be provided by the Training Organization, which would also benefit from the additional interaction with vendors and from comprehensive training of system engineers. The assessor addressing system engineer qualifications and training should consider whether any deficiencies are the result of inadequate Training Organization resources.
Management Cause Analysis
2B.2: Training instructors should demonstrate at a high cognitive level, the requisite technical knowledge. They are role models for those individuals in training that will eventually perform safety-related tasks in the facility. For this reason, instructor qualifications should, as a minimum, match job requirements, several of which can be considered "soft requirements." For example, the degree to which an instructor can convey the facility nuclear safety culture and the self initiative of the instructor are two job performance measures which can be observed only while training activities are in progress. The instructor must possess excellent communication skills as well as technical competence to be effective. The qualifications and quality of instructors have a direct effect on the adequacy of personnel resources in the Training Organization.
2B.3.2: The number of training courses offered may or may not be adequate, caused by a management failure to recognize the need for additional Training Organization resources. Training programs should be developed in a systematic, proactive manner where each training requirement for each person is determined and implemented based on a needs analysis. Planning for nuclear facility training should be based on established learning objectives and job task analysis. This planning should include providing the requisite array of training courses typically documented through a training implementation matrix. Sometimes these courses are made possible through self-training programs and provision of self-training equipment resources such as reference material or vendor-supplied cutaways of equipment.
2B.3.4: One way of determining whether the training organization has adequate resources is to assess the individual training opportunities and scheduling of training for a range of individuals. Job task analysis should have been performed to identify the training needs. The assessor may determine whether a specific job has less than required or no training courses or activities directly related to developing a strong level of expertise. If training management does not routinely inquire about training requirements of individuals and then make arrangements to meet those requirements, the individual's performance level may be limited and may not develop the skills needed in the future. One of the more efficient ways to use available resources for actual performance of tasks is on-the-job training. The objective of each individual training program is to develop and maintain the knowledge, skills, and abilities required by the job analysis.
2B.3.7: If data are used to assess the effectiveness of a training program, they can include indicators relative to the training resources provided. For example, Training Organization staffing can be assessed by reviewing the total percentage of authorized positions that are filled (e.g., 5% of authorized positions for minimum staffing requirements are not filled), by reviewing the average time a training position is vacant before filling, and by determining the percentage of Training Organization vacancies filled by outside contractors. Other indicators are the average educational level and years of experience of the Training Organization staff relative to the main work force. The best nuclear facility training programs are often those that use the most experienced and highly motivated people as instructors and role models.
2E.1.1: If there are deficiencies in resources provided to the Training Organization, this may be simply due to failure of management to meet identified requirements. To the extent that nuclear safety issues arise that reflect a lack of training, either the quantity or quality of training resources may be deficient. If nuclear training resources are deficient, the nuclear operations and other line activities are deficient almost by definition. Management must consistently provide an excess of training resources and opportunities at each nuclear facility in order to achieve an adequate level of nuclear safety. Because adequacy is a subjective notion, the level of nuclear facility training capacity needs to be determined systematically using cost-benefit determinations and other qualitative or quantitative measures (such as training self-assessments).
2E.1.3: The adequacy of personnel and material training resources can be assessed by looking at how the resources are allocated relative to their nuclear safety significance and risk to the public and workers. A logical, documented, and consistent prioritization process of allocating training resources should be established. Resource loading charts for training requirements in each of the departments of the facility should be discussed with the individual departments, weighted based on safety significance, and trended to see whether any adjustments might be required in the future. The effectiveness of how well resources were allocated can be evaluated by looking at the historical differences between training budget requests and allocated training resources.
2F.1.3: If the resources in the Training Organization are numerically adequate, they may still be deficient from the perspective of nuclear safety culture. Each instructor must have the level of motivation and positive attitude needed to pass on good attitudes and practices to those personnel receiving formal training. The instructor should clearly convey the safety culture, preferably through facility-specific examples to which the trainees can relate. This ability is best developed though personal experience. Also, it is not adequate to leave training to the Training Organization. For example, training on formal communications conducted in the classroom is of no value unless it is vigorously demanded throughout the organization. In this regard, every individual in the facility organization is a member of the Training Organization. Managers themselves should routinely conduct training for new personnel as well as experienced personnel on communications formality and their expectations regarding the actual implementation of the high standards discussed in the classroom. One way of determining whether management is doing its training-resource job is identify managers (including top managers) and supervisors who do not conduct formal training on a routine basis. Where managers and supervisors are not conducting training on safety culture topics, the level of commitment to nuclear safety will be inconsistent in the organization, allowing plant personnel to be inconsistent in their performance.
References
DOE Order 5480.18A, Accreditation of Performance-Based Training for Category A Reactors and Nuclear Facilities.
DOE Order 5480.20, Personnel Selection, Qualification, Training, and Staffing Requirements at DOE Reactor and Non-Reactor Nuclear Facilities.
Draft NS-40 Reference Guides, Chapter 12, Training.
NRC Inspection Procedure 41500, Training and Qualification Effectiveness.
Related Assessment Activities: TA1, TA2, FA6
Related MPPA Hierarchy Sections: 2C.2, 2C.3, 2C4.1, 2C5.1, 2D.1.1, 2D.2.1, 2D.3.1, 2F.2
Basis
Management controls are needed for the training function of an organization in order to carry out those training programs important to nuclear safety in a consistent manner. A well-structured training organization requires a systematic job task analysis, a set of defined learning objectives, and training design and implementation based on learning objectives. If training programs are structured in an informal or haphazard fashion, certain tasks will be performed differently by one individual and another or from group to group. The tasks are likely to be done in an inefficient manner. Furthermore, if the tasks were performed inconsistently because of a disorganized and inconsistent training program, this may accelerate degradation of equipment and/or facilities.
Clear lines of authority and responsibility as well as an accountability system must be established for the training function of an organization. The accountability system defines goal setting, reporting, tracking, and implementing a corrective action process. Putting these controls in place will greatly aid in maintaining a consistent, well-defined training program that would give each organization a more efficient and effective use of resources.
Symptoms
• Work practices are performed inconsistently for complex tasks that require more than one individual (e.g. filling, venting, and returning to service a large cooling water system). Workers do not meet the job performance measurements for complex tasks.
• Significant numbers of personnel errors resulting from inconsistent applications of on-the-job training.
• The mechanics of safety-related routine tasks are taught by word of mouth as evidenced by the use of little or no documented guidance during formal training.
• Safety-related upgrade tasks are performed ad hoc, and good practices or lessons learned are not incorporated into a continual job task analysis process.
• Frequent facility or equipment forced outages are experienced due to incorrect, poor, or inconsistent operating, maintenance, and engineering practices.
• Key divisions within each organization (e.g., operations, maintenance, technical support) do not have a designated training coordinator or a structured, formalized periodic training program.
• Recurrent errors are made in the performance of tasks over a long period of time because lessons learned training is not conducted and good practices are not passed on to all affected individuals.
• Divisional training programs are not reviewed, assessed, and updated by line management on at least an annual basis. The evaluation and revision of job performance measures is not documented nor formalized.
• Training programs within divisions are inactive (as evidenced by little or no training records), and although divisional training coordinators are assigned with formal authority and responsibility, supervision has failed to follow through, reviving training and making training coordinators accountable.
Functional Area Coordination
TA1: The training function of an organization will undoubtedly suffer if the total facility training program lacks organization, planning interactions with other organizations, and systematic feedback for training program improvements. Without a strong centralized training function (particularly at large, complex facilities), the training function of the divisions within the organization will not flourish, although the divisions may have a well-established procedure of responsibilities, authorities, and accountabilities that they in fact implement. Support is clearly needed from a strong facility training program.
TA2: A very telling measure that can determine the effectiveness of established procedures and methods for responsibilities, authorities, and accountabilities within the training function is a review of the facility training records. If training records indicate inconsistencies, incompleteness, and poor traceability, it is very likely that the training function has been neglected and certainly not implemented to the extent necessary. Poor documentation indicates a lack of formality and consequently a loss of vital training information, particularly if certain training is especially valuable to several groups or individuals.
FA6: If authorities and responsibilities are not clear and an accountability system is not in place for fire protection training, an assessor may find a training function deficiency in this activity. Either regular fire protection training may not be provided to facility staff or the observed training sessions may be inadequate.
Management Cause Analysis
2C.2: A Lead Program Secretarial Officer (PSO) should be designated and assigned line management responsibility and accountability for the headquarters and field operations of multi-program field offices. Part of the Lead PSO's responsibility is to ensure that roles, responsibilities, reporting relationships, and conflict resolution procedures for the training function are formally defined for the Field Office Managers (FOM), Field Program Managers (FPM), and the field office. Without a formalized program, individuals in the field will not know what the official training position is because they will not have a centralized document to determine who is in charge. Also, these same individuals may establish their own mini training program on an ad hoc basis that might not be consistent with the manner in which other individuals perform activities or tasks. Accountability should also be clearly defined within the same official documents that specify responsibilities and authorities.
2C.3: The FOM is responsible to PSOs for implementation of training programs at their sites. In order to perform effectively, the FOM must understand the goals and objectives established by the PSO, and be accountable to the PSO on institutional and cross-cutting issues. If the FOM does not have a good understanding of the PSO's training objectives, there is a distinct possibility that training programs within the FOM's sphere will be inconsistent and deficient when compared to other sites.
2C4.1/2C.5.1: Accountability must be clearly defined and documented for FPMs and Managers/Directors Area/Site Office with respect to the training function. A cohesive, organized training effort would generally not be sustained without a formal method of accountability. An established accountability process is much more effective in identifying and correcting deficiencies, and should include goal-setting for the training function. An assessor can evaluate whether accountability processes are followed as defined in management agreements by observing the Manager's/Director's day-to-day implementation, verification, and reporting of training program activities.
2D.1.1/2D.2.1/2D.3.1: The PSO, Field Office Manager (FOM), and contractors have in place management processes and programs to carry out responsibilities for personnel selection, qualification, training, and staffing requirements for the training function. These management processes and programs should be clear enough that selection of training personnel is efficient and provides the highest returns. If line management has not established or implemented these controls, the quality of training instructors may be deficient or the knowledge, skills, and abilities of the instructors may be uneven from person to person.
2F.2: It must be understood by all individuals that part of the authority, responsibility, and accountability of a successful training program lies with them as well. This principle should be formally conveyed to all individuals along with continued development of a safety culture where safety problems, standards, and objectives are identified and addressed. Within the facility-wide training program (e.g., general facility training), this concept should be discussed.
References
DOE Order 5480.18A, Accreditation of Performance-Based Training for Category A Reactors and Nuclear Facilities
DOE Order 5480.20, Personnel Selection, Qualification, Training, and Staffing Requirements at DOE Reactor and Non-Reactor Nuclear Facilities
Draft NS-40 Reference Guides, Chapter 12, Training
NRC Inspection Procedure 41500, Training and Qualification Effectiveness
Related Assessment Activities: EA4, FA6, MA1, TA1, TA2, TA3, TA4, TA7
Related MPPA Hierarchy Sections: 2A, 2A.3, 2B.2.3, 2B.2.5, 2B.3.1, 2B.3.5, 2B.3.7, 2B.3.8, 2B.4, 2D.2.1.7, 2D.3.1
Basis
Personnel involved in nuclear safety-related work possess qualifications and level of knowledge that are extremely important to the operating viability of a facility. A periodic program that refreshes, updates, and upgrades individual level of knowledge and verifies adequacy of knowledge (testing) is needed to ensure that the facility can be operated, engineered, and maintained safely. Continual evaluation and revision of training based on the performance of personnel in the job setting contributes to an effective training program. The program also provides a historical record to evaluate the validity of the training program.
Complete training records of individuals must be maintained in order to continue to have a means to indicate that a government facility has been operated safely, effectively, and efficiently by trained and qualified individuals who continually upgrade their level of knowledge and maintain pace with the facility changes that have occurred throughout the facility's operating history.
Symptoms
• Individuals with responsibilities related to nuclear safety have lapses in qualification and certification without formal approval as evidenced by a review of personnel training records.
• Continuing training programs neglect job-related knowledge, skills, and abilities and do not keep pace with new information and changes in areas such as job-related technical areas, normal, abnormal, and emergency procedures, site emergency preparedness, and in-house and industry operating experience.
• Continuing training does not reflect current needs or identified problem areas in order to correct adverse trends. Periodic needs analysis of associated jobs has not been performed.
• Continuing training is not conducted or infrequently conducted for key nuclear operations and hazards that are appropriate for the facility as evidenced by weak level of knowledge of actual hazards in the job setting.
• Increased number of personnel errors can be attributed to a lack of knowledge of a certain aspect of a task as evidenced by review of occurrence reports.
• Recurring training for personnel involved with nuclear operations and hazards does not include written and oral examinations or walkthroughs to verify level of knowledge.
• Personnel training records are incomplete, are not readily accessible, or are not stored in appropriate storage facilities (e.g., fire-proof locked safe). Training records do not contain information required in accordance with DOE Order 5480.20.
• Personnel that take formal examinations after recurring training fail at an excessive rate, indicating that training is either too infrequent or inadequate or that testing is not administered frequently enough during the training. Excessive failure rate may also indicate that the formal test is too rigorous for the learning objectives established for the training module.
• Training for personnel does not make use of facility-like props or materials that lend a real-facility atmosphere (e.g., the simulator is not patterned after the control room). The training is not designed and implemented using realistic scenarios.
Functional Area Coordination
EA4: The effectiveness of the retraining program in improving and maintaining the level of knowledge of system engineers can be assessed through this activity. Personal interviews of system engineers can reveal whether the recurring training is inadequate or too infrequent, or whether the level of knowledge of the system engineer needs to be upgraded to fulfill the job requirements. Inadequate system engineer knowledge may also indicate that the facility does not periodically evaluate and upgrade training evaluation based on performance in the job setting.
FA6: The assessor reviewing staff fire protection training can assist in determining whether the Training Organization is effective in providing training support. Since fire protection is directly related to nuclear safety, qualifications and training of facility staff regarding fire protection training should be assessed. The evaluator should verify that periodic training and testing are performed to maintain and assess level of knowledge, should ensure that training records are maintained.
MA1: Deficiencies in material conditions and excessive levels of hazardous conditions noted during a plant assessment could indicate that personnel do not receive recurring training and testing to maintain and assess their level of knowledge regarding nuclear hazards. The assessor conducting facility assessments may identify problems where recurring employee training could provide supporting guidance for routine corrective actions to be initiated by plant personnel.
TA1: The recurring training and testing needed by some personnel to maintain and assess their level of knowledge of nuclear operations is an indication of how well the Training Organization plans, schedules, and coordinates training with other organizations. Many observations can be made that might indicate a deficiency in this area. Two examples are lapsed qualifications because of missed training and poor attendance because of scheduling conflicts with other departmental duties.
TA2: Incomplete or deficient training records may indicate that recurring training and testing of level of knowledge are not being conducted for those involved in nuclear safety and operations. Training records must be maintained on each individual having responsibilities related to nuclear safety. An assessor can verify whether training records are available on personnel holding nuclear safety-related positions. This does not ensure that training is adequate and complete. This evaluation only verifies that training records are correctly and accurately retained, and that recurring training and testing has been conducted.
TA3: Nuclear safety fundamentals and safety culture support are at the core of certified operator training. An assessor may have to probe beyond just observation of training activities of certified operators to form an overall assessment of the adequacy of nuclear safety training. Review of training records and operating history may identify how nuclear safety issues are addressed, and provide more insight into the adequacy of the training.
TA4: Radiation protection training, including practical applications, should be compared with actual facility practices to see whether the training is realistic and reflects the latest facility practices promulgated by the health physics organization. Part of the review should include whether periodic training is conducted frequently enough. A barometer of the effectiveness of radiation protection training can be the number of contamination occurrences or other radiation protection related incidents.
TA7: General facility training provides a baseline level of knowledge that all facility employees need. The training should also be held periodically for everyone (typically annually) to help personnel keep pace with the evolution of the facility, nuclear safety issues, and DOE Orders. Testing should be administered to trainees in general facility training in order to evaluate trainee mastery of the material. The content of the training should emphasize safety culture, communications, and inquisitiveness.
Management Cause Analysis
2A: Personnel that are involved in nuclear operations and hazards should be challenged and pushed to improve or at least maintain their level of knowledge. Headquarters, field office, and contractor personnel should have adequate and current knowledge of the design, operation, material condition, and hazards of their nuclear facilities. This creates a cross-cut of knowledge which can raise the baseline knowledge of the entire facility to a high level.
2A.3: Line managers should also receive recurrent training and testing to maintain proficiency concerning those facilities for which they are responsible for. Without this training, line managers are unlikely to provide effective oversight and guidance, which may lead to early identification of level-of-knowledge problems.
2B.2.3: Personnel receiving recurrent training should be tested as well to verify their level of knowledge. This can be the specified process for documenting the judgment that an individual's qualifications meet the job requirements, if performed in a periodic manner. If a testing process is not established or documented, it becomes evident during a training records review. Additionally, the effectiveness of certain aspects of the training program is weakened and an assessment may uncover overall weak level of knowledge in a certain segment of the training program.
2B.2.5: Without an effective and tangible means to evaluate an individual's qualifications against the job requirements (e.g., by means of a test, walkthrough, or individual records review), no mechanism exists to identify and upgrade the individual's qualifications or level of knowledge to allow the individual to perform his/her job in accordance with requirements. Evaluation and revision of training based on performance of trained personnel in the job setting is a necessary step to ensure an effective training program. Indicators of a deficiency in this area may be an increased number of personnel errors attributed to lack of knowledge, or an extended duration taken to perform tasks considered routine for the job position.
2B.3.1: An approved and documented training program is the first step to establish recurring training and testing for level of knowledge on nuclear operations and hazards. An approved, documented training program instills formality and discipline and should include a periodic training program to demonstrate proficiency in a nuclear-safety-related job or task. Also, the maintenance of individual training records is established typically through the formal training program. The approved and documented training program can be compared for facility nuclear activities with the actual practices in the field to verify compliance with procedure. If aspects of the process are not being implemented, actions should be underway to either change or comply with the process (particularly if it is logical and advances nuclear safety).
2B.3.5: Part of the recurring training that personnel involved in nuclear operations should receive is classroom studies, simulator training, on-the-job training, and plant familiarization. All of these training tools can be used to periodically test the level of knowledge of nuclear operators. These different tools provide a training platform that is more closely modeled after the actual plant, and is more effective than classroom studies alone. The tools must be continually updated to reflect actual plant characteristics, which are dynamic as well. If recurring training does not include as many of these tools as is available for the facility, the training program will not achieve adequate effectiveness.
2B.3.7: Some of the data used to analyze the effectiveness of the training program are found through a review of the training records maintained for each individual. Also, a qualitative review of test results taken from recurring training modules can be an important indicator of the training program's effectiveness. Testing and records review are a concrete means of analyzing training effectiveness and should be a part of a facility's training program.
2B.3.8: If a program for retraining of operations and maintenance personnel does not exist, the overall level of knowledge of these personnel may degrade to the degree where nuclear safety can be seriously affected. For example, operations and maintenance personnel may lose the benefit of lessons learned from other incidents or personnel errors and make similar errors. Industry-wide improvements and plant upgrades that help make the operators and maintenance individuals job easier might be overlooked or not even realized. Retraining encourages more interest and in depth study, and generally raises the overall understanding of job requirements and complex tasks. Without retraining, one might be able to observe an operations or maintenance group that has slipped into a routine of maintaining the status quo with very little personnel interaction and inquisitiveness.
2B.4: DOE program managers and supervisors require a training, recruiting, and staff development program that helps sustain a strong nuclear safety culture. Retraining and testing are periodically required to maintain and/or improve staff and supervisory level of knowledge with respect to nuclear safety. Training records that show no performance of training nor document only periodic nuclear safety training may indicate a lack of definition in the training program.
2D.2.1.7: The Manager, Field Office (MFO) or his assigned staff will review the certification and recertification of shift supervisors, senior reactor operators, and fissionable material handlers at Category A reactors and high-hazard nonreactor nuclear facilities. Internal procedures and/or standards will be established to conduct the review to assess their level of knowledge. An absence of the MFO's oversight may indicate a deficiency in the priorities assigned to training.
2D.3.1: Contractors must establish management processes and programs similar to and consistent with the DOE training, recruiting, and staff development program for the same facility. Contractor training programs include written procedures for written and oral examinations, and operational evaluations to demonstrate that certified operators and supervisors possess the level of knowledge and skills required for certification. Many more findings can be made with contractor programs as with DOE programs, all of which must comply with DOE Order 5480.20.
References
DOE Order 5480.18A, Accreditation of Performance-Based Training for Category A Reactors and Nuclear Facilities.
DOE Order 5480.20, Personnel Selection, Qualification, Training, and Staffing Requirements at DOE Reactor and Non-Reactor Nuclear Facilities.
Draft NS-40 Reference Guides, Chapter 12, Training.
NRC Inspection Procedure 41500, Training and Qualification Effectiveness.
Related Assessment Activities: TA2, TA3, TA5
Related MPPA Hierarchy Sections: 2A.1.1.1, 2A.1.4.2, 2A.3.2, 2B.3.9, 2D.2.1.5, 2D.3.1.9, 2F.2
Basis
DOE Order 5480.20 requires operators and technical support personnel who protect special nuclear materials and radioactive materials to maintain a thorough understanding of the facility safety analysis report (SAR) and probabilistic risk assessments (PRA). These documents comprise a large part of the safety basis or safety envelope of a facility. To safely operate, control and maintain a facility, an individual has to understand the rationale behind safety margins and the several design and operating approaches employed throughout the facility. Additionally, the SAR and plant-specific PRA form the basis for emergency preparedness responses.
If no plant safety analysis report and probabilistic risk assessment training is given, there a higher probability exists that a facility may be exposed to more personnel errors and nuclear safety violations. Operators and maintenance personnel are likely to perform tasks with less understanding of the basis of why tasks are performed or processes are followed in a certain manner.
Symptoms
• Training courses for operations, maintenance, engineering, fire protection, and health physics groups associated with safety analysis report bases or understanding of plant-specific PRA are not formally included.
• A systematic job task analysis has not been conducted for training courses in SAR bases and PRA as evidenced by a lack of consideration of these topics.
• A significant number of errors is noted while conducting maintenance and operations tasks and is due to a misinterpretation of maintenance and operating procedures as evidenced by review of Occurrence Reporting Process System.
• Operations and maintenance do not know where the SAR and plant PRA are located or they do not have ready access to these reference documents.
• Modifications prepared for the facility that affect nuclear safety do not appropriately consider the safety significance of the changes. The quality of safety evaluation reports prepared for modifications may be not address the requirements of the SAR or may miss important areas of the PRA that are affected by the modification.
• Training test questions of formal examinations for certified operators which originate from SAR bases or plant-specific PRAs are not used.
• Modification packages affecting nuclear safety do not include PRA studies or neglect to cover all affected areas of the SAR, even if they are readily available.
Functional Area Coordination
TA2: A review for completeness of facility training records should include an assessment of whether training programs impart details of the SAR and plant-specific PRAs to personnel that protect special nuclear materials and radioactive materials. The records should indicate that training modules have been prepared for these items, test questions related to the SAR and PRAs have been periodically asked, and in particular, that safety margins and safety bases are clearly understood. If assessment of training records indicates that these areas have been lacking, deeper understanding of nuclear safety principles related to the technical facility design may be nonexistent.
TA3: Certified operator training should cover fundamentals of nuclear safety and safety culture support. The distinction between what is related to nuclear safety and what is not can be found, in part, in the SAR and plant-specific PRAs that have been documented over the history of the facility. Additionally, design and operating margins on safety limits need to be communicated to certified operators. Certified operator examinations should include SAR and PRA related questions.
TA5: Training for auxiliary operators should cover fundamental of nuclear safety and safety culture support, even where less emphasis would be placed on SAR and plant-specific PRA training of auxiliary operators than of certified operators. Because auxiliary operators perform several important tasks that protect special nuclear materials and/or radioactive material, the deeper the understanding of safety margins and safety bases an auxiliary operator has, the greater the appreciation will be for the possible implications of incorrect operation or manipulation of equipment on overall facility nuclear safety.
Management Cause Analysis
2A.1.1.1: In order that personnel have adequate and current knowledge of the design, operation, material condition, and hazards of their nuclear facility, an accurately documented, current SAR must be available on the facility. The training program can accurately convey the important aspects of the facility that are contained in the SAR. By reviewing and comparing parts of the SAR with training courses observed, it can be verified whether training has kept abreast with SAR changes or whether training is conducted at all.
2A.1.4.2: An evaluator should determine whether there is a PRA for the facility before attempting to answer this question. If no facility PRA exists for good justification, PRA training is not required. If a facility PRA exists, the evaluator can determine whether it is up to date by verifying whether all significant amounts of radioactive and hazardous materials identified are in actual facility locations, and that the measures used to prevent release of these materials are still employed.
2A.3.2: A line manager must have a good general knowledge of the facility or facilities for which he/she is responsible. Some of the tools the line manager uses to obtain that knowledge originates from a description of the safety envelope of the facility. The SARs and plant-specific PRAs are two of the many documents that form the basis of the safety envelope. If training is not given to line managers on SARs and plant PRAs, the line managers will likely be less effective in their oversight abilities to identify actual or potential nuclear safety concerns.
2B.3.9: The training program for nuclear operators must include germane parts of any facility PRA and facility accident management. A nuclear operator must know what the safety margin is in order not to overreact or underreact to an abnormal occurrence. Whether training is given and at what level can be ascertained by questioning nuclear operators on the current safety margins based on a relatively recent facility PRA.
2D.2.1.5/2D.3.1.9: Temporary deviations from facility SARs regarding facility staffing requirements must be approved by the Manager, Field Office (MFO). The contractor must request approval from the MFO of temporary deviations to the SAR. Keeping an updated, official version of the SAR with these changes ensures that employees responsible for protecting special nuclear materials and radioactive materials are cognizant of staff approved to perform nuclear safety tasks. Training programs should cover the details of facility staffing requirements in the SAR. An assessor can verify that SAR facility staffing requirements match the actual facility staffing.
2F.2: Personnel must have a working knowledge of the safety standards applicable to their functional responsibilities, which includes knowledge of the applicable SAR sections and applicable plant PRAs that affect safety margins related to their job requirements. There should also be an understanding of the broader DOE safety objectives outside of the scope of their assigned responsibilities. The training program should contribute to this knowledge base by providing SAR and plant PRA safety bases training. A strong facility safety culture is encouraged by this type of training.
References
DOE 5480.18A, Accreditation of Performance-Based Training for Category A Reactors and Nuclear Facilities.
DOE 5480.20, Personnel Selection, Qualification, Training, and Staffing Requirements at DOE Reactor and Non-Reactor Nuclear Facilities.
DOE 5480.23, Nuclear Safety Analysis Reports.
Draft NS-40 Reference Guides, Chapter 12, Training.
NRC Inspection Procedure 41500, Training and Qualification Effectiveness.
INPO 85-006, Principles of Training System Development. t
Related Assessment Activities: TA2, TA3
Related MPPA Hierarchy Sections: 2B.2.5, 2B.3.5, 2B.3.6, 2D.1.1.2, 2D.2.1.1, 2D.2.1.4, 2D.3.1.2, 2D.3.4.28
Basis
Facilities may be non-operational for a variety of reasons, including initial startup and construction, or during an extended period of shutdown. Before a facility can operate at full capability, a contingent of certified operators must be available to operate the facility around the clock. Practically speaking, many of the requirements of certification can only be simulated to the most realistic extent possible before a facility becomes operational. Additionally, periods of extended routine operation occur during which operators do not have the opportunity to conduct evolutions on the facility for training. Therefore, a provisional qualification program is required to conditionally qualify operators when necessary. However, qualified operators and certified operator trainers must follow up and perform these tasks or practical factors when actual on-the-job performance is possible (e.g., the facility is operational).
To ensure that provisionally certified operators actually perform evolutions or tasks, the operator training program must have a formal process to systematically follow up after provisional qualification and facility restoration. DOE Order 5480.20 discusses the use of a training implementation matrix (TIM) for this application to ensure all the requirements are defined and identified or exceptions noted. This systematic follow up concept is routed in the need to continually improve operator level of knowledge and to enhance nuclear safety culture of certified operators.
Some certified operator practical training can never actually be performed. Some examples of this type of training are implementation of emergency preparedness operating procedures, loss of coolant accidents (if applicable to the facility), and other accidents or incidents at the facility that present too severe a challenge to the plant. In cases such as these, realistic simulation is the best type of certified operator training possible.
Symptoms
• Operator errors occur at an ever increasing frequency because of inexperience or unfamiliarity with certain evolutions as evidenced by operator hesitancy and lack of confidence while performing a task. This may be caused by a deficiency in the training design and implementation that has not been identified through a systematic evaluation of training in the job setting.
• Operator training records indicate a lack of experience with certain routine evolutions or tasks (e.g., no practical factors recorded in facility startup and shutdown).
• The certified operator practical training program is not formalized. For example, no on-the-job training qualification record or signoff exists and practical factors are not recorded in training records as having been performed.
• Operator level of knowledge is nearly all "book knowledge" and certified operators do not fully understand and appreciate actual plant responses or plant conditions. This may be evident during an observation of complex job setting evolutions or simulator drills.
• Certified operators do not have a good understanding of what the duration of certain tasks or evolutions is as established by job performance measures, particularly those performed by auxiliary operators out in the field outside the confines of the certified operator's work area.
• Operator level of knowledge during recurrent training is deficient as evidenced by tentative manipulation of controls during infrequent evolutions (particularly in the conduct of simulator exercises).
• Operator learning objectives that are derived from job task analysis lack a variety of implementing tools such as practical factors and actual on-the-job training.
Functional Area Coordination
TA2: A review of the facility certified operator training records should reveal whether simulated practical factors are subsequently performed in the actual facility, if at all possible. Such review helps the assessor determine whether the certified operator training records are complete as well as whether the practical factors were actually performed in the plant. The time between facility restoration and completion of practical training in the actual plant may be an indication of the effectiveness and timeliness of the follow-up training, and whether there is a systematic process to complete practical training.
TA3: Nuclear safety fundamentals and nuclear safety culture support training for the certified operator should be taught using the most realistic means possible. If only simulated training is possible because of facility conditions, this should be considered acceptable. However, when plant conditions eventually allow for training to be performed in the plant and it is a task or evolution that can rationally be performed, the training records should track such practical training for this task or evolution to closure. Nuclear safety culture and understanding are enhanced when practical training is performed, if at all possible, in lieu of simulated training.
Management Cause Analysis
2B.2.5: Where certified operators may be provisionally qualified, a formal mechanism must be established to ensure that practical training is completed as soon as practicable. Follow-up actions should be systematically scheduled to ensure that actual plant training is not neglected. The scheduling of this training may be based on facility availability. An evaluator can inspect certified operator training records to assess the discipline involved to complete training and to ensure that qualifications match job requirements. The implementation of practical training must match the training design and learning objectives.
2B.3.5: The certified operator training program should include, if available, classroom studies, the use of simulators, on-the-job training, and plant familiarization that lead to formal facility nuclear operator approval. Simulators should not be relied on heavily unless it is impractical to perform the training in the actual plant. On-the-job training and plant familiarization are generally the most effective training tools and should comprise a large part of formal operator certification. Where plant conditions do not allow in-plant training to be performed for formal certification, a means of tracking and following up on this training deficiency must be established. The TIM for operators is an appropriate tracking mechanism. This will contribute to the formality and record keeping of the facility nuclear operator approval process.
2B.3.6: Operator training courses should have a high percentage of enrollments that result in personnel certification to perform related jobs or tasks. Operator qualifications must be formally certified because facility nuclear operators are directly involved in the operation and control of nuclear safety-related work. They also project (to an outside assessor) the facility's nuclear safety culture and how emergent issues are confronted. To pass an operator training course, an individual normally receives a formal certification. Where temporary or provisional certification must be given to support facility restart or initial startup, and all practical training cannot be performed, the provisionally certified personnel must complete their training eventually in order to receive full, formal certification.
A high percentage student certifications may imply that the training is not challenging enough or generally does not meet the requirements of the job. An assessor can determine whether this is a problem either by directly examining certified operators or reviewing the recorded examinations to verify adequate level of knowledge.
2D.2.1.4: In the event that practical training is not performed as scheduled and certification extensions are required to run the facility, management will have in place a procedure to extend certification on a limited basis. An evaluator should look at specific cases where facility management extends operator certifications and assess the approach and circumstances used in the decision.
2D.1.1.2/2D.2.1.1/2D.3.1.2: The training implementation matrix (TIM) defines and describes the site-specific selection, qualification, and training requirements, including those derived from DOE Order 5480.20. The Program Secretarial Officer (PSO) and the Manager, Field Office (MFO) should review and approve the TIM before implementation. Further, the contractor will have procedures approved the MFO for granting individual exceptions to specific training or qualification requirements. The TIM includes any exceptions taken for requirements which are not implemented. An assessor can determine whether practical factors are an exception and whether there is adequate justification in not performing them by reviewing the TIM.
2D.3.4.28: Part of the practical training for certified nuclear operators that is incorporated in formal training is on-shift training. On-shift training is conducted in accordance with the formal training objectives. As assessor may review the procedure for the training program that describes how on-shift training is conducted. Then the assessor may observe an actual on-shift training exercise to verify that training objectives are properly implemented in the job setting.
References
DOE Order 5480.18A, Accreditation of Performance-Based Training for Category A Reactors and Nuclear Facilities.
DOE Order 5480.20, Personnel Selection, Qualification, Training, and Staffing Requirements at DOE Reactor and Non-Reactor Nuclear Facilities.
Draft NS-40 Reference Guides, Chapter 12, Training.
NRC Inspection Procedure 41500, Training and Qualification Effectiveness.
INPO 85-006, Principles of Training System Development.
Related Assessment Activities: TA2, TA4, TA7, FA6
Related MPPA Hierarchy Sections: 2B.2, 2B.3.1, 2D.2.1.2, 2D.3.1
Basis
Maintenance & Operating (M&O) contractors at some facilities employ subcontractor personnel in key areas such as operations, maintenance, engineering, health physics, and fire protection. When used to support facility operations, subcontractors must undergo the same training (as contractors and DOE field personnel filling the same positions) or a similar training program based on identical standards used by the facility training program. This standard is to ensure consistent and safe job performance as well as a state-of-the-art level of knowledge by all participants.
Most types of training should be the same for DOE personnel, the M&O contractor, and subcontractors. This includes general facility training, health physics and radiation protection training, and facility-specific fire protection training. Typically, these types of training are combined into indoctrination training for individuals newly arrived at the facility.
Training for technical positions is highly specialized and usually requires identical training programs for employees and subcontractors alike because these technical positions are invariably related to nuclear safety. Generally, the more specialized or safety-related the position is, the more it is likely to require training programs of higher similarity.
Symptoms
• Large inconsistencies exist between the training records of subcontractors and others performing the same job or task function. The design of the training programs is markedly different between the two organizations.
• Inconsistent operating philosophies and approaches exist between the shift controlled by subcontractors and shifts controlled by DOE and M&O staff. Again, the evidence may be found by comparing the documented design and implementation of each organization's training program.
• Performance of subcontractor groups is markedly different from that of the DOE or M&O contractor groups. The performance could be rated either better or worse. A possible cause may be inconsistent job performance measures between groups for the same job.
• General maintenance practices vary between subcontractor groups or individuals and other groups or individuals as evidenced by different job performance measures, one of which does not meet the requirements of DOE 5480.18.
• ALARA and other health physics practices are inconsistent between the different groups (e.g., donning and removing anti contamination clothing).
• Level of knowledge pertinent to a specific job varies widely as revealed by oral examination and interrogation of individuals performing the same job functions. A difference in documented task learning objectives between the two groups may exist and one may not be in accordance with DOE 5480.18.
• Training requirements for subcontractors are not established and certified by the facility Training Organization and are not documented in auditable files.
Functional Area Coordination
TA2: Facility training records should include subcontractor training records, and they should be reviewed for consistency, completeness, and ability to reconstruct individual training records. The subcontractor training records should be as complete and auditable as other facility training records. There should also be consistency between subcontractor and other group records, because consistency helps demonstrate the similarities between the subcontractor and the facility training programs (if they are two distinct programs).
TA4: By observing radiation protection training in the classroom and comparing the training to the performance of practical applications, the assessor can verify whether different and inconsistent practices exist between subcontractors and other facility groups. Inconsistencies are less likely to exist where everyone receives the same indoctrination training. However, when comparing classroom training with what is actually done in the facility, one might find an inconsistencies between different groups may exist, because individuals who have been exposed to many different facilities may carry those different practices from facility to facility. Some of these practices may be inconsistent with practices of the current facility, and disrupt established flow and quality of work and worker protection activities.
TA7: The content of nuclear safety training aspects in the general facility training curriculum will give the reviewer and observer an idea of the minimum acceptable facility training standards for subcontractor workers. General facility training should be identical for subcontractors, DOE personnel, and M&O contractor alike, and training should therefore not be segregated. Because nuclear safety training is the most important aspect of general facility training, insights should be received from all groups. An evaluator can easily identify whether this requirement is being met .
FA6: Consistent fire protection training should be provided to entire facility staff, much in the way that general facility training is conducted. The emphasis should be placed on the safety-related aspects of fire protection. The learning objective of protecting the health and safety of the public and workers should form the basis of fire protection training.
Management Cause Analysis
2B.2: Subcontractor personnel must possess the same qualifications for the same jobs as do DOE and contractor personnel. Both groups of personnel should have equal technical competence, commitment, discipline, and high standards of professional and personal excellence. If there is an inconsistency between the different groups, then the training of one or the other group, or the standards used may be deficient or outdated. Training records may be audited and a sample comparison made between individuals from the two groups. One reason for a disparity between the two groups may be that one group has not completed a process for review and update of job requirements and incorporation of the update into formal training. Training program evaluation and feedback may not be conducted by one group.
2B.3.1: Clearly, an approved and documented training program for facility nuclear activities must be readily available and used. Ideally, both the subcontractors and the facility training program are identical. However, a subcontractor probably has a corporate training program that is not facility-specific. Some differences are allowed, but these differences cannot arise from a different use of standards. An assessor can verify that facility and subcontractor standards are identical, and then subjectively determine, for a given position the subcontractor is filling at the plant, if the training programs are similar enough. The greater the association the job has with nuclear safety, the more similar the training programs have to be.
2D.2.1.2: The Manager, Field Office (MFO), should ensure that contractors and subcontractors alike implement the applicable requirements in DOE 5480.20, Personnel Selection, Qualification, Training, and Staffing Requirements at DOE Reactor and Non-Reactor Nuclear Facilities. The order specifies several types of training required for individuals working at nuclear facilities, and also provides qualification requirements (e.g., minimum education and experience, special requirements) for several job positions. Subcontractors filling these job positions are required to meet the training standards of DOE 5480.20. The assessor may find that there are documented results of audits by the MFO reviewing the existing status of contractor compliance with DOE 5480.20.
2D.3.1: Subcontractor management must have in place the management processes and programs to train subcontractor personnel to comply with DOE 5480.20 using the same or a comparable training program. This is particularly important in the case of safety-related positions. Whether subcontractor management oversight and direction are sufficient can be ascertained by observing the subcontractor training program (if different from the rest of the facility) through classroom activities and evolutions.
References
DOE Order 5480.18A, Accreditation of Performance-Based Training for Category A Reactors and Nuclear Facilities.
DOE Order 5480.20, Personnel Selection, Qualification, Training, and Staffing Requirements at DOE Reactor and Non-Reactor Nuclear Facilities.
Draft NS-40 Reference Guides, Chapter 12, Training.
NRC Inspection Procedure 41500, Training and Qualification Effectiveness.
INPO 85-006, Principles of Training System Development.
For those who would like a more complete understanding and involvement with Technidigm.org: