Project Payback Analysis and Organizational Feasibility

U. S. Department of Education Institute of Education Sciences NCES 2005–160 Feasibility of a Student Unit Record System Within the Integrated Postsecondary Education Data System Research and Development Report March 2005 Alisa F. Cunningham Institute for Higher Education Policy John Milam HigherEd. org, Inc. Cathy Statham Project Officer National Center for Education Statistics U. S. DEPARTMENT OF EDUCATION Margaret Spellings Secretary INSTITUTE OF EDUCATION SCIENCES Grover J. Whitehurst Director NATIONAL CENTER FOR EDUCATION STATISTICS Grover J.
Whitehurst Acting Commissioner The National Center for Education Statistics (NCES) is the primary federal entity for collecting, analyzing, and reporting data related to education in the United States and other nations. It fulfills a congressional mandate to collect, collate, analyze, and report full and complete statistics on the condition of education in the United States; conduct and publish reports and specialized analyses of the meaning and significance of such statistics; assist state and local education agencies in improving their statistical systems; and review and report on education activities in foreign countries.
NCES activities are designed to address high priority education data needs; provide consistent, reliable, complete, and accurate indicators of education status and trends; and report timely, useful, and high quality data to the U. S. Department of Education, the Congress, the states, other education policymakers, practitioners, data users, and the general public. We strive to make our products available in a variety of formats and in language that is appropriate to a variety of audiences.

You, as our customer, are the best judge of our success in communicating information effectively. If you have any comments or suggestions about this or any other NCES product or report, we would like to hear from you. Please direct your comments to: National Center for Education Statistics Institute of Education Sciences U. S. Department of Education 1990 K Street NW Washington, DC 20006–5651 March 2005 The NCES World Wide Web Home Page is: http://nces. ed. gov The NCES World Wide Web Electronic Catalog is: http://nces. d. gov/pubsearch Suggested Citation Cunningham, A. F. , and Milam, J. (2005). Feasibility of a Student Unit Record System Within the Integrated Postsecondary Education Data System (NCES 2005–160). U. S. Department of Education, National Center for Education Statistics. Washington, DC: U. S. Government Printing Office. For ordering information on this report, write: U. S. Department of Education ED Pubs P. O. Box 1398 Jessup, MD 20794–1398 or call toll free 1–877–4ED–PUBS; or order online at http://www. dpubs. org Content Contact: Cathy Statham (202) 502–7383 cathy. [email protected] gov Executive Summary This report examines the feasibility of implementing a student unit record (UR) system to replace the student–related components of the Integrated Postsecondary Education Data System (IPEDS). The feasibility study was initiated by the National Center for Education Statistics (NCES), a part of the Institute of Education Sciences (IES) within the U. S.
Department of Education (ED), in response to growing interest within the postsecondary education community for more accurate measures of net price and graduation rates, especially measures that take into account institutional mission and student mobility. This interest parallels a growing congressional desire to hold postsecondary institutions accountable for student outcomes. private institutions, or students who leave an institution and transfer across state lines. Many governmental and other organizations also maintain UR systems on specific groups of students.
For example, the National Student Loan Data System (NSLDS) within the office of Federal Student Aid (FSA) compiles information on all recipients of federal student loans, including verification of enrollment by academic term. In addition, the National Collegiate Athletic Association (NCAA) collects UR data on 1,800 institutions with Division I, II, or III varsity athletic programs, and about 2,800 colleges and universities currently contract with the National Student Clearinghouse to perform enrollment verification and other services using student UR data uploaded from member institutions.
At IES/NCES, the Integrated Postsecondary Education Data System (IPEDS) is the core postsecondary education data collection program, designed and implemented to meet its mission to report on the condition of postsecondary education in the United States. IPEDS is a single, comprehensive system that encompasses over 10,000 institutions whose primary purpose is to provide postsecondary education (including roughly 6,700 institutions that have Program Participation Agreements with ED for Title IV federal student financial aid programs and are required by statute to report to IPEDS).
The IPEDS system collects institution–level iii Background This discussion of the feasibility of a UR system at the federal level is occurring within the context of the development of other UR systems for students attending postsecondary institutions. Unit record systems are maintained by most colleges and universities to track registration for courses, academic performance, degree and certificate completion, financial aid, and other purposes. A number of states began to develop UR systems in the mid–1980s and use UR data for analysis and program evaluation.
Today, 39 states have at least one student UR system. A limitation of state UR systems, however, is that most do not include data on students attending Executive Summary data in the areas of enrollment, program completions, graduation rates, faculty, staff, finances, institutional prices, and student financial aid. The use of aggregate data has some limitations in comparison with UR data, such as the inability to track the academic progress and experiences of individual students, and therefore to study the longitudinal enrollment of different types of students.
Despite its comprehensiveness, the IPEDS system cannot measure many of the evolving trends in postsecondary education that are necessary for sound policy decisions. The current IPEDS framework cannot accurately capture changing enrollment and completions patterns in the postsecondary education sector, especially given increasing numbers of nontraditional students, and cannot describe the prices various types of students face after financial aid is taken into account.
To do so, it would be necessary to collect accurate student– level information on persistence systemwide (i. e. , regardless of institution and nationwide), multiple enrollment, part–time enrollment, transfer, and attainment. It would also be necessary to collect student–level information on prices and financial aid, in order to calculate net prices that take into account the individual circumstances of each student. By its very nature, a UR system would enable the collection of data that would lead to more accurate estimates of these variables.
In addition, a UR system would allow the development of a whole range of new measures, such as net prices for specific groups of students, graduation rates that take into account institutional missions, persistence rates that consider student mobility and a iv systemwide perspective, measures of enrollment patterns for nontraditional students, and time to degree by field of study. Goals and Design of the Feasibility Study
In exploring the feasibility of a UR system, the study attempted to investigate whether such a system could be constructed technically and effectively, given the knowledge about UR systems already accumulated at the state and institutional levels. In addition, the feasibility study tried to explore whether such a system should be developed by the federal government. To do so, the study solicited input on several dimensions, including privacy and confidentiality, institutional burden, coordination, technical issues, and timing.
As part of the feasibility study, three Technical Review Panels (TRPs) were designed to gather feedback and ideas from different perspectives related to the study, and included representatives from the following groups: 1) states, state systems, private systems, and private associations of colleges and universities; 2) institutions, particularly institutional researchers and registrars; and 3) other stakeholders, including the national postsecondary education association community, federal agencies, units within the ED, and vendors such as administrative information system developers.
In addition, the contractor developed an architecture and flow of operations for a proposed student UR system, as well as a list of potential data Executive Summary elements that might be collected under such a system. In reading this report, it is important to keep in mind that any redesign of IPEDS to develop a UR system would require legislative authorization through amendments to the Higher Education Act (HEA) and funds would have to be appropriated by Congress to implement the system. ould be used to calculate institutional summary totals for each school, with information about enrollment, completions, graduation rates, financial aid, and price. Four types of files would be submitted. • Header files: These data provide individually identifiable information such as name, Social Security Number (SSN), date of birth, address, race/ethnicity, and gender that are attached to an individual student’s record. These files would be required at least once for every student.
New header records would be submitted as needed to document any changes in these key data. • Enrollment/term files: These data include program information such as number of courses and credits attempted, major field of study, start and end dates, and attendance status. The files would be required three to four times a year, and institutions would be allowed to upload files more frequently if they wished. • Completions files: These data include information on degree completions and the date of completion. The files would need to be uploaded at least once per year. Financial aid files: These data include information on financial aid received from federal, state, and institutional sources. Information on price of attendance would also be included with the financial aid file. These data also would need to be uploaded at least once a year. Proposed Redesign of IPEDS If authorized and funded, the proposed UR system would replace the student–related components in the current IPEDS collection—Fall Enrollment, Completions, Student Financial Aid, and Graduate Rates—as well as the price of attendance variables collected in the Institutional Characteristics component.
The UR system would be designed to include all of the variables necessary to replace those components and calculate institution–level estimates for the Peer Analysis System (PAS). The collection process for nonstudent–related components in IPEDS would remain the same. It is difficult to describe exactly what the UR system would look like before the design process is undertaken. Such a process would involve numerous technical review panels and input from campuses, university systems, and state coordinators, particularly from states with UR systems.
Generally, the UR collection system would be designed to collect individually identifiable data through files that are submitted electronically by institutions. The files v Executive Summary In addition, in the first year of an IPEDS UR collection, additional files would need to be submitted in order for NCES to complete the historical calculations that are part of the Graduation Rate Survey. Depending on program length, these could include up to six years of data for key pieces of information.
For each submission of data, the IPEDS keyholder at an institution or coordinating agency would submit data electronically through the IPEDS collection system, similar to the process that exists currently. After submission, NCES would review the data to make sure they are consistent within the file and with prior submissions. Schools would work with the IPEDS Help Desk to match all records, and any that do not match would have to be resolved. The UR data would then be summarized in online institutional reports, which would also be checked for consistency, before the keyholder “locks” or finalizes the submission.
The UR data would then be moved from the collection system to the permanent database storage system. The full UR database would only exist in this permanent storage area, which would not be accessible via the Internet and would be subject to high IES/NCES levels of protection for confidentiality and security. Ultimately, aggregate estimates would be calculated from the full UR database and moved to the PAS, where they would be stored as institution–level data. Individually identifiable data would remain within the permanent storage vi 1 system.
The only allowable redisclosures of individually identifiable data would have to be specifically authorized in the HEA legislation, including. • Enrollment verification for the National Student Loan Data System (NSLDS): The UR system would be used to verify enrollment for students who are receiving federal student loans. Currently, this verification is being done either by institutions themselves, or by organizations such as the National Student Clearinghouse. • Verification of subsequent enrollment to the IPEDS keyholder: The UR system would be used to redisclose individually dentifiable data back to the initial keyholders and to state/system coordinators, in order to give something back to institutions. Data on the subsequent enrollment of students who left the first institution in the previous year would be redisclosed to the keyholder, including the institution of subsequent enrollment, date, attendance status, attainment, and date of attainment. 1 • Record mismatches: During the process of data collection for the UR system, mismatches between data records and other types of edit failures would have to be resolved.
This would involve sending individually identifiable information back to the IPEDS keyholder. These Redisclosure of student information to the original institutions could take place over a longer time period if this was decided by a future design Technical Review Panels and NCES. Executive Summary types of edit failure resolutions would be essential to the data integrity of the database. Other uses of the data would not involve the disclosure of individually identifiable student information.
For example, while ensuring the confidentiality of the data, NCES could generate aggregate reports for the Office of Postsecondary Education (OPE) using the UR data (e. g. , to generate aggregate measures of persistence, transfer, and attainment for various types of federal student aid recipients, such as those attending on a part–time basis). It would also be possible to add new derived variables to the PAS, used by institutional researchers and other analysts. Each of these derived variables would be reviewed for potential disclosure risks prior to their release on the PAS.
Such variables could include new definitions of net price; new measures of graduation rates that better take into account the missions of postsecondary institutions and the mobility of students across institutions; new definitions of time to degree, including transfer calculated for various fields of study; variables that describe enrollment by field of study and program length; and completions by field of study. postsecondary institutions already submit UR data electronically to private organizations; and postsecondary institutions that are Title IV participants are required to upload information on federal aid recipients to the FSA.
Nonetheless, in feedback from institutions, states, associations, and other stakeholders, it is clear if a UR system is legislatively authorized, certain concerns must be dealt with and resolved in the design phase of implementation. Privacy and confidentiality Concerns have been raised about student privacy and the confidentiality of individually identifiable student data under a federal UR system. ED, IES, and NCES have always taken seriously the importance of safeguarding student data, but a UR system raises questions about students’ rights to withhold or control personal information.
This is particularly the case for students who do not receive federal student aid. However, these students benefit indirectly2 from federal student aid funds, which support all programs, and benefit directly from state appropriations at public institutions and the tax–exempt status of private, not– for–profit institutions. Additionally, data on nonaided students are a critical element to compute graduation rates, retention measures, and other indicators. Information on nonaided students would be necessary in order to compare these measures with information on students receiving student aid.
Challenges to Implementing a UR System Technically, UR could be done at most institutions in the long term, after investment of time and financial resources. This can be inferred from the fact that 39 states have compiled UR systems in some form; thousands of vii 2 Tuition at these schools is probably lower than it would be if they were not the beneficiaries of tax–exempt status and state appropriations. Executive Summary In addition to misgivings about student privacy, there are practical, technical concerns about unauthorized access to the data by hackers and identity theft.
This is particularly true given the proposal to use SSNs as one of several personal identifiers that are necessary for matching student records. The use of SSNs would be essential to a UR system to accurately link together student information on financial aid, enrollment, and completions, as well as records from various institutions. Enrollment verification for the FSA already includes the use of SSNs as a student identifier. An additional measure of enrollment intensity at the start of each term (such as full– or part–time) would also be collected to atisfy FSA requirements. Despite these concerns, IES/NCES is well suited to protect the data, given the strict limits of the legislation regarding data confidentiality under which it operates. IES/NCES legislation protects the privacy of individuals, making wrongful disclosure a Class E felony punishable by up to five years in jail and a $250,000 fine. NCES has experience in working with individually identifiable data through its various sample surveys, and has created the structures and procedures necessary to prevent unauthorized disclosure of such data.
In fact, there are no cases where individually identifiable data collected by NCES have been wrongfully disclosed by an employee, a contractor, or a restricted licensee, or of cases in which hackers have breached IES/NCES firewalls. If collected, the data would be technologically protected and secure, and would not leave NCES unless viii allowed by law. Under the Patriot Act, the Attorney General and the Department of Justice could conceivably obtain access to UR data in order to fight terrorism. Students on whom data are held would be able to “opt out” of the redisclosure of subsequent enrollment information.
Institutional burden The additional burden of a UR system can be divided into two categories: initial implementation and subsequent operations. The burden of initial implementation is expected to be higher than the costs of subsequent operations. A field test would be necessary in order to make sure the system works, to anticipate and address problems that would be encountered, and to develop all necessary features in the system prior to implementation. About 1,200 to 1,500 institutions would be required to participate in the field test and report using both the old and new IPEDS collection system.
Although NCES would make every effort to notify selected institutions early, participating institutions would need to make changes in their reporting systems within a relatively short time frame, depending upon the desire of Congress for an implementation schedule. In the full–scale implementation, many institutions would need to upgrade information technologies and assign staff to comply with new reporting requirements. Staff would need to be trained in the use of these systems and the details of reporting procedures.
Some institutions would need to rely on vendors to provide upgrades to existing Executive Summary software, build their UR extracts, or pay for changes to legacy information systems. These additional activities would likely increase software costs. Obtaining historical GRS files for all cohorts in the first year would present a burden (although these same files are needed now to calculate the GRS locally). The initial burden on small institutions is likely to be relatively high, unless the institutions are part of a larger system or state association.
The additional costs of subsequent operations under a UR reporting system are expected to be lower than the costs of initial implementation. Keyholders would need to coordinate with offices on campus to gather data, run internal checks to make sure data make sense, submit data to NCES several times per year, and work with the IPEDS Help Desk to reconcile record mismatches and discrepancies in data. Some mismatches of records could be difficult to resolve, especially if there are numerous records. It is very difficult, at the conceptual stage, to make cost estimates with any degree of precision.
Costs would differ widely among postsecondary institutions, depending on whether they are in state UR systems, whether they currently upload to organizations such as the National Student Clearinghouse, whether they use local or proprietary administrative information systems, and the extent of their IT and institutional research capability. There would be a decrease in burden after the initial implementation of a UR system, as postsecondary institutions would no longer need to track and maintain records on GRS cohorts for six years or ix fill out the current IPEDS student– related components.
If a UR system were implemented, it would be important to take into account these various issues during the design phase of implementation so as to minimize institutional burden. There are different ways to offset the cost and burden of a UR system. One funding mechanism, Administrative Cost Allowances (ACAs), is used to help defray the cost of administering federal student aid programs. 3 A similar funding mechanism could be put in place for a UR system. Technical issues Technical issues were also raised as a potential challenge to the implementation of a federal UR system.
The proposed system would include the creation and maintenance of a database of millions of student records, with new records added every year. In addition, the system would require the uploading of large files from postsecondary institutions to NCES, using multiple forms of security to protect against unauthorized disclosures of data. NCES currently has most of the hardware and software necessary to implement a UR system, including current equipment used in the web–based IPEDS collection as well as servers capable of storing large amounts of student data.
One necessary addition would be database storage, to be located offline in a secure 3 Institutions currently receive over $150 million in Administrative Cost Allowances (ACAs), which is provided to help cover the cost of administration of federal programs such as Pell Grants and campus–based aid. Executive Summary site and protected by physical and software firewalls. There would likely be greater technical challenges for postsecondary institutions, with the extent varying among the registrar, institutional research, and financial aid offices, which sometimes utilize different and incompatible information systems.
Institutions using both legacy and proprietary student information systems would need to make software conversions or updates. For the smallest schools, an Excel template could be provided to collect data and generate the data file needed for submission. Although the technical issues could present a problem, these schools currently find a way to do uniform reporting for FSA financial aid eligibility and NSLDS loan deferment. The proposed UR system would also use XML4 technology for the submission of data files to NCES, although it is likely that ASCII files would be accepted in the early years of implementation.
Some postsecondary institutions have already adopted XML and are using it in their exchange of data with other organizations. On the other hand, many institutions do not currently use XML and training would be required on the use of this technology. Nonetheless, the FSA has already mandated that institutions begin 4 submitting data to the office using XML by 2005–06. Coordination Coordination of the flow of information presents a multitude of challenges in implementing a UR system. For example, a UR system might not work well within the existing IPEDS structures in some states.
Most state UR systems are based on specific census dates. If multiple header and/or enrollment files need to be submitted at different points in time to capture total enrollment, this would involve a change in workload for both institutions and systems. Special TRP meetings should be held during the UR design phase in order to leverage existing UR systems whenever possible in order to meet both federal and state/system requirements and needs. This will prevent unnecessary duplication of effort and reporting, and ensure that any federal UR system maximizes the lessons that have been learned through years of state UR reporting.
Timing In implementing a UR system, the timing of data collections would have to be addressed. If a UR system were authorized in 2005, a field test would then be administered in 2006–07, followed by full–scale implementation in 2007–08. The project timetable is designed to yield data relatively quickly while avoiding potential problems associated with an expedited timeframe. A phased implementation could also be considered to provide additional time to address problems during x XML is a “markup language,” or mechanism for identifying structures within a document or data file.
It employs tags to identify data elements, thereby facilitating the seamless exchange of data. In other words, it allows users to describe data and deliver it across a network, through the creation of common records across disparate databases. Executive Summary implementation. To respond adequately as part of the field test, it might be necessary for institutions to examine the utility of their administrative information systems for the purposes of producing UR extracts and to address some of the burden issues mentioned above such as training and staffing.
Early notification for the selected institutions would be crucial for the institution’s ability to respond in a timely and accurate fashion. It is possible that NCES could draw the sample of institutions immediately after legislative authorization to allow selected institutions almost a year to prepare. Since the UR system is based on individually identifiable records, it must comply with the Office of Management and Budget (OMB) requirement for collecting race/ethnicity data with a two–question format.
A by–product of the UR system is that schools that have not yet implemented this change will need to do so to meet OMB Statistical Policy Directive No. 15, Race and Ethnic Standards for Federal Statistics and Administrative Reporting. Another important issue is operational—how to time data collection schedules, while minimizing conflicts with other reporting schedules. The proposed UR system likely would collect enrollment records once per term. However, some institutions do not have standard terms; for example, courses may be offered on a rolling basis or on six–week terms.
Institutions could choose to upload data more frequently, especially for the purpose of enrollment verification for student loan programs. It would be necessary to find a method of xi specifying a whole range of flexible term reporting options, perhaps by asking institutions to document all possible term sequences using the IPEDS Institutional Characteristics component. Degree and certificate completions would likely be collected with only one file per year, although institutions with several commencement periods might wish to submit multiple files over the year.
In some cases, awards are recorded months after the relevant students have stopped attending institutions; degree dates then reflect the date the degree was awarded rather than when the degree was finished. In designing the timing of data collections and the periods of reference for the data, it would be useful to align the completions data with the enrollment data necessary to calculate graduation rates so that completions records can be matched to comparable enrollment records. Student financial aid information also would likely be collected in only one file per year.
Data submitted in an academic year would be from the previous year’s award cycle. It would be important to time the collection of financial aid data so that it does not conflict with the institution’s aid packaging period, which is the busiest time of year for financial aid offices. In addition, the treatment of summer sessions varies by institution, especially regarding whether summer sessions would follow or lead the submission of an annual data file. All of these timing issues would be addressed during the design phase of UR implementation, should a UR system be authorized.
In the proposed UR system, Executive Summary collection schedules would not need to be on a uniform schedule, but rather could be geared to a schedule that works best for individual institutions. In other words, institutions with different calendars or financial aid packaging schedules could submit data to NCES on different cycles. decisions, including the improved calculation of net prices; and more accurate measures for institutional accountability and program effectiveness, including enrollment, persistence, transfer, and attainment rates by program of study.
Policymakers would be able to monitor in real–time federal student aid programs (such as Pell Grants) and variations in aid packaging. The study also has attempted to highlight some potential benefits to institutions, researchers, consumers, and other users of NCES data. The study did not attempt to address every challenge or make recommendations about how each aspect should be addressed. Nor did the study document specific organizational positions regarding the obstacles a UR system might face.
Rather, it provided a framework for policymakers to understand the potential costs and benefits of a UR system as they discuss whether it should be considered. The central defining question of the feasibility of a UR system in IPEDS is not a “could” question. It is a “should” question, asking whether the federal government should develop a system that is based upon individually identifiable information about enrollment, financial aid, and attainment. This system would, for the first time, give policymakers and consumers much more accurate and comprehensive information about postsecondary education in this country.
Some of the benefits of a UR system include the collection of new data that would measure the success rates of students at institutions to which family and federal xii Conclusions As this report has outlined, a central question for a UR system is “Could it be done? ” Have the information technologies and infrastructures at the campus and state levels matured, could the current IPEDS web–based reporting system be adapted to a UR system, and would there be adequate technical and legal protections in place at IES/NCES?
The report has addressed some of the technical and system problems associated with the design and development of a new IPEDS UR system. At the technical level, a UR system could be done at most institutions given time for implementation. The feasibility study also addressed the “Should it be done? ” question, providing a framework for the discussion of issues inherent in this question. These issues constellate in several areas of concern—privacy, burden, coordination, technology, and timing—which would need to be addressed and resolved in the design phase of a UR system should policymakers decide to authorize and fund such a system.
Finally, the feasibility study outlined areas of federal interest: better information for informed consumer Executive Summary student aid monies flow, provide more accurate consumer guidance, and improve federal programs that support those families and students. In addition to benefits, the feasibility study found a number of significant issues that would need to be overcome before a UR system could be implemented, including objections about student privacy, confidentiality of data, new institutional burdens, coordination within and outside of institutions, and timing issues. iii Foreword This report examines the feasibility of implementing a student unit record (UR) system to replace the student–related components of the Integrated Postsecondary Education Data System (IPEDS). These components currently are based on aggregate institution–level data collected through IPEDS. The feasibility study was initiated by the National Center for Education Statistics (NCES), a part of the Institute for Education Sciences (IES) within the U. S. Department of Education, in response to renewed interest within the higher education community for improved data.
The findings in the report are based on several components. Three Technical Review Panels (TRPs) were designed to gather feedback and ideas from different perspectives related to the study, and included representatives from the following groups: 1) states, state systems, private systems, and private associations of colleges and universities; 2) institutions, particularly institutional researchers and registrars; and 3) other stakeholders, including the national postsecondary education association community, federal agencies, units within the U. S.
Department of Education, and vendors such as administrative information system developers. In addition, the experiences and architecture of existing UR systems at the state level, other federal agencies, and private organizations were compiled. space xiv Acknowledgments The authors wish to thank all of those who contributed to this report. We would particularly like to thank our colleagues at the Institute for Higher Education Policy and at HigherEd. org for their feedback and assistance, including Jamie Merisotis at the Institute for Higher Education Policy, and Connie Holohan, Mike Roma, and Andy Simpson at HigherEd. rg. We would like to thank Michael Lance of Benchmark Associates and Tod Massa of the State Council of Higher Education for Virginia. We also are grateful for the feedback and support provided by Cathy Statham, Dennis Carroll, and Susan Broyles at the National Center for Education Statistics (NCES) throughout the project. We appreciate the time and effort of the individuals who participated in the three Technical Review Panels held as part of this study, as well as the numerous individuals who sent personal comments and thoughts about the study.
We also would like to acknowledge the careful review and thoughtful comments provided by the following individuals at various stages of the study: Paula Knepper, Sam Barbett, Marilyn Seastrom, and Jan Plotczyk from NCES; David Bergeron and Dan Madzelan from the Office of Postsecondary Education; and LeRoy Rooker and Frances Moran from the Family Policy Compliance Office (FPCO). xv Table of Contents Page Executive Summary …………………………………………………………………………………………. ii Foreword ……………………………………………………………………………………………………….. xiv Acknowledgments…………………………………………………………………………………………….. xv List of Tables ………………………………………………………………….. ……………………………. xviii List of Figures…………………………………………………………………………………………………. xix Chapter 1—Introduction …………………………………………………………………………………….
Overview………………………………………………………………………………………………………. Reasons for Feasibility Study…………………………………………………………………………. Background …………………………………………………………………………………………………… Unit Record Systems in Operation ………………………………………………………………….. Current IPEDS Framework…………………………………………………………………………..
Future Data Needs ……………………………………………………………………………………… Context of IES/NCES Operations …………………………………………………………………. Organization of this Report…………………………………………………………………………….. Chapter 2—Proposed Redesign of IPEDS…………………………………………………………… Brief Description of Unit Record System…………………………………………………………..
General Architecture…………………………………………………………………………………… Redisclosures…………………………………………………………………………………………….. Analysis for OPE……………………………………………………………………………………….. Other Possibilities………………………………………………………………………………………. Alternatives Using IPEDS Aggregate Components ……………………………………………..
Other Alternatives to Unit Records ………………………………………………………………….. Chapter 3—Issues Related to the Development of a Unit Record System ………………. Privacy and Confidentiality ……………………………………………………………………………. Burden………………………………………………………………………………………………………… Initial Implementation …………………………………………………………………………………
Subsequent Operations ……………………………………………………………………………….. Technical Challenges…………………………………………………………………………………….. Coordination………………………………………………………………………………………………… Timing………………………………………………………………………………………………………… Variation Across Institutions …………………………………………………………………………..
Chapter 4—System Architecture……………………………………………………………………….. The State Unit Record Collection Model …………………………………………………………. Assumptions About System Architecture………………………………………………………….. Collection System…………………………………………………………………………………………. Schedule of Data Collection ………………………………………………………………………….. vi 1 1 3 6 6 11 16 19 22 23 23 23 26 27 28 29 30 33 34 37 38 39 42 43 44 47 49 49 51 52 53 Table of Contents Page File Preparation and Submission…………………………………………………………………… Edit Process………………………………………………………………………………………………. Movement of Data to Permanent Storage and Aggregates to PAS………………………. Process of Matching Records………………………………………………………………………..
Issues in the Collection Process ……………………………………………………………………. Redisclosures and Other Data Uses………………………………………………………………….. Training………………………………………………………………………………………………………. Help Desk……………………………………………………………………………………………………. Software ………………………………………………………………………………………………………
Hardware…………………………………………………………………………………………………….. 58 63 64 67 69 71 79 79 80 80 Chapter 5—Conclusions …………………………………………………………………………………… 83 References……………………………………………………………………………………………………….. 87 Appendix A—Technical Review Panels ……………………………………………………………. A-1 Appendix B— Estimates of Burden…………………………………………………………………..
B-1 xvii List of Tables Page Table 1 2 List of proposed variables to be collected in an IPEDS unit record system, by file type and IPEDS requirement ………………………………………………………………….. …. 54 Schedule of data collection ………………………………………………………………………….. 56 Appendix B B1 Selected comments received from institutions regarding burden associated with implementing unit records system, by selected topic …………………………………… B-2 viii List of Figures Page Figure 1 2 3 4 5 6 7 8 States with unit record systems ……………………………………………………………………. 9 Registration, file preparation and submission, and post–file–lock activity …………. 59 Adjustment of Peer Analysis System for unit record transactions and corrections . 66 Student record match subroutine ………………………………………………………………… 68 Enrollment verification for the National Student Loan Data System…………………. 3 Subsequent enrollment disclosure to institutions …………………………………………… 75 Program reports for the Office of Postsecondary Education…………………………….. 77 NCES sample survey files (NPSAS, BPS, and B&B) ……………………………………. 78 xix xx Chapter 1—Introduction This report examines the feasibility of implementing a student unit record system to replace the student–related components that currently are based on aggregate institution–level data collected as part of the Integrated Postsecondary Education Data System (IPEDS).
The feasibility study was initiated by the National Center for Education Statistics (NCES), a part of the Institute of Education Sciences (IES) within the Department of Education (ED), in response to renewed interest within the higher education community for improved data. The feasibility study was conducted between October and December 2004. This report describes the findings of the feasibility study. Overview If a student unit record (UR) system were to be implemented, it would allow the collection of high–quality data for student–related information in IPEDS, especially related to net prices and graduation rates.
By virtue of collecting data at the student level, a UR system would lead to more accurate estimates that take into account both nationwide trends happening across institutions as well as developments within institutions. The current IPEDS framework cannot accurately capture changing enrollment and completions patterns in the postsecondary education sector, especially given increasing numbers of nontraditional students and the mobility of students.
It also cannot describe the prices various types of students face after financial aid is taken into account. In addition to producing the same aggregate estimates that are already collected through IPEDS, a UR system would enable a number of additional estimates that would capture new dimensions of postsecondary education. These new measures could better capture the tracking of students across institutions, unduplicated national headcounts, and compute net prices that take into account student characteristics and enrollment patterns.
In exploring the feasibility of a UR system, this study attempted to investigate whether such a system could be constructed technically and effectively, given the knowledge about UR systems already in place at the state and institutional levels. In addition, the feasibility study tried to explore whether such a system should be developed 1 Chapter 1 — Introduction by the federal government. To do so, the study solicited input from various sources on several dimensions of the issue, including privacy and confidentiality, institutional burden, coordination, technical issues, and timing.
Three Technical Review Panels (TRPs) were designed to gather feedback and ideas from different perspectives related to the study, and included representatives from the following groups: 1) states, state systems, private systems, and private associations of colleges and universities; 2) institutions, particularly institutional researchers and registrars; and 3) other stakeholders, including the national postsecondary education association community, federal agencies, units within the U. S.
Department of Education (ED), and vendors such as administrative information system developers. (See appendix A for agendas and participants. ) In addition, the experiences of specific states, private organizations, and other entities that have built or maintained existing UR systems were compiled. A revision of IPEDS would need to consider the effective practices of already existing UR systems and maintain an ongoing dialogue with State Higher Education Executive Officers (SHEEOs), systems, and the states.
Also as part of the feasibility study, the contractor (HigherEd. org, Inc. ) developed an architecture and flow of operations for a proposed student UR system, as well as a list of potential data elements that might be collected under such a system. In reading this report, it is important to keep in mind that any redesign of IPEDS to develop a UR system would require authorization through the Higher Education Act (HEA) and appropriation by Congress.
This feasibility study was initiated in order to explore whether a UR system could, in fact, be developed, as well as what types of challenges existed to the successful implementation of such a system. The study did not attempt to address every challenge or make recommendations about how each aspect should be addressed, but rather provided a framework for policymakers to understand the potential costs and benefits of a UR system as they discuss whether it should be considered. 2
Chapter 1 — Introduction Reasons for Feasibility Study The feasibility study, reflecting a renewed interest in a UR system at the federal level, is the culmination of several trends in postsecondary education during the 1990s. • annual price increases at postsecondary institutions that have exceeded increases in inflation indexes such as the Consumer Price Index (CPI); • policy concerns about the impact of price increases on consumers and on student aid programs; • a growing congressional interest in holding institutions accountable for outcomes, starting with graduation rates for tudent athletes and campus crime reporting; • a demand for better and more timely data to inform policymaking and consumer choices; and • the desire of many postsecondary institutions for more accurate measures of net price and graduation rates, especially measures that take into account institutional mission and student mobility. Congress has attempted to address these trends in several reauthorizations of the Higher Education Act. The 1992 HEA Amendments created a “National Commission on College Costs” to study the problem of annual increases in prices at institutions beyond increases in the CPI.
Such increases in price were an issue both for consumers and for Congress, which each year faced increased appropriations for federal student aid programs. The commission’s report distinguished between cost and price of attendance, recommended more accurate and timely data on costs, prices, and student aid, and looked at the relationship of student aid programs to cost increases. The 1992 Amendments also included “Student Right–to–Know” legislation, mandating graduation rate information for all students.
In response, NCES began the Graduation Rates Survey (GRS) component of IPEDS collecting data on graduation rates on first–time, full–time students, within 150 percent of the nominal time to degree or completion. The 1998 HEA amendments instructed NCES to conduct a “Study of College Costs,” which included an analysis of whether student aid programs were themselves a factor in driving up costs and an analysis of net prices, focusing on the relationship of 3 Chapter 1 — Introduction rising sticker prices and the differential net price paid by students and their families.
Net prices reflect the prices paid after financial aid is taken into account. Although there is substantial debate surrounding the issue of which definition of net price is the best to use in examining access or affordability, there is agreement about the fact that it is difficult, if not impossible, to measure net prices on an aggregate level. The net price study (Horn et al. 2002) showed that prices were flat both for low–income students taking grant aid into account and for middle–income students with both grants and loans.
The 1998 HEA Amendments mandated a redesign of IPEDS, making it a significant element in institutional accountability. 5 IPEDS was charged with collecting data on institutional prices, changes in prices over a three–year period, and student aid. NCES was also tasked with making this consumer information readily available online, along with graduation rates. To fulfill this task, NCES created IPEDS College Opportunities on Line (COOL), the Department of Education’s provision of information on all Title IV institutions.
IPEDS itself became a web–based data collection, to insure more timely data for policymakers and consumers (see further discussion below). All of these changes reflected congressional interest in accountability and having better information on college prices and net prices. In the context of the current reauthorization of the HEA, price increases, particularly in the public sector, led some in Congress to consider an “Affordability Index” to reign in price increases.
Under the proposal, institutions whose prices had increased more than twice the rate of increase in the CPI over a two–year period would be required to meet higher standards of accountability. They would have had to report on why prices had increased and outline steps to reduce the rate of increase to remain eligible for campus–based portions of Title IV of the HEA, where federal student aid programs are authorized. Although the Affordability Index proposal has been dropped, the interest in outcomes measures such as graduation rates has remained.
For example, a recent report by the Education Trust (Carey 2004), A Matter of Degrees, as well as a congressional oversight hearing, have focused on institutional persistence and graduation rates and on the limits of the current IPEDS Graduation Rates component in providing accurate information for institutional accountability. In addition, 5 The mission of and data collected by IPEDS and its predecessor, the Higher Education General Information Survey (HEGIS), have shifted over time. See further discussion below. 4
Chapter 1 — Introduction the postsecondary education community has shown a renewed interest in better information on graduation rates, which would include new data on nontraditional students who attend part time or otherwise delay their enrollment and have gaps in attendance, as well as on students who attend more than one institution, transfer, or coenroll at multiple institutions. If Congress were to use graduation rates and time to degree as accountability measures for institutions, the consensus is that the GRS in its present form is inadequate.
Mission–specific measures would take into consideration the goals of the institution, such as offering two–year, transfer programs; serving part–time, adult learners; or tailoring workforce, noncredit training to the needs of business and industry. Furthermore, the current system treats nongraduates as dropouts, when they may have in fact persisted or completed their educational program at another institution. More accurate data are necessary for more nuanced policy decisionmaking, toward the goals of improving student performance and informing students and parents about the true costs of college.
The American Council of Education (ACE), the Association of State Colleges and Universities (AASCU), and the State Higher Education Executive Officers (SHEEO) sent letters to congressional leaders, asking that IES/NCES conduct a “feasibility study” of a data system, derived from URs, that would provide mission–specific data on enrollment patterns of all students and outcome measures such as institutional persistence, completion rates, and time to degree, along with detailed information on student aid that would make possible accurate calculations of “net price” for students.
Taking into account individually tailored financial aid packages—including the packaging of aid from federal student aid programs—would allow the federal government for the first time to assess accurately the relationship of various student aid programs to persistence. The debate on the feasibility of a UR system at the federal level is occurring within the context of the development of other UR systems for students attending postsecondary institutions.
The following section describes some of these efforts, as well as the current IPEDS framework, future data needs, and the context of IES/NCES operations related to the protection of individually identifiable data. 5 Chapter 1 — Introduction Background Unit Record Systems in Operation It is important to distinguish between the two types of data that are available for analysis: (1) summary or aggregate data; and (2) microdata, the raw or unit record (UR) data that are summarized or “rolled up” into aggregate data.
For example, an aggregate report may document the number of bachelor’s degrees awarded by an institution, where UR data would document the data that go into the report; in this case, individually identifiable information about each degree recipient. A UR system could then document students’ demographic, enrollment, attainment, and financial aid information, as individually identifiable records if desired. For each type of data collected, schools would submit one record per student per term per institutional identifier.
For many years, colleges and universities have maintained computerized recordkeeping through the use of administrative information systems. Typically, specialized admissions software is used to monitor student applications and acceptances, in addition to human resources software for hiring and paying employees and registrar software to keep track of course enrollments, grades, and awards/degrees. In order to submit URs to a federal, state, or other data collection, schools must create electronic extracts or snapshots of their recordkeeping data from these different administrative information systems.
These extracts are created using special software tools and can include whatever variables are desired. Extracts represent the selected data and records as of a specific point in time when the files are cut. In order to complete aggregate summaries in reporting to governing boards, state agencies, or other entities, institutions have two basic choices: (1) run a computer program (or query) against the live, administrative information system to produce a summary report; or (2) create an extract of the data needed and use these records to produce a summary report locally.
For both choices, schools must then engage in a significant amount of review and clean up of data to ensure that they can be aggregated accurately to reflect the institution at the point in time (or census date). In cases in which institutions must submit UR data directly rather than in summary reports, schools have two choices about the initial cleanliness of their file submissions. They may either (1) submit the raw data they obtain from their administrative information systems (which are called transaction files because they 6
Chapter 1 — Introduction represent ongoing transactions or interactions with the live database); or (2) submit files on which they have conducted further editing (in which case they are called analytical files because they have been scrutinized from an analytical perspective). The different approaches are important to recognize because, if submitted, transaction files may be missing some data or include unexpected values, such as invalid Classification of Instructional Programs (CIP) codes. The resulting summary report may be different than expected for the institution as a whole.
However, increased work is necessary to create analytical files, resulting in greater burden on institutions. The distinction between aggregate and UR data is important because there are inherent limitations to using aggregate data, just as there are issues of data integrity for transaction versus analytical files. Aggregate data collected at one level of analysis cannot be used for lower levels of analysis, such as how the data differ among various groups of students or how students move between institutions (unless this is specifically included at both levels).
With aggregate data, it is difficult (if not impossible) to examine relationships among variables or to recompile data for different reporting needs (National Postsecondary Education Cooperative 1998). Many governmental and other organizations maintain UR systems on specific groups of students. For example, in the Department of Education, NCES conducts sample surveys of postsecondary students (such as the National Postsecondary Student Aid Study, [NPSAS]) in which it collects UR information from the institution for each student in the sample.
The National Student Loan Data System (NSLDS) within the office of Federal Student Aid (FSA) compiles information on all recipients of federal student loans, including verification of enrollment by term. FSA also has detailed data on all federally aided students, which represent more than half of full–time undergraduates. The Office of Postsecondary Education (OPE) collects student–level information on the recipients of specific program funds, such as GEAR UP, Upward Bound, and Talent Search, for program evaluation.
Other branches of the federal government also collect student information. For example, the National Science Foundation (NSF) conducts its Survey of Earned Doctorates annually from all individuals receiving research doctoral degrees from U. S. institutions. The Internal Revenue Service (IRS) requires colleges and universities to annually submit individually identifiable student data on tuition and related expenses and 7 Chapter 1 — Introduction scholarships/grants, for all enrolled students, in case they or their parents claim a Hope or Lifetime Learning tax credit.
This same information also is used to send 1098–T forms to students for use in preparation of their tax forms. More recently, the Department of Homeland Security (DHS) has created the Student and Exchange Visitor Information System (SEVIS) in order to maintain information on nonimmigrant students and exchange visitors from the time they receive their visa documents until they complete their programs. Under SEVIS, colleges are required to provide regular electronic reports confirming enrollment in postsecondary institutions to the DHS.
State and local governments, which are the primary funding sources for public colleges and universities, have tied this funding to requirements for collecting increasing amounts of data on students attending institutions in their states. A growing number of states began to develop UR systems in the mid–1980s, and some states have 30–year histories of using UR systems for analysis and program evaluation. In fact, according to a recent study (Ewell et al. 003), 39 states have at least one student UR system; some states have more than one, because separate UR systems are maintained for sta