National Pupil Database


In the United Kingdom, the National Pupil Database is controlled by the Department for Education, based on multiple data collections from individuals age 2-21 in state funded education and Higher Education in England. Data are matched using pupil names, dates of birth and other personal and school characteristics, including special educational needs, disability, and indicators for free school meals, a child in care, and families in the armed forces. Personal details are linked to pupils' attainment and exam results over a lifetime school attendance.
In October 2018 the database contained . It is deemed by the Department to be “one of the richest education datasets in the world". This is just one of the distributed datasets that the Department for Education controls, and separate from the further Individualised Learner Record in the Learning Records Service, for example.
Schools use Management Information Systems to collect and analyse pupil level information at local level. Data from these systems are used to complete the termly school census returns provided to Local Authorities or directly to the Department for Education three times a year. The National Pupil Database has expanded in its scope of the items collected, and from children of a wider age range over time. Data once stored in the National Pupil Database, are never deleted.
, passes collected from universities to the Department for Education, where it is linked to individuals' school records in the National Pupil Database, expanding the lifetime record for millions of people that the Department retains indefinitely.
The National Pupil Database referred to here, covers only pupils in state schools in England. However similar systems operate across the rest of the United Kingdom.
The pupil level data are personal confidential data which include sensitive personal data as defined by the Data Protection Act 1998. The National Pupil Database contains:
Identifiers: the pupil, school and local authority identifiers. With effect from 2010/11, the pupil’s UPN remained consistent throughout their time in school and remain permanently on the record.
Fixed pupil characteristics which are expected to stay the same, such as date of birth. However some characteristics which may be expected to be static, for example ethnicity, may be inconsistent within an individual record. Ethnicity was found in 2016 to have been ascribed by school staff in some counties, even to the point of overriding parental and pupil choices to refuse to provide the optional information as recommended to schools by Brighton and Hove City Council ahead of the autumn school census. This option was changed in system providers' design to remove fields 'ascribed by' as a result.
Time-varying pupil characteristics: these may change in each sweep of the census because they describe the circumstances of the pupil at that point in time.
Key stage test results and other attainment data: data at the ages of five, Early years foundation stage profile or seven, eleven 14, 16 and 18.
School type characteristics: these describe the school the pupil attends at each sweep of the census. School staff data is not included in this census for the National Pupil Database, but through the separate . Workforce personal data includes identifiers such as names and National Insurance numbers and characteristics such as ethnic group, employment contracts and remuneration details, qualifications and absence information.
There are about 400 possible variables to collect on individual pupils. The full national code sets of all the items of data that can be collected on individual children can be downloaded from the Department for Education, listed in the , including health and SEND.

Use of pupil level data

For uses of Key Stage attainment datasets and School Census dataset see also England: School Census. Raw pupil level personal data are held in the Department for Education National Pupil Database. contain data which are identifying, and too sensitive or disclosive to be published, although these data are given out to third parties in raw form.
David Cameron announced in 2011, the government would be “opening up access to anonymised data from the National Pupil Database .” This was an expansion to other third parties, since these data had already been used for many years and extensively by academic .
Since 2012, Secretary of State has had powers to share raw data from National Pupil Database under terms and conditions with named bodies and third parties who for the 'purpose of promoting the education or well-being of children in England are conducting research or analysis, producing statistics, or providing information, advice or guidance', and who meet the Approved Persons criteria of the 2009 Prescribed Persons Act, updated in 2012/13.
The data when released, however are not anonymised, but are sensitive and identifying. "According to centrally held records at the time of writing, from August 2012 to 20 December 2017, 919 data shares containing sensitive, personal or confidential data at pupil level have been approved for release from the National Pupil Database. For the purpose of this answer, we have assumed the term sensitive, personal or confidential uses of information to be data shares classified as either Tier 1 or Tier 2 as set out in the National Pupil Database area on GOV.UK. There were 95 data shares approved between March 2012 and this classification system being introduced."
In a presentation to the NPD User group in September 2016, the Director of the DfE Data Modernisation group acknowledged the release of sensitive data: "People are accessing sensitive data, but only to then aggregate. The access to sensitive data is a means to an end to produce the higher level findings.”
The data items for release are classed into four tiers by the Department for Education, as described in the NPD User Guide. Following the change of legislation, releases of the data since 2012 from the Department for Education to third parties have not been anonymous, but have been of identifiable and highly sensitive, identifiable and sensitive, aggregated but may be identifying due to small numbers and identifying non-sensitive items. Raw, closed data are released on a regular basis to third parties, and the majority of releases are of Tier 1 and 2 data.
A list of completed and those in the pipeline, are published on a quarterly retrospective basis.
Government uses of the data are based on a model of data sharing, passing raw data from one location to another, which is viewed by some as 'obsolete'. Intra departmental transfers of data include to the Cabinet Office for preparation of Electoral Registration Transformation work in 2013, to in the National Citizen Service, and for use in the Troubled Families programme, as well as arms length bodies such as NHS Digital for a survey mailed home to 300,000 15 year olds in 2014. Not all government uses of the data are recorded in the Third Party Release Register, such as internal use. The volume of Police and first made public through Freedom of Information requests in 2016, were first officially published by the Department, in the Third Party Release Register in December 2017, under "". Police requests were only documented going as far back as July 2015. This omits police access to records before this date, as noted in a made by Nick Gibb, Minister of State for School Standards, on the numbers of pupils data released to the Home Office and police. “Information supplied by the Data Modernisation Division of the DfE has been identified as containing incorrect facts in the response provided to Parliamentary Questions concerning the volume of children’s records passed onto the police and the Home Office and in figures quoted during a House of Lords Debate on the 31 of October 2016 on the Education Regulations 2016. “
Of the documented 887 requests for identifiable data that have been through the DMAP request process in March 2012 – December 2016, only 29 have been for aggregated data, according to analysis by the NGO defenddigitalme. There were 15 rejected applications between March 2012 and September 2016, including from the Ministry of Defence to target its messaging for recruitment marketing. Approved uses include identifying and sensitive data released to Fleet Street papers, “to pick interesting cases/groups of students," and about 60% of applications approved for identifying and sensitive, pupil level data, were from think tanks, charities, and commercial companies.
The Telegraph newspaper was granted identifying and sensitive data in 2013, for all pupils in the KS2, KS4 and KS5 cohorts for the years 2008-2012.
Academic uses of school census data make up about 40% of the requests for identifying, pupil level data, processed through and approved by the DMAP process.. The raw data are sent to the requestor's own location. There is no charge made for fulfilling requests. "DfE does not charge for data, nor does DfE charge for the processing and delivery of extracts to customers."
There is however no transparency of the volume of how many children’s data have been given away in approved uses either, because,
“the Department does not maintain records of the number of children included in historic data extracts.”
Public interest research use of pupil level data through other routes of access to the data, include projects linking individual data together with other education and employment data from citizens' interactions with other government departments and public services. For example, is made up of information from the National Pupil Database, the Individualised Learner Record, the Higher Education Statistics Agency, Her Majesty’s Revenue and Customs data, The National Benefit Database, the Labour Market System and Juvos, the unemployment research database. Further work by DfE compares self-reported salaries from the 2008/09 DLHE survey with earnings data from the LEO dataset coming directly from HMRC tax records.
In June 2018, the UK Parliament gave powers to the Office for Students through the Higher Education and Research Act 2017 regulations 2018 No.607 to distribute personal data to thirteen third party organisations. In 2019 The Higher Education and Research Act 2017 Regulations 2019 will expand which data that may be, and will include the entirety of the National Pupil Database and Alternative Provision data. In debate, Shadow Secretary for Higher Education Gordon Marsden MP, asked the government whether, "it the intention of the new regulations that through the new data powers they give OfS to receive data in regulations 28 and 32 they can also enable the distribution by OfS of population-wide personal data?" The data in question, "includes the personal, confidential data of every pupil from state education since 1996, past, present and future and in perpetuity—over 25 million people, and growing every year—distribution to its own third-party prescribed persons, including potentially Pearson Education Ltd, among other commercial parties, for such wide-ranging company purposes, through the powers of last year’s regulations, which set out who the OfS could give data to, and for purposes defined only by that ."
Since legislation changed over time to permit new uses and access to personal data by new third parties, over 15 million people whose data was already in the National Pupil Database and who had already left school pre-2012, have not been informed how their personal data may be used, for what purposes, and by whom, such new Regulations demonstrates.

Controversial collections and distribution of identifying personal data from the database

In July 2015, the Department for Education and Home Office Border Removals Team agreed a Memorandum of Understanding to share pupil data including names, date of birth, gender, home address and school address for up to 1,500 children a month, from the last 5 years of their records, for various purposes of direct interventions.
This policy became public knowledge through the expansion of the school census in October 2016 which added to the collection.
In October 2017, the Department for Education confirmed in that, information obtained from the National Pupil Database was used to contact families to "regularise their stay or remove them" and confirmed in January 2019 that this policy continues.
An expansion of the Alternative Provision census starting in January 2018, added further sensitive data to the National Pupil Database including pregnancy, physical and mental health, and a code for young offender, as reason for transfer out of mainstream education. The indicates that the age group has been lowered. "Within the AP census, pupils should be aged between 2 and 18 - those pupils born between 01/09/1998 and 31/12/2015."
Campaigners and charities that the changes would lead to sensitive details being collected without the knowledge of parents and pupils, in breach of data protection law and raised concerns that "there are not enough safeguards to ensure that sensitive data does not end up being passed on to third parties and damaging the privacy of those it covers."

New data access model for data re-use by third parties

The sharing of identifying pupils’ personal data with third parties was in May 2018 for three months.  The Department for Education halted the distribution of personal information about  school children in England, to restart it aligned with a Five Safes model, according to the Office for Statistics Regulation recommendations. Although this was intended as an improvement towards safer pupil data, in spring 2019 data , more than six months after the safer model was introduced.
The new infrastructure was part of a set of recommendations made by the UK Statistics Authority in 2018, which included that the Department carry out a Data Protection Impact Assessment. A summary was published in May 2019. It included recognition of the risk that people, “may not be aware that their personal data may be shared with other organisations.”
In May 2019, the Department for Education released the first . It revealed that , for students from Higher Education.
The in interim investigation findings in autumn 2019, that, "This investigation has demonstrated that many parents and pupils are either entirely unaware of the school census and the inclusion of that information in the NPD, or are not aware of the nuances within the data collection, such as which data is compulsory and which is optional. This has raised concerns about the adequacy DfE’s privacy notices and their accountability for the provision of such information to individuals regarding the processing of personal data for which they are ultimately data controllers.”

Data request process

Access is granted through an applications process to the Department for Education Education Division and internal Data Management Advisory Panel, and is subject to requesters complying with terms and conditions imposed under contractual licence arrangements. The DMAP was first published in July 2016 by the Department for Education, but became obsolete after a 2018 panel reconfiguration.
The Department for Education application procedures for handling requests for data from the National Pupil Database, from March 2012, enabled interested parties to request extracts of data from the National Pupil Database using forms available on . for researchers and third-party organisations who have received DfE approval for applications for data extracts are completed before users are sent the password protected data.
The sensitive and identifying items that require DMAP approval include name, date of birth, postcode, candidate numbers, Pupil Matching Reference, detailed types of disability, indicators of adoption from care, reasons for exclusions.
There is no ethics committee review for the release of identifying or sensitive data directly from the National Pupil Database by the Data Management Advisory Panel or Education Division.
There was no privacy impact assessment of the National Pupil Database for over twenty years, until 2019.

Legal basis for the release of pupil level data by the Department for Education

Some of the history behind its collection, use and changes to legislation are outlined in a presentation given at an Open Data Institute ODI Friday lunchtime talk:
The release of data permitting pupil level release of individuals’ identifiable data to third parties from the National Pupil Database was updated by 2013 changes to legislation. , and , together with the 2009 Prescribed Persons Act, were amended in and , to allow the release of individual children’s data to third parties. Which data items are involved is based on the 2006 Act around the register data a school must hold, which has subsequently had many amendments.
The Data Protection Act 1998, in particular, Principle 1, sets out a fairness obligation which cannot be set aside merely because of the presence of a legal basis such as a Statutory duty. On October 1, 2015, this latter point was again made explicit for public bodies in the judgment of the Court of Justice of the European Union in the in which it ruled that “ must be interpreted as precluding national measures…which allow a public administrative body of a Member State to transfer personal data to another public administrative body and their subsequent processing, without the data subjects having been informed of that transfer or processing,” i.e. individuals must be informed when public bodies share personal data and why.
For sensitive data an additional condition from must also be met to justify a legal basis for disclosure. These conditions are a high bar, for example, in the interests of justice.
The Data Protection Act 1998 gives research exemptions for the purposes of statistical and historic research purposes, most significantly on the principles of indefinite retention and data minimisation, as well as Subject Access rights, for as long as data are processed for the legitimate interests of the Data Controller. To qualify for the research exemption, the research must be able to comply with the following ‘relevant conditions’:
that the data are not processed to support measures or decisions with respect to particular individuals, and
that the data are not processed in such a way that substantial damage or substantial distress is, or is likely to be, caused to any data subject.
Campaigners from the children's privacy NGO defenddigitalme, have questioned whether this legal basis is met for some releases between 2012 and 2017 from the National Pupil Database and whether new uses put the research status of the National Pupil Database at risk.
As observed in 2014 by independent experts, "the central concern is that parents and pupils themselves are not sufficiently aware of the way the data is being shared with third parties." "There appears to have been no concerted effort to bring the consultation or the NPD initiative to the attention of parents or pupils."
In November 2019, : "the issues that the DfE experienced with the collection of the nationality data in terms of parent and pupil awareness of the optional nature of the collection of that data has highlighted concerns regarding compliance with articles 12, 13 and 14 of the GDPR. Our view is that the DfE is failing to comply fully with the GDPR in respect of these articles.
The investigation has demonstrated that many parents and pupils are either entirely unaware of the school census and the inclusion of that information in the NPD, or are not aware of the nuances within the data collection, such as which data is compulsory and which is optional. This has raised concerns about the adequacy DfE’s privacy notices and their accountability for the provision of such information to individuals regarding the processing of personal data for which they are ultimately data controllers.