NSHG (17:2) 10

Minutes of the National Statistics Harmonisation Group (NSHG) Meeting 11th July 2017

1100 to 1330 hrs ONS, 1 Drummond Gate, London, SW1V 2QQ

Attendees: Apologies: Steve Ellerd-Elliott (Chair) MoJ Andrew Phelps ONS Alison Reynolds DCMS Becky Tinsley ONS Becki Aquilina ONS Bonang Lewis ONS Charlie Wroth-Smith ONS Jill Pobjoy ONS Darren Barnes (video – Newport) ONS Darren Stillwell DfT Dawn Camus BEIS Emma Dickinson ONS Elishama Tizora (video Newport) ONS Fiona Aitchison ONS Emily Knipe ONS Jacob Wilcock DFID Emma Morgan (audio – NISRA) NISRA Jane Simcock DEFRA Helena Rosiecka ONS Jonathan Swan ONS Ian Sidney ONS Karen Hurrell EHRC Ian O’Sullivan (video Newport) ONS Katrina Morrison ONS Jay Sheppard (video Newport) ONS Lisa Walters Welsh Gov Jill Morton (audio – Scotland) NRS Martin Ralphs ONS Joy Preece (video Newport) ONS Matthew Minifie ONS Julian Austin DoE Michelle Monkman ONS Kim Bradford-Smith DFID Paul Benson HSE Marina Wright (video Newport) ONS Rafael Mastrangelo HMRC Matthew Steel (video Newport) ONS Rebecca Hulme Ofsted Nick Forbes (video Newport) ONS Sabena Khan BEIS Penni McClure ONS Scott Edgar DCLG Pete Betts ONS ONS Rachel Rees (audio – Wales) Welsh Gov Roger Smith (video – Newport) ONS Samaa Elsandabessee ONS Steve Webster NHS Digital Thomas Sims DCLG Mark Herniman (Secretariat) ONS 1.0 Welcome and Introduction – Steve Ellerd-Elliott

The Chair welcomed everyone to the meeting and thanked those who have left the NSHG for their contribution. New members were also introduced to the group.

2.0 Minutes and Actions from the previous meeting, NSHG (17:1) 04 – Steve Ellerd-Elliott

There were no comments from the previous minutes. All the actions were complete except for two and these will continue to be progressed outside of the meeting.

1

NSHG (17:2) 10 3.0 Update on Harmonisation Work

a. Verbal Update, NSH SG / SPSC Meeting Report – Steve Ellerd-Elliott The main points were:

• NSH SG - the last meeting was held on 28th June 2017 - the revised Harmonised Principle for Personal Well-being was formally approved - Harmonised business definitions for Total Turnover, Number of Employees and

Employment Hours Worked were formally approved - The communication plan was discussed including what projects the Harmonisation

Team should be involved in - the next meeting will take place 28th September

• SPSC - the next SPSC meeting will take place on 18th July - the Chair and Tim White will be looking at the implications of the refresh of the

National Statistics Code of Practice - NSHG members were encouraged to respond to the code of practice consultation

especially in relation to harmonisation (https://gss.civilservice.gov.uk/blog/2017/07/uk-statistics-authority-launches-consultation-refreshed-code-practice/)

- a review is currently taking place looking at the GSS Policy Store and how this is being used

b. NSHG (17:2) 05, Review of the NSHG Workplan – Mark Herniman

The following additional points were discussed: • the ONS Ethnic Group Forum (EGF) is looking at the transformation of ONS social

surveys and the 2021 Census with the aim of ensuring definitions, questions and outputs from both programmes are harmonised

• the Harmonisation Team have been providing guidance to the Race Disparity Unit as there are some government Departments who are not using the Ethnicity Harmonised Principle; this has caused issues with comparing data and highlights the importance of harmonisation

• the GSS Methodology Symposium takes place in 12th July; Becki and Ian (Harmonisation Team) will be presenting on social and business harmonisation

c. NSHG (17:2) 06, Communications Plan – Becki Aquilina Becki presented the Communication Plan and there were no comments. d. Verbal Update, Business Harmonisation – Ian Sidney

Ian presented his update and the following additional points were discussed: • the Business Harmonisation Task and Finish Group (BHTFG) are looking for new

members across the GSS who produce or use business related variables • Steve Jobling (NHS Digital) has left the BHTFG and has been replaced by Robyn

Wilson ACTION 1 – NSHG members to make Ian Sidney (ian.sidney@ons.gov.uk) aware of potential members for the BHTFG across the GSS known to produce or use business related variables ACTION 2 – Steve Webster to provide Ian Sidney with contact details for Robyn Wilson as the replacement representative on the BHTFG e. NSHG (17:1) 02, Update on Good Practice Team & Harmonisation – Martin Ralphs

2

NSHG (17:2) 10 Martin was unable to attend the meeting but the update was presented. The following

additional points were discussed: • members were encouraged to use the ‘Slack’ app which is an instant messenger type

collaboration tool • the Harmonisation Team is currently involved in an internal transformation which has

resulted in the Good Practice Team, Harmonisation and the Quality Centre all sitting within one division (Best Practice and Impact)

4.0 Verbal Update, Data Collection Transformation Programme (DCTP) Online Survey Question

Harmonisation – Ian O’Sullivan Ian explained that DCTP is working on transforming social survey questions. It was identified by the GSS Online Surveys Group that Harmonisation is important and further collaboration is required between DCTP and the Census to encourage increased Harmonisation. The following points were discussed: • harmonised principles should cater for different modes of collection and should provide at least

a top level of comparison for different data sources • it is recognised there may need to be ‘trade-off’ between the level of detail required for

interviewer led questions and online questions • the focus should be on harmonisation, not standardisation, and how the differences are

considered when developing online questions • it was noted that all revisions to existing harmonised principles or new principles must follow the

Harmonisation Process which includes conducting research, consulting throughout with stakeholders and achieving agreement via the NSHG & harmonisation steering group

• there is potential for the redesign of questions for other modes as online questions are developed

• there is potential for some tension caused by the process of getting overall agreement when different governing processes (LFS, Census, etc) are involved

• It was noted that demographic and ethnicity informal groups including NSHG topic leads and representatives from DCTP and the Census have been formed to share research, developments and to discuss harmonisation issues; these are working well and could be used as a model for other topics

• such collaboration provides topic leads with an understanding of the differences and similarities in questions and will better inform them to make changes to the harmonised principle, whether the question has derived from Labour Force Survey (LFS), Family Resources Survey (FRS), Census or other surveys

• DCTP offered to present their potential online question findings at the next NSHG meeting • members were reminded harmonisation ensures there is comparability between different

sources of data and that changes should not be made in isolation • it was suggested a meeting between Ian O’Sullivan and Charlie Wroth-Smith would be useful to

discuss the harmonisation process for online questions ACTION 3 – Charlie Wroth-Smith to set up a meeting with Ian O’Sullivan to discuss the harmonisation process for online questions 5.0 Presentation, Business Survey Question Bank – Jay Sheppard

Jay provided a presentation and the following additional points were discussed: • the project is in its early stages and was set up to enable all ONS’s business survey

questions to be in one place so they may be accessed electronically • good progress has been made and the survey question bank currently includes around 90

surveys with up to 450 forms 3

NSHG (17:2) 10

• the data is ‘scraped’ from business survey pdfs • this work may lead on to be part of the ONS ‘authoring tool’ for electronic surveys, to enable

stored questions to be re-used • there is potential this could become a GSS tool in the future • members were informed of the ONS Data Science Campus • the Ministry of Justice have undertaken a similar project to produce a ‘PQ Tool’ where PQs

can be viewed via a ‘similarity index’ to give a selection by topic • it was suggested it may be useful to have a combined social and business question bank

ACTION 4 – Secretariat to circulate ONS Data Science Campus link to members 6.0 NSHG (17:2) 08, Census Topic Update (inc post 2017 test) – Helena Rosiecka

Helena presented her paper and the following additional points were discussed: • sexual identity – no testing was conducted on the ‘prefer not to say’ option on the

harmonised principle • sexual identity – if the ‘other’ category and the text box is included, how responses are

processed and output needs to be considered • the results of question testing for the 2017 Census Test will be published this December

(high level results) • Scotland are looking at the same issues faced by England and Wales and there will be

differences in some of the questions due to country specific requirements and user needs (for example religion and language)

• Scotland are working with the other nations on sexual orientation and gender identity questions and are considering a question on transgender

• a draft test plan is being finalised and includes testing up until 2018 by which time questions will be proposed

• all nations are working closely on topic groups and harmonisation • NISRA are researching interactions between the sexual identity question and the Northern

Ireland religion question in the autumn which may have an impact on whether Northern Ireland can include a sexual identity question

• a question on frequency of Irish language and Ulster Scots is being tested by NISRA as part of the language questions this summer – this will be an additional question

• the group was reminded of the need to be aware of each other’s work, particularly where harmonisation needs to be considered

7.0 Verbal Update, GSS Data Discovery Programme – Darren Barnes

Darren provided an update, explaining that there are different portals for data and this presents problems for users trying to identify the correct portal to use. The aim of the Data Discovery Project is to research and standardise the way the data is managed and disseminated with a view to making it easier for users. To do this, there needs to be an understanding of respective departments’ internal processes. The research has identified what needs to be done and has been collated into a report for the National Statistician. The main themes identified were; data is hard to find, lack of understanding of how data can be used with other data (because of difference between them), non-consistent formatting, harmonisation issues (because metadata is not used correctly) and where there are differences these are not explained.

4

NSHG (17:2) 10

Going forward, one of the ‘quick wins’ identified was to gather every output in the GSS and try to understand these so commonalities could be identified. The next phase will be an ‘alpha’ build to enable users to access a range of data from across the GSS. The project will provide links to give members access to the list of GSS outputs, some visualisations and a prototype search engine built to research underlying data.

ACTION 5 – Darren Barnes to provide links to the Harmonisation Team for the list of GSS outputs, visualisations and the prototype search engine

Ensuring definitions, concepts, labels and metadata are used correctly is important and the project is looking at bringing data together from across departments by theme. The first theme being looked at is ‘Trade’ and work will begin in September. It is hoped this work will be finished in 6 months, resulting in pilot software to enable focus on a single topic. It is also hoped to come up with recommendations on how to take the project forward and make more scalable across the GSS. The project is keen to work with the NSHG and the Harmonisation Team to provide a steer on how best to take the work forward. Information on any user groups the NSHG may be able to provide would be useful to help progress the work.

ACTION 6 – Members to provide Darren Barnes with contact details for any ‘user Groups’ that may be useful to the Data Discovery Team

By the time of the next NSHG meeting it is hoped to have something more tangible to show regarding the search engine or present on the impacts of non-harmonisation. The following additional points were discussed:

• Data.gov.uk should have included a definitive list to find out about statistics and data available across the government but unfortunately it only really offers ‘signposting’ to other government departments to find the data

• looking to the future, either GOV.UK or data.gov.uk may build the finished product the GSS Data Discovery Project are working on

• in terms of harmonisation, the work being conducted may help to bridge the gap with admin data and overcome issues caused by trying to harmonise at the final stage of the process when producing admin data

• in summary, the project is trying to expose the issues with data, so in the future using the data becomes easier

8.0 NSHG (17:2) 09, NSHG Topic Lead Updates – Topic Group Leads The Chair the topic leads for their reports. Benefits and Tax Credits – Thomas Sims Thomas provided a verbal report explaining there are changes to the welfare system that are not

reflected within the existing Harmonised Principle. The changes have now been made on the Family Resource Survey (FRS) and need to be transferred across to the principle. The affected benefits are the ‘bereavement’ benefits. The Harmonised Team will contact Thomas to effect the changes to the Harmonised Principle.

ACTION 7 – Harmonisation Team to contact Thomas Sims to effect the changes to the ‘bereavement’ benefit Consumer Durables – Joy Preece Joy presented a report and there were no comments or questions.

5

NSHG (17:2) 10 Crime and Anti-Social Behaviour – Fiona Aitchison

Fiona presented a report and there were no comments or questions.

Demographic Information – Emily Knipe Emily presented a report and there were no comments or questions. Economic Activity – Roger Smith Roger provided the following verbal update:

• the LFS drives the Economic Activity through the harmonisation route • dramatic changes will be made to the questions because of the transformation work and

the critical consideration will be in relation to any discontinuity • consultations are taking place on the transformation work • LFS is a European requirement and some of the questions are driven by that and this

causes some conflict, so clarity is being sought • there is a question about self-defined employment status which would have to be asked by

the LFS and SILC – the same question was rejected by the Census for good reasons • a consultation will be circulated to users about what the framework of definitions should be

and the types of aspects in the labour market that should be considered • in the future, there will be no need to collect data on government training schemes • initial work is taking place attempting to match LFS data to see what Admin Data can be

used in the future • the membership of the topic group needs to be reviewed

Education – Julian Austin/Tony Clarke Julian, as a new topic lead was unable to produce a report. There were no comments or questions. EILR – Becki Aquilina (temp) Becki presented a report and there were no comments or questions. Gender Identity – Michelle Monkman

Michelle was unable to attend the meeting but was represented by Samaa Elsandabesee. There were no comments or questions.

Health, Disability and Carers – Steve Webster

Steve presented a report and the following additional points were discussed: • Department for International Development (DFID) have been doing a lot of work around the

sustainable development indicators to ensure there are standards that different countries can sign up to

• internationally, countries are signing up to the Washington Group question on disability status but the UKs position on this is not clear

• ideas are welcome from the group on where to follow this up and which department in the UK is responsible

ACTION 8 – NSHG Members to contact Kim Bradford-Smith if they are aware of any department which has an interest in the Washington Group Housing and Tenure – Scott Edgar Scott was unable to be at the meeting and there were no comments or questions.

Income – Matthew Minifie Matthew had presented a report and there were no comments or questions. 6

NSHG (17:2) 10 Migration, Country of Birth and Citizenship – Penni McClure Penni presented a report and there were no comments or questions.

Pensions – Bonnie Lewis There was nothing to report for this period. Personal Well-being – Matthew Steel There was nothing to report for this period. Social Capital – Katrina Morrison

Katrina presented a report and there were no comments or questions. 10.0 Any Other Business

There were no items of AOB. 11.0 Next Meetings Dates of the next meetings are:

• Wednesday 8th November 2017, 1100 to 1330 hrs at ONS Drummond Gate, Meeting Room 3.

• Tuesday 13th March 2018, 1100 to 1330 hrs at ONS Drummond Gate, Meeting Room 3

ACTION 9 – Members are requested to let the Secretariat know if they have any items for inclusion on the agenda for the next NSHG meeting Annexes:

A. List of Actions B. Presentation Slides – Business Harmonisation Question Bank

7

NSHG (17:2) 10

ANNEX A

ACTIONS FROM THE NSHG MEETING – 11th July 2017

No Para Action Responsible Status 1 3.0 NSHG members to make Ian Sidney

(ian.sidney@ons.gov.uk) aware of potential members for the BHTFG from Government Department known to produce or use business related variables

All Members

2 3.0 Steve Webster to provide Ian Sidney with contact details for Robyn Wilson as the replacement representative on the BHTFG

Steve Webster

3 4.0 Charlie Wroth-Smith to set up a meeting with Ian O’Sullivan to discuss the harmonisation process for online questions

Charlie Wroth-Smith

Complete

4 5.0 Secretariat to circulate ONS Data Science Campus link to members

Secretariat

5 7.0 Darren Barnes to provide links to the Harmonisation Team for the list of GSS outputs, visualisations and the prototype search engine

Darren Barnes

6 7.0 Members to provide Darren Barnes with contact details for any ‘user Groups’ that may be useful to the Data Discovery Team

All Members

7 8.0 Harmonisation Team to contact Thom Sims to effect the changes to the ‘bereavement’ benefit

Harmonisation Team

8 8.0 NSHG Members to contact Kim Bradford-Smith if they are aware of any department which has an interest in the Washington Group

All Members

9 11.0 Members are requested to let the Secretariat know if they have any items for inclusion on the agenda for the next NSHG meeting

All Members

8

ANNEX B

1

Business SurveyBusiness Survey Question Bank

Project Jay SheppardJay Sheppard

Data Analyst Apprentice11/07/2017

ContentProject BriefProject Brief BenefitsInvolvementProgressCurrent StagegPossible Outcomes

Presentation Slides – Business Harmonisation Question Bank

NSHG (17:2) 10

2

Project Brief• Convert all of ONS’ business surveys (including

all form types) to a machine readable format toall form types) to a machine readable format tocreate a shared question library

• Validate ‘scraped’ surveys with survey owners toensure questions are captured correctly

• Once captured the next phase is to begin toperform text analytics to short list similarperform text analytics to short list similarquestions and relevant admin data attributes

Benefits• Enable harmonisation efforts by identifying

question wording redundancy and similaritythrough text analyticsthrough text analytics

• Enable use of admin data to supplement surveydata by identifying likely candidate surveyquestions that admin data attributes couldsupplement through text analysisProvide input into I T development to manage• Provide input into I.T. development to managequestion bank/library

• Be a resource generally to the wider ONS

3

Involvement• Collaborative effort between many departments

within the ONS:H i ti U it– Harmonisation Unit

– Big Data Team– DCTP– Data Science Campus– eQ Team– DST– DST

Phase 1Currently

at this stage for stage oPhase 1.

Questions already scraped

have allowed us to explore Phase 2.

4

Progress - Phase 1• Proof of concept surveys signed off• Currently scraped over half the surveys – 18

h b i t ll lid t dhave been internally validated• Validation of JSON files is an ongoing process

within team to ensure mistakes are identified• Developed a visualisation tool to display JSON

files in a HTML format• Once validation within team is complete we can

use HTML format to validate with owners

Example of Survey in JSON Format

All associated info. to a question is capturedSection Question Sub Question

5

Ph

Phase 2 Created an initial matrix of similarity scores between

each question

Phase 2

Currently in the discussions over the design of a

similarity visualisation tool.

Progress - Phase 2• Created a sample set of 1,500 questions• Began text analytics and produced initial

i il it ith th tisimilarity scores with these questions• Further research into text analytics; stop-words

and stemming• Discussions with relevant teams on use cases

for outcome tool• Initial designs of those outcome tools

6

Possible Outcomes• Discussed three possible use cases:

– Search InputType in search criteria and program returns allType in search criteria and program returns allquestions and associated information that match

– Question similarity finderSelect a question and program returns all questions thatsimilar via a similarity score

– Comparison toolProgram highlights differences in multiple questions

Example of Possible Outcome

7

Thank you for listening.

AnyAny Questions?