Pilot write up

Consisted of one mainstream school teacher, three from a specialist provision and a trainee teacher to gather a broad spectrum of feedback.

Why do you do a pilot study? You do it in order to make sure that your methodology will generate the desired results. It is a reliability and validity check for your research method. 

The pilot survey was based upon a combination of questions regarding the testing of subject knowledge (Ludi, Patel, other) as well as testing the knowledge and classroom experience of teachers (Shinohara, swiss, indian).

It was conducted with...to get a cross representation of those taking the survey.

Think-aloud - Then you may sit with these participants, and make sure that they understood the survey questions correctly. Then, you will make sure that the way they answer the survey questions is exactly how the questions are intended to be answered.

As well as giving me invaluable practice in using the online survey tool, the pilot highlighted several areas where changes needed to be made, including the content of the questionnaire, its format, the distribution process and obtaining informed consent. 

The most significant change was adding extra questions in terms of being able to check that the representative sampling was varied - questions about geographical location and educational setting.

The pilot also introduced me to the language and teachers used, for example they relied on online resources and repositories rather than textbooks, referred to learners as pupils and were not fully familiar with what the term digital accessibility awareness meant, also wording on introductory information needed to be more plain English for those not familiar with the subject.

Refer to learning more about the language and culture of the setting.

The detailed changes made as a result of this pilot are highlighted in a table in Appendix C.

QUOTES TO USE

The first step to developing a questionnaire is to perform a literature review. There are two primary purposes for the literature review: (1) to clearly define the construct and (2) to determine if measures of the construct (or related constructs) already exist. A review of the literature helps to ensure the construct definition aligns with related theory and research in the field, while at the same time helping the researcher identify survey scales or items that could be used or adapted for the current purpose. (Artino et al, 2014)

However, the researcher unavoidably brings his/her perspectives and biases to this definition, and the language used in the literature may be technical and difficult to understand. Thus, other perspectives are needed. Most importantly, how does the target population (the patient from the previous example) conceptualize and understand the construct? (Artino et al, 2014) - the creation of/the teaching of - assumptions that AT was Immersive reader.

THINK ALOUD - The think-aloud technique requires respondents to verbalize every thought that they have while answering each item. Here, the interviewer simply supports this activity by encouraging the respondent to keep talking and to record what is said for later analysis (Artino et al, 2014)

In addition, the necessity for respondents to understand each item in the way it was intended by the survey creator is integrally related to the validity of the survey and the inferences that can be made with the resulting data (Artino et al, 2014).

The next challenge is to write a set of clear, unambiguous items using the vocabulary of the target population (Artino et al, 2014)

Wilson and McLean (1994) suggest researchers need to clarify the primary objective of the questionnaire and the kinds of data relevant to the concepts and constructs of the research (Cohen, Manion and Morrison, 2000).

Questionnaires or surveys are mainly ‘one offs’ in that they are often not repeated so they need to be an accurate tool and fit for purpose (Denscombe, 1998, p.155).

In most cases researchers don’t get a second chance to follow up issues that they might have missed in a questionnaire (Denscombe, 1998, p.161).

From an ethical point of view, the researcher needs to provide sufficient background information about the research and the questionnaire. It is no good assuming respondents know how to answer the questions, because mistakes can invalidate a whole questionnaire (Denscombe, 1998, p.159).

There is a need to pilot questionnaires to refine their content, wording and length to ensure they are capturing the right responses, but also enable participation by not being to cumbersome (Cohen, Manion and Morrison, 2000).

Sampling in questionnaires needs to consider reliability and validity. An under-represented skewed sample is too small and can distort the data for generalisation and prohibit a valid analysis (Morrison, 1993; Cohen, Manion and Morrison, 2000).

Ideally, the questionnaire should be piloted on a smaller sample of intended respondents, but with a sample size sufficient to perform systematic appraisal of its performance (Rattray and Jones, 2007).

Less structured questions, such as word-based or open ended questions can capture specific or rich personal data (Cohen, Manion and Morrison, 2000). 

A completely closed questionnaire does not always offer the relevant or exhaustive pre-set categories of response or offer participants to add remarks or explanations to their responses (Cohen, Manion and Morrison, 2000).

- These proved useful in gaining some early insight into how teachers think.

The ordering of questions is important, because early questions can set the tone or mindset of respondents (Cohen, Manion and Morrison, 2000, p.257)

If a questionnaire covers several topics it is good practice to introduce the purposes of each topic for the respondent, so they can develop more involvement and engagement with it and provide clear instructions how to answer questions, for example choose one of the following answers (Cohen, Manion and Morrison, 2000, p.258).

Piloting a questionnaire helps to increase its reliability, validity and practicality (Cohen, Manion and Morrison, 2000; Openheim, 1992; Wilson and McLean, 1994). It serves to check the clarity of questions, gain feedback on the response categories for closed questions, check the time it takes to complete it, identify redundant questions and offers an opportunity to try out a system of analysis (Cohen, Manion and Morrison, 2000).

COVER SHEET

The purpose of a cover sheet is to indicate the aims of the research, to assure respondents of confidentiality and to encourage them to respond (Cohen, Manion and Morrison, 2000).

A major advantage is that questionnaires can be emailed to respondents, saving time in comparison to interviews (Teddlie and Tashakkori, 2009, p.232). - However what is said in the cover sheet and email is just as important to promote buy-in.

Some of the most interesting results can come directly from quotes generated by open ended questions (Teddlie and Tashakkori, 2009).