PCK bits Mixed methods, triangulation and Questionnaires

 (Denscombe, 1998)

Triangulation involves the practice of using things from more than one perspective and using different methods or sources of data. The principle behind it is to get a better understanding of the thing that is being investigated if it is viewed from different positions (p.134) Denscombe, 1998.

The most popular or common form of triangulation adopted by social researchers is to use findings from on method to contrast or correlate findings with another (Denscombe, 1998, p.135).

The benefits of triangulation are that findings can be corroborated or questioned by comparing data and findings can complement to add something new or different to add to what is already known by using another (Denscombe, 1998)

These can lead to improved accuracy, authenticity or validation, offer a fuller picture to enhance the completeness of findings, or help develop a line of enquiry to complement or build upon findings from one source to another (Denscombe, 1998).

Questionnaires can be designed to collect information which can be used subsequently as data for analysis (Denscombe, 1998).

Questionnaires or surveys then to be ‘one offs’ in that they are often not repeated so they need to be an accurate tool and fit for purpose (Denscombe, 1998, p.155).

Questionnaires may need to gain permission from those in authority in the form of informed consent (Denscombe, 1998).

From an ethical point of view, the researcher needs to provide sufficient background information about the research and the questionnaire. It is no good assuming respondents know how to answer the questions, because mistakes can invalidate a whole questionnaire (Denscombe, 1998, p.159).

In the right tool, responses on a web based questionnaire can be easily converted into a spreadsheet or database for analysis, which offers the benefit of speed and accuracy in terms of collating results (Denscombe, 1998, p161).

In most cases researchers don’t get a second chance to follow up issues that they might have missed in a questionnaire (Denscombe, 1998, p.161).

The advantage of open questions is that the information gathered is likely to reflect the full richness and complexity of view held by the respondent that can be expressed and articulated in their own words (Denscombe, 1998, 166).

Closed questions provides data that is uniform and in a format that lends itself to being easily coded, quantified and compared (Denscombe, 1998, p.166).

Questionnaires are economical in that they can supply a considerable amount of research data, are easier to arrange and offer a relatively low cost, low effort method of data collection in terms of money and time (Denscombe, 1998, p. 169).

Pre-coded questionnaires can bias findings towards the researchers ways of seeing things because they impose a structure of questions and answer options assumed by the researcher (Denscombe, 1998,p 170).

Denscombe (1998, p.172) offers a checklist for the production of a research questionnaire (ADD SCREEN GRAB IN APPENDICES DOC).

ME: Using the questionnaire as a test of knowledge rather than specifically opinions or attitudes. Open questions used for questions relating to prior experiences or perceptions.

————-

Questionnaires used alongside secondary sources such as interviews help to develop a more complete picture when answering a complex research question and in some cases help for comparisons or to make predictions, especially in the instances of preparing for pedagogic roles and to find out what sorts of pedagogical knowledge is held and what correlates with that knowledge (Nind, Curtin and Hall, 2016, p.82).

Within PCK Shulman (1986) drew attention to questions of how teachers formulate explanations, decide on teaching content for the subject, how to represent learning and how to interact with the understandings or misconceptions that learners might have.

Shulman’s model specifically highlights the understudied form of teacher’s knowledge of how content can be transformed for teaching into a form that is comprehensible to learners (Shulman, 1986; 1987).

The concept of PCK has taken hold amongst research on teacher education (Kind, 2009).

Kind, V. (2009) Pedagogical content knowledge in science education: perspectives and potential for progress. Studies in Science Education, 45 (2) pages 169-202

When it comes to the craft of teaching, it is difficult knowledge to represent as it is tacit and hard to get at (Lewthwaite and Nind???; Traianou, 2006).

Traianou, A. (2006) Understanding teacher expertise in primary science: a sociological approach, Research papers in education, 21 (2) pages 63-78

It is not something easily observed but can be made visible through a combination of observing teachers situated in context, through interviews and reflections (Nind, Kilburn and Wiles, 2015), and something that the process of lesson study research offers (Nind, Curtin and Hall, 2016).

Nind, M., Kilburn, D and Wiles, R (2015) Using video and dialogue to generate pedagogic knowledge: teachers, learners and researchers reflecting together on the pedagogy of social research methods. International journal of social research methodology.

Gaining access to the educational thoughts and decision making of others towards their practice is intrinsic to the endeavour of many social scientists (Calderhead, 1981, p.211) but in pedagogical research this takes on a very practical importance to help understand the thinking and decision making behind it (Nind, Curtin and Hall, 2016. p.161).

Calderhead, J (1981) Stimulated recall: a method for research on teaching. British journal of educational psychology, 51, pages 211-17

Nind, Curtin and Hall (2016, p.267) recommend a set of research techniques that offer ideal insights for studying teacher’s thoughts, processes, evaluations, problem solving and decision making for PCK. These mainly relate to think aloud methods to capture thinking in the moment, these can access introspective higher order mental processes for planning teaching (pedagogy as planned) and lesson study to observe teaching in action (pedagogy as enacted) and the dialogue of reflection to uncover evaluative thinking after the event (pedagogy as experienced).

One advantage of questionnaires being anonymous is that they can encourage more honesty (Cohen, Manion and Morrison, 2000)

There is a need to pilot questionnaires to refine their content, wording and length to ensure they are capturing the right responses, but also enable participation by not being to cumbersome (Cohen, Manion and Morrison, 2000).

Sampling in questionnaires needs to consider reliability and validity. An under-represented skewed sample is too small and can distort the data for generalisation and prohibit a valid analysis (Morrison, 1993; Cohen, Manion and Morrison, 2000).

Morrison, K. R. B (1993) Planning and accomplishing school centred evaluation. Norfolk. Peter Francis Publishers

Questionnaires are widely used and useful instruments for collecting information and can provide structured or numerical data that can be comparatively straightforward to analyse compared to that of other research methods such as interviews (Wilson and McLean, 1994; Cohen, Manion and Morrison, 2000).

Wilson, N. and McLean, S. (1994) Questionnaire Design: a practical introduction. Newtown Abbey, Co Antrim. University of Ulster Press

Ethically, respondents cannot be coerced into completing a questionnaire and their participation would benefit from the process of informed consent as well as the right to withdraw from completing it (Cohen, Manion and Morrison, 2000).

Wilson and McLean (1994) suggest researchers need to clarify the primary objective of the questionnaire and the kinds of data relevant to the concepts and constructs of the research (Cohen, Manion and Morrison, 2000).

Highly structured closed questions are useful to generate frequencies of response to enable statistical analysis or comparison (Oppenheim, 1992).

Oppenheim, A. N (1992) Questionnaire design, interviewing and attitude measurement. London: Pinter publishers Ltd

Less structured questions, such as word-based or open ended questions can capture specific or rich personal data (Cohen, Manion and Morrison, 2000).

A completely closed questionnaire does not always offer the relevant or exhaustive pre-set categories of response or offer participants to add remarks or explanations to their responses (Cohen, Manion and Morrison, 2000).

Multiple choices questionnaires offer a range of likely choices relevant to a given statement or question, and yes or no answers require a decision from respondents (Cohen, Manion and Morrison, 2000).

Open ended question responses might contain the ‘gems’ of information that otherwise might not have been caught using closed questions and put the response more firmly in the hands of respondents to craft. Open questions contain the hallmark of qualitative data that can offer an aspect of authenticity, honesty, richness and depth to the questionnaire (Cohen, Manion and Morrison, 2000).

Completing a fully open ended questionnaire is very time consuming for participants and makes it more difficult for the researcher to make comparisons and compile clear correlations (Cohen, Manion and Morrison, 2000).

The ordering of questions is important, because early questions can set the tone or mindset of respondents (Cohen, Manion and Morrison, 2000, p.257)

Completing a questionnaire in some circumstances can be seen as a learning process in which respondents can become more at home with the task as they proceed and build up confidence and motivation (Cohen, Manion and Morrison, 2000, p.257).

If a questionnaire covers several topics it is good practice to introduce the purposes of each topic for the respondent, so they can develop more involvement and engagement with it and provide clear instructions how to answer questions, for example choose one of the following answers (Cohen, Manion and Morrison, 2000, p.258).

The purpose of a cover sheet is to indicate the aims of the research, to assure respondents of confidentiality and to encourage them to respond (Cohen, Manion and Morrison, 2000).

Piloting a questionnaire helps to increase its reliability, validity and practicality (Cohen, Manion and Morrison, 2000; Openheim, 1992; Wilson and McLean, 1994). It serves to check the clarity of questions, gain feedback on the response categories for closed questions, check the time it takes to complete it, identify redundant questions and offers an opportunity to try out a system of analysis (Cohen, Manion and Morrison, 2000).

Teddlie and Tashakkori, 2009

Questionnaires offer a very efficient, cost effective and quick way to gather data (Teddlie and Tashakkori, 2009, p.232).

A major advantage is that questionnaires can be emailed to respondents, saving time in comparison to interviews (Teddlie and Tashakkori, 2009, p.232).

However, researchers will instead need to spend time using follow up methods such as reminders and re-emailing non-respondents if this proves difficult (Teddlie and Tashakkori, 2009).

Questionnaires can offer similar similarities to interviews, in that they can seek to determine attitudes, feelings, experiences and beliefs and can Simultaneously collect a mix of quantitative and qualitative data (Teddlie and Tashakkori, 2009).

Mixed methods questionnaires can include predetermined responses of closed questions and open ended questions to illuminate some aspects of the phenomenon under study (Teddlie and Tashakkori, 2009, p.235).

Using strands of quantitative and qualitative data from both open and closed questions can mean the analysis of one strand can inform the analysis of another (Teddlie and Tashakkori, 2009, p.235).

In this research the strands are teachers prior knowledge and experience as well as an assessment of subject matter knowledge.

Some of the most interesting results can come directly from quotes generated by open ended questions (Teddlie and Tashakkori, 2009).

Tests are techniques to assess knowledge or ability often by using multiple choice questions. When researchers use tests as a data collection strategy it often is used as a dependent variable and can generate quick quantifiable numerical data (Teddlie and Tashakkori, 2009).

Questionnaires can offer a quick turnaround of data (Teddlie and Tashakkori, 2009)

In sequential forms of exploratory mixed methods research design, often the first stage of the data collection may not typically use the same participants as later in the study. The purpose of the quantitative stage is to generate some results to sample a population to offer key predictors, indicators or distinguish key characteristics (Creswell and Plano Clark, 2007).

Another paper:

Rattray, J. and Jones, M.C., 2007. Essential elements of questionnaire design and development. Journal of clinical nursing, 16(2), pp.234-243.

https://onlinelibrary.wiley.com/doi/full/10.1111/j.1365-2702.2006.01573.x

Closed questions which are commonly used may restrict the depth of participant response (Rattray and Jones, 2007)

To allow respondents to expand upon answers and provide more in-depth responses, free text response or open questions may be included. Respondents may welcome this opportunity. However, whilst this approach can provide the interviewer with rich data, such material can be difficult to analyse and interpret (Rattray and Jones, 2007).

Ideally, the questionnaire should be piloted on a smaller sample of intended respondents, but with a sample size sufficient to perform systematic appraisal of its performance (Rattray and Jones, 2007).

—-

Artino Jr, A.R., La Rochelle, J.S., Dezee, K.J. and Gehlbach, H., 2014. Developing questionnaires for educational research: AMEE Guide No. 87. Medical teacher, 36(6), pp.463-474

https://www.tandfonline.com/doi/full/10.3109/0142159X.2014.889814

(Artino et al, 2014)

Before creating a questionnaire, however, it is imperative to first decide if a survey is the best method to address the research question or construct of interest. A construct is the model, idea or theory that the researcher is attempting to assess (Artino et al, 2014).

In education, many constructs of interest are not directly observable – student satisfaction with a new curriculum, patients’ ratings of their physical discomfort, etc. Because documenting these phenomena requires measuring people’s perceptions, questionnaires are often the most pragmatic approach to assessing these constructs (Artino et al, 2014)

The first step to developing a questionnaire is to perform a literature review. There are two primary purposes for the literature review: (1) to clearly define the construct and (2) to determine if measures of the construct (or related constructs) already exist. A review of the literature helps to ensure the construct definition aligns with related theory and research in the field, while at the same time helping the researcher identify survey scales or items that could be used or adapted for the current purpose. (Artino et al, 2014)

Formulating a clear definition of the construct is an indispensable first step in any validity study (Cook & Beckman Citation2006). A good definition will clarify how the construct is positioned within the existing literature (Artino et al, 2014)

Furthermore, it is important to acknowledge that reliability and validity are not properties of the survey instrument, per se, but of the survey’s scores and their interpretations (Artino et al, 2014)

However, the researcher unavoidably brings his/her perspectives and biases to this definition, and the language used in the literature may be technical and difficult to understand. Thus, other perspectives are needed. Most importantly, how does the target population (the patient from the previous example) conceptualize and understand the construct? (Artino et al, 2014)

The goal of this step is to write survey items that adequately represent the construct of interest in a language that respondents can easily understand. One important design consideration is the number of items needed to adequately assess the construct. There is no easy answer to this question. The ideal number of items depends on several factors, including the complexity of the construct and the level at which one intends to assess it (i.e. the grain size) (Artino et al, 2014)

The next challenge is to write a set of clear, unambiguous items using the vocabulary of the target population (Artino et al, 2014)

The think-aloud technique requires respondents to verbalize every thought that they have while answering each item. Here, the interviewer simply supports this activity by encouraging the respondent to keep talking and to record what is said for later analysis (Artino et al, 2014)

In addition, the necessity for respondents to understand each item in the way it was intended by the survey creator is integrally related to the validity of the survey and the inferences that can be made with the resulting data (Artino et al, 2014).