Current Visitors By Country
OpenED on Twitter
|Module 5 A1 : What is the question|
Target audience: Business students embarking on a dissertation.
Duration: 30 hours, over 3 weeks
Module Chair: Dr Pascale Hardy, HES-SO, Switzerland
In any given area of study, the research methods used are often closely related to the questions being asked. These questions are themselves influenced by the paradigms that affect what is thought to be good science or good research at any one time, and by policy concerns that affect the way funding bodies distribute the resources for research.
This module gives a flavour of the nature of research in business and management research studies, and offers a set of readings for shared scrutiny. These readings are intended to provide you with an introduction to approaches and techniques that are often found in empirical research in the area.
Through the readings you will explore the relationships between research questions and the methods used to investigate them. You will also consider how research methods are categorised, and learn about key issues relating to epistemology, validity, reliability and objectivity.
A. Forum for general communication: Module 5A1
C. Live sessions at UStream (Dates to be announced at the forum)
The Assignment for this module is to critique the study described in Reading 5 (in Week 3 of the module activities below). Specifically:
You will upload the Assignment to the Assignment Directory.
Assignments will be "peer assessed", so make sure that you present your results in a clear and logic way that is understandable for others! Criteria for assessing assignments is is given in the Peer Assessment section below.
Please note: This module will show as completed in your "Self-Print Certificate" only when you have submitted your Assignment.
This module is divided into 21 activities, taking in total 26 hours to complete. You then need to allow a further 4 hours to complete the assignment. You may also want to allow additional time for IRC chats or other live interactions, depending on how many there are, perhaps around an hour a week.
We start Module 5A1 with two contrasting readings.
The first reading this module is a paper by A. Alexandra Michel from the USA about the management of “cognitive uncertainty” in investment banks. The research is qualitative. It is a long paper, but we think you will find it written in an accessible style that explains the research and associated concepts clearly.
Activity 1: Reading the abstract (10 minutes)
Read the abstract of the paper first. We will return to the role of abstracts later in the module, but for now just make a note of unfamiliar terms. There is no need to investigate these terms on the internet yet: the paper will likely explain them.
Activity 2: Reading the introduction (30 minutes)
Read the introduction: to page 3. Some introductions give an overview of the structure of the paper. This introduction simply outlines the area of the research. Try to identify the research question(s) being addressed by the research.
Activity 3: Finding out about distributed cognition (1 hour)
Activity 4: Identifying the methods (1 hour)
Activity 5: Identifying the findings (90 minutes)
Read the Findings section (pages 11-36). Make notes on the key claims.
Activity 6: Finishing reading the paper (1 hour)
Read the Discussion section (pages 36-45). Again, make notes on the key claims.
a. Could the research questions have been better addressed using different methods?
b. How well did the evidence support the claims? Are alternative explanations possible?
c. How could the claims have been tested more strongly?
d. Could other research questions have been asked of this data?
d. Are there any ethical issues associated with the research?
f. What are the implications (if any) for practice, policy or further research?
Reading 2: Kautonen (2008) Link to Reading 2
The second reading this week is a paper by Teemu Kautonen from Finland comparing younger and older entrepreneurs. It draws on a survey looking at factors that influence the emergence of new enterprises. Although it is a relatively short paper, it is statistical in nature, and might need a little unpacking if you do not have a statistical background. Some help with this unpacking is provided in the activities below.
Activity 8: Reading the abstract, introduction and review of previous research (1 hour)
Now comes the difficult part if you don’t have a statistical background. If you are unfamiliar with inferential statistics, certain parts of this paper will appear baffling. There are mentions of “chi-square” (sometimes written as “χ2”), “analysis of variance” (sometimes referred to as “ANOVA”), “statistical significance”, “the .05 level” and something called “the Tukey HSD test”. The letters “M”, “F”, “df” and “sd” appear to represent different numbers at different times. And, perhaps most bewildering of all to the outsider, “Despite reaching statistical significance, the actual differences in the mean scores of the ‘pull’ motives between the groups were rather small, as indicated by the small effect size (calculated using eta squared) of 0.01 for both variables” (p. 9).
If you are baffled by such things, then at this stage you should (for now) take the authors on trust when they write things like “The data contained four variables relevant to this paper, which are reasonably normally distributed and pass the Levene’s test of homogeneity of variances” (p. 9).
It may be helpful to know that the “mean” of a set of numbers is an average; and that “s.d.” stands for “standard deviation”, which is a measure of how much variation there is in a set of numbers. If you didn’t know these things, there is a PowerPoint file available entitled “Descriptive Statistics” that you should look at this week (Activity 9).
Furthermore, if “chi-square”, “analysis of variance” and “the .05 level” are new to you, then there is a second PowerPoint file entitled “Inferential Statistics” that you will be directed too later on. This is a situation in which consulting Wikipedia might not be a good move, because its entries on statistics (at the time of writing) are a bit technical. The slides are a gentler introduction to inferential statistics.
It’s important to be able to grasp the basic ideas and terminology of statistical hypothesis testing in order to interpret many useful studies in the literature, but it’s not essential that everyone becomes a statistics expert. It is possible to understand the gist of the paper without a fine-grained understanding of all the statistical terms mentioned.
The key idea here is that you can learn something about a set of scores by looking at a sample of those scores and calculating a meaningful number (called a “statistic”) from that sample. Which statistic you choose to calculate depends on what assumptions you can make about the particular situation. Then, you can test a claim about the scores by determining mathematically how likely it is for a theoretical sample picked at random to produce the value you calculated.
So, for example (glossing over some of the complexity), in Kautonen (2008) the researcher wants to test whether there are differences in motivation between three categories of entrepreneurs. So three sets of scores measuring entrepreneurs’ motivation were collected, and a χ2 statistic calculated from those scores. Given various assumptions about the data, the researchers can then determine mathematically how likely it is for χ2 to be that value or a more extreme value by chance. If it’s really not very likely, then the hypothesis that the three sets of scores are about the same looks shaky. In such situations, researchers start to refer to “statistically significant” differences. This doesn’t necessarily mean large differences; it means that the differences are unlikely to have arisen by chance.
Activity 9: Brushing up on statistics (1 hour)
If you do not use statistics very often, read the presentation slides entitled “Descriptive Statistics”. These slides introduce “descriptive statistics”; i.e. how to describe the properties of a set of data. The presentation defines different kinds of scales, such as ordinal and ratio; it explains what a frequency distribution is; it defines the mean, median, variance and standard deviation which are very often used in research papers; and it introduces correlation.
Activity 10: Identifying the methods (30 minutes)
Activity 11: A first read of the rest of the paper (90 minutes)
Congratulations! You have now finished Week 1. Please complete the learning reflection form for this week.
In this second week of Module 5A1, you will finish your reading of Kautonen (2008) by exploring the statistics and reflecting on it critically. You will also read a paper that uses a “mixed methodology approach” and compare the three papers you will have read so far.
Activity 12: Exploring the statistics in Kautonen (2008) (2 hours)
If you’ve had to skip over the statistical bits of the paper, now is the time to find out what they mean, using a second set of presentation slides provided here. You don’t need to know how to calculate chi-square or ANOVA, but you do need to know what they can tell us about a data sample, and the circumstances under which they can be used.
Activity 13: Reflecting on Kautonen (2008) (90 minutes)
Reading 3: Ryan (2009) Link to Reading 3
The third reading of this module is a paper by Maria Ryan from Australia, examining the influence of people’s attachment to their town on shopping behaviours. It uses a combination of qualitative and quantitative research methods: essentially in-depth semi-structured face-to-face interviews on the one hand and a telephone survey on the other.
The paper does not go into great detail on the data, but we think it is helpful in highlighting some of the differences between qualitative and quantitative methods.
Activity 14: Reading the paper (2 hours)
You now have experience of reading a couple of papers, so as before, make notes on...
a. the research question(s)
b. key terms, texts, concepts, alternative viewpoints
c. the empirical setting, the methods of data collection, the types of data collected, the methods of data analysis
d. the key claims
Activity 15: Reflecting on the paper (2 hours)
Comparing Readings 1, 2 and 3
The module so far has provided you with an opportunity to read three examples of research outputs and different approaches. We want you now to compare them.
Activity 16: Comparing the readings (1 hour)
You will be adding further empirical studies to this comparison next week.
Activity 17: Discussing the comparison (30 minutes)
Now in the Module 5A1 forum, discuss whether, in relation to these three papers, the nature of the research questions asked has affected the choice of research methods used.
Categorising research methods
The French philosopher Auguste Comte is usually credited with the invention of social science back in the nineteenth century. When Madge (1953) came to review and categorise ‘The tools of social science’ over a century later, he felt able to define four categories:
We have taken these four categories, unpacked them a little and given examples in Table 2.
Not every research study fits easily into Madge’s typology. For example, in the 1950s researchers were able to compare the behaviour of people in one geographic area where TV was not available with those in another region where it was (Himmelweit et al., 1958). This might be seen as a ‘naturally occurring experiment’. In our classification in Table 2 this could be called a non-interventionist quasi-experiment!
Activity 18: Categorising the studies met so far (1 hour)
From our reading of the literature, almost all “new” research methods can be categorised in terms of Madge’s original four-point typology, or somewhere in Table 2. The “newness” is more in the use of information and communication technology (ICT) to collect and analyse data in new ways, rather than a fundamental change in methodology. It is not like, say, in neuroscience, where functional magnetic resonance imaging actually collects new types of data using totally different measurement techniques (e.g. Haier et al., 2009, who looked at the effects on the brain of playing Tetris).
Each method, whether “new” or “old”, has its advantages and disadvantages. What is “appropriate” will depend upon your and your participants’ access to technology, budget, timing, population to be studied, topic, etc. From a scientific point of view it is more interesting to consider whether, all other things being equal, the different methods might produce different results.
It is actually very difficult to define exactly what is meant by a “new” research method:
Congratulations! You have now finished Week 2. Please complete the learning reflection form for this week.
In this final week of Module 5A1, you will find out a little more about epistemology, validity, reliability and objectivity; and read two further papers (one quantitative, one qualitative).
At the end of Week 2 you compared the characteristics of quantitative methods and of qualitative methods. This is a notoriously hot topic in the social sciences.
Sometimes a sharp epistemological distinction is drawn between quantitative and qualitative research. For example, Guba and Lincoln (1989) argue against the dominance of essentially quantitative “positivist” research in studying social situations. Positivism is characterized as a belief in the application of a particular model of the methods of the natural sciences. In this model the verification of hypotheses, numerical measurement, tests of statistical significance and experiments are typically central. Guba and Lincoln suggest that such methods are too crude and artificial to generate useful insight in the study of human perceptions and interactions. Numerical measurement is bound to focus on only a small number of predetermined variables at a time. Guba and Lincoln urge researchers instead to undertake predominantly qualitative research in natural settings, without preconceived hypotheses.
Conversely, other authors have emphasized “evidence-based research”, in which randomized control trials are seen as the gold standard, as in the medical sciences. Too much social research, such authors argue, is small scale and overly dependent on subjective interpretation. Consequently, it fails to cumulate into a rigorous body of knowledge. Time and time again, businesses have undertaken major changes in their practices without convincing quantitative empirical evidence that they work, that the side effects are understood, and that the practices they replace are inferior. Only through carefully controlled experimental studies, it is said, can research provide managers with robust empirical evidence that can rise above the level of over-elaborate anecdote.
These two “paradigms” of qualitative and quantitative research have been characterized in different ways. There is also much argument in philosophical circles about the validity of the distinction itself (Pring, 2000). In recent years more attention has been paid, not so much to the argument about which paradigm is most appropriate to social science research, but to how the strengths of the various methods employed can be used alongside each other most effectively.
So when Ryan (2009) talks about how “The use of both qualitative and quantitative methods in this research has enhanced the understanding of measurement issues of both place and community attachment constructs” (p. 113), she is attempting to bridge the distinction. The aim is to gain the strengths not only of large-scale quantitative research that uses publicly-verifiable criteria and statistical tests, but also of context-sensitive qualitative research that might provide deeper insight into human behaviour.
Reading 4: Kariv (2009) Link to Reading 4
The fourth reading of this module is a paper by Dafna Kariv from Canada, who used a questionnaire to examine relationships between gender, managerial performance and business success.
Activity 19: Critiquing the paper (2 hours)
Here are some additional questions you may like to consider:
A note on validity
In Activity 19 you were asked to consider “How convinced were you by the research?” This could have been phrased “How valid is the research?” The concept of “validity” is an important part of the discourses of research methods. Campbell and Stanley’s (1963) text has been influential in relation to the idea of validity in experiments and quasi-experiments. They describe validity as the degree to which a study supports its conclusions. They subdivide validity in a number of ways, including “external validity”, “internal validity” and “construct validity”. They also list a variety of “threats” to the various types of validity.
However, other (overlapping) distinctions have been made by other researchers and the term can be used in very different ways depending on the discourse. There are particular differences in relation to discourses of, for example, surveys, experiments, ethnography, action research, grounded theory, and ideologically committed research. It is best to be aware that “validity” is a contested term; but also to be aware of the common distinctions that are made. This awareness will help you to understand published critiques of research methods and enable you to ask critical questions of research studies yourself, such as:
Activity 20: Investigating validity (90 minutes)
Reading 5: Zott & Huy (2007) Link to Reading 5
The final reading of this module is a paper by Christoph Zott and Quy Nguyen Huy, who are based in France. They focus on symbolic actions, which typically convey values in some way, and explore whether entrepreneurs are more likely to acquire resources for new ventures if they perform symbolic actions.
Activity 21: Critiquing the paper (2 hours)
Here are some additional questions you may like to consider:
A note on objectivity
Earlier you were asked to investigate the difference between validity and reliability. You will have found that a test of reliability is whether researchers using the same instruments will come up with the same but not necessarily valid results. However, this begs a further, larger question: To what extent are researchers, in fact, able to obtain knowledge of an external world that is independent of the research activity?
Research processes and researchers’ interpretations are not somehow free of the values, biases and power relations of the social world simply because we wish it so.
Here is what two scholars have said about objectivity.
“Objectivity is one of the most cherished ideals of the educational research community. In fact it is so important that if our work is accused of being subjective, its status as a source of knowledge sinks slowly into the horizon like a setting sun. Yet, though we use the term objective with ease in our conversations and in our literature, its meaning is not particularly clear, nor … are the consequences of the tacit, almost unexamined assumptions upon which it rests.”
Eisner (1992, p. 9)
“It turns out, then, that what is crucial for the objectivity of any inquiry – whether it is qualitative or quantitative – is the critical spirit in which it has been carried out. And, of course, this suggests that there can be degrees for the pursuit of criticism and refutation obviously can be carried out more or less seriously. ‘Objectivity’ is the label – the ‘stamp of approval’ – that is used for inquiries that are at one end of the continuum.”
Phillips (1989, p. 36)
Congratulations! You have now finished Week 3. Please complete the learning reflection form for this week.
Assignments will be "peer assessed", so make sure that you present your results in a clear and logic way that is understandable for others!
Please note: This module will show as completed in your "Self-Print Certificate" only when you have submitted your Assignment.
For each part of the task, high marks should be awarded to answers that are accurate, comprehensive, convincing, insightful and scholarly.
Being "scholarly" means using logical arguments; avoiding non sequiturs; substantiating claims; and using accurate spelling, grammar and referencing.
Furthermore, each part of the task also has a specific success criterion for achieving the highest marks:
The module makes extensive use of The Open University's module H809 "Practice-based research in educational technology" which has been made available within the Creative Commons framework under the CC Attribution – Non-commercial licence. These materials are available under the same licence.
The PowerPower slides “Descriptive Statistics" and “Inferential Statistics” were created for H809 by Professor John Richardson, The Open University.
Extra Service Directory
Latest Forum Posts
- New session when?
annerichter 16.5.2013 22:45
- engineering course
kilmith12 17.1.2013 9:11
- Re:Guest Lecture UNU: Replay Now Available !!!
Sunyjuarez 16.1.2013 7:29
- Greet & meet autumn 2012
Leonie 20.11.2012 17:30
- Activity: Analyze the stakeholders of ten comp...
m_shepler 13.11.2012 23:26