Grad School Update/Methodology Rant
Haven't done one of these for a while, so we're about due. Stuff is about to get crazy go nuts around here, seeing as we only have five weeks left in the semester (or something like that). It is about this time that I get swamped with studying for finals, finishing up projects, and running subjects in the lab like an ill-tempered constipated baboon surviving on a diet consisting mainly of mountain dew and pop rocks (WARNING: the preceeding metaphor is neither accurate nor to be taken as a genuine evaluation of my diet which is in fact decent).
Research is taking an interesting twist as my advisor and I have been cooking up a potential new project that would attempt to show some scientific evidence for the idea that "you pay attention to what you want to pay attention to." It sounds simple, but in reality it's going to be like squeezing a camel through the eye of a needle. People often wonder why scientists try to prove "common sense" with scientific research. The reason we do this is because all too often, common sense can be flat out wrong.
Well, that and we're supposed to be skeptical. But in this situation, we (my advisor and I) feel that there may be a legitimate concept behind the idea that our volition (our will) drives and directs our attention mechanism to whatever we want to pay attention to in our environment.
Here's where it gets tough. We have to contend with three very well established theories of attention in the field:
Object-based attention (attention attends only to specific objects as a whole)
Attribute based attention (attention attends to features of an object and then integrates them into a whole)
Space based attention (you attend to a set area of space - sort of like a spotlight - and whatever happens to be there at the time).
Here's the basic idea for the proposed experiment: We create a set of stimuli that we hold constant, and then present it to subjects in three conditions:
1. A condition where we ask them to look for and respond to the object itself
2. A condition where we ask them to look for and respond to a feature of the object
3. A condition where we ask them to look at a field of space and respond to the object there
What we would hope to find is that under the different direction conditions using the exact same stimuli in all three conditions, the responses to the stimuli would change from direction condition to direction condition. According to the respective theories themselves, this should not happen as we are attending to the exact same stimuli, and the three theories are just three approaches to the same basic effect.
So, yeah, if we can show that the participants response times to the objects (which will be our dependent measure) have different means for all three conditions, then BAM! We have some support for our hypothesis that volition directs attention (as the direction conditions told them to attend willfully to different things) and could have a really great publishable paper.
Here's the big, big, BIG problem: My advisor has been thinking about this experiment for years. The reason she hasn't run it? Coming up with a stimulus that fits all three theories yet could fit the requirements of the task is going to be as hard as hell.
It may not seem like it, but there are hundreds of little considerations we have to deal with (more than I care to get in to) that eliminate the useage of certain types of objects.
So, as a result of this, my job is, over the next few months (yes, it will take that long), I have to try and come up with a stimulus design that fits all three theories AND fits the proposed task. What this will entail is a HELL of a lot of reading and research. And I mean a HELL of a LOT of reading and research so I can get a good grasp of what we call Construct Validity (which is a nice way of saying I understand the theory backwards and forwards) for all three theoretical concepts so I can even start to try and build this stimulus.
The cool thing about it is I'm super-excited to get to work on it.
Of course, me having told all of you this means that I can't run ANY of you as participants, so you can feel relieved about that. The reason I can't is because you now all know my hypothesis, so if I were to run you, your expectations on the project regarding what kind of response would best be helpful to us would result in you shaping your responses to help. It's called the good subject effect, and it completely ruins data. (It's also because of this that we eject participants who try to guess the hypothesis/purpose of the study. We try to structure the experiments so that they have no idea in hell what is happening. If they think they've figured it out, right or wrong, and start to shape their responses to fit their idea of what is happening, it also screws up the data as they will tailor their responses to fit what they think is happening. This is the validity threat called Hypothesis Guessing.
Both are good ways to ruin a researchers day and make them hate you with the fire of a thousand suns because:
1. You've just wasted their time, probably about two hours of their day, in which they could have been working on something else OR running a subject who didn't try to sabotage their study
2. You've just wasted your time, because the data you gave them cannot be used and running you was completely pointless
The trouble with hypothesis guessing and the good subject effect, of course, is that you absolutely cannot keep the subject from trying to figure out what the purpose of the study is. In fact, every single subject by virtue of basic human curiosity will try to figure it out at some point. There are , of course, ways we can structure the experiment to prevent them from forming an idea one way or another about what is being measured in the study. If the researcher has done their job right, the correct response when they ask you "what do you think the study was about" should be "I have no freaking clue what just happened and I want to pound my head against a wall." If a majority of the subjects can say anything else to them, then they've goofed up somehow.
These, among many other things, are the ideas and concepts and threats to the validity of our studies that we have to tackle whenever we build an experiment. It takes a LOT of time to get it done right, so you can imagine how frustrating it is for us to have to deal with subjects who either try to wreck it for us deliberately (they do exist, and we hate them) or just don't take it seriously enough to make an effort to respond at all.
There is, of course, one other implication of me telling you all this that I hope is a bit obvious: Because I want to talk to all of you about this, and I know that some of you will find this topic very interesting, I have to maintain anonymity on this blog. If I were to, say, post the hypothesis of my experiment in a public forum with my name attached to the blog, then my students and/or potential subjects could find it, learn the hypothesis, and subsequently ruin the subject pool for my experiment. Ah, but if I remain anonymous, the subject pool is kept in the dark, and I am free to share what I am tinkering about on with all of you happily and freely.
Research is taking an interesting twist as my advisor and I have been cooking up a potential new project that would attempt to show some scientific evidence for the idea that "you pay attention to what you want to pay attention to." It sounds simple, but in reality it's going to be like squeezing a camel through the eye of a needle. People often wonder why scientists try to prove "common sense" with scientific research. The reason we do this is because all too often, common sense can be flat out wrong.
Well, that and we're supposed to be skeptical. But in this situation, we (my advisor and I) feel that there may be a legitimate concept behind the idea that our volition (our will) drives and directs our attention mechanism to whatever we want to pay attention to in our environment.
Here's where it gets tough. We have to contend with three very well established theories of attention in the field:
Object-based attention (attention attends only to specific objects as a whole)
Attribute based attention (attention attends to features of an object and then integrates them into a whole)
Space based attention (you attend to a set area of space - sort of like a spotlight - and whatever happens to be there at the time).
Here's the basic idea for the proposed experiment: We create a set of stimuli that we hold constant, and then present it to subjects in three conditions:
1. A condition where we ask them to look for and respond to the object itself
2. A condition where we ask them to look for and respond to a feature of the object
3. A condition where we ask them to look at a field of space and respond to the object there
What we would hope to find is that under the different direction conditions using the exact same stimuli in all three conditions, the responses to the stimuli would change from direction condition to direction condition. According to the respective theories themselves, this should not happen as we are attending to the exact same stimuli, and the three theories are just three approaches to the same basic effect.
So, yeah, if we can show that the participants response times to the objects (which will be our dependent measure) have different means for all three conditions, then BAM! We have some support for our hypothesis that volition directs attention (as the direction conditions told them to attend willfully to different things) and could have a really great publishable paper.
Here's the big, big, BIG problem: My advisor has been thinking about this experiment for years. The reason she hasn't run it? Coming up with a stimulus that fits all three theories yet could fit the requirements of the task is going to be as hard as hell.
It may not seem like it, but there are hundreds of little considerations we have to deal with (more than I care to get in to) that eliminate the useage of certain types of objects.
So, as a result of this, my job is, over the next few months (yes, it will take that long), I have to try and come up with a stimulus design that fits all three theories AND fits the proposed task. What this will entail is a HELL of a lot of reading and research. And I mean a HELL of a LOT of reading and research so I can get a good grasp of what we call Construct Validity (which is a nice way of saying I understand the theory backwards and forwards) for all three theoretical concepts so I can even start to try and build this stimulus.
The cool thing about it is I'm super-excited to get to work on it.
Of course, me having told all of you this means that I can't run ANY of you as participants, so you can feel relieved about that. The reason I can't is because you now all know my hypothesis, so if I were to run you, your expectations on the project regarding what kind of response would best be helpful to us would result in you shaping your responses to help. It's called the good subject effect, and it completely ruins data. (It's also because of this that we eject participants who try to guess the hypothesis/purpose of the study. We try to structure the experiments so that they have no idea in hell what is happening. If they think they've figured it out, right or wrong, and start to shape their responses to fit their idea of what is happening, it also screws up the data as they will tailor their responses to fit what they think is happening. This is the validity threat called Hypothesis Guessing.
Both are good ways to ruin a researchers day and make them hate you with the fire of a thousand suns because:
1. You've just wasted their time, probably about two hours of their day, in which they could have been working on something else OR running a subject who didn't try to sabotage their study
2. You've just wasted your time, because the data you gave them cannot be used and running you was completely pointless
The trouble with hypothesis guessing and the good subject effect, of course, is that you absolutely cannot keep the subject from trying to figure out what the purpose of the study is. In fact, every single subject by virtue of basic human curiosity will try to figure it out at some point. There are , of course, ways we can structure the experiment to prevent them from forming an idea one way or another about what is being measured in the study. If the researcher has done their job right, the correct response when they ask you "what do you think the study was about" should be "I have no freaking clue what just happened and I want to pound my head against a wall." If a majority of the subjects can say anything else to them, then they've goofed up somehow.
These, among many other things, are the ideas and concepts and threats to the validity of our studies that we have to tackle whenever we build an experiment. It takes a LOT of time to get it done right, so you can imagine how frustrating it is for us to have to deal with subjects who either try to wreck it for us deliberately (they do exist, and we hate them) or just don't take it seriously enough to make an effort to respond at all.
There is, of course, one other implication of me telling you all this that I hope is a bit obvious: Because I want to talk to all of you about this, and I know that some of you will find this topic very interesting, I have to maintain anonymity on this blog. If I were to, say, post the hypothesis of my experiment in a public forum with my name attached to the blog, then my students and/or potential subjects could find it, learn the hypothesis, and subsequently ruin the subject pool for my experiment. Ah, but if I remain anonymous, the subject pool is kept in the dark, and I am free to share what I am tinkering about on with all of you happily and freely.