UPDATE: 12 August, 2016. This post looks at the draft Framework. For a review of the approved version of the frame “Scholarship as Conversation” please visit https://senseandreference.wordpress.com/2016/08/12/revisiting-the-framework-is-scholarship-a-conversation/
A few weeks ago I wrote that I was not too thrilled with the “threshold concept” theory underpinning the new ACRL information literacy framework. And though I hinted at the six threshold concepts put forth by the ACRL, I chose not to criticize them directly. Why? Well, it’s because the six concepts themselves seem like good things for students to learn. Just to recap, the six concepts identified by the ACRL framework are:
- Scholarship is a Conversation
- Research as Inquiry
- Authority is Contextual and Constructed
- Format as a Process
- Searching as Exploration
- Information has Value
Taken at face value, these seem like six important insights; six things we presume information literate persons should be familiar with. Granted, I’m not buying into the threshold concept business, but they seem like they could be what we used to call foundational or core concepts.* That is to say, they’re really important. Maybe even the most important things to understand when becoming information literate (though there are probably others too). And they are the core of the ACRL’s new approach to information literacy. These six concepts–quite independent from the notion of threshold concepts–are going to play a huge role in library instruction, assessment, and so on. Really, in another five years or so we’ll all have these memorized and hearing people say “format as a process” at LOEX will be no big deal (except to the grammar police).
Perhaps the thing that most interests me is that these concepts have gone almost entirely unchallenged. Other than some slight snark on Twitter, I haven’t seen anyone really dig into these core concepts with a critical eye. Basically, I’m reading about a lot of excitement and the occasional “I’ve been teaching this concept for years, thank god the ACRL finally recognizes it” going on. But who’s calling shenanigans? (If, indeed, there are shenanigans to be called.) [EDIT: I just finished writing this when I saw that Jacob Berg called shenanigans on the ethical dimensions of the “Information has Value” TC. Go check out his post. It’s a good read.]
Well, shoot, I guess I’ll just have to call them.
Starting with this post, I’ll take a look at each core concept in turn and figure out what to make of it. Again, I do think these are important concepts, but I just don’t like to see important ideas go untested. So, I’m going to play devil’s advocate and poke at the framework in the hopes that I can make sense of it. Don’t think I’m going to be entirely negative here: there are a few threshold concepts I really like. And the ACRL task force should be commended for thinking outside of the box. I just want to poke around in the hopes that any weaknesses in the concepts are addressed prior to formal adoption of the framework. So, that’s what I’ll do. But first, an explanation of how the framework is set up.
Anatomy of a frame
Each of the six “frames” in the new framework follows the same structure:
- A summary overview of the threshold concept
- A bulleted list of “knowledge practices”
- Related “metaliteracy learning objectives”
- Dispositions
- Self-assessments
- Possible assignments
The summary overview is further divided into a short (one sentence) description of the concept followed by a paragraph long explication. The knowledge practices section details the skills that a person will have after acquiring the threshold concept in question. The metaliteracy learning objectives have something to do with social media, collaboration, metacognition, and…um…actually, I have no clue what’s going on here. Moving on, the dispositions section outlines what I assume are meant to be desired intellectual dispositions like curiosity, open-mindedness, self-awareness, and so on. The self-assessments seem to be concrete activities designed to test whether a person has mastered a particular threshold concept, and the possible assignments are simply suggested classroom activities.
Given that I don’t understand the reasoning behind the metaliteracy objectives, I won’t speak to them. And, since the self-assessments and possible assignments seem more like optional curricular suggestions, they don’t seem to be necessary in order to understand the threshold concepts. That leaves just the concept itself, the knowledge practices, and the dispositions as the real meat of the framework. And, since it makes sense to go in the order that the threshold concepts are presented in the framework, let’s start by looking at the first frame…
Scholarship is a Conversation
Overview
Let’s take a look at the first threshold concept of the new Framework: scholarship is a conversation. From the draft framework:
Scholarship is a conversation refers to the idea of sustained discourse within a community of scholars or thinkers, with new insights and discoveries occurring over time as a result of competing perspectives and interpretations.
While many questions can be answered by appeal to a single, authoritative source–the capital of a country or the atomic number of an element, for example–scholarly research resists simple answers. Rather, scholarship is discursive practice in which ideas are formulated, debated, and weighed against one another over extended periods of time. Instead of seeking discrete answers to complex problems, scholars understand that a given issue may be characterized by several competing perspectives. Far from a unified body of uncontested knowledge, the scholarly record is better understood in terms of a conversation in which information users and creators come together to negotiate meaning, with the expert adding his or her voice to the conversation. The expert understands that there may not be a single uncontested answer to a query and, hence, is inclined to seek out the many perspectives in a scholarly conversation, not merely the one with which the expert already agrees.
The idea of the scholarly conversation has been floating around for a while now: in 2011 Barbara Fister introduced the idea by way of Kenneth Burke,** explaining that “what we’d like students to have is some sense of scholarship as conversation in progress and some tips on how to figure out who’s talking and how to pick up the threads without getting completely lost.” Scholarly articles are not meant to be consumed independently from one another. Indeed, the entire scholarly record is a self-supporting framework of interconnected ideas. So the story goes. Of course, the idea that scholarship is a conversation is nothing new; in addition to Burke, Michael Oakeshott talked about it in “The Voice of Poetry in the Conversation of Mankind” and Richard Rorty used the idea quite extensively as he argued for the primacy of hermeneutics over epistemology. It’s also a wildly popular concept among postmodern Christian apologetics. And of course, taken broadly, the discursive nature of learning has been with us since Socrates. So, again, it’s not a new idea: students need to move away from looking for simple answers to complex problems and instead realize that every researcher brings some new insight to the scholarly conversation and new theories or perspectives (not necessarily answers) arise in negotiation. And so it goes: we expose students to the idea that scholarship is a conversation, they have an “A-ha!” moment, and they run off to seek multiple perspectives on challenging issues.
But, there’s a tension in this threshold concept that I can’t seem to find a way around: is the claim “scholarship is a conversation” meant to be taken literally or figuratively? Either way leads to some uncomfortable results for the ACRL’s framework.***
Scholarship is literally a conversation
If the threshold concept is taken literally, then all we have to do to explain scholarship is look to the definition of a conversation. The venerable OED defines a conversation as an “interchange of thoughts and words; familiar discourse or talk”**** and the entry in Wikipedia explains that a conversation is “a form of interactive, spontaneous communication between two or more people who are following rules of etiquette.” Averaging these out, we can say that a conversation consists in three things:
- an exchange of information
- between two or more people
- following a particular etiquette
That last bit about etiquette is actually pretty interesting, and it’s worth stopping to mention Grice’s conversational maxims, simplified as: be as informative as possible, be truthful, be relevant, be clear. I’ll let you go down the rabbit-hole of conversational implicature on your own. But the point is that a conversation follows informal rules that help distinguish it between other exchanges of ideas. For example, in a conversation there is an informal rule of etiquette that all those involved participate. If one person does all the talking, that’s not a conversation, it’s a monologue. There’s also a rule that participants should seek agreement or, at the very least, coherence. If the conversants are trying to prove each other wrong, we call that a debate or an argument, not a conversation. You get the idea.
Anyway, there certainly are some librarians out there who hold a literal version of “scholarship is a conversation.” Dave Lankes of New Librarianship fame comes to mind. And the idea is that when I read your article, cite it in my paper, and then someone cites me later on, there’s a real conversation taking place over time. Amplified throughout the world of scholarly communication, we can say that there is a massive exchange of information between millions of scholars following formal and informal rules (e.g., citations, peer-review, etc.). Hence, scholarship is a real conversation, structurally no different from hanging out with your coworkers around the water cooler.
But, is it really? I don’t think so. I mean, when you think of a conversation, don’t you usually think of something more like that casual water-cooler talk or shooting the shit with your friends or even talking about deep ideas with your peers? “Hey, I was thinking…” “Yeah, sort of like this book I read…” “Cool, reminds me of this blog post…” You get the idea. And though I don’t know how best to illustrate it, I shouldn’t have to: I think we all know what a regular conversation is. But, when we start to take away that real-time exchange and we start to allow that citations months, years, or decades apart count as information exchange (i.e., “ideas are formulated, debated, and weighed against one another over extended periods of time”) or when we extend normal conversational etiquette to include things like peer-review, citation styles, and the like, aren’t we stretching the normal definition of conversation awfully thin? It seems to me that the long timescales, the rigid rules for publication, the choke-points on access, the necessary evils of copyright…all work against the casual exchange of what we normally assume is conversation. Basically, publishing scholarly articles is such a different thing from going back-and-forth on Twitter that it only dilutes the meaning of “conversation” to say that scholarship is literally a conversation. It dilutes it to the point that there’s nothing to be gained by calling scholarship a conversation: if scholarship is just any old exchange of ideas, then everything is a conversation, from debating to delivering a monologue to writing fiction to making movies, to painting a mural. How is that helpful?
It isn’t, because the entire reason we make a distinction between conversation, monologue, debate, scholarship, and so on is because those differences matter.
Scholarship is only figuratively a conversation
But, perhaps this threshold concept isn’t meant to be taken literally. Perhaps it’s just a metaphor; scholarship is like a conversation. This seems more plausible; it allows us to keep a more intuitive definition of what a conversation is. It also allows there to be some differences between scholarship and normal conversation, so long as there are more similarities on balance. So, for example, that thing about the weeks, months, or years between the points and counter-points of the “scholarly conversation” is more palatable: scholarship is just a really slowed-down conversation. And just like conversations involve multiple voices, scholarship requires multiple voices, it isn’t a solitary activity.
I think that the scholarship/conversation metaphor does have its place as a rhetorical or pedagogical device because, at heart, calling scholarship a conversation is a means of simplifying what is otherwise a rather complex subject area. When I teach first-year students about finding sources, I do use the scholarly conversation metaphor precisely because it allows me to set aside some of the more complicated issues surrounding peer-review, publish-or-perish, open access models, journal subscriptions, and so on. In a weird way, the metaphor allows me to step away from expertise. See, I don’t start from the position of the expert, as the Framework suggests we should. I start from the position of the novice: how do they see scholarship? Well, we know that novice researchers tend to think in binary terms of pro/con, right/wrong, for/against (actual request: “I’m writing about date rape on campus and I found stuff against it, but I’m having trouble finding articles that support it”). and they tend to seek out all and only that evidence that supports their own beliefs (“I need stuff that shows how gays shouldn’t marry”) . They tend to think that if they can form a research topic about something, then there must be scholarship surrounding it (“I need a peer-reviewed article about parking on the UTC campus”). They tend to get hung up on what is and isn’t considered scholarly (“It doesn’t have “Journal” in its title, so it isn’t scholarly”). Hopefully this stuff rings true to library instructors.
Anyway, the conversation metaphor is useful in helping students break out of those bad habits and misconceptions. Helping students see that some research areas actually contain multiple perspectives is a good thing. So is helping students realize that good research means maybe changing your own beliefs. And these are the parts of the threshold concept I think work best; I do this stuff when I teach. For example, one of our visual rhetoric activities for first-year composition has them make the move from research being like this
to research being more like this
And we have the whole conversation about looking for perspectives instead of concrete answers. So, on this point I think the ACRL’s threshold concept is on point. There are some similarities between conversations and scholarly communication and pointing these out is a good pedagogical tactic.
But, here’s the thing: I don’t pull the conversation metaphor with more advanced students. The metaphor is a great way of simplifying scholarship, but once researchers get past the more naive misconceptions it’s the differences between conversations and scholarship that become important. In a sense, the expert researcher needs to know how scholarship is not like a conversation. Here are a few examples of how scholarship is not a conversation:
- Conversations are not mediated; scholarship is. In a traditional conversation, all parties are able to speak freely and without interference other than self-censorship. But, scholarship is mediated through processes like peer-review, which impose filters between an author and her readers. When peer-review works well this is a good thing and one of the foundations of modern scholarship. But there is no analogue in a conversation. Experts need to understand how the scholarly record is mediated.
- Conversations are symmetrical; scholarship is not. Conversations require that familiar back-and-forth. I say A, you respond B, I respond C, etc. Remember, if only one person is talking, it’s only a monologue (NTTAWWT). But, scholarship lacks this conversational symmetry. If I publish a paper and someone cites my work, are they responding to me? Or are they using my ideas to further their argument? Sure, sometimes scholars directly address one another. Those of you interested in information science probably remember the epic exchange between Marcia Bates and Birgir Hjørland in JASIS&T a few years back. Here’s a summary. Heck, even this rinky-dink little blog was involved in a little reciprocal exchange when I criticized an article by John Budd and he addressed my post in the Journal of Documentation. Link here. But, for the most part, scholarship is not about “information users and creators com[ing] together to negotiate meaning” as the ACRL puts it. It’s more asymmetrical, with most scholarship proceeding through accretion of ideas, not negotiation of ideas. Speaking of which…
- Conversations may negotiate meaning; scholarship doesn’t. I’ll grant that in some disciplines–maybe critical theory or history or something else in the humanities–scholars often “negotiate meaning.” Though, to be fair, I’m not too sure what it even means to negotiate meaning in a scholarly context. Is the idea that every single fact, theory, or belief is up for negotiation? That the goal of scholarship is trying to figure out what theories and facts mean? That doesn’t seem right. Look at the millions of scientific papers out there that simply report the findings of empirical study. Sure, sometimes there are competing theories: until recently there were competing schools of thought about whether neanderthals had a carnivorous or an omnivorous diet. So, a bunch of scientists vaporized some caveman turds and found that neanderthals were omnivores. Case closed. And not through negotiation, but through experiment. The vast majority of scientific scholarship isn’t about negotiating answers or new discoveries, it’s about reporting the answers or discoveries found through research. Are we really supposed to teach students that even empirical studies really just boil down to conversational negotiations? [EDIT: Donna Lanclos has pointed out that this isn’t exactly how science works, and she’s right. I oversimplified. My point is that meaning in science isn’t negotiated through the scholarly record the same way it is in a conversation: it’s determined via experiment and empirical data. Ideally, the evidence should be what determines what the scientific community believes, not some conversational negotiation. Granted, science, as a human activity, is constantly affected by non-evidential biases and negotiations do occur. But by and large the sciences should be judged by the quality of the evidence, not the coherence of the conversation. Where the scholarly record comes in is at the level of independent confirmation (i.e., replication), pattern recognition, inductive reasoning, and (if needed) debunking via more compelling evidence. The posts I wrote on social constructionism a few years ago may shed some light on my position. Post 1. Post 2.]
I guess my point is that the metaphor “scholarship is a conversation” only works if we approach scholarship in a highly simplified way. That works for first-year students…gotta start slow. But, experts know better: the structural realities of contemporary scholarship negate comparison with a conversation.
One last issue with the overview: it seems to equivocate between scholarly communication and the act of researching. Sort of how the word “research” can mean either the act of researching or the product of researching. You see this equivocation clearly as the concept moves from discursive practice (act) to competing perspectives (product) to negotiating meaning (act) to artifacts like books and articles (products). So, is it the act of researching that is a conversation? Or are the products of research the conversation? The overview could be clearer here.
Knowledge Practices
Moving beyond the overview, we get the “knowledge practices” which are the abilities that information literate researchers possess. Overall, I think these are a bit far-fetched. Consider the first three:
- Identify the contribution that particular articles, books, and other scholarly pieces make to disciplinary knowledge.
- Summarize the changes in scholarly perspective over time on a particular topic within a specific discipline.
- Contribute to scholarly conversation at an appropriate level (local online community, guided discussion, undergraduate research journal, conference presentation/poster session).
Identify the contribution that a particular article makes to its field? Summarize changes in scholarly perspective over time in a given field? These seem to require subject mastery, implying that information literacy only applies within a particular discipline and cannot be interdisciplinary or cross-disciplinary. That’s a strong statement for the ACRL to suggest and I can’t agree with it. Contribute to scholarship at the appropriate level? Again, I suppose I’m not information literate in any area outside of library science because I don’t contribute to other disciplinary conversations. These implications are unacceptable because one of the great strengths of information literacy (and one of the reasons librarians push it) has always been that becoming information literate allows us to evaluate information without having to be experts on every topic under the sun. I really hope these knowledge practices get a major overhaul in the next revision
Dispositions
- Seek out conversations that are taking place in their area of research.
- Suspend judgment on the value of a particular piece of scholarship until the larger context for the scholarly conversation is better understood.
- Recognize that scholarly conversations take place in a variety of venues.
- Value user-generated content and critically evaluate contributions made by others.
- See themselves as contributors to scholarship rather than only consumers of it.
- Understand the responsibility that comes with entering the conversation through participatory channels.
Now, I really like the dispositions. Students should absolutely “seek out conversations” taking place in their area of research and they should absolutely suspend judgment until they’ve learned more. I agree with these. Though, the former disposition I may reword as “students should be inquisitive” and the latter is basically the idea that evaluating sources gets easier as you acquire more domain knowledge. Good points, but not necessarily ground-breaking. Same goes for recognizing that scholarship takes place in a range of venues…though I don’t see how that’s a disposition. I suppose the fourth disposition is a bit hard to figure out: what does it mean to “value user-generated content? Value how? Which users? And the fifth disposition might be improved by changing it to “potential contributors to scholarship.” I don’t think you have to publish scholarly work to be information literate. But, overall, I find the dispositions more compelling than the rest of the document. Nice work.
Verdict: Is scholarship a conversation?
There are some choice bits of wisdom in the first frame, but also some rather sweeping simplifications. The beginning, “while many questions can be answered by appeal to a single, authoritative source…scholarly research resists simple answers,” is spot on and valuable. Some research questions can be answered directly, some research questions are more contentious. I’d say the information literate student should be able to recognize the simpler factual research from the more complicated perspectives-based research questions. Another way to say it: the information literate expert is discriminating. But, unfortunately, the ability to discriminate between the various types of research and information needs doesn’t appear in the framework. Perhaps the next draft? I also appreciate that the threshold concept focuses on the inclination “to seek out the many perspectives in a scholarly conversation, not merely the one with which the expert already agrees.” Though, I wish it were clearer that not all perspectives are equally valid. And even if the knowledge practices actively undermine the interdisciplinarity of information literacy, the dispositions are worth holding on to.
In sum, I fell like this threshold concept is more about rhetoric and pedagogy than anything else and I’m definitely going to continue to use the scholarship/conversation metaphor when working with novice researchers. But, the oversimplification only goes so far; scholarship as conversation is a concept that is of minimal value for intermediate and expert researchers. This is a concept you acquire and then move past as you become more knowledgeable about scholarly communication and treating it as a threshold concept just muddies the waters.
What do you think?
* Recall that threshold concepts are supposed to somehow go beyond foundational or core concepts.
** Burke, Kenneth. 1941/1967. The philosophy of literary form: studies in symbolic action. Baton Rouge: Louisiana State University Press.
*** The same question–literal or figurative–can be asked of other threshold concepts.
**** Actually, that’s the sixth definition. Other definitions abound, but, for example, I don’t think the ACRL is suggesting that scholarship is a conversation in the sense of “sexual intercourse or intimacy.” Though, now that I think about it, that would certainly liven things up in library instruction.
Even if it does oversimplify things, the “scholarship as conversation” metaphor helps doctoral students too, mainly because they often get confused about what the context of a literature review is all about. Many think they have to get everything that’s out there but, actually they don’t. Getting them to focus on key ideas related to their thesis so they can actually enter the conversation with new studies and new knowledge that they create helps them figure out what they have to do, which is synthesis, not just analysis. After all, you don’t know that what you are saying OR doing is new unless you thoroughly figure out the “conversation,” right? You have to figure out where your work fits in to even be able to write about it in the early chapters of a thesis. Don’t knock oversimplification if it can help even one student conceptualize what they need to do to get “unstuck” then use it.
I actually agree with you. I probably should have clarified that the novice researcher can be anywhere from a first-year student to a doctoral student. And I have definitely used the metaphor with grad students. I think the larger point I wanted to make is that once a person “gets” the metaphor, you can move on to ever more complex issues surrounding scholarship. Eventually, when that doctoral student becomes an “expert researcher” they will realize that scholarship isn’t *really* a conversation.
Ultimately, I think “scholarship is a a conversation” is a great starting point. I just disagree with the ACRL that it’s an ending point.
Great post and lots of things to think about here, thanks. Just a couple thoughts:
> In a traditional conversation, all parties are able to speak freely and without interference other than self-censorship.
One could argue that the analogue in traditional [literal and figurative] conversations is that they are mediated by social, cultural and behavioral norms, power differential, intent and motivation, and loads of other contextual factors. And sometimes, forcible interference in the form of violence or threats. In that sense, a scholarly conversation via peer reviewed articles may be much freer than, for instance a conversation between a boss & employee, or parent & child, or police officer & alleged perpetrator. The unfolding conversation is governed by expectations to conform to one or more of the above factors, just as it is in the publication of scholarly works–both refereed and unrefereed.
In your above comment to Susan, you state “I think the larger point I wanted to make is that once a person “gets” the metaphor, you can move on to ever more complex issues surrounding scholarship…Ultimately, I think “scholarship is a a conversation” is a great starting point. I just disagree with the ACRL that it’s an ending point.”
Isn’t this more or less what the “threshold concept” concept is meant to convey? Not that this is an ending point, but that an information literate person must understand the transactional nature of scholarly communication as a critical step to an “enlarged understanding” (Draft 2, June 2014, lines 25-27) of the latter.
What would you think about rewording this to “Scholarship proceeds through discourse?”
Hi clarke, great questions.
On the issue of conversations, I think you’re right that power has a way of mediating conversations, so my characterization is an oversimplification. I wrote the post in haste and I’ll be sure to acknowledge the cultural/social mediation in future posts. And I also agree that the peer-review process can sometimes be “free” in a way that conversations are often not. But, isn’t this just further evidence that conversations and scholarship are categorically different things? If power affects traditional conversations and scholarship in different ways that seems like more evidence to ditch the metaphor.
On the issue of the threshold concept, it’s true that threshold concepts are liminal concepts that lead to enlarged understanding. But the literature on threshold concepts is clear that these concepts are bounded, transformative, and irreversible. If the scholarship/conversation metaphor is a threshold concept, then it has to be bounded, irreversible, etc. And that’s a problem because if we want to gain a deeper understanding of scholarship the metaphor has to be reversible. We have to be able to see all the ways that scholarship is *not* like a conversation if we want to be experts.
And I like the idea that scholarship proceeds through discourse a little better.
Thanks for your reply, Lane.
> isn’t this just further evidence that conversations and scholarship are categorically different things? If power affects traditional conversations and scholarship in different ways that seems like more evidence to ditch the metaphor.
First, I agree with your point, in the “Scholarship is literally a conversation” section, that the word “conversation” connotes something cavalier, brief, and temporary. Although I would prefer “discourse,” I use “conversation” below because that’s what exists in the draft–plus, the meaning’s been substantially changed in LIS since Lankes’s Atlas.
Actually, I think it could be argued that the types of mediation that exist in all conversations does indicate that scholarly communication is a type of conversation (i.e., in the larger set of all figurative conversations, scholarly communication is a subset). In that sense, power & mediation is expressed in several ways:
– Format conformity in journal articles (lit review, methods, findings, etc.)
– Publication norms (e.g., humanities publish in monographs & sciences in journals; the changing norms surrounding data sharing;)
– The role of tenure & promotion in publication (requiring faculty to publish in certain journals with certain impact factors; how this is related to open access & serials crisis issues)
– The various pressures exerted by peer review (forcing out/censoring unconventional views, see also the many issues brought up by advocates for open peer review)
This could be touched on when teaching the ‘Authority is Constructed’ section of the framework. Most of it is approaching irrelevance to first year undergrads, but is of vital importance to grad students and faculty as well, and it belongs in any advanced discussion on scholarly communication.
But my larger point is, if the framework is to argue that scholarship is a conversation/discourse, then it should also draw the analogy to the ways that these forces exert control over that discourse.
I like Clarke’s idea of “Scholarship proceeds through discourse” a lot better than “Scholarship is a conversation.” For one thing, it gets more at the idea of scholarship making progress rather than just being a back-and-forth without moving toward a goal. It also backs away from identifying scholarship as a conversation, which is a very strong claim, as opposed simply to proceeding through conversation.
Paul, those are really good points. Scholarship is progressive, not just back-and-forth. It proceeds through conversation without being conversation. I kind of want to rewrite the post to highlight these. Thanks. 🙂
I think we are all on the same page with regard to “conversation” as a helpful metaphor. Whether it is an actual “threshold concept” or not is another question entirely. Perhaps “discourse” does capture the more complex issues surrounding advanced research & scholarship.
Absolutely. It’s a wonderful metaphor and I use it almost daily with students!
But basing a foundational concept on a metaphor? That just seems wrong.
Sidenote: I was just thinking about cultural norms surrounding conversations. I wonder if this threshold concept is meant to imply that scholarship is a specific culturally-determined type of conversation (i.e., Western, educated, white, middle-class, etc.), or if it implies that scholarship is relative to cultural norms surrounding conversations (i.e., scholarship is different in each culture). Hmmm. Just thinking…
[…] “Conversation at Caffe Nero” by ktylerconk on Flickr A few weeks ago I wrote that I was not too thrilled with the “threshold concept” theory underpinning the new ACRL information literacy framework. […]
[…] It seems to me that the long timescales, the rigid rules for publication, the choke-points on access, the necessary evils of copyright…all work against the casual exchange of what we normally assume is conversation. […]
Thank you, Lane, for starting a serious conversation about the new ACRL Information Literacy Framework. In a bit, I’ll disagree with you on a fundamental assumption. But that only means that your discussion goes far enough to make it clear where we disagree.
One should always attempt to give the most charitable interpretation of any opinion or writing one is criticizing or analyzing. In the case of the IL Framework, the general concepts of “metaliteracy” and “thresholds” are vague or empty. So these general concepts does not allow useful criticism.
But by the same token, empty or vague ideas are unlikely to be misleading. IL instructors may be able to muddle through to an effective pedagogy, as long as they have sufficient academic freedom to allow them to experiment.
So if the slogan, “Scholarship is a conversation,” is merely vague or metaphorical, that’s not as bad as being misleading. Unfortunately, as you explain, it is usually misleading to consider scholarly communication as a conversation.
As you observe, this slogan is most misleading in science. Neanderthal turds are evidence. They might be conversation-stoppers. They are not part of a conversation. They are not really even a conversational topic.
The slogan is misleading in humanities and social sciences too. In the humanities, different kinds of evidence are used, for example, to support an argument that an interpretation of a literary work is a good interpretation. It’s not like saying, “I like this,” without any evidence. If one is going to justify an aesthetic judgment then one will end up giving reasons for it.
But even in science, the kinds of evidence vary greatly by discipline. Evidence used in mathematics is nothing like evidence in evolutionary biology, except insofar are evolutionary biology uses some results from mathematics.
You say that “one of the great strengths of information literacy (and one of the reasons librarians push it) has always been that becoming information literate allows us to evaluate information without having to be experts on every topic under the sun.”
I’m afraid I don’t agree that is possible to evaluate information (at least not in any non-superficial way) unless one understands the evidence for it or against it, or more generally, what makes that information correct or not, or useful or not.
Even though you explicitly say that IL has this power, I don’t think you really believe this. Since you recognize that it is misleading to think of scholarly communication as a conversation without evidence, I think you probably will also recognize that IL cannot teach us (or students) how to evaluate information without knowledge.
What should IL instructors teach then? An answer would require more than a response to a blog post. It would require discussion of the aims of IL instruction. It would require discussion of sample assignments that have been shown to be doable by real undergraduate students. Again, I think IL instructors can muddle through to an effective pedagogy, as long as they have academic freedom to do so.
Meanwhile, there is a huge gap between the ambition of the assignments suggested in the Framework and the vague or empty metaphors, such as “Scholarly is a conversation,” offered by the Framework. The suggested assignments in the Framework seem pitched more-or-less at the level of doctoral research:
• Trace the development of scholarship on a particular topic using the traditional ‘information cycle’ model with the ‘invisible college’ and print publication outlets.
• Conduct an investigation of a particular topic from its treatment in the popular media, and then trace its origin in conversations among scholars and researchers.
• Have students select a seminal work on a topic, and then identify sources that preceded and continued the conversation, analyzing the impact of the seminal work on the field.
I suggest that the misleading idea that IL can enable us to evaluate information without having disciplinary knowledge explains the gap between the vague or empty ideas of the Framework and these lofty goals.
When it comes to evaluating information about which we do not have disciplinary knowledge, we get thrown into the paradox of expertise. In the Charimides, Socrates asks how we can trust the medical advice of a doctor without being able to evaluate the expertise of said doctor. The problem is that it would seem we have to have at least as much expertise as the doctor in order to evaluate his or her reliability. So why rely on the doctor’s expertise in the first place?
I think the way we get around this paradox is in identifying two different levels of evaluation. At the first level, we evaluate the truth of a claim in light of the evidence for or against it and as you point out this requires disciplinary knowledge. But, at the second level, we evaluate not the truth or the falsity of a claim but, rather, how much credence we should put in it. How reliable is the source? Are there any nearby and relevant defeaters? Does trusting the claim require further inquiry?
I see IL as living at this second level of evaluation. I’m with you: I can’t help someone evaluate a specific claim without appealing to some level of disciplinary knowledge. But, I can help someone evaluate and look for common marks of trustworthiness. That Neandertal turd article? I can’t directly evaluate if it’s findings are true because I’m not an expert on gas chromatography and anthropology. But, I can evaluate its context to the extent that I am willing to trust the findings: peer-review, author credentials, methodology, place of publication, etc.
I think that there are a great many things that we are asked to believe despite not having disciplinary knowledge regarding them. From climate change to the hydro-geological effects of fracking to the impact of the ACA on the economy, there are a lot of things about which I possess only rudimentary disciplinary knowledge. But, I can still come to good reasons to accept certain disciplinary claims over others based on non-disciplinary markers. I think that’s what IL should get at.
Thoughts?
I agree. Much of the time we are teaching other non-experts those markers we ourselves rely upon without deep content knowledge. Let the “turd” experts debate the methodology and the veracity of the study. Our knowledge about the context of the information as it is presented is still very valuable and our perspective as non-experts can also be valuable as advocates for the students.
The fact that the article about Neanderthal turds made it past peer review does not tell us whether the claims made in the article are true. Same with credentials, or the place of publication.
If we are evaluating the claims about Neanderthal turds, presumably we want to know whether they are true.
The methodology, on the other hand, often is the kind of thing that a good researcher can check. Is this methodology standard is this field? Or are its results inconclusive? Is there disciplinary consensus? Can we find an authoritative textbook explanation of the methodology?
I completely agree that we have to shift to evaluation of the reliability of the method.
Or reliability of the testimonial source. But I don’t think that we should think of the authors of the Neanderthal turd study as merely giving testimony. They have reduced our reliance on the reliability of their testimony about the turds by applying a method that other experts can check.
About climate change, fracking, or economics, IL instructors can teach students to rely on the indirect clues of peer review, appearance in authoritative journals, or maybe slightly more informative: citation counts. We do that when that’s the best we can do.
But IL instructors can also go a bit further. I am not satisfied to teach students just to rely on authority. I would say that *solely* to teach reliance on authority is “authoritarian.”
IL instructors can also teach students how to research content, not just indirect clues or indicators of authority. This puts a greater burden on IL instructors to be able to find examples of claims for which good research methods can be applied to find the basis for these claims, and to make these learnable by non-experts.
One example I’ve used in class is the fact that archaea are phylogenetically distinct from bacteria, discovered in 1977 by Carl Woese.
The methodology was comparison of 16S ribosomal RNA, which then became standard in microbiology. I never studied biology, but I can establish that this method is standard. That’s just library research.
The neat thing is that students can use NCBI nucleotide databases to confirm Woese’s discovery.
Another example I’ve used in class is the fact that income inequality in the US is increasing. This fact can be checked using IRS data in the Statistical Abstract. In this case, there is reliance on the authority of the IRS. Did they lie to us?
But apart from the this assessment of authority, we also rely on the IRS’s expertise in handling the data. Federal agencies document what they do.
I’m reluctant to apply epistemology here. Presumably all claims in natural science can be defeated by some contradictory information. It is unclear what would count as a defeater in the context of scientific research, where there are a lot of mixed signals: defeaters are defeated by other information, etc.. Also, I’m reluctant to assume that reliability in the context of scientific research has to do with subjective credence. What are the prior probabilities?
But none this suggests skepticism. Maybe epistemology does not get down that far into the weeds.
I agree that the peer-review process doesn’t confer truth. However, my approach to teaching novices doesn’t hinge on truth. Rather I try to focus on warrant, and I do think that peer-review can be a source for warranted beliefs (albeit an imperfect one…aren’t they all). I have to think it over, but my gut instinct is that we can’t infer the truth of a claim without disciplinary knowledge.
But, we’ve still got to decide between competing beliefs and I think IL is a small step towards at least getting students to limit themselves to warranted beliefs.
And I’ll totally concede that we can’t focus solely on authority.
Still, I work primarily with first-year students who lack both disciplinary knowledge and discernment. Though I am limited in my abilities to help with the disciplinary part I think I can help with discernment.
Your Woese example is instructive: we don’t have to be full-blown microbiologists to decide whether to accept Woese’s work.
Finally, I think epistemology does have some value here, though I don’t get into particulars with students. As you can probably infer, I’m a reliabilist following Goldman, and I think students can handle some basic concepts from his work on social epistemology.
I’d continue, but the kids are begging for dinner.
Thanks for the comments/corrections/clarifications!
How much are we warranted in believing something because it has passed peer review?
In math journals, peer review involves checking the proofs. So in math, we do obtain a warrant from the fact that a result passed peer review.
But in other disciplines, peer review does not check for correctness. I suppose peer reviewed claims are more reliable than other claims. I suppose we’d test that by comparing the claims made in commercial publications with peer reviewed journals. Sports Illustrated might turn out to be more reliable than Science or Nature, because Sports Illustrated confines itself to making well-confirmed claims.
Peer review is a quality control mechanism. It just says that the claims being made in peer reviewed papers passed a test making them worth considering.
I’m very far from being a skeptic, however.
With respect to climate change, laypeople are warranted in believing the conclusions of the IPCC’s Assessment Reports because these are a scientific consensus, which is stronger reason than just passing peer review. Also, if we examine the IPCC Assessment Reports, we find reasoning and justification.
The fact that the claims passed a test making them worth considering is but one step to warrant isn’t it? I’m never going to give credence to a claim solely on the basis of its having passed peer-review, nor would I ever dismiss a claim because it did not pass peer-review, but peer-review, when present, should play into our epistemic probability calculus when evaluating scientific claims, shouldn’t it?
And I’m not sure that SI confines itself to easily verified claims. The boxscores sure. But the cover stories? The draft speculations? The team rankings? There’s an awful lot of speculation in SI. Whatever the case, the presence of quality control of any sort (and SI has an editorial team) should play a role in our evaluations.
And of course laypeople are warranted in believing the IPCC reports, despite their lacking peer-review. There are many roads to (epistemic) Damascus.
Basically, I agree with you.
Lane, I also agree with much what you say, especially about IL. The specifics of epistemology are very debatable. But applying epistemology is nevertheless helpful in getting clearer about IL.
[…] information about a topic and how that information might be accessed. In other words, “seek out conversations that are taking place in their area of research” and “identify which formats best meet particular information […]
Dear Lane
I am the senior librarian for information literacy at the University of the Western Cape, Cape Town, South Africa. I really enjoyed this discussion of the various IL Frames and just love the way you unpacked each one and critiqued it. I spent this year teaching our team of Faculty Librarians the threshold concepts and have also started engaging them in workshops. Our objective is to embed the frames in teaching and learning on campus and we will also be involving academic teaching staff in this endeavour. This week I shall be using this blog for discussion in our monthly reading club! Thanks!
Shehaamah:
Thanks for reading and sharing! Keep in mind that a new draft is coming out very soon with some substantive changes to the threshold concepts. Honestly, it wouldn’t surprise me if yet another draft is issued in a few months. I’ll publish some more commentary in the near future.
Thanks for reading!
[…] https://senseandreference.wordpress.com/2014/07/10/is-scholarship-a-conversation/ […]
I’m not aware of ALL the conversation here 🙂 however my simple take on the idea of scholarship being a conversation (in the context of also being a threshold and potentially at some time troubling) is that one piece of scholarship (say an article) is not everything; it’s just part of the conversation. article 1 says: Steam engines are very powerful; (but that’s not everything) article 2 says: (maybe) steam engines are historical; (but that’s not everything) article 3 says: (maybe) steam engine technology had some relevance for the design of steam turbines, that are still in use today–I’m making all this up 🙂 and the overall conversation, and whatever else is said in the conversation, is the scholarship. So, the “find” of article 1 (for the new student) has not yet given the sense of the conversation. I may not be conveying the main idea of how scholarship is a conversation. I think the surprising thing is that the “find” is not the end of getting information.
[…] Scholarship is a Conversation […]
[…] Scholarship is a Conversation […]
[…] Scholarship is a Conversation […]
[…] Here’s what I originally had to work with: […]
[…] Scholarship is a Conversation […]