There are plenty of reasons to be wary of the AI tidal wave: the ethical use of copyrighted information, bias in search results and generated text, and false information are a few of them. The environmental impact of AI tech, the exploitation of workers, job loss, and privacy concerns are a few more (see educator Leon Furze for an excellent outline). And yet…it’s here! I don’t know about you, but when I first tried chatting with ChatGPT over a year ago now, I had a goosebumps feeling that it would be part of my life going forward. As an editor and writing consultant, my goal is to understand how scholarly publishing is responding to AI, build my own critical AI literacy, and support clients to do the same.
In that spirit of critical literacy, I present a research tool I recently encountered, Consensus. I found it, like other AI tools, both really cool and troubling.
Above: Consensus returned this response to my question: “Is climate change making cherry trees bloom earlier?” Below this header, it provided AI-generated snapshots of each study. Alt text: The Summary on the left reads “These studies suggest that climate change, particularly global warming and urbanization, is causing cherry trees to bloom earlier, with projections indicating this trend will continue and potentially impact cultivation.” The Consensus Meter on the right shows Yes, Possibly, and No bars with Yes at 91%, Possibly at 9% ,and No at 0%.
An Answer Engine
Consensus is an “answer engine,” meaning it combines an AI chatbot with a search engine to give you both a generated text response and citations with links to sources. Consensus is built over Semantic Scholar, a nonprofit database of over 200 million academic articles founded in 2015. Ask Consensus your research question, and it searches Semantic Scholar to generate a three-part response: a one or two-sentence summary answer, a graph of the findings from top papers on a “Consensus Meter” (Yes, Possibly, and No), and AI-generated snapshots of each study.
Consensus’ vision is “to make expert knowledge accessible and consumable for all.” Scholars and practitioners who suffer from information overload are using Consensus and similar tools to keep up-to-date with new research. I’d use it if I’m just starting to research a new topic and am looking to get a temperature check on debate.
Pros and Cons
Encouragingly, Consensus’s search doesn’t use generative AI; rather, its AI function extracts data from studies, so it won’t hallucinate false information. However, Consensus does use ChatGPT-4, generative AI, to create the synthesis of the studies and one-sentence snapshots of each article. Potentially, that generated text could misrepresent the research.
Other grains of salt: An answer engine like Consensus is not meant to replace searching discipline-specific databases during an in-depth research project or systematic review. And it doesn’t offer access to the full text of journal articles behind paywalls; you still need your library for that. At the same time, by generating snapshots of studies from the database, these tools make more information available in a more-digestible way as compared to Google Scholar or traditional databases.
I’m intrigued by the effort by both Semantic Scholar and Consensus to “promote equal access to science” by using AI to power research and save scholars time. But, as with other AI writing and research tools, I’m troubled by the potential they have to make the process more opaque. I’ll be reading more about how developers at Semantic Scholar and elsewhere are working to reduce bias in search results and AI-generated text. If you try Consensus before my next AI update, let me know what you think.
Disclosure: I’ve adopted the strategy of disclosing if and how I use AI in the writing process. Many journals now require that disclosure, and I do it in order to build trust with readers. For this piece, I copied an earlier draft into the free version of ChatGPT and asked it to report any inaccuracies. I made a couple edits after searching for more resources based on its feedback. I didn’t incorporate any generated text into the article.