Machine-learning algorithms that generate fluent language from vast amounts of text could change how science is done — but not necessarily for the better, says SP, a specialist in the governance of emerging technologies at the University of Michigan in Ann Arbor
In a report published on 27 April, SP and other researchers try to anticipate societal impacts of emerging artificial-intelligence (AI) technologies called large language models These can churn out astonishingly convincing prose, translate between languages, answer questions and even produce code to the corporations building them — including Google, Facebook and Microsoft — aim to use them in chatbots and search engines, and to summarize documents. (At least one firm, Ought, in San Francisco, California, is trialing in research; it is building a tool called 'Elicit' to answer questions using the scientific literature)
They are already controversial They sometimes parrot errors or problematic stereotypes in the millions or billions of documents they’re trained on And researchers worry that streams of apparently authoritative computer-generated language that’s indistinguishable from human writing could cause distrust
Parthasarathy says that although they could strengthen efforts to understand complex research, they could also deepen public skepticism of science She spoke to Nature about the report
How might they help or hinder science?
I had originally thought that they could have democratizing and empowering impacts When it comes to science, they could empower people to quickly pull insights out of information: by querying disease symptoms, for example, or generating summaries of technical topics
But the algorithmic summaries could make errors, include outdated information or remove nuance and uncertainty, without users appreciating this If anyone can use they to make complex research comprehensible, but they risk getting a simplified, idealized view of science that’s at odds with the messy reality, that could threaten professionalism and authority It might also exacerbate problems of public trust in science And people’s interactions with these tools will be very individualized, with each user getting their own generated information
Is not the issue that they might draw on outdated or unreliable research a huge problem?
Yes, But that doesn’t mean people won’t use that They’re enticing, and they will have a veneer of objectivity associated with their fluent output and their portrayal as exciting new technologies The fact that they have limits — that they might be built on partial or historical data sets — might not be recognized by the average user
It's easy for scientists to assert that they are smart and realize that they are useful but incomplete tools — for starting a literature review, say Still, these kinds of tool could narrow their field of vision, and it might be hard to recognize when and they get something wrong
They could be useful in digital humanities, for instance: to summarize what a historical text says about a particular topic, But these models’ processes are opaque, and they don’t provide sources alongside their outputs, so researchers will need to think carefully about how they’re going to use them I’ve seen some proposed usages in sociology and been surprised by how credulous some scholars have been
You must be logged in to post a comment.