ChatGPT for Research Writing: Game Changer or Ethical Risk?

Artificial Intelligence (AI) has become a critical part of our lives and the rapid development of AI-powered tools has helped to automate and simplify some challenging and tedious tasks today. This is true even in the world of science and research. For researchers, AI writing tools like ChatGPT have emerged as a viable option to better academic writing. A medRxiv preprint that evaluates ChatGPT’s performance and has ChatGPT as an author says the OpenAI tool displayed a “high level of concordance and insight in its explanations,”¹ suggesting the key role large language models can play in research writing, education, and even decision-making.

As the language tool gains global attention, we explore how ChatGPT can help researchers in the academic writing process, delving into the limitations and ethical issues surrounding the use of such AI tools to write research papers.

What is ChatGPT?

ChatGPT is a pre-trained language model developed by OpenAI. It uses a variant of the Transformer architecture, which was introduced in the paper “Attention Is All You Need” by Vaswani et al. The model is trained on a large dataset of human-generated text and is fine-tuned for specific tasks such as question answering, language translation, and text completion. ChatGPT is capable of understanding and generating human-like text, making it useful for a wide range of natural language processing tasks. It can be fine-tuned to different tasks and is available via OpenAI’s API and in various open-source frameworks.

Launched on November 30, 2022, ChatGPT is taking the world by storm with its fascinating ability to understand key user inputs and generate human-like responses as conversational dialog. See for instance the detailed answer it provided above in response to the question “What is ChatGPT?”

Apart from ChatGPT, OpenAI, an AI firm based in San Francisco, California, is also known for creating DALL-E, an AI tool that gained fame for being able to turn text into images.

How does ChatGPT work?²

Large language models are designed to accurately predict the next word in a sentence, which allows them to write entire pages of content. These AI models are neural networks trained on massive amounts of human-generated text to learn how to perform any tasks. ChatGPT, for instance, is trained on 570 gigabytes of text and has 175 billion parameters (its predecessor GPT-2 had only 1.5 billion parameters), which helps it perform tasks it was not explicitly trained for. Apart from being the largest language model ever trained, ChatGPT comes with additional Reinforcement Learning with Human Feedback (RLHF), which gives it a clear advantage over earlier tools. It is no wonder then that this is being seen as a breakthrough model for content generation.

The OpenAI chatbot is currently in a “research preview” phase where users can try it for free and share feedback that allows it to improve how it answers questions. People the world over have been using ChatGPT extensively over the past 45 days or so and the internet is flooded with talk about how well or poorly the AI tool has performed to specific cues. Some are already comparing it to Google.

Why researchers are turning to ChatGPT

Students and researchers have been increasingly turning to AI tools to help make their academic writing easier and more efficient. OpenAI’s ChatGPT, which delivers intelligent-sounding and well-written text much like a real human, has emerged as a powerful tool to help researchers, especially non-native English speakers, with the academic writing process.

Here’s a quick look at some ways researchers can use ChatGPT.⁴

1. Creating a research paper outline

ChatGPT is a great assistive tool for researchers and can produce a well-structured outline or an impressive title for your PhD thesis or research paper based on user inputs. Researchers can evaluate this suggested outline and modify it as needed before they start writing.

2. Writing research paper abstracts

Researchers often find it difficult to showcase their work effectively in the form of a well-rounded abstract. This is another area where ChatGPT proves useful – in fact, in a recent study, abstracts generated by ChatGPT managed to fool scientists and even passed key plagiarism and AI output detection checks.⁵

3. Translating text in one language to another

ChatGPT is great for natural language processing tasks like generating translations of text in one language to another. Researchers may need to evaluate its performance against human-generated translations to ensure complete accuracy.

4. Rewriting difficult passages of text

ChatGPT is also a good option for researchers, especially non-native English speakers, to rewrite translated text and generate definitions or analogies for common concepts. Researchers struggling to convey complex ideas in a clear, succinct manner can also use ChatGPT to turn existing text into well-written content that retains the original message and is free of language and grammatical errors.

5. Summarizing long articles or documents

The OpenAI chatbot helps busy researchers with their literature search and reading by generating brief summaries of full-text articles and long documents. These short, concise overviews help researchers quickly grasp ideas and identify research papers that may be relevant to their work.

6. Suggesting variations for experimental design

Language models like ChatGPT can help with developing experimental design, offer interesting variations based on an inputted description or provide a new perspective on established experimental protocols.

7. Training for natural language processing tasks

ChatGPT is a versatile language model that can be fine-tuned and customized for specific tasks such as training models for understanding language, text generation, dialogue systems, data augmentation, and pre-training.

Limitations of using ChatGPT for research writing

While ChatGPT is a powerful tool for research writing, there are some limitations researchers need to consider when using it.

1. ChatGPT is incapable of generating original ideas, it only creates text based on the patterns it sees in training data. This raises the risk of plagiarism as the AI-generated text may not include references or citations that is critical for research writing.
2. This chatbot is a statistical model generating text based on patterns in the training data and is only as good as the prompts it receives. Since it cannot understand the meaning of the content, this could mean that it occasionally suggests plausible sounding but incorrect or nonsensical answers.

3. The OpenAI chatbot may not always represent the specific domain of the research, which can lead to inaccuracies or inconsistencies in the research content generated. This is a problem for researchers, who need to generate text that requires specialized knowledge in a specific field.
4. ChatGPT may occasionally generate text that is biased or offensive if the training data contains such biases. Hence, it is important to be aware of this limitation and take the required steps to eliminate this when using the model for research writing.
5. Finally, ChatGPT has limited knowledge of the world and global events that occurred after 2021. It is important to remember that any content generated by this AI chatbot needs in-depth checks and editing to ensure it meets key submissions requirements.

“What we have today is essentially a mouth without a brain,” says Yejin Choi, a computer scientist at the University of Washington and the Allen Institute for Artificial Intelligence.⁷ Figuring out how to teach large language models causal reasoning, common sense, or moral judgement remains a major challenge for researchers.

Ethical concerns surrounding the use of ChatGPT for academic writing

Academic writing is a critical part of the research process and requires careful consideration and attention to detail. ChatGPT can deliver prompt-based text in a few seconds, which can save time and boost productivity, but it is important to consider the ethical implications of using AI tools to do your research writing for you.

Risk of biases seeping into the writing

While additional safety mitigations have been put into place,⁸ ChatGPT is trained on existing datasets, and if these contain negative information or biases, it could seep into the chatbot’s results. This is a problem for researchers who need to be aware of potential prejudices and check the answers or text provided to ensure they are accurate and unbiased.

Potential of content being plagiarized

AI tools can help researchers quickly create a draft of a paper, but they may also be used to quickly copying text from sources without giving them the proper credit. As researchers, it is important to be mindful of this potential for plagiarism and take the necessary steps to ensure that all sources are properly cited and credited.

Inclusion of offensive or harmful text

A related concern is the potential for offensive or harmful text to be generated by ChatGPT. If the training dataset contains any offensive or harmful language, the OpenAI chatbot may reproduce that language in the text it generates. This can be a problem for researchers who are responsible for ensuring the text is respectful and inclusive.

Concerns about quality of academic text

ChatGPT can automate academic writing, and even generate citations and references, but it doesn’t guarantee reliable, quality research. Academics must be mindful of this deficiency and carefully edit the automated text to make sure it is reliable, replicable, scientifically accurate, and meets all the stringent quality standards required for submission.

In conclusion, the use of AI tools like ChatGPT for writing research papers should be done responsibly and ethically. Researchers must make the best decisions for their research by understanding that these tools are meant to enhance the researcher’s capabilities, not replace them altogether. Much like Pedro Domingos, author of The Master Algorithm, said, “It’s not man versus machine; it’s man with machine versus man without. Data and intuition are like horse and rider, and you don’t try to outrun a horse; you ride it.”

References:

1. Kung TH, Cheatham M, ChatGPT, et al. Performance of ChatGPT on USMLE: Potential for AI-Assisted Medical Education Using Large Language Models. medRxiv 2022. https://www.medrxiv.org/content/10.1101/2022.12.19.22283643v2
2. Montti R. What is ChatGPT And How Can You Use It? Search Engine Journal – Digital Marketing, December 2022. https://www.searchenginejournal.com/what-is-chatgpt/473664/
3. Angela Yu on Twitter. https://twitter.com/yu_angela/status/1599808692085743616
4. Hutson, M. Could AI help you to write your next paper? Nature, Technology Feature, October 2022. https://www.nature.com/articles/d41586-022-03479-w
5. Else, H. Abstracts written by ChatGPT fool scientists. Nature, News, January 2023. https://www.nature.com/articles/d41586-023-00056-7
6. Fiora Aeterna on Twitter. https://twitter.com/FioraAeterna/status/1599760552686862336
7. Hutson, M. Robo-writers: the rise and risks of language-generating AI. Nature, News, March 2021. https://www.nature.com/articles/d41586-021-00530-0#ref-CR2
8. ChatGPT: Optimizing Language Models for Dialogue. OpenAI, November 2022. https://openai.com/blog/chatgpt/
9. Steven T. Piantadosi on Twitter. https://twitter.com/spiantado/status/1599462375887114240

Editage All Access is a subscription-based platform that unifies the best AI tools and services designed to speed up, simplify, and streamline every step of a researcher’s journey. The Editage All Access Pack is a one-of-a-kind subscription that unlocks full access to an AI writing assistant, literature recommender, journal finder, scientific illustration tool, and exclusive discounts on professional publication services from Editage. 

Based on 22+ years of experience in academia, Editage All Access empowers researchers to put their best research forward and move closer to success. Explore our top AI Tools pack, AI Tools + Publication Services pack, or Build Your Own Plan. Find everything a researcher needs to succeed, all in one place – Get All Access now starting at just $14 a month!