Research findings published in July 2023 have revealed that participants using ChatGPT were able to write business documents faster, at a better quality. The improvements were made on the basis that a person needs to check the AI-generated text, then edits and corrects it, according to Jakob Nielsen, international expert of user experience (UX) techniques.
The US researchers, Noy & Zhang, found that ChatGPT substantially raised productivity: The average time taken decreased by 40% and output quality rose by 18%. Inequality between workers decreased, and concern and excitement about AI temporarily rose. Workers exposed to ChatGPT during the experiment were twice as likely to report using it in their real job 2 weeks after the experiment and 1.6 times as likely 2 months after the experiment.
(GPT is an acronym meaning Generative Pre-trained Transformer. (I’m sure that clarifies everything!) GPT uses deep (machine) learning to produce human-like text once it’s provided with some initial text as a prompt. Put simply, generative AI can create new content, text, audio, video, images, and even code.)
This was one of three studies:
- Customer support
- Business professionals writing business documents
The three studies were very different, but arrived at the same conclusions, which is very impressive, according to Nielsen.
Writing business documents
A total of 453 college-educated business professionals engaged in mid-level professional writing tasks. The writers included marketers, grant applicants, data analysts, HR professionals, and managers. They were assigned to write two business documents within their field. Examples included press releases, short reports, analysis writing, responding to customer service requests, rough drafts of emails, and editing, which were understood to represent the type of work in which they were usually engaged.
Most of the participants were using ChatGPT for the first time. Around 30% had used it previously.
Participants were asked how they had allocated their time across three phases of the writing activity: brainstorming (ideation/creativity), writing a rough draft, and polishing their text. A review revealed that they spent less time on writing the initial draft text and more time editing to reach the final result. The conclusion is that ChatGPT’s main contribution was to save users substantial time on the production of draft text.
Bar chart shows the average time spent on the three stages of writing a document:
- deciding what to do (called “brainstorming” by the researchers)
- generating the raw text for the first draft
- editing this draft to produce the final polished deliverable.
The top bar shows the average times spent by users who employed ChatGPT, whereas the bottom bar shows the average times for users who wrote their document the normal way, without AI assistance. The difference between the two “brainstorming” time estimates is not statistically significant.
This chart is based on recalculated data from Noy and Zhang (2023).
Jakob Nielsen, who analysed the key differences between the two conditions (no AI or with AI), said that participants without AI assistance spent much more time generating their initial draft text. This left them relatively little time for editing. In contrast, the participants who received help from ChatGPT were very fast in generating the draft text because most of that work was done by ChatGPT. Faster text entry allowed the AI-using participants the luxury of substantially more thorough editing. This additional editing is likely the explanation for the higher-rated quality of the documents produced by these users.
AI will not replace humans. The best results come when the AI and the person work together, for example, by expediting the production of draft text, leaving human professionals to focus on editing and polishing.
He observed that one of the exciting findings from this research is that generative AI also has the effect of narrowing the gap between the worst and the best performers.
10 pros and 10 cons of ChatGPT for PR pros
Overall, how would PR pros best use AI? Here are some of the positives and negatives of using ChatGPT for PR:
10 great uses of AI-assisted writing (assuming proper human prompts)
- First drafts of documents
- Outlines for meetings or documents
- Text summaries (of long-form content)
- Potential headlines
- Topic ideas
- Break writer’s block
- Interview preparation
- Transcribe audio
- Explaining complicated subjects or processes
10 downsides of AI-assisted writing
- Repetitive or vague
- Not up-to-date
- Made-up ‘facts’
- No source references
- Potential plagiarism
- Lacks human perspective, critical thinking
- Still requires human editing
- Lack of personalization or perspective (for brands)
- Can be just as biased (or more) than humans
- Google may consider it spam or de-rank it