ChatGPT reliance considered harmful


As Socrates famously said in his arguments against the (then novel) invention of writing:

[the craft of writing will] create forgetfulness in the learners’ souls, because they will not use their memories; they will trust to the external written characters and not remember of themselves. The specific which you have discovered is an aid not to memory, but to reminiscence, and you give your disciples not truth, but only the semblance of truth; they will be hearers of many things and will have learned nothing; they will appear to be omniscient and will generally know nothing; they will be tiresome company, having the show of wisdom without the reality.

I have tested Chat-GPT extensively and find it mostly useless. These are the things I think it is good at:

  1. Generating interesting Book Titles

It can give you some great ideas to start with, but as you say, they tend not to be interesting. At the moment, I think it’s quite awful as a search engine replacement. It struggles to answer >90% of my questions with factually correct answers.

Chat-GPT is good at fabrication, but only particular types of fabrication. It’s bad at writing essays and the like due to its tendency to ramble and not expand on ideas properly. I tried to use it for fiction and it invariably wrote itself into glaring plot holes. These fields are too complex for Chat-GPT to get right.

This is why the book titles task is great for Chat-GPT. It’s basic creative work, but tiresome and laborious.


Beautifully put @Seirdy, thanks for sharing.

I do think chabots can sometimes be useful as a thinking tool. I have found that using them (and I’ve tried quite a few) as an always available sounding board is sometimes useful. It can help me consider topics from different angles in a faster way. In that respect, for certain avatars, is better than using ChatGPT.


Echo @Colin here, in particular:

Use it to think, and your ideas will be disposable.

Also seeing a syndicated to… link point right here which is much appreciated.

Sounds like the intro to a walks into a bar joke but I was sat with a doctor, a neuroscience PhD, and a writer recently and decided to whack out the tool to show them. They very much found that:

Unfortunately, it’s often convincingly incorrect

This in particular perturbed the first two.