Interview with Jürgen Schmidhuber

Schmidhuber is widely recognized as one of the early trailblazers of deep learning and neural network research.

An AI data center that is worth $500 billion today will only be worth $50 billion in five years, and $5 billion in 10 years. Soon, smaller, cheaper computers with increasingly efficient open-source AI software will do what the large data centers do today. So AI will not be controlled by a few big AI utilities. Everyone will own cheap but powerful and transparent AI that improves their lives in many ways.

2 Likes

It would be cool (if a little frightening) if this comes to be. I always thought Cowboy Bebop would be a more fun cyberpunk universe to live in than umm, probably 1984, I, Robot, or some other dystopian society ruled by corporations, but I haven’t read any of those books because they sound kind of depressing…

So AI will not be controlled by a few big AI utilities. Everyone will own cheap but powerful and transparent AI that improves their lives in many ways.

I’m sure the same thing was said about mainframes and microcomputers back in the day, and while that did come true, corporations regained power again with the very mainframe-like “Cloud”.

That being said, given that LLM tech seems to be plateauing[1], it seems like the only big breakthroughs left are figuring out how to shrink the most capable tech down into local models—which already exist today, but aren’t quite as good as the huge models—and figuring out how to reduce the environmental footprint.

Unsolicited AI commentaryI'm somewhat skeptical about generative AI use cases. I've mainly found them useful for generating code (which usually needs to be fixed). I do a lot of writing for fun, but I would never use generative AI for it because that takes a lot of the fun out of it. I don't even use a grammar checker because they're distracting, and I didn't use a spellchecker for a few years. The latter was a clear mistake, though it was mostly out of laziness as I hadn't figured out I could include `set spell spelllang=en_us` in my `init.vim`...

That being said, when editing professionally, I’ve been known to rely on generative AI for alternative ways to phrase a very awkward passage someone else has written. I rarely use the suggestions as-is, but it gets me thinking. What is the author really trying to say, and how can I help them say it eloquently? The LLM rarely “understands”, but it helps to have more ideas.

Japanese➜English machine translation tools occasionally come up with almost passable translations. I wouldn’t say LLMs have improved these tools’ accuracy or rendering much from the little experience I’ve had with them, but I was never a big user of machine translation tools in the first place as I’ve put years into understanding the language to a degree that an English rendering rarely matches.

So, I’ve given LLMs a go and I kind of get it, but I just can’t get excited about them. When I think of a future with AI, all I can think of is WALL-E.

And yes, I need to make “vibe-based” judgements using popular movies as metaphors as I don’t understand the science behind LLMs.

I did really like most of the Terminator movies aside from Genisys—the most recent one was actually pretty good!


A totally unrelated rant about a use of AI that depressed me The most depressing thing to me was reading fanfiction that I became certain several paragraphs in was mostly AI-generated, but the author gave no indication that it was. Fanfiction is something people have always written for the fun of it, or to scratch an itch, and to read something the author was trying to pass of as their own is...I dunno, demoralising?

And then I kept seeing it. For the first time, I used the block function for a few authors on AO3.

I’ve read a lot of badly-written fanfiction; one of my favourites has bad grammar. But I’ve built a tolerance for that because the story is worthwhile. The only things that make me nope out immediately are failing to add commas before dialogue tags and a glut of spelling errors.

This strange breed of new fics have good grammar and no spelling mistakes, but they’re so damn rambly and the prose is needlessly purple. They just aren’t fun to read. They feel cheap.

Okay, sorry, I needed to rant about that. People are free to use whatever tools they want to write whatever they want, and some of my favourite authors readily admit to using those tools to help them (and it’s clear that it’s help, not a substitute). I really wish everybody would admit to it, because it makes me feel cheated, at least, when I figure it out.

I’m actually surprised I don’t see more of it.


  1. [citation needed], but I haven’t seen any big improvements lately. I’m even more skeptical of agents, having tried it out. At least ChatGPT immediately had use cases when I saw it for the first time. ↩︎

In response to your comments, I had a recent positive experience with machine learning where Copilot helped me decipher a Git string I was unfamiliar with.

I mostly read nonfiction. In general, there is a lot of crap writing. And, I include my writing in that. But, if I can come back to something a year later and both read it without trouble and understand why I thought it was important, then I’m satisfied with what I wrote.

I’ve had positive experiences with ChatGPT in trying to get layouts to work properly; in combination with StackOverflow, it works fairly well in most cases. Perhaps if I were a more competent web designer, this would be less true :slight_smile:

That seems like a good rule to live by. There is certainly writing of my own I can’t say the same thing for…

1 Like