Thank you for your questions and responses. There’s a lot of confusion about what AI is, so first some context before addressing points raised.
Much of the confusion is fueled by media, blitzscalers and Big Tech, which often present Generative AI as either a panacea or a threat. People here are aware of this hype, but to keep things grounded, here’s a brief overview.
Today, “AI” usually refers to Generative AI, especially for text prediction. This means generating text by predicting the next token — a task performed by Large Language Models (LLMs) using transformer-based decoder architectures, like OpenAI’s GPT. These are the basis for today’s chatbots.
Earlier transformer models, like Google’s BERT, used encoder-only designs and are not used for text generation, but for tasks like classification.
Generative LLMs are known to confidently output incorrect information — so-called hallucinations. This is an inevitable byproduct of the decoder design. Reducing this remains an active area of research.
Finally, LLMs are just one part of the broader AI landscape, which was more accurately called machine learning before 2017 — a term coined in 1959, reflecting decades of diverse models and techniques.
At Mojeek it is NOT our mission to generate information.
Our mission to provide links to sources of information. We do use machine learning where it helps to do this, as explained below.
With reference to posts above:
We did not change the core algorithm of Mojeek, but we did include a semantic scoring in the overall scoring used for ranking results. This improved the relevance of search results, for what is still a keyword based search engine. For the semantic scoring we use a local encoder-only model similar to BERT. That is not AI in modern parlance.
These use our own internal scoring algorithm used for ranking. So no AI in modern parlance is used.
We do believe in the value of “good-old classic search results” and unlike everyone else it seems, we are not aiming to be an answer engine. Where resource light, preferably local, machine learning models and techniques can help us be a better search engine we use them, and would be foolish not to do so.
We use generative AI, in two experimental cases, as follows
The first is the Summary feature which:
- Is a feature specifically designed to assist users in more efficiently locating the search results to click through too.
- Is designed to distill information from only the search results.
- Is off by default
- Has a button which can be removed from the display and a tab which can be removed (in the preferences menu)
The second is the Related Queries feature which are testing but have not rolled out.
As for the rest of Mojeek which is of course a vast array of IP built up in-house for 20 years, we do use some machine learning techniques and models. And of course we do develop our own algorithms. These models/agorithms are all run locally and consume low amounts of power.
“No Tracking. Just Search” is mission statement not a cliche.