@mike Feedback

[Colin]

@mike

Index Quality

As part of my experience, I noticed that Focus draws my attention to existing problems. For example, if I search The Worldfact Book at cia.gov, some of the links are out of date because the Factbook was moved to a new location. Or if I search docs.microsoft.com, I get results in languages I can’t read. Similar experiences might give others a bad impression of Mojeek.

I know that the search team is looking at some particular issues that there might be with the indexing of The Worldfact Book. What would help here is some specific examples, if you can share; email me on colin@mojeek.com if you prefer. We are working on language detection at present so this will improve in future. For now you can use &lr=EN and that should help; let us know if you don’t mind.

@mike

I think the general problem is that by searching a fraction of the index, Focus is using a fraction of the indexing speed. In my case, I’m including sites which means I’m looking at a very small fraction of the index. And I’m getting a bad experience because there is not enough overall indexing speed to keep the fraction of the Internet that I’m looking at with Focus up-to-date.

Yes, a fraction of a fraction of the Web, is an issue we are only too aware of with Focus. Indexing speed (better stated as frequency) is, of course, a fundamental issue in crawling/indexing. As it happens we are looking at improved intelligence for handling that. And we have new servers on order, but still this will be remain a challenge.

On the topic of infrastructure, we are doing a great job with our own current small-scale infrastructure but improvements can be made. Most folks assume we need a bigv data centre to index >5 billion search engine. I would love to know the fraction that Google/Bing use for indexing compared with the total resources used for all their search engine ecosystem; data collected, hungry ML models, ad tech.

[Colin]

@mike

Collection Policy

When I search with Focus, I am sending parameters foc={name} which can identify me personally. Will there be a section in the privacy policy which specifically addresses Focus?

Yes, on general release. Great point. We’ll consider what might be done.

I think I’ve now covered all your feedback in this thread. Let us know if we missed something. Many thanks - this is just the feedback we hoped for.

[mike]

@Colin

@mike

Focus engines seem similar to custom search engines in Chromium-based browsers. I think one advantage to Focus is that it can bring custom searches to Firefox and mobile.

Could you expand on what you refer too. What do Chromium-based browsers have that Firefox and mobile don’t have?

In Chrome, you can right-click on a search field and add it as a search engine.

You can then go into your settings and edit the search engine. For example, I might add the terms below so that if I enter the name of a country, I am sent to that country’s entry in the World Factbook.

site:https://www.cia.gov/the-world-factbook/countries/ !ducky

This process lets me add search engines and customize them in any Chromium-based browser.

With regard to customization, Focus is similar to Chrome because I can add terms to my search and reuse those searches later. Right now, I can add the equivalent of site: to Focus engines. But, hopefully, there will be more customization available in the future.

In my case, I use Firefox and Vivaldi on the desktop. And I use Safari on an iPad.

All three of these behave differently.

Vivaldi is the most customizable. I can create custom search engines like my example above.

Firefox is less flexible because a site has to publish an OpenSearch plugin before I can add it as a search engine. And those engines can’t be customized.

Mobile is the least flexible because I have a fixed set of search engines to choose from. And I can’t add to those. For example, if I want to search using Mojeek, I have to visit mojeek.com.

Ultimately, Focus does not let me do anything I could not do before. With enough typing and bookmarking, I can do all of my customized searches everywhere.

What I thought was potentially useful about Focus was that I had the same interface across browsers and platforms. And Focus makes it easy to customize searches for Mojeek. For example, this is the first time that I’ve had customized searches in Firefox.

The potential benefits of Focus make sense to me because I’m using three different browsers in three different ways. I can imagine that if you just use Google and don’t do any advanced searching then you won’t understand or care about this view.

---

So the answer to your question is that Chromium-based browsers allow their searches to be customized similar to what Focus can do now. And, if you view Focus as a unified interface, then Focus adds customized searches to Firefox and mobile. That is how I arrived at my comment.

Let me know if this makes sense.

[mike]

@Colin

@mike

Backup

I think the average person will be confused by the fact that Focus engines are excluded from any cloud synchronization. When people ask about this, it would be helpful to have a one-click backup and one-click import process ready as a counterpoint. They should also know that Mojeek chose this process to preserve their privacy.

At this stage, the average person is not going to use Mojeek but I guess you mean that the “Backup All” functionality as it is, won’t be intuitive as it requires a) one-click, b) copy (somewhere) c) paste later (from the somewhere). I’ll leave it to the more technical folks to consider how we can provide it more intuitively whilst still privacy-preserving. Thanks for researching this and sharing these findings:

I was thinking about a group where they synchronize their computer with the cloud, lose their computer to some misfortune, recover their bookmarks, recover their passwords, but lose their Focus engines because Focus engines fall outside the scope of the backup. I’m guessing that cookies are probably one of the things not being backed up by Apple, Google, and Microsoft. And anyone affected might be confused and inconvenienced.

I think backup should be something addressed explicitly because it could be an issue. But awareness can help a lot. I don’t see the harm in explaining the issue if someone goes looking for that information.

[mike]

@Colin

@mike

It would be useful if I could include a region &reg=us with an engine.

You can put &reg=us in the URL and it will persist, at least whilst you keep the current browser tab open. But I’m guessing you know that. Can you be more specific as to how it doesn’t work?

I simply made the association that if the equivalent of site: could be associated with a Focus engine then region could be an additional parameter saved in the future.

Off the top of my head, this could affect researchers.

In my case, I got the idea because the results from docs.microsoft.com come back in various languages. I think I mentioned that elsewhere. Region does not affect those results anyway.

Finally, as a practical matter, I search in new tabs. So it sounds like region would not persist as it exists today.

---

@Colin

@mike

It would be helpful if I could save an entire search with a variable and have my search term replace the variable. For example, boilerplate %s site:example.com and my search term replaces %s.

Don’t quite understand, sorry. What is boilerplate? Is this a Focus thing, or for full web search too?

If the point of Focus is to save and reuse search parameters later, then I’d like to try adding one of my customized Chromium searches once Focus supports it.

As a working example, here is a search I can do from duckduckgo.com.

the united states site:https://www.cia.gov/the-world-factbook/countries/

Then, if I wanted to save and reuse that search later, I can progressively modify it.

# Search for the US in The World Factbook. 
the united states site:https://www.cia.gov/the-world-factbook/countries/

# Replace my search terms with the placeholder %s
%s site:https://www.cia.gov/the-world-factbook/countries/

# Send me immediately to the first result. 
%s site:https://www.cia.gov/the-world-factbook/countries/ !ducky

Here I use %s which is the placeholder or variable used in search engines saved in Google Chrome. This allows you to take any search URL, replace the search terms (the united states) with a variable (%s), and reuse the search later.

If I properly encode the last iteration and save it as a search engine in Chrome, the %s is replaced by any terms I type. And the search is performed.

Here is the search URL I extracted from Vivaldi.

https://duckduckgo.com/?q=%s+site%3Ahttps%3A%2F%2Fwww.cia.gov%2Fthe-world-factbook%2Fcountries%2F+!ducky

Now that I have this custom engine in Vivaldi, I can enter the following in my address bar instead of visiting DuckDuckGo. It will perform the same search as above. And, if I choose a different country, I will be sent directly to that country’s Factbook page instead. Here, cia is the keyword associated with the search.

cia the united states

I find customized searches like this to be a useful feature in Chrome.

My suggestion is for Focus to eventually support arbitrary search operators. I just used boilerplate as a catchall to mean anything I might want to include in a saved search.

At this point, it is not clear how far the company wants to take Focus in terms of what operators or static search terms will be supported in the future.

---

@Colin

@mike

Browser Integration

There does not seem to be a good way to integrate Focus engines with the search function in Web browsers. I’m not sure if OpenSearch plugins would be a good solution for this. But there is a way to update OpenSearch plugins.

I would assume issues/opportuntiies here are no different for Focus from Web search, but I might be wrong. We’ll consider.

The issue I perceived here was that Focus engines for individual users will be changing frequently. I believe OpenSearch plugins are relatively static.

And if you think through the process of creating a search plugin on the fly for a Focus engine and then having to void that out and replace it, I don’t think anyone thought of that use case before. It seems like it would be a mess. That’s why I thought there would be no good way to integrate Focus with the Web browser.

---

@Colin

I know that the search team is looking at some particular issues that there might be with the indexing of The Worldfact Book. What would help here is some specific examples, if you can share; email me on colin@mojeek.com if you prefer. We are working on language detection at present so this will improve in future. For now you can use &lr=EN and that should help; let us know if you don’t mind.

I will try these out with the issues I’m having and report back later.

---

[itsMe]

@mike

Firefox is less flexible because a site has to publish an OpenSearch plugin before I can add it as a search engine. And those engines can’t be customized.

i’m going kind of off-topic here, and technically, by default, that’s essentially true, but pretty much any search engine can be added and edited in Firefox with extensions such as Search Engines Helper or mozlz4-edit - things could be allot easier if Mozilla wanted, but i think they make it difficult because they don’t want people fooling with their revenue stream

[mike]

Examples of Index Quality

PowerShell

I frequently reference the syntax and usage for PowerShell cmdlets at microsoft.com.

I set up this Focus Engine for PowerShell Docs.

{"PwshDocs":"i=docs.microsoft.com"}

Below is an example of a page I might visit. This one is for the Get-ChildItem cmdlet.

This is what I would expect to see as the first result. This is what will help me while I am working on a scripting problem. This is the official documentation for a commercial software product.

https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.management/get-childitem?view=powershell-7.2

If I search for “get-childitem” using my Focus Engine, here are the top three results.

powershell - Get-ChildItem | Copy-Item - Server
  https://serverfault.com/questions/1055051/get-childitem-copy-item
  Computers = hostname get-psdrive -PSProvider " FileSystem " ` | foreach
  {Write-Output $Computers $_.Root;get-childitem $_.Root -include ...

Get-ChildItem gci - PowerShell - SS64.com
  https://ss64.com/ps/get-childitem.html
  By default, Get-ChildItem gets only non-hidden items, use the -Force
  parameter to include both hidden and non-hidden items in the results.

colors - Powershell: Properly coloring
  https://stackoverflow.com/.../.../powershell-properly-coloring-get-...
  ... Get-ChildItem $args " ) | ForEach-Object { if ($_.GetType().Name -eq
  'DirectoryInfo') { $Host.UI.RawUI.ForegroundColor = 'Magenta' echo $_ ...

At the moment, it does not look like my “Sites to Search” are being respected.

But here is a typical example I was seeing before.

One way to make sense of these results is to ask yourself which one would you click on. Here, I don’t see anything related to my search.

Download tools from NuGet (Developer Guide for
  https://docs.microsoft.com/en-us/dynamics365/customerengagement/on-premises/developer/download-tools-nuget
  ... Get-ChildItem ./Tools | Where-Object {$_.Name -match
  'Microsoft.CrmSdk.XrmTooling.PackageDeployment.Wpf.'} move
  .\Tools\$pdFolder\tools\*.* ...

Download tools from NuGet (Microsoft Dataverse) -
  https://docs.microsoft.com/en-us/power-apps/developer/data-platform/download-tools-nuget
  ... Get-ChildItem ./Tools | Where-Object {$_.Name -match
  'Microsoft.CrmSdk.CoreTools.'} move
  .\Tools\$coreToolsFolder\content\bin\coretools\*.* ...

Télécharger les outils depuis NuGet (Microsoft
  https://docs.microsoft.com/fr-fr/powerapps/developer/data-platform/download-tools-nuget
  ... Get-ChildItem ./Tools | Where-Object {$_.Name -match
  'Microsoft.CrmSdk.CoreTools.'} move
  .\Tools\$coreToolsFolder\content\bin\coretools\*.* ...

There are a few problems here.

• I don’t have the result I am looking for.
• There are results in a foreign language.
• I have no way to ask Mojeek to specifically search https://docs.microsoft.com/en-us/powershell/

If I manually add the language parameter &lr=EN, that improves the results by limiting them to microsoft.com/en-us/. But I still don’t have the Docs article I am looking for.

Merriam-Webster

I look up spellings and definitions using the US English language dictionary Merriam-Webster.

I set up this Focus Engine for Merriam-Webster.

{"Define":"i=merriam-webster.com"}

Here is an example of a page I might visit. This one is for the definition of the word “prose”.

https://www.merriam-webster.com/dictionary/prose

If I search for

prose site:merriam-webster.com

here are the top three results.

Prose poem Definition & Meaning -
  https://www.merriam-webster.com/dictionary/prose%20poem
  Other Words from prose poem Example Sentences Learn More About prose poem ...
  the Definition of prose poem to Facebook Share the Definition of prose ...

Prose Definition & Meaning - Merriam-Webster
  https://www.merriam-webster.com/dictionary/prose
  More Example Sentences Phrases Containing prose Learn More About prose ...
  Post the Definition of prose to Facebook Share the Definition of prose on ...

21 Synonyms & Antonyms of PROSE -
  https://www.merriam-webster.com/thesaurus/prose
  Post more words for prose to Facebook Share more words for prose on Twitter
  ... The first known use of prose was in the 14th century

This is a pretty good result. The definition I am looking for is the second result.

But, looking ahead, one problem here is what I want is not the first result. So, for example, I should not use a hypothetical !ducky bang while searching the dictionary.

Another problem is that less common entries do not show in the results.

Here is an example.

exempli gratia site:merriam-webster.com

Results 1 to 6 from 6 in 0.01s

  Browse the Dictionary A-Z: Letter E (page 34) |
    https://www.merriam-webster.com/browse/dictionary/e/34
    ... exemplarism exemplarity exemplary exemplifiable exemplification
    exemplificative exemplificatory exemplified exemplify exemplifying exempli
    gratia ...

  Eg Definition & Meaning - Merriam-Webster
    https://www.merriam-webster.com/dictionary/eg
    stands for exempli gratia in Latin, which means “for example.” It
    introduces one or more examples that illustrate something stated, such
    as:

  Exemplify | Definition of Exemplify by
    https://www.merriam-webster.com/dictionary/exemplify
    1 : to show or illustrate by example anecdotes exemplifying those virtues
    ... of or serve as an example : embody she exemplifies ... exempli gratia

  I.e. vs. E.g.: What is the Difference? |
    https://www.merriam-webster.com/words-at-play/ie-vs-eg-abbreviation...
    is short for exempli gratia , which means "for example." E.g. ... stands
    for exempli gratia in Latin and means “for example.” Just like the ...

  Digging Up the Latin Roots of 14 Abbreviations |
    https://www.merriam-webster.com/words-at-play/origin-of-latin-abbre...
    comes from the Latin phrase exempli gratia , which translates literally as
    "for example," and it is used in English with that meaning to introduce
    ...

  Ie Definition & Meaning - Merriam-Webster
    https://www.merriam-webster.com/dictionary/ie
    stands for exempli gratia , which means "for example." It introduces one or
    more examples that illustrate something stated, as in "Submit a sample of
    ...

No more results.

These results are ok. For example, “exempli gratia” appears in all of them.

But, there is an exact hit.

https://www.merriam-webster.com/dictionary/exempli%20gratia

If this is not convincing, here is a better example of a missing definition.

generic site:merriam-webster.com

Here, https://www.merriam-webster.com/dictionary/generic does not appear anywhere in the results.

The World Factbook

If I am reading an article and a country which is unfamiliar to me is mentioned, I like to look up the country in The World Factbook which is a publication of the US Central Intelligence Agency.

I set up this Focus Engine for The World Factbook.

{"Countries":"i=www.cia.gov"}

Here is an example of a page I might visit.

https://www.cia.gov/the-world-factbook/countries/united-states/

The main problem with the World Factbook results are that Mojeek indexed them right before the service was moved to a new location.

So if I search for the United States, the formerly correct page is the top result. But visiting that page now gives a 404.

https://www.cia.gov/library/publications/the-world-factbook/geos/print_us.html

[Colin]

Thanks @mike for the clarifications on backup, region setting, boilerplate/parameters and browser integration. Much appreciated.

Also for the explanation and comments in the section where you say:

@mike

Ultimately, Focus does not let me do anything I could not do before. With enough typing and bookmarking, I can do all of my customized searches everywhere.

Whilst the other parts makes sense, this part doesn’t. I will try to clarify with a question…

How you do search across multiple sites in “custom search engines in Chromium-based browsers”?

For example, how would you combine these two searches into one?
word site:reddit.com
word site:en.wikipedia.org
something like this:
word site:(reddit.com AND en.wikipedia.org)
which is what Focus allows you easily do.

[Josh]

1. Powershell

@mike

There are a few problems here.

• I don’t have the result I am looking for.
• There are results in a foreign language.
• I have no way to ask Mojeek to specifically search https://docs.microsoft.com/en-us/powershell/

and

@mike

At the moment, it does not look like my “Sites to Search” are being respected.

It looks as if we’re rejecting this URL and have therefore not yet indexed it, it’s longstanding and we’re looking at it now to see if we can improve it. On the second quote post, Focus was disabled at this point which was causing this to not be respected; we’ve re-enabled it now.

2. Merriam Webster

@mike

I look up spellings and definitions using the US English language dictionary Merriam-Webster.

@mike

Another problem is that less common entries do not show in the results.

Here is an example.

exempli gratia site:merriam-webster.com

At different times over the weekend servers were being restarted, which meant at different times there were some results missing. Repeating this search now shows on our end the result you pointed out up top:

If you’re able to check it also.

@mike

If this is not convincing, here is a better example of a missing definition.

generic site:merriam-webster.com

Here, https://www.merriam-webster.com/dictionary/generic does not appear anywhere in the results.

Looking into it, you are correct, the definition page for the word generic is not in the index. MW is a very big site, as with most dictionaries, so we can end up with some gaps as we’re not prioritising them over other sites and relying upon links and limits to get them.

3. The World Factbook

No answer as of yet but we’re looking into why this hasn’t been recrawled and removed.

[mike]

@Colin

How you do search across multiple sites

I agree with your sentiment. Focus can search an exclusive list which contains multiple sites.

However, I search single sites. And I can recreate that experience even on mobile by visiting a given site and typing any search operators.

I never thought to try, but, for reference, it is apparently possible to search multiple sites with the proper operators.

guns (site:pbs.org/newshour OR site:bbc.com)

https://duckduckgo.com/?q=guns+(site%3Apbs.org%2Fnewshour+OR+site%3Abbc.com)&df=2022-05-22..2022-05-30

https://www.pbs.org › newshour › nation › how-does-u-s-gun-policy-compare-with-the-rest-of-the-world
  How does U.S. gun policy compare with the rest of the world?
  The United States, with less than 5 percent of the world's population, has 46
  percent of the world's civilian-owned guns, according to a 2018 report by the
  Switzerland-based Small Arms Survey ...

https://www.pbs.org › newshour › nation › governors-divided-on-gun-control-mirroring-the-nations-partisan-split
  Governors divided on gun control, mirroring the nation's partisan split ...
  Scott signed a law in 2018 limiting the capacity of firearm magazines and
  raising the general age to buy guns to 21, with exceptions for 18- to
  20-year-olds who undergo a firearms safety course.

...

https://www.bbc.com › news › 61599697
  Texas shooting: US gun control claims fact-checked - BBC News
  The issue of controls on gun ownership is being debated in the United States
  once again, after a gunman opened fire in a school in Uvalde, Texas, killing
  19 children and two teachers. US ...

https://www.pbs.org › newshour › tag › police-response-to-uvalde-massacre
  police response to uvalde massacre | PBS NewsHour
  Politics May 30. Watch 8:40. Tamara Keith and Amy Walter on the debate over
  guns after the massacre in Uvalde. NPR's Tamara Keith and Amy Walter of the
  Cook Political Report with Amy Walter join ...

https://www.bbc.com › news › world-asia-india-61629133
  Sidhu Moose Wala: The murdered Indian rapper who 'made sense of chaos'
  Drawing heavily from the genre of gangster rap, his music was a jumble of
  gritty opulence - measured in guns and fancy sports cars - as he made sense
  of life around him. His songs offered ...

---

@Josh, I was able to get the same result for exempli gratia.

[mike]

World Factbook Titles

The new World Factbook pages are missing their titles.

---

One problem is that the <title/> tag is about 200 kilobytes into the page.

Another problem is that the page uses the <title/> child of <svg/>.

[Josh]

Can you check if this is still happening? I have this page as Crawled Today and it seems fixed.

EDIT: Yep, can confirm here, a fix from a few weeks ago has managed to get round to making this work properly.

[mike]

It is working for me too. Thanks for the help.

[mike]

Focus Share Button

I had a problem with my search results. So I clicked on the Feedback button. And this URL was returned as part of my submission:

https://www.mojeek.com/search?q=survive&foc=Define

The URL contains my search term and the name of the Focus engine. But it does not provide any details about my Focus engine. Not knowing the parameters of my Focus engine (like what sites are included) makes it difficult for @Josh to respond to Web feedback which includes a Focus engine.

Question

Is it possible to include a share button as part of the Focus search results?

So if I typically see this URL when using Focus:

https://www.mojeek.com/search?q=survive&foc=Define

then clicking the share button would produce the following text in some modal field or offer to populate my clipboard.

https://www.mojeek.com/search?q=survive+site%3Amerriam-webster.com

This would allow me to share my search results in a standard way.

And this URL could be used in the feedback form instead of including the Focus parameter.

I’m not sure how site clustering would differ between these two URLs. And a share button might create a JavaScript requirement in the browser.

[mike]

Keyboard Navigation

From mojeek.com, I expect to type search terms and then use the keyboard arrow keys to navigate to the Focus engine.

I noticed that in Firefox I can press Tab to navigate between UI elements. But this is cumbersome.

An example where something like this works is the Firefox address bar where left and right move the cursor within the search terms and up and down change the selected search engine.

Demo of Firefox address bar.

One problem is I am not sure how a solution like the Firefox address bar would interact with autofill or typed history in various Web browsers.

[Josh]

Raised on both accounts. For the moment with the first point: with a smaller testing group, sharing is not so bad providing the person doing the sharing is comfortable with letting me know what a Focus contains, but you’re completely right on this being something that would be much better if Submit interacted with it.

EDIT: on the tabbing point also, we made version one of Focus as simple as possible, relying upon no JavaScript; our planned developments going forward will not be so tied to this.

[mike]

Focus Search Operator

It would be useful to trigger Focus from a search operator.

This would let me type and execute my search without a mouse click.

Earlier, I mentioned it is difficult to integrate Focus into the web browser. There, I mentioned creating a custom search engine or publishing OpenSearch plugins.

But, by using a search operator, I could use the search built into the web browser without any modification. I would just have to include the Focus operator in a standard Mojeek search box.

[Josh]

Cheers for this one @mike, and for illustrating it there. I’ve raised it as a suggestion.

[mike]

Focus Widget Bug

There is a minor bug with the Focus widget.

The widget offers to remove sites from the default template. You can see this after searching with a default template.

But the default templates are read-only. This should be acknowledged in the user interface.

There are three ways to go:

1. Report that the default templates are read-only.
2. Offer to create a new Focus engine.
3. Silently create a new Focus engine and remove the site.

I think a simple error message is the least confusing and makes the fewest assumptions.

Mike

[Josh]

Thanks for this one @mike as well as the useful suggestions, it has been raised and we’ll sort it out.