What hesitations do you have about paying for it?
How much would it be worth to you in $/month?
As a paying customer, what features would you expect?
Why is this a bad idea?
This is a quick & rough market feasibility assessment. Thanks for your answers.
I’m skeptical of this since a good general purpose search engine is also a good rationalist-site search engine with the right prompt engineering. Using (and paying for) multiple search engines seems worse than just appending “rationalist” or “lesswrong” to the end of a query in a normal search engine.
Personally I also don’t think limiting to rationalist sites in general in really that useful in general. I occasionally search for specific rationalist sites because I want to find an article I already know about, but I don’t think I’ve ever wanted to a limit a more general query to rationalist sites.
Unfortunately, I’ve found that appending rationalist to queries doesn’t get the desired results. Instead you get this: link
If you could limit your search results to sites with a higher level of epistemics, would that be more compelling? There might be default set of sites, which you could customize and submit requests for additions to the corpus.
What price point would change your mind? Is the idea compelling enough that you would try a demo?
Can you give an example of a case where I would want to do this? I have trouble thinking of one.
Some related but different cases that are already handled by normal search engines are:
Finding an article on LessWrong—search for “something about the article lesswrong”. A rationalist-specific search engine is overkill for this. Note that this actually works decently well to find articles that aren’t on LessWrong, like how https://www.google.com/search?q=the+toxoplasma+of+rage+lesswrong returns a Slate Star Codex article as the first result
Finding a relatively unbiased article about a recent political case—search for “the political event” then scan the first page for sites I think are relatively unbiased. Politics generally doesn’t get as much coverage in Rationalist spaces so I would expect a Rationalist-only search engine to fail to find anything at all most of the time (and personally I don’t think Rationalist sites are systematically better at political coverage than the decent mainstream outlets even when it does exist).
I don’t mean to be overly negative though, and it’s possible there is a use-case / audience for this. Just giving honest feedback on why I wouldn’t find this compelling since that seems to be what you were wanting.
Use cases: superconductor, Ukraine war, LLM development, diet or exercise, dealing with anxiety, etc. But you would only get results from a curated list of sites with higher epistemic standards.
I should have been more explicit in my initial post. I was relying on the word “rationalist” to do too much.
No worries about negativity. It is exactly what I want, so thank you.
That’s the wrong search query, you’re asking google to find pages about the Ukraine War that also include mentions of the term “rationalist”; you’re not asking google to search for rationalist discussions of the Ukraine War. Instead I’d do something like this.
Yes, but instead of searching one domain (lesswrong), it would search ~100+ curated domains. Google currently limits the domains to ten.
I have never in my life paid for a search engine, even though I’m not at all happy with the results from Google or Bing, because I detest subscriptions and because I had the impression that there aren’t any paid search engines good enough to blow those big ones out of the water. (That said, this impression might be mistaken.)
And as an individual I wouldn’t pay for a search engine just limited to rationalist sites. That kind of thing would provide me a bit of utility, but certainly not enough to warrant a paid subscription (yuck).
Also, when I browse, I usually want to have exactly one search bar where I can enter any arbitrary queries and get a result, without having to use a separate search engine for specific queries. The only situation where I am delighted to use another search engine is geizhals.de, a German price comparison site which produces really neat tables for comparison shopping of electronics. And that’s a case where the search provides obvious monetary value (via savings on purchases) and is nonetheless as free as anything else on the Internet (i.e. financed by ads and affiliate links etc.).
Overall, I can’t imagine that there would be a sufficient market for such a search engine among the user base, so I figure your actual customers (or funders) would have to be those rationalist and EA sites or grantmakers themselves.
E.g. I really don’t like the search function on LW[1], to the point that I prefer to use third-party search engines like Google to search for LW stuff. But I don’t know whether the LW team (and by extension the EA Forum team, who use the same software) also considers their search to be a problem, and if so, if they would be able and willing to pay for an improvement.
PS: If you’re considering using AI to improve search, another idea which could improve discoverability would be to auto-tag the LW corpus. The tags seem like a good system, but asking authors and readers to manually tag posts just doesn’t work.
As an example, searching for “harry potter and the methods of rationality” displays 3 random users, then 9 supplementary posts, and only then lists the “The Methods of Rationality” sequence, which is book 1 of HPMOR. This is the default search order sorted by “Relevance”. Sorting by karma makes the results even worse, e.g. the top result is now the preface to Rationality: AI to Zombies.
What utility would it need to provide to change your mind?
How about a search of sites curated for a higher level of epistemics? Can you think of any searches you might do where that would be useful?
Suppose it cost between $2-6 a month? Or, what price point would be enticing for you to try it?
I have so many bookmarks that I’m drowning in (mostly rationalist-adjacent) stuff to read, and I don’t immediately see how even a fantastically powerful search engine could help here.
For me to even consider paying a subscription, I must obviously get way more value out of it each month than I put in in money. That means it must save me significant chunks of time or money. I don’t immediately see how a search engine which is limited to a subset of websites is supposed to do that, or what it is supposed to offer that’s orders of magnitude better than just sorting LW by the posts with the most karma.
And even if your product managed to meet that bar, then you’d need to fulfill a second requirement, namely that there must not exist any competitors who can do what you offer except better, cheaper, or for free (as in, financed via ads etc.). E.g. Google is free, and supports limiting queries to specific domains.
Also consider that essentially no individual users on the Internet currently pay for search; it’s considered a free service. And the offer you’ve been describing so far sounds worse, namely more expensive and more limited. Maybe there’s a way to produce so much value that a subscription is a slamdunk, but I haven’t heard any sufficiently impressive value proposition yet.
That’s why I figured that an economically more sensible approach would be to offer helping to improve the search on LW and the EA Forum. Those sites are already home to a significant fraction of the rationalist userbase, plus it seems much easier for them to generate significant extra value from better search.
EDIT: And fundamentally, so far I have the impression that you already have some technical solution in mind, and now you’re searching for a problem to solve. From what I understand of startups (not much), isn’t that exactly the wrong approach? E.g. usual recommendations for startup ideas sound more like “solve a problem you have yourself”.
>drowning in stuff to read
Suppose you wanted to find content on prioritizing what you read by people with similar interests or with higher standards than most writers in the google search results.
Do you expect a search of LW will be more likely to deliver what you want than a search of LW +100 other sites?
>Google is free, and supports limiting queries to specific domains.
The limit is ten sites.
>just search LW or EA
What if you could do both in one place plus search all these and the ACX’s blogroll and similar sites?
>solution in search of a problem
Google search quality seems to not satisfy a large number of people: link. Not to say this idea will fix that for everyone.
>value delivered
What if it cost $1-6 a month? Would you try it if it was free? Would you donate once or regularly if you liked it?
When I am looking for rationalist content and can’t find it, using Metaphor (free) usually finds what I want (sometimes even without a rationalist-specific prompt. Could be the data it was trained on? In any case, it does what I want.)
Don’t there already exist extensions for google that you can use to whitelist certain websites (parental locks and such)? I’d think you could just copy paste a list of rationalist blogs into something like that? This seems like what you are proposing to create, unless I misunderstand.
I would not pay for it. Some thoughts:
I don’t find myself having a need here. Doing what Brendan said with appending “rationalist” or “lesswrong” or some other DuckDuckGo-fu to my search queries usually works. Sometimes I find myself browsing through tag pages here on LW too. Other times I’ll look at sites like Stanford Encyclopedia of Philosophy or Stack Exchange. And with the advent of LLMs, I find myself increasingly using Claude instead of web searches.
I suspect pretty strongly that this isn’t the case, but I might be failing to see the usefulness of a search engine limited to rationalist sites. I think concrete examples of when you think it would be useful would be helpful. Do any come to mind?
One somewhat large consideration is that, when I search for something, I type the query into my URL bar and then hit enter. I have my browser (Firefox) configured to automatically search DuckDuckGo when I do that. I think most browsers do something similar by default. So for normal, quick searches, I think the trivial inconvenience is actually pretty important and the rationalist search engine would be something I’d try only if the first approach of using the URL bar didn’t work.
I’m not sure how common this is, but I at least feel like there’s some sort of cognitive overhead in play. Yet another monthly bill. Yet another think in my bookmarks. Yet another thing to keep in the back of my mind. I suspect I’m in something like the 99th percentile in how minimalist I am, but I also suspect that these sorts of thing are non-trivial considerations for maybe 10-70% of others.
Personally, I would like to spend less time browsing the web. Suppose that the rationalist search engine was an improvement. If so, I think it’d be some sort of “one step forward and two steps back”. Two steps back because I’d like to do less of this semi-productive web browsing; one step forward because (in this hypothetical) it’d make the semi-productive web browsing more productive.
Hey Adam, please review some of replies I’ve made to other commentators for issues I don’t address here.
>ease of use
A keyboard shortcut, chrome extension that serves the results in a side bar or some other spot, autocomplete in the search bar, or bookmark would remove that friction.
If I want to go to lesswrong, I hit ctrl-t for a new tab, type “les” and chrome completes the url. The same would apply.
>cognitive overhead
I do not think about those things for something that delivers me consistent value. If the starting premise is “I don’t value this.” It doesn’t matter what comes after it.
>less time browsing
Wanting to spend less time doing semi-productive browsing isn’t something a better search engine can fix—unless it’s because the result quality is the reason the time is semi-productive.
Good point. I see the need to set that up as a downside but ultimately a relatively small one.
Yeah, that makes sense too. I was thinking along the lines of “I don’t think this would provide me a lot of value, but maybe it’d provide a little value. But if it provides a small amount of value, the small downsides might outweigh the small value.”
I’m assuming this would basically be like a wrapper that takes a search term, concatenates it with a bunch of google “site:site1 OR site:site2...” boolean terms, then displays the results. If so, then sure, I’d like to have such a string pre-formed, but I don’t think it’s something that valuable to me personally, or something I’d pay for. If not, I’d be curious to hear more about what you mean.
It would be nice to be able to form such a search for a set of sites of my choosing (text box, comma separated list, something like that), or a list of recommendations for sites I might want to include but not know about yet (checkboxes?).
Maybe a way to sort or filter by level of technical rigor or mathematical ability assumed?
Maybe an explanation, for each site searched, why that site is included, and a sentence or two about what to expect from things found on that site?
My understanding is that google limits the search space to ten sites.
>set of sites of my choosing
Perhaps a standard set of sites that could be customized with the option to submit requests for additions.
Thanks for the ideas.
I didn’t know google had that limit, thanks.
At my last job we had some data scientists build a tool to convert a relatively intuitive interface for inputting Boolean search terms to the specific forms needed for APIs of various data sources. We used it for patents, papers, that kind of thing. Sometimes the search strings ended up having to be 5-10 lines long. That was mostly when a term could be used lots of ways and we only needed one of them, or when there were combinatorically many ways of combining sets of terms to mean the overall same thing. So, I do think there can be a lot of value in prompt engineering for searches in targeted contexts.
Do you think LLMs will get to a point of being able to do this relatively well with the right prompts?
I think creating such a thing is a few hours of work with https://programmablesearchengine.google.com/about/ . Generally, those Google-based custom search engine don’t find much use.
Tried that previously. It limits the search results and it doesn’t rank the results it simply spits out first results it finds on the first domain it searches.
You can give the one I made a try to see what it mean. Don’t be fooled by the number of pages it lists at the bottom—that’s fake: https://cse.google.com/cse?cx=cced60b51960f6137