I recently listened to an interview with Liron Shapira—whose first startup achieved a huge valuation and then came to nothing. And he was blunt in mentioning it failed because it wasn’t useful for any single person.
They had an abstract idea of something that seemed exciting and didn’t check to see if it was of much use to any single individual. He claimed many startups fail this way, and most founders would probably be much better off finding one person and trying to build something they find helpful enough to use regularly—and then expand from there.
Conditional on this being true, individuals who long for non-existent software are highly undervalued! So please post requests for software that does not exist but you think would be of great immediate value to you.
And if we get enough posts, perhaps some devs will actually implement some. And who knows, maybe we can nucleate a startup or two!
more a piece of glue, but I want a self-hosted AI to basically do experiments on me. I want to tell it a goal that I have, and get its feedback on figuring out how to measure that goal, and then it’d track things about me (either ambiently by me giving it access to home automation, credit card statements, etc, or actively by texting me and asking questions or giving reminders). Ideally it’d ask me to make small/easy modifications to my day-to-day life to test how effective various changes are at moving me toward my goals with minimal active effort on my part. I basically want to outsource the executive function and consistency required to benefit from the whole Quantified Self thing, which seems like it should be absolutely possible these days.
The problem is that I really want it self-hosted—I’d buy it terabytes of storage, and I’d spend around $1000 for dedicated hardware that it could run on, and if it worked really well I’d consider upgrading it for more of the same to the tune of 10k-100k over time—but I absolutely don’t want that intimate of a software system to ever be emitting data onto the internet or acting like it’s owned by someone else and only rented by me.
Been exploring local models lately, and I might be interested in working on a 7B-13B model version of this, potentially with scaling up to preferred models, if I could find the time and compensation.
A $1000 computer would be low for what you want, and $10,000 would be high, unless you need the kind of self-tuned reasoning capability only truly accessible on enterprise graphics cards; a 7B model or quantization of it would work best with at least 8GB vRAM (or unified memory? I’d go higher on unified memory and not make it dedicated hardware), and 34B 4-bit quantizations look like they should fit in 24GB vRAM, which is about the limit for consumer grade nvidia cards (best for compatibility in all aspects). The most efficient pricing specifically for inferencing GGUF format is found in Macs, currently: the unified memory works perfectly well for it, and they go up to 128 … wait no, 192GB shared memory (metal memory about 75% or 145GB), so if you maximize that and storage … an 8TB, 192GB Mac Studio computer will run you about $8000 and theoretically should be able to run even the largest current open weight models (though training and so forth is a greater challenge).
Cool! Unfortunately I’m not really sure if the idea itself is compatible with turning a profit—modern business models would push for it to leak data or include ads in ways that would defeat the purpose.
I’ll eventually get one of the good macs if I have to, but I’m giving that decision another year or so to become clearer whether or not it’ll be really necessary in the long run.
I’ve also heard some very promising things about eventually being able to do a one-time investment of renting fancy compute for initial training, and then compressing the trained model to run on smaller hardware.
Yeah, there’s a reason I specified ‘compensation’ rather than ‘profit’. :) Executive function assistants of some kind could be useful for me too, but whether it’d be useful enough to put the work into it as its own reward … well, that’s a question.
And, yeah, if you want to either rent the GPU yourself or have someone do training for you, and you don’t mind the training data going into the cloud, that’s the best way to do it. Tuning takes more compute than inference, in general.
(I don’t think personally identifying training data is particularly helpful for tuning; you’re trying to get methods, approach and formatting down, not so much memory, though it may pick up on a few things. Not to mention if you ever felt like letting it out of the box and sharing your helpful assistant. Retrieval-augmented context is better for memory outside pretraining.)
Quantized models have good performance/speed tradeoffs—a 4, 5 or 6 bit quantization of a larger model still captures most of the performance improvement over smaller models (that would fit in the same memory without quantization) of equivalent quality otherwise. You can indeed run inference on much larger models than you can train.
Writing polite but short emails that have a single intention is hard. The fewer words you use, the more can be wrongly inferred about the tone you hoped to convey. You want to save your recipient’s time and energy and to do that consistently, but it’s difficult to know if people will read something you didn’t intend to say.
While Claude and GPT-4 often understand exactly what I mean when I feed them poorly written word salad that is both long-worded and not acceptable to send, they don’t yet do a good job of removing what I want because of what seems to owe to how instruction-tuning and RLHF weight verbosity. Maybe this is easy to fix with the correct prompt, but no prompt I’ve tried has been universal.
I want software that can totally remove my personality and the extra context of what else I might be thinking about from a piece of writing, while also double checking every interpretation of my tone . I want to be the Bruce Lee of emails.
A simple, understandable, and fully customizable web browser (like nyxt but usable as a daily driver.
A program or service that allows multiple disparate communication services (email, Signal, Discord, Whatsapp) to all be interfaced with from one service (imagine a web page as your default client), that exposes API endpoints and allows users to write their own clients (so you can use it in emacs, for example). Bitlbee exists but I haven’t gotten around to figuring out how to set it up yet
A “personal assistant” style AI system that tracks your tasks, your agenda, your mood and preferences, the events and things you may be interested in, and suggests you things at the right times and in the right contexts.
The above are easy optimizations compared to the stuff below:
Internet access everywhere, no exceptions, via wireless connectivity. Imagine Starlink but internet provided to you everywhere, reliably. Also you don’t have to deal with the nonsense of switching SIM cards as you travel across continents.
A fully open source hardware and software stack for devices that are actually usable.
An e-ink display portable device (such as a laptop or a tablet) that uses relatively customizable operating systems like Linux or BSDs.
Quoting another comment I made:
Make a hyperphone. A majority of my alignment research conversations would be enhanced by having a hyperphone, to a degree somewhere between a lot and extremely; and this is heavily weighted on the most hopeworthy conversations. (Also sometimes when I explain what a hyperphone is well enough for the other person to get it, and then we have a complex conversation, they agree that it would be good. But very small N, like 3 to 5.)
https://tsvibt.blogspot.com/2023/01/hyperphone.html
It’s difficult to understand your writing, and I feel like you could improve in general at communication based on this quote. The concept of a hyperphone isn’t that complex—the ability to branch in conversations—so the modifiers “well enough”, “complex”, and “very small N” make me believe it’s only complex because you’re unclear.
For example, the blog post you linked to is titled “Hyperphone”, yet you never define a hyperphone. I can infer from the section on streaming what you imagine, but that’s the second-to-last section!
The point of the essay is to describe the context that would make one want a hyperphone, so that
one can be motivated by the possibility of a hyperphone, and
one could get a hold of the criteria that would direct developing a good hyperphone.
The phrase “the ability to branch in conversations” doesn’t do either of those.
One software that would unify my accounts across Telegram, Signal, WhatsApp, Viber, Facebook Messenger, Skype, Teams, Zoom, and dozen other similar applications, so that I don’t have to install them all and have every discussion in a separate application.
(Like in the old times when we had ICQ, AOL messenger, MSN messenger, etc., and then there was Pidgin that communicated to all of them.)
this exists for some of those—texts.com
Thanks you! Also, alternatives
A browser extension that colors reddit usernames red if the user is a likely bot.
User friendly financial software that can help with saving and budgeting. And some sort of software that can optimize career potential.
File explorer where I don’t type all of
D:/F1/F2/F3/F4/Xto get/open folder or file X, but I type (part of)F2andF4and it immediately (yes, indexing etc.) offers me X as a result (and maybe the few others that fit the pattern)If I have an insane amount of subfolders/files, maybe it indexes better those with recent or regular access.
Extension: A version on steroids might index even file-seek results of files and index on (my or so) most common searched words. Find if that’s a bit too extravagant.
Useful in traditional file structures, as we then type/think/remember less. Plus, it might encourage a step towards more tag-based file organization which I feel might be useful more generally, though that’s just a potential side-effect and not the basic aim.
It sounds like you want fzf (or similar). It’s not arbitrarily fast, but it’s pretty fast and I think there are somewhat faster competitors. It doesn’t use indexing because indexing wasn’t needed for pretty fast speeds.
A search engine that gives results like Google but before everything turned to sludge.
Have you tried Kagi? It was too expensive for the utility it provided me, but I think it is worth at least trying it out.