Software engineer from Ukraine, currently living and working in Estonia.
I mainly specialize in computer vison & robotics. https://grgv.xyz/.
Sergii
Karma: 147
The problem is that most hackerspaces are not this:
”warehouse filled with whatever equipment one could possibly need to make things and run experiments in a dozen different domains”most hackerspaces that I’ve seen are fairly cramped spaces full of junk hardware. which is understandable: rent is very high and new equipment is very expensive.
but would be cool to have access to something like:
https://www.facebook.com/watch/?v=530540257789037
Thanks, it’s an inspirtional pitch, I can relate.
And my observation from this kind of communities (hackerspace, engieering/hacking conference), is that a large fraction (I think majority?) of participants are much more interested in the tech itself rather than in applcations. There is also not that much drive for novelty and innovation.
I think that there should be space for exploration and learning, but to me, wizardry is about getting things done, solving actual practical problems.
For example, at hackaday.com, there are cool projects, but a large fraction of the (extremely talented) hackers are building yet another 8bit computer.
You should at lest give the credit to the inventors: https://reason.com/2019/01/16/how-scientology-recruits-inside-florida/
I have found that when using Anki for words/language learning, I frequently can’t remember the correct translation exactly, but can guess the translation as one of top-3 options. In fact, this works well for me—even knowing vaguely what the word means is very useful.
does anyone else uses Anki with non-exact answers?
I have similar issues, severity varies over time.
If I am in a bad place, things that help best:
- taking care of mental health. I do CBT when i’m in worse shape, and take SSRIs. YMMV. both getting dianosed and treated are important. this also includes regular exercise and good sleep. what you have described might be (although does not have to be) related to depression, anxiety, attention disorders.
- setting a timer for a short time, can be as short as 1min, and doing one of the avoided tasks for just that 1 minute. it kind if “breaks the spell” for me
- journaling, helps to “debug” the problems, and in most cases leads to wring down plans / intervations / resolutuons
Ha, thinking back to childhood I get it now, it’s the influence of the layot of the school daily journal in USSR/Ukraine, like https://cn1.nevsedoma.com.ua/images/2011/33/7/10000000.jpg
I have similar thing for week days, but somehow with a weird shape?
in general, it’s a similar cycle, but flipped horizontally, going left to right:
on top it’s: sun, sat
on the bottom: mon, tue, wed, thu, fri
the shape connecting days goes downwards from sun to mon, tue, wed, then upwards to thu, then down to fri, then up to sat, sun, closing the loop.not sure is this makes any sense )
unfortunately, AI research is commercialized and is heavily skewed by capitalist market needs,
so it’s still going to be all in for tryinng to make an “AI office worker”, safety be damned, until this effors hit wome wall, which I think is still plausible.
One possibility is that at some point AI products capabilities would be constrained by compute cost.
At that point, alignment “features” coule become a competitive advantage, so companies would invest much more in alignment.
The latest short story by Greg Egan is kind of a hit piece on LW/EA/longtermism. I’ve really enjoyed it. “DEATH AND THE GORGON” https://asimovs.com/wp-content/uploads/2025/03/DeathGorgon_Egan.pdf
in the long-term this could move the country toward the draconian censorship regimes, restrictions on political opposition, and unresponsiveness to public opinion that we see today in England, France, and Germany
I don’t know much about freedom of speach in US, but all the free speech indexes that I’ve found with a quick search show that European countries are ahead of US. Am I missing something?
https://rsf.org/en/index
https://ourworldindata.org/grapher/freedom-of-expression-index
https://futurefreespeech.org/who-supports-free-speech-findings-from-a-global-survey/
Apperently it’s more efficient to do it other way around, to compile programs into transformers, which are then useful as refecene and ground truth when analyzing “real” transformers.
See usage of TRACR in “Towards Automated Circuit Discovery for Mechanistic Interpretability” https://arxiv.org/pdf/2304.14997, for example.
I have similar experience, but I don’t think it’s a problem—my approach to learning a language is to first accumulate enough recognized words (thousands), and then to read a lot. In my experience lots of reading improves both recognition and recall.
There are several ways to explain and diagram transformers, some links that were very helpful for my understanding:
https://blog.nelhage.com/post/transformers-for-software-engineers/
https://dugas.ch/artificial_curiosity/GPT_architecture.html
https://peterbloem.nl/blog/transformers
http://nlp.seas.harvard.edu/annotated-transformer/
https://sebastianraschka.com/blog/2023/self-attention-from-scratch.html
https://github.com/markriedl/transformer-walkthrough?ref=jeremyjordan.me
https://francescopochetti.com/a-visual-deep-dive-into-the-transformers-architecture-turning-karpathys-masterclass-into-pictures/
https://jalammar.github.io/illustrated-transformer/
https://e2eml.school/transformers.html
https://jaykmody.com/blog/attention-intuition/
https://eugeneyan.com/writing/attention/
https://www.jeremyjordan.me/attention/
In abstract sense, yes. But for me in practice finding truth means doing a check in wikipedia. It’s super easy to mislead humans, so should be as easy with AI.
I agree with the possibility of pre-training platoeing as some point, possibly even in next few years.
It would change timelines significantly. But there are other factors apart from scaling pre-training. For example, reasoning models like o3 crushing ARC-AGI (https://arcprize.org/blog/oai-o3-pub-breakthrough). Reasoning in latent space is too fresh yet, but it might be the next breakthrough of a similar magnitude.
Why not take GPT-4.5 for what it is, OpenAI has literally stated that it’s not a frontier model? Ok, so GPT-5 will not be 100x-ed GPT-4, but maybe GPT-6 will be, and it might be enough for AGI.
You should not look for progress in autonomy/agency in commercial offings like GPT-4.5. At this point OpenAI is focusing on what sells well (better personality and EQ). I think they care less about a path to AGI. Rapid advances towards agency/autonomy are better gauged from academic literature.
I agree that we should not fall for “vibe checks”.
But don’t bail on benchmarks, many people are working on benchmarks and evals, there is constant progress there, benchmarks are getting more objective and harder to game. Rather than looking at benchmarks that are pushed by OpenAI, it’s better to look for cutting-edge ones in academic literature. Evaluating a SOTA model with a benchmark that is few years old does not make sense at this point.
LLMs live in an abstract textual world, and do not understand the real world well (see “[Physical Concept Understanding](https://physico-benchmark.github.io/index.html#)”). We already manipulate LLM’s with prompts, cut-off dates, etc… But what about going deeper by “poisoning” the training data with safety-enhancing beliefs?
For example, if training data has lots of content about how hopeless, futile and dangerous for an AI it is to scheme and hack, it might be a useful safety guardrail?
I made something like this, works differently though, blocking is based on a fixed prompt: https://grgv.xyz/blog/awf/
What about estimating LLM capabilities from the length of a sequence of numbers that it can reverse?
I used prompts like:
”please reverse 4 5 8 1 1 8 1 4 4 9 3 9 3 3 3 5 5 2 7 8“
”please reverse 1 9 4 8 6 1 3 2 2 5”
etc...
Some results:
- Llama2 starts making mistakes after 5 numbers
- Llama3 can do 10, but fails at 20
- GPT-4 can do 20 but fails at 40
The followup questions are:
- what should be the name of this metric?
- are the other top-scoring models like Claude similar? (I don’t have access)
- any bets on how many numbers will GPT-5 be able to reverse?
- how many numbers should AGI be able to reverse? ASI? can this be a Turing test of sorts?
His later work, “I am a strange loop”, I think is more interesting for insights on consciousness.