Software Engineer (formerly) at Microsoft who may focus on the alignment problem for the rest of his life (please bet on the prediction market here).
Sheikh Abdur Raheem Ali
Karma: 588
No, humans have much more in common with each other than they have with AIs, drawing faction lines based on contemporary geopolitics is a mistake, U.S and China will both soon be under attack from rogue AIs (see https://openai.com/index/hugging-face-model-evaluation-security-incident/ for a recent case with ExploitGym) and we will need a bilateral verifiable treaty on safety. I think it’s more accurate to model the rogue AIs as germs (as opposed to modeling them as phones).
Great work, well done!
Thank you for sharing. Giving good criticism is really difficult in general, I’ve found that the expected norms when communicating with a grantmaking/fundraising audience can feel much more reserved than the register I’d use with technical or executive staff. I’ve settled into a pattern where I write very candid feedback, post it, then filter/translate the original through an LLM and link to the sanitised/”professional” version, which doesn’t seem ideal at all.
I try to be sharp and sincere but the directness of the language which comes natively to me can in some contexts come across as caustic. That said, I don’t have the full picture, but from what you’ve described I think Longview was wrong to rescind the invitation. Your dialect is very clear from your public writing, and if that was a problem for them then they shouldn’t have invited you in the first place.
I found five submissions under review. One was satire, two were real alignment papers. One seemed like slop. The last one I’m not equipped to evaluate but it seemed like a decent causal graph theory paper. Based on this, it seems like progress is coming along, and I think it would be good if it kept developing along the same direction. Also, this is the only journal I’m aware of with a sane submission format.
This is interesting work but I don’t understand the link between the signal obfuscation and the brain computer interface.
to get the benefits of mismatch
What are these benefits?
Using humans to conduct vishing is highly unprofitable at US wages
Scammers are not paying US wages. It’s usually call centres in the Philippines or something.
It has been about four and a half months since the announcement. You described these as “build-in-the-open” updates, and in March said initial submissions might open as early as April. The public GitHub organization still has no repositories, the latest blog update was April 28, and I can’t tell from the main site whether the journal is currently accepting real submissions.
I tested Scholastica and found that creating a bare submission portal is nearly trivial. Obviously, creating a credible journal involves much harder work—recruiting editors and reviewers, establishing governance and policies, building the review process, and earning legitimacy. Could you share a concrete status update: what exists today, whether any manuscripts are in the pipeline, the revised launch timeline, and what is currently gating launch?
I completely disagree with your stance on the importance of replications in ML/AI.
Edit: sanitised version https://www.lesswrong.com/posts/msnGbm52ZcG3xYcFo/an-alignment-journal-coming-soon?commentId=ERYiLi5wLKXYcot3o
It has been 5 months since this post. I am sure you are working hard but where are your results? There is nothing public on github. Has this journal done anything other than write three blog posts about the journal? I just went on scholastica and created a journal named “awegw”, which has already received three submissions within one minute, all from a prolific author named Brian Cody, although unfortunately I had to reject the manuscripts for being lorem impsum text (not even slop!). If I added a description, list of editors (applications open), and submission guidelines for authors, then I can be Editor-In-Chief of a real journal (one of over 1300!) and start taking articles from authors while this project would still be stuck with a contact us form. Would love to learn more about what the bottleneck is here.
China’s open-source models are the only option if you want to do any kind of whitebox research work.
Is this true given that U.S companies such as Thinking Machines and Meta release some open-weight models?
Would the merge-and-assist clause in OpenAI’s charter violate Section 7 of the Clayton Act?
My understanding is that if the DOJ and FTC choose not to challenge a merger, the merger may proceed unless another party successfully challenges it. But I’m still confused about some open questions:
How should the relevant market be defined?
Would frontier AI labs constitute a distinct antitrust market?
How would market concentration (HHI) be measured?
How should innovation competition be analysed?
Could AI safety benefits qualify as merger efficiencies?
The GPT 5.5/5.6 models score very competitively on security-agent benchmarks like ExploitBench, but it’s possible a small advantage could snowball. I’m surprised that the SpaceXAI-Anthropic compute partnership happened, and Demis Hassabis’s “A Framework for Frontier AI and the Dawning of a New Age” (Jul 14) speaks of competitive dynamics in a way that makes me think about a hypothetical DeepMind-OpenAI consolidation.
Also, could U.S frontier labs merge with their Chinese counterparts (e.g as part of a coordinated slowdown)?
I think that I would like to see this framework applied to a hypothetical war between humanity and AIs, to analyse current geopolitical conflicts, and also on a smaller scale for modelling competition that isn’t war.
One-Pager Brief on Pangram Labs
I wish that I had written this post myself. I’ve been thinking about this topic a lot over the past three weeks. Thank you so much.
Is scalable AI text detection & response (such as pangram) bad because it puts selection pressure on LLM outputs to be more humanlike?
We do not see significant accuracy uplift from self play optimization overall.
Isn’t this trivial due to the anthropic principle as accuracy uplift from self-play optimisation would FOOM via RSI?
Yes, I think that having two phones is essential for my workflow.
Thanks for this info dump. Very cool. I hope you can improve the aerobic exercise section at some point.
Great to see this level of transparency and effectiveness, and especially impressive for a student-run organization!