Hi, I’m Cormac. I am currently trying to find my way towards the most impact I can have in AI existential risk.
Corm
Karma: 126
What Are My Values?
“I’d encourage you to steal from it. perhaps clone it in ../ and tell claude code to look at it as needed. note: a major todo for me is getting it to get comments, which it doesn’t do now.”
Working on this right now
The original search before I went with the references based approach got a couple posts, but I think clearly not enough. I couldn’t figure out a good systematic way to get posts, but I will definitely spend some more time thinking about this and add a tag for LW/AI Alignment posts,
No it did not! I’ll take a look
Heretic uses ablation, which requires editing all of the weight matrices. My quick assessment is that the heretic codebase as it currently exists couldn’t deal with K2.5 out of the box because K2.5 does some weird things that heretic isn’t baseline designed to handle. I do think that it would be possible to get Heretic working on K2.5 with real effort put into it. The largest Heretic’d models on HF are a fifth the size of K2.5 and it looks like they still get around 30⁄100 refusals (this is not surprising because ablation is simply harder with MoE models) compared to my 0%, although my guess is they have less KL divergence than my approach which is more of a throw the kitchen sink at the problem vibe. Heretic uses an automatic optimization process to find the best coefficients for each abliteration, Claude thinks the runpod costs for this could easily go over $100.
Thanks for the suggestion! Most of https://transformer-circuits.pub/ have an arXiv and got picked up, but it seems like I missed two of them. It looks like I don’t have anything from https://distill.pub/ which I will work on.
White-Box Attacks on the Best Open-Weight Model: CCP Bias vs. Safety Training in Kimi K2.5
I Had Claude Read Every AI Safety Paper Since 2020, Here’s the DB
Side by Side Comparison of RSP Versions
“Our fair city is poised to allow 6-story buildings citywide by an 8-1 vote. In context that is a huge change. Under the old rules only 350 units (!) total were expected over 15 years and 85%+ of the existing housing wouldn’t have been legal to build. Here’s a primer on the changes. They had to compromise a bit on setbacks and lot size to get it over the finish line, but it still seems great.” I wrote about the specifics of this if anyone wants to hear more/cool data visualization maps.
https://ascendantnewyork.substack.com/p/what-nyc-can-learn-from-cambridge
what did you use to generate the images in this post?
I am become Matt Levine, destination for content relevant to my interests.
You don’t even need to go to London for mundane utility, there’s and “AI Mart” in LIC.
Have you considered doing random spot checks. Feels like even 3x per year gets 80% of the value.
Match group acquired OKCupid in 2011, ever since then it has been destroying OKCupids functionality and moving it closer and closer to a generic swipe dating app.
Here’s some major mundane utility, huge if true: Automatically analyze headlines for their implications for stock prices, get there first and earn a 500% return
I read the article about this and their assumptions are insane. Normally stock news happens after the market closes. The way returns on the next day are calculated is just (end of day price for the day)/(end of day on the previous day). It is importantly not (end of day)/(opening price) of the stock. This is why stocks are able to very frequently open up many percentage points due to news happening overnight.
So, all they did was say based on the news we will get long if chatGPT says good and get short if chatGPT says bad, and the way we will calculate our returns is just whatever the returns for the next day are. But remember returns for the next day are based on the close for the previous day. So, they are assuming that if news comes out at 6pm they are able to buy or sell the relevant stock at exactly the 4pm closing price. This is a fucking insane assumption. Not only are markets very thin in the extended 4pm-8pm hours and pre open 4am-9:30am hours. But responding to news during non core trading hours (any time not 9:30-4:00) is already an extremely standard thing that hedge funds do. So yes this would be true if there was some magical market maker that was willing to market make at unlimited size in either direction without doing any fading in response to news or someone trading against them during the normally very thin non core trading hours.
Would be nice to be able to have a summary after finishing for each question of what I estimated, what the correct answer was, and points scored.
title: bundling not bunding
For anyone specifically looking for a phone game, I have recently found Slice & Dice: https://tann.itch.io/slice-dice
to be very compelling, consistently interesting choices and good to jump in an out of while riding public transit for 20-30 minute spurts.
Talking to strangers is more enjoyable than people think. Update accordingly. Notice that if you already know about how much people enjoy talking to strangers, but not how much they expect to enjoy it, then this new information becomes bad news, but I presume it is instead good news. Does this mean it is ‘that easy’? No, but it’s also not that hard.
I read the 2014 paper that this NYT article refers to. They ran the experiment on METRA which is a very white commuter line from Chicago Suburbs into downtown Chicago. Sample size less than 200, mean age 49 (SD 13). So it’s unclear how generalizable this study is to other contexts.
It is especially interesting that the NYT article chose to use the 2014 paper when the same author published a 2021 paper with basically the exact same setup but in the London Underground (and with a ~4x larger sample size) which I would expect to have a significantly less homogenous makeup (and therefore possibly more generalizable) than METRA. This is probably b/c while the 2021 paper did find that people somewhat underestimated how much they would enjoy talking, the control group had a larger underestimate of how much they would enjoy doing control things. So there was a larger difference between actual enjoyment and expected enjoyment in the group that was told to do whatever they normally do on their commute than the group that was told to talk to whoever sat next to them.
I will say that ignoring the difference between expected enjoyment and actual enjoyment being inconclusive here, in the london underground paper people expected to most enjoy talking to strangers AND did most enjoy talking to strangers. So it is somewhat interesting that people generally choose to not talk to others despite both expecting and realizing the most enjoyment by talking to others. Maybe there’s some tail risk considerations here where better avg commute isn’t worth 1/10k chance of talking to someone who decides to stalk you etc.
Okay, I finished the first pass at this: https://sladebyrd.com/ai-safety-db/posts
any thoughts?