Professionally, AI, science, AI4Science, Safety4AI. Also human ecology and indonesian death metal remixes.
See danmackinlay.name for more words about background and my now page for bonus stuff.
Dan MacKinlay
Karma: 501
Thanks! Yes. I think the integration-of-compute-over-time into spend captures some of that, which is kind of the value proposition of that approach. Making it more explicit would be more strictly useful. I think that we would then need to have a model for “how much” the compute we have already done makes the compute we will do next more efficient, we now know that Conjugate Gradient can solve linear systems fast and don’t need to do it the
And then the next fork: which are more capability-enabling than safety enabling seems very important, and also very difficult. In fact one reason I went for a very abstract model is that I kinda wanted to integrate that out. nurse a vibe that it might in addition to being difficult, be path dependent (depending on which algorithms arrive with which hardware or geopolitical regime) and if that needs to be accounted for, the model here would be unhelpful.
Yeah, I did, but it’s not accessible through the API yet. I filed a ticket with the lightcone team
Should our journal publish AI-drafted manuscripts?
Who computes?
@Yonatan Cale, I got a helpful user for you! (I can vouch for Gunnar’s helpfulness)
The Alignment Journal: Organization, Personnel, and Scope
Thanks for the sanitised version. It is substantially more… tactful.
Yes, these are good questions. The tl;dr is that a whole bunch of stuff has been happening. Some of it has been slower than we hoped. But other parts of it have been harder to do in the open that we hoped. Nonetheless, we are getting close.
I don’t want to do too much foreshadowing of future debriefs (stay tuned for upcoming “do’s and don’t’s of journal founding”), so here is a short concretization of current roadblocks:
1. The main component of an academically respectable journal is the human technology of getting a good board who are
a. academically respected (in some research community)
b. have time to volunteer on a project, and
c. concur on the scope, aims, and methods of the journal
This turns out to be an intrinsically sensitive and difficult process, which we have not spoken much about.
The main reason for that is that it is super important that the board be credible and that the staff and editors have a high-trust relationship and a good process for agreeing on stuff.
It has not felt like a wise move for the founders to unilaterally declare exactly what the board will do, before the board itself has decided — otherwise they are not an editorial board
Thus we’ve had no flashy announcements of “new journal policy X” while we sort all that out.
One mild friction here has been bringing together people from inside the big institutions (universities, labs) and from outside (independent researchers, research non-profits etc). That is tough, but I think worth it if we want to live up to our pitch.
Nonetheless, we have made a heap of progress there and we are looking forward to sharing who and what our shiny new board is as soon as we can, which will be when they sign off on it.
2. Software! Also happening and also hard but progressing. As you’ve noted, the testing version of the software is online, thanks to the efforts of @Yonatan Cale, who has been labouring away.
As also noted, the article “under review” are not real articles. I suspect that the reason we are testing using those will be relatively clear, but to spell it out: we can’t easily do a “real” review.
A full review requires anonymity and many hours of human time— the amount of time that you would only commit if you were credibly expecting a possible real publication result at the end.
People don’t have maybe-articles and hours of time up their sleeves, typically.
Further, the real review is supposed to be partially blinded, which is difficult (although not impossible) to arrange while getting UX feedback from the submitting authors and reviewers. Going to great lengths to fake that out has not seemed like a great use of time, but— watch this space.
Anyway, thanks for the prod!
We are, by the way, open to help. If you want to beta test the software, or you have excellent contacts, especially amongst well-credentialled university professors, please get in touch on our volunteer form, or just email contact@alignmentjournal.org
Yes, I want some math too. I think we could cash this out in some basic mixture models, even. Are you still working on it?
This is as good a place as as any to mention George Hosu’s The world won’t end, but we should be ashamed for trying. He doesn’t lean so much on deep evolutionary analogies as wonder what very specifically has caused us to see soo many bringing-about-the-apocalypse-in-order-to-avoid-it dynamics.
The leading analogy is: What was it that lead to us handling nuclear weapons by racing and biological weapons by … not?
Since publishing this we noticed the following initiative: https://future-science.org/mirror
Mirror: An Automated Journal of AI Interpretability is a fully automated journal of AI interpretability. This journal features original research composed, conducted, and written entirely by LLMs analyzing LLMs. Much of the research published in Mirror falls within the category of “mechanistic interpretability,” in which model behaviors are decomposed into operations in the model’s internal representation space, but any rigorous research advancing our understanding of LLMs is welcome, be it mechanistic, behavioral, or theoretical.
I would be curious to know if others get value from this “fully automated journal”
An Alignment Journal: Adaptation to AI
An Alignment Journal: Features and policies
We do not yet plan to support replications of empirical work. Organisationally, there is a desire to keep opening scope tight and theoretical to avoid having diffuse messaging at start up
Personally, I would make a case that replications are not as important in the ML/AI research as in the sciences of the physical (although this depends somewhat on what we mean by “replications”)
That said, I think that there is a strong argument for replications generally, and maybe in this field too, and if the Editorial Board agreed with that, then that is what we would do. I am beholden at this point to mention the connection to the UnJournal work that David has mentioned elsewhere in these comments.
I like this idea aesthetically. I foresee some challenges in making “staking” something that won’t trigger alarms in the existing research bureaucracies that host many of our potential authors. If you have clever ideas for how to handle that I would be curious to hear.
This publication bias story in ML is a whole can of worms which I would love to open at some point. tl;dr it is a problem, but the field has semi-accidentally mitigated many of the worse excesses of it. There is an IMO massively under-regarded work on this— Moritz Hardt’s Machine Learning Benchmarks, which I will write a LW review of some day if I have time.
Yes, I’m excited to see what we can learn David’s experience, especially given the incentive designer’s insight that he brings to this. We also, collectively, have some experience with the ILIAD conferences which was a precursor experimenting with alternative compensation mechanisms. See Proceedings of ILIAD: Lessons and Progress for some analysis of that project.
We are trying to do both, in that we are attempting to be a bridge between LW and wider scientific communities. Where do you feel our tone might be excluding domain scientists?
An Alignment Journal: Coming Soon
@megasilverfist there are quite a few of us based in Melbourne. HMU.
Now that I’ve said that out loud, it occurs to me that we could add in software as a self-excitation term in the incremental version of the model, like the Hawkes process. I will try to carve out some time to revisit that.