Senior Research Scientist at NTT Research, Physics & Informatics Lab. jessriedel.com , jessriedel[at]gmail[dot]com
JessRiedel
Karma: 965
An Alignment Journal: Adaptation to AI
I found one associated with Aurelia Song :
“Whole brain emulation system”
Abstract: Systems and methods to allow for generating a simulated humanoid. The simulated humanoid operates in a simulation space with simulated humanoid having a whole brain emulation module. The whole brain emulation module includes a virtual stimuli input module that is configured to receive or capture stimuli input data. The whole brain emulation module includes an encoder that is configured to translate the stimuli input data into a simulated functional neurodata frame. The whole brain emulation module also includes a brain state module that maintains a current brain state corresponding to a current functional neurodata frame of the simulated humanoid.
Inventor: Viktor Toth, Connor Flexman, Aurelia Song, Maximilian Jakob Schons, Robert Bolkow, Michael Andregg, Alexander D. Wissner-Grosshttps://patents.google.com/patent/US12536425B1/en
(This of course does not seem to be economically relevant to Nectome.)
We do not expect to be immediately overrun by slop submissions and reviews when the journal launches, but this may become a bigger issue as the journal grows.
As an interested reader, I would prefer having a filter for low quality AI content to none, if only to be comforted by the knowledge that I’m less likely to be reading slop.
To be clear, we mean that in the short-term we expect to be able to desk-reject low-quality submissions by hand, whether AI-generated or otherwise. We never want to publish it, and we expect to mostly spare reviewers having to read it. The open question is how quickly we will need to develop automated tools to maintain these standard without putting undue burden on our editors.
An Alignment Journal: Features and policies
+ risk of being locked in if your preferred provider changes,
The contract is transferable, so if Nectome becomes successful (many patients in the future) you presumably should be able to recover a large fraction of the contract value at that time.
(Obviously if the procedure becomes cheap then you won’t recover as much, but that’s inherent to the “pay far in advance for a discount on current prices” bet, regardless of your provider preferences changing.)
Thanks, this is specific and useful. I think it’s less that we’re attempting to target LW and more that it’s just how we tend to talk. We’ll work on keeping the word choice more conventional and professional.
I’m interested to hear more. Would it mostly be for practical reasons (financial sustainability), or to reduce the submission of bad work that wastes editor/reviewer time?
We’re been discussing scope a lot, and this is indeed a big question. Some considerations:
We can only do a good job reviewing papers in a given field if we’ve got a good editor in that field. So scope will be constrained by the practical question of who we are able to get.
Justified or not, it’s a bit perilous for a new journal in mathematical/technical fields to also publish papers in fields with less objective criteria, especially in a new field like alignment with contentious boundaries. Even a slight perception of softness could hurt us in the beginning.
Of course, even in pure math importance is a subjective criteria, but perception doesn’t necessarily track this. And some sorts of philosophy (formal logic) can have pretty objective standards, but I think this is not the sort of philosophy you’re interested in.
It’s possible this can be mitigated with journal sectioning (e.g., Alignment: Mathematical, Alignment: Empirical, Alignment: Philosophical, etc.), but it’s dicey and hard to do right, especially when the journal is new and not yet established.
Regarding scope, we always need to ask: Would this sort of work benefit from review? Could reviewers meaningfully improve the work? Could we establish a reputation where publication (or the contents of the reviewer abstract) was a useful, credible signal to other researchers?
There’s no point in starting a journal if we exclude the sort of work that actually matters.
Incidentally, if someone wanted to help make the case for philosophy in the journal, a very useful thing would be to compile a list of papers (which could be a mix of published in traditional journals and not, and need not be strictly on alignment) to serve as exemplars of what should be included.
Yea, I can definitely see the selective reporting problem, which goes beyond the problem of negative results being unfairly denied publication. But to combat selective reporting, you’d really need to require preregistered experiments, which is more of a collective-action problem between journals, since if any of them allow un-preregistered experiments, the authors can just publish there. (Of course, you can try and convince the broad community to ignore all experiments that aren’t preregistered, but if you can do this then you’ve already won; the journals will be strongly incentivized to follow suit.)
Required preregistration is just very cumbersome and difficult to do for exploratory science; it really seems only feasible for the later stages of things like medical trials or big contentious question requiring a decisive experiment.
Sounds like a lot more risk of bias (and appearance thereof) than it’s worth. At the least, I figure you’d need to have a disclosure on every paper authored by an employee of the company, as well as conflict-of-interest rules making sure the action editor and reviewers were un-biased. Would be a pain, and still suspect. (Here’s GPT’s summary of how existing journals handle this, most commonly in medical research: https://chatgpt.com/share/69a992c3-75d8-8002-a592-a8053ee1cdbe )
An intermediate and more plausible case would be personal donations from a former or current employee of a frontier company; we expect many to be philanthropically motivated in the coming years. Imo, this is something we’d consider, but I haven’t thought about it much yet. We’re set for funding for the first year.
If we are successful in standing up a good and well-respected journal, I expect there will be many funders interested in supporting us. (And if we’re not successful, the issue is moot.) So I’m not too worried about getting backed into a corner where our only option to keep running is money from a potentially biasing source. We’d ideally like a broad diverse base of funders, like the arXiv.
An example, for what it’s worth: Quantum journal is relatively new physics arXiv-overlay journal (10 years old) that runs on volunteers effort and modest publication fees (~$700). They didn’t want the fees to be a barrier to submitting, so they have a very easy process for getting the fees waived; you basically just have to ask. My understanding is that they still have not been overrun with slop, and whenever I am asked to review the papers are reasonable quality. So it does not seem they are foisting the slop handling onto reviewers. Desk-rejection by the editors appears to be enough.
Hmm. Ultimately it would be up to the editorial board, but here’s why I personally think these features are probably low priority given their nontrivial cost: (1) I presume we are talking about numerical experiments, and I expect the foundational/conceptual topics we want to publish on are less vulnerable to publication bias than, say, experimental psychology or economics. It would be more like pre-registering numerical math papers. That said, if you think the alignment literature has big problems with publication bias, I’d be interested to hear more. (2) Our primary audience is other researchers. Often, journals are motivated to provide press abstracts to induce popular coverage (by making a time-pressed journalist’s life easier, as with a press release), and increasing popular coverage is not one of our goals. It can also be a corrupting influence (although there are steps that we could take to reduce this). High quality popular-science journalist will generally take the time to talk to the authors and outside researchers to get the story right.
It depends on few factors, but April at the earliest for initial submissions. Publication will almost certainly be on a rolling basis (no discrete issues). Our ambitious goal is to drive the submission-to-publication time down to something like a month, but it will require combining several new tricks so it won’t be that fast at the beginning.
An Alignment Journal: Coming Soon
Proceedings of ILIAD: Lessons and Progress
I’d also be surprised.
No, vacuum decay generally expands at sub-light speed.
Vacuum decay is fast but not instant, and there will almost certainly be branches where it maims you and then reverses. Likewise, you can make suicide machines very reliable and fast. It’s unreasonable to think any of these mechanical details matter.
This work was co-authored by Jordan Stone, Darryl Wright, and Youssef Saleh, whose names appear on the EA Forum post but not on this cross post to LW.
Initial support is being provided by the AI Safety Tactical Opportunities Fund.