I now think it is plausible that Wolfram sued “over literary conventions”:
I suspect that Wolfram just wanted to reveal the relevant proof himself, first, in his book NKS (A New Kind of Science), and that Matthew Cook probably was contractually obligated to allow Wolfram to do that.
Given that the two parties settled, and that Cook published his paper about his proof in Wolfram’s own journal (Complex Systems), two years after NKS was published, seems to mostly confirm my suspicions.
The ‘components’ of our diet, e.g. meat, potatoes, etc., are very different now than earlier, and more different over the last 100 years than prior periods too.
I suspect people that are doing diets like this tho are much less obese, e.g. the Amish.
I’ve weirdly been less and less bothered since my previous comment! :)
I think “planecrash” is a better overall title still, so thanks for renaming all of the links.
Huh – I wonder if this has helped me since I made a concerted effort to eat leafy greens regularly (basically every day).
I always liked the ‘fact’ that celery has net-negative calories :)
I do also lean towards eating fruit raw versus, e.g. blended in a smoothy. Make-work for my gastrointestinal system!
I think you’re making an unsupported inferential leap in concluding “they seem oddly uninterested in …”.
I would not expect to know why they haven’t responded to my comments, even if I did bring up a good point – as you definitely have.
I don’t know, e.g. what their plans are, whether they even are the kind of blogger that edits posts versus write new follow-up posts instead, how much free time they have, whether they interpreted a comment as being hostile and thus haven’t replied, etc..
You make good points. But I would be scared if you ‘came after me’ as you seem to be doing to the SMTM authors!
It just seems to me that the SMTM authors are doing a very bad job at actually pursuing the truth
I think – personally – you’re holding them to an unrealistically high standard!
When I compare SMTM to the/a modal person or even a modal ‘rationalist’, I think they’re doing a fantastic job.
Please consider being at least a little more charitable and, e.g. ‘leaving people a line of retreat’.
We want to encourage each other to be better, NOT to discourage them from trying at all! :)
I was, and still am, tho much less, excited about the contamination theory – much easier to fix!
But I think I’m back to thinking basically along the lines you outlined.
I’m currently losing weight and my model of why is:
I’m less stressed, and depressed, than recently, and I’ve been able to better stop eating when I’m satiated.
I’m exercising regularly and intensely; mainly rock climbing and walking (with lots of decent elevation changes). It being sunnier and warmer with spring/summer has made this much more appealing.
I’m maybe (hypo)manic (or ‘in that direction’, i.e. ‘hypohypomanic’; or maybe even ‘euthymic’). I’m guessing recent sunlight-in-my-home changes triggered this (as well as the big recent drop in stress/depression).
I would love to see a study of weight gain in modern hunter-gatherer people that provides an experimental group with ‘very palatable’ food. I think I would be willing to bet that they would gain some weight.
I do also suspect that hunter-gatherers engage in a LOT of fairly strenuous physical activity. Walking – and living in a dense urban walkable city (in my case NYC) – does seem like maybe one of the most feasible ways to try to match that much higher level of overall physical activity. (Rock climbing is also pretty strenuous!)
I also thought it was (plausibly) a ‘friendly challenge’ – we should be willing to bet on our beliefs!
And we should be willing to bet and also trust each other to not defect from our common good.
The challenge did specify [emphasis mine]:
up to $1000
I think they’re a proponent of the ‘too palatable food’ theory.
Thanks!
I’ve definitely downgraded the (lithium) contamination theory. I’ll still take a (very modest) 100:1 bet on it tho :)
In regard to your (implied) criticism that SMTM’s blog post(s) haven’t been edited, it occurred to me that they may not be a ‘edit blog posts’ person. That seems related to their offered reasons for refusing bet challenge, i.e. ‘we’re in hypothesis exploration mode’. They might similarly be intending to write a follow-up blog post instead of editing the existing one.
(I actually prefer ‘new post’ versus ‘edit existing post’ as a blogging/writing policy – if there isn’t a very nice (e.g. ‘GitHub like’) history diff visualization available of the edit history.)
I’ve been asking various people this same question basically and am still looking for (more) concrete recommendations. (Several people basically answered ‘work for a finance/investment/trading company’, which is … not ideal!)
I’m tentatively planning on doing a more intense search for this soonish and I’ll comment here, or supply an answer, if I find anything that seems promising.
I’ve already explained why that’s an anti pattern. If you had rejected the very idea of magnetism when magnetism wasn’t understood, it would not now be understood.
I’m rejecting the idea of ‘qualia’ for the same reason I wouldn’t reject the idea of magnetism – they both seem (or would seem, for magnetism, in your hypothetical).
I’m rejecting ‘mysterious answers’, e.g. “Theres not supposed to be a good explanation of qualia”.
Who said otherwise? You seem to have decided that “qualia are subjective experiences and we don’t understand them” means something like “qualia are entirely and irredeemably subjective and will be a mystery forever”.
Sorry – that’s not what I intended to convey. And maybe we’re writing past each other about this. I suspect that ‘qualia’ is basically equivalent to something far more general, e.g. ‘information processing’, and that our intuitions about what ‘qualia’ are, and the underlying mechanics of them (of which many people seem to insist don’t exist), are based on the limited means we have of currently, e.g. introspecting on them, communicating with each other about them, and weakly generalizing to other entities (e.g. animals) or possible beings (e.g. AIs).
I also suspect that ‘consciousness’ – which I’m currently (loosely/casually) modeling as ‘being capable of telling stories’ – and us having consciousness, makes thinking and discussing ‘qualia’ more difficult than I suspect it will turn out to be.
Maybe we will have qualiometers one day, and maybe we will abandon the very idea of qualia. But maybe we won’t, so we have no reason to treat qualia as poison now.
What I’m (kinda) ‘treating as poison’ is the seemingly common view that ‘qualia’ cannot be explained at all, i.e. that it’s inherently and inescapably ‘subjective’. It sure seems like at least some people – tho maybe not yourself – are either in the process ‘retreating’, or have adopted a ‘posture’ whereby they’re constantly ‘ready to retreat’, from the ‘advances’ of ‘objective investigation’ and then claim that only the ‘leftover’ parts of ‘qualia’ that remain ‘subjective’ are the ‘real qualia’.
I’m not sure what you mean by “how high the relative tech capabilities are of the AGI”.
I think the general capability of the AGI itself, not “tech” capabilities specifically, are plenty dangerous themselves.
The general danger seems more like ‘a really powerful but unaligned optimizer’ that’s ‘let loose’.
I’m not sure that ‘agent-ness’ is necessary for catastrophe; just ‘strong enough optimization’ and a lack of our own capability in predicting the consequences of running the AGI.
I do agree with this:
It’s not supposed to be a self-contained cinematic universe, it’s supposed to be “we have little/no reason to expect it to not be at least this weird”, according to his background assumptions (which he almost always goes into more detail on anyway).
But specific P problems can still be ‘too hard’ to solve practically.
I agree that “verification is much, much easier than generation”.
But I don’t agree that verification is generally ‘easy enough’.
I’ve long interpreted Eliezer, in terms of your disagreements [2-6], as offering deliberately exaggerated examples.
I do think you might be right about this [from disagreement 2]:
By the time we have AI systems that can overpower humans decisively with nanotech, we have other AI systems that will either kill humans in more boring ways or else radically advanced the state of human R&D.
I do like your points overall for disagreements [1] and [2].
I feel like there’s still something being ‘lost in translation’. When I think the of the Eliezer-AGI and why it’s an existential risk, I think that it would be able to exploit a bunch of ‘profitable capability boosting opportunities’. I agree that a roughly minimal ‘AGI’ probably/might very well not be able to do that. Some possible AGI can tho. But you’re also right that there are other risk, possibily also existential, that we should expect to face before Eliezer’s specific ‘movie plots’ would be possible.
But then I also think the specific ‘movie plots’ are besides the point.
If you’re right that some other AI system mega+-kills humans – that then is the “nanotech” to fear. If it’s a foregone conclusion, it’s not that much better in the cases where it takes the AI, e.g. several years to kill us all, versus ‘instantaneously’.
I also have a feeling that:
Some (minimal) ‘AGI’ is very possible, e.g. in the next five (5) years.
The gap between ‘very disruptive’ and ‘game over’ might be very small.
I guess I disagree with your disagreement [7]. I think partly because:
AI systems that can meaningfully accelerate progress by generating ideas, recognizing problems for those ideas and, proposing modifications to proposals, etc.
might be AI systems that are “catastrophically dangerous” because of the above.
I think maybe one disagreement I have with both you and Eliezer is that I don’t think an AI system needs to be ‘adversarial’ to be catastrophically dangerous. A sufficiently critical feature missing from the training data might be sufficient to generate, e.g. an idea, that can be apparently reasonably verified as aligned and yet lead to catastrophe.
I am very happy that you’re asking for more details about a lot of Eliezer’s intuitions. That seems likely to be helpful even if they’re wrong.
I’m skeptical of your disagreement [19]. Is it in fact the case that we currently have good enough abilities at verifying, e.g. ideas, problems, and proposals? I don’t feel like that’s the case; definitely not obviously so.
I think I’ve updated towards your disagreements [18] and [22], especially because we’re probably selecting for understandable AIs to at least some degree. It seems like people are already explicitly developing AI systems to generate ‘super human’ human-like behavior. Some AI systems probably are, and will continue to be, arbitrarily ‘alien’ tho.
For your disagreement [23], I’d really like to read some specific details about how that could work, AIs reasoning about each other’s code.
Overall, I think you made some good/great points and I’ve updated towards ‘hope’ a little. My ‘take on your take (on Eliezer’s takes)’ is that I don’t know what to think really, but I’m glad that you’re both writing these posts!
I think I’m missing a LOT of context you have about this. I very well could be – probably am – missing some point, but I also feel like you’re discouraging me from voicing anything that doesn’t assume whatever your point is. Is it just that “Stephen Wolfram is bad and everyone should ignore him.”? I honestly tried to investigate this, however poorly I might have done that, but this comment comes across as pretty hostile. Is it your intention to dissuade me from writing about this at all?
I don’t what norms you’re referring to!
But, AFAIK, some commercial norms do in fact condone or allow for aggressive enforcement of NDAs, even where there are is (obviously) “zero money at stake”. I also guessed that the point of the lawsuit was maybe (partially) about ‘enforcing NDAs generally’ too, which is also one way it could ‘accomplish good’ even tho it didn’t prevent dissemination of the info in the paper itself.
If a member of any other research team, in defiance of their team leader (‘lead/principle investigator’), published a paper describing work done on a team project, is there no recourse or punishment expected?
Do researchers have a blanket right to publish at any time, on their own, about work they’ve done as an employee? That’s surprising if true! I wouldn’t think that’s true even in purely academic organizations.
How do you know that Wolfram’s ‘original plan’ wasn’t to allow Cook to publish the paper of his proof after the release of NKS? Is there some reason why making Cook wait was unconscionable? I’m sincerely curious! What’s the big deal? Why was Cook waiting so bad that of course he should have been able to publish his paper whenever he wanted? Or was making him wait a year fine, but three (or more) not?
Why didn’t Cook just retract his paper and settle with Wolfram immediately? I couldn’t find any relevant details about the suit and it appeared to me that the records were sealed, so I’d guess only the people involved could answer.
I don’t understand your theory of Cook. Why did he publish the paper when apparently Wolfram, his employer, didn’t want him to do that? Why didn’t he just retract or withdraw his paper? Why was he in the right in all of this? (Was he in the right about this?) Should Wolfram have just ignored Cook publishing the paper? Why shouldn’t he have asserted his legal right to control when the paper was published? What’s different about this specific paper than others, in both academia and commercial research? What am I missing?
I am very sympathetic to the lawsuit being “cruel and vindictive”! But I’m also sympathetic to being “cruel and vindictive”. If a (hypothetical) employee of mine openly defied my orders, e.g. released info about work I’d paid them to do and told them not to publish – I’d be pissed! If they had signed a legal agreement to NOT do that kind of thing too, I suspect I’d think of suing them myself. That the project was a non-fiction book doesn’t seem like it’d make this that much different. If nothing else, Cook deprived Wolfram of being able to market NKS as announcing that particular result.
As for the citation/credit generally, NKS is a large book and, AFAICT, it was NOT intended to be an ‘academic’ work. Had it been written in that ‘style’, I’d guess it would have been WAY too big to have been published at all.
Is there in fact a reasonable way that Wolfram could have cited other’s works, and described the contributions to the book itself, that would work not only for the work done by Matthew Cook, but everyone else involved?
I still don’t know very many details of all the contributions to just the Rule 110 proof paper!
(I don’t even know what academic paper authorship means. AFAIK, it is somewhat specific to the field
I am happy to admit that it’s certainly very likely that Wolfram failed to properly cite other’s work, or fairly credit their contributions. I am very sure that he does in fact have a reputation for that.
I can’t “almost hear the teeth being pulled” but I’m even more confused – is Wolfram’s own citations ‘merely’ as bad as that done by many other PIs? I can’t tell what standard you’re judging him by, or how well I should expect him to fare by it, especially given his reputation for doing poorly in this area.
What does the distribution of how well others do in this regard look like? Are, e.g. 95% of researchers, or authors of ‘pop science’ books, basically ‘perfect’? Is Wolfram uniquely terrible?
I also can’t see what you mean by Wolfram “jamming himself in” in any of the quotes you provided.
I also don’t know that Cook didn’t do other “technical content” besides the Rule 110 proof.
I thought Wolfram was clear about NKS being a ‘team effort’ so I still think it’s (somewhat) reasonable that a much longer and more detailed description of more precise contributions were omitted.