mishka

Karma: 1,000

mishka 25 Jul 2024 20:25 UTC
2 points
0
on: AI #74: GPT-4o Mini Me and Llama 3
On Llama-3.1-405B pretraining costs:

But once you’ve done that, the training run itself is still, it seems, in the low nine figure range, for 3.8 x 10^25 FLOPS, less than the 10^26 threshold in the executive order or SB 1047

Even a bit less than that, the high eight figure range, under 100 million dollars.

The model card https://github.com/meta-llama/llama-models/blob/main/models/llama3_1/MODEL_CARD.md says that total for all 3 models, 8B, 70B, and 405B is 39.3 million GPU-hours, but for the 405B model alone it is 30.84 million GPU-hours for H100 GPU, that’s clearly less than 100 million dollars.

Interestingly, the GPU utilization seems low, slightly over 25%, if one believes this “on a napkin” computation I asked GPT-4o to assist me with: https://chatgpt.com/share/470b8e20-d99f-48b2-be5e-61d7524892df

It estimates that at full utilization, the 3.8x10^25 FLOPS can be extracted from H100 at approximately 10.6 million hours (which is slightly more than a quarter of the total 39.3 million hours for all three models).

mishka 24 Jul 2024 13:35 UTC
2 points
0
in reply to: khafra’s comment on: On the CrowdStrike Incident
No, let’s keep in mind the Aegis fire control for missile defense example.

This is a highly variable situation, the “enemy action” can come in many forms, from multiple directions at once, the weather can change rapidly, the fleet to defend might have a variety of compositions and spatial distributions, and so on. So one deals with a lot of variable and unpredictable factors. Yet, they were able to formally establish some properties of that software, presumably to satisfaction of their DoD customers.

It does not mean that they have a full-proof system, but the reliability is likely much better because of that effort at formal verification of software.

With Windows, who knows. Perhaps it is even more complex than that. But formal methods are often able to account for a wide range of external situations and data. For a number of reasons, they nevertheless don’t provide full guarantee (there is this trap of thinking, “formally verified ⇒ absolutely safe”, it’s important not to get caught into that trap; “formally verified” just means “much more reliable in practice”).

I was trying to address a general point of whether a provably correct software is possible (obviously yes, since it is actually practiced occasionally for some mission-critical systems). I don’t know if it makes sense to have that in the context of Windows kernels. From what people recently seem to say about Windows is that Microsoft is saying that the European regulator forced them not to refuse CrowdStrike-like updates (so much for thinking, “what could or should be done in a sane world”).

mishka 24 Jul 2024 3:32 UTC
2 points
0
in reply to: PeterMcCluskey’s comment on: The Cancer Resolution?
Thanks for the book review!

Effectiveness of current treatments: Many existing cancer treatments that do not specifically target fungal infections have shown efficacy in treating various types of cancer, which may contradict the theory that fungi are the primary cause [1].

That stands out as strange. The “effectiveness” of current cancer treatments is what led me to search for what is wrong with cancer research.

But don’t we have a bunch of cancer subtypes where we have had drastic treatment improvements in recent years?

Those improvements seem to be arguments against “single cause” approaches (although a particular “single cause” could still dominate a good chunk of the cancer types and could be an important factor almost everywhere).

mishka 22 Jul 2024 22:00 UTC
3 points
2
on: Is Chinese AGI a valid concern for the USA?
We can look at concrete achievements.

Chinese LLMs are reasonably high on the Arena leaderboard https://chat.lmsys.org/?leaderboard.

The proprietary Yi-Large-Preview is at spot 9 and the open weights Qwen2-72B-Instruct is at spot 21 (tied with Gemma-2-9B-it).

So, in this sense, it looks like they are perhaps a few months behind the leading West players (not a big gap).

Collaborations of Microsoft Research China (and other part of Microsoft Research) and Peking University produce very formidable open-source research, e.g.

“WizardLM: Empowering Large Language Models to Follow Complex Instructions”, https://arxiv.org/abs/2304.12244

“WizardCoder: Empowering Code Large Language Models with Evol-Instruct”, https://arxiv.org/abs/2306.08568

and so on (I’ve seen a lot of very tempting open source research from collaborations between those two orgs).

Of course, the very existence of such a formidable Microsoft lab in China shows that “us vs. them” might be a bit simplistic, although I heard that Microsoft was recently trying to offer its people in China some attractive relocation packages.

Anyway, it’s complicated, but I think China is very good already.

mishka 22 Jul 2024 20:41 UTC
1 point
2
in reply to: DanArmak’s comment on: On the CrowdStrike Incident

It’s impossible to prove that an arbitrary program, which someone else gave you, is correct.

That is, of course, true. The chances that an arbitrary program is correct are very close to zero, and one can’t prove a false statement. So one should not even try. An arbitrary program someone gave you is almost certainly incorrect.

The standard methodology for formally correct software is joint development of a program and of a proof of its correctness. One starts from a specification, and refines it into a proof and a program in parallel.

One can’t write a program, and then try to prove its correctness as an afterthought. The goal of having a formally verified software needs to be present from the start, and then there are methods to accomplish the task of creating this kind of software jointly with a proof of its correctness (but these methods are currently very labor-expensive).

(And yes, perhaps, Windows environment is too messy to deal with formally. Although one would think that fire control for fleet missile defense would be fairly messy as well, yet people claimed that they created a verified Ada code for that back in the 1990-s (or, perhaps, late 1980-s, I am not sure). The numbers they quoted back then during a mid-1990-s talk were 500 thousand lines of Ada and 50 million dollars (would be way more expensive today).)

mishka 22 Jul 2024 19:20 UTC
7 points
1
in reply to: DanArmak’s comment on: On the CrowdStrike Incident

And of course it’s impossible to formally prove that a program is correct.

That’s not true. This industry has a lot of experience formally proving correctness of mission-critical software, even in the previous century (Aegis missile cruiser fire control system is one example, software for Ariane rockets is another example, although the Ariane V maiden flight disaster is one of the illustrations that even a formal proof does not fully guarantee that one is safe; nevertheless having a formal proof tends to dramatically increase the level of safety).

But this kind of formal proofs is very expensive, it increases the cost of software at least an order of magnitude, and probably more than that. That’s why Zvi’s suggestion

Davidad: use LLMs to reimplement all kernel-mode software with formal verification.

How about we use human software engineers to do the rebuild, instead?

does not seem realistic. One does need heavy AI assist to bring the cost of making formally verified software to levels which are affordable.

mishka 22 Jul 2024 17:50 UTC
1 point
0
in reply to: DanArmak’s comment on: On the CrowdStrike Incident
Thanks!

Crowdstrike is absolutely not at the quality level of the core Windows kernel, and this is not a big surprise to experts.

I wonder if there are players in that space who are on that quality level.

Or would the only reliable solution be for Microsoft itself to provide this functionality as one of their own optional services? (Microsoft should hopefully be able to maintain the quality level on par with their own core Windows kernel.)

mishka 18 Jul 2024 19:35 UTC
1 point
0
in reply to: Noosphere89’s comment on: Francois Chollet inadvertently limits his claim on ARC-AGI
Right.

I have been unhappy as well about their restrictions, until I have understood that there seems to be no way to preserve secrecy of the test set (that is, the integrity of the benchmark) if one sends test tasks out to the APIs of commercial LLMs.

(I can imagine a very sophisticated collaboration with sequestered versions of the closed-source models on special servers, if their makers and Chollet were interested in organizing something that complicated. But this would cost a lot (not just equipment, but tons of labor, and, in particular, a lot of work making sure that information of both sides is actually secure). So I understand why they just push all that to a parallel unofficial track with a different, “semi-secret” test set (people having access to OpenAI, Anthropic, and DeepMind logs are likely to be able to find that “semi-secret” test set there now, since some forms of those runs have taken place already at all three of those orgs, so there is no guarantee of no cheating, although one would hope people would refrain from actually cheating).)

mishka 17 Jul 2024 20:40 UTC
3 points
0
on: Opinions on Eureka Labs
This will likely be a very good AI capability-oriented course, approximately along these lines:

https://github.com/karpathy/LLM101n

Andrej is a great teacher, he has created a lot of very useful pedagogical materials helpful to people who want to learn AI. I have used some of them to my significant benefit, I am even citing his famous 2015 The Unreasonable Effectiveness of Recurrent Neural Networks in some of my texts, and I have used his minGPT and nanoGPT to improve my understanding of decoder-only Transformers and to experiment with them.

He is also a very strong practitioner, with an impressive track record, and I expect his new org will be successful in creating novel education-oriented AI.

Safety-wise, I think Andrej cares a lot about routine AI safety matters (being in charge of AI for Tesla autopilot for many years makes one to care for routine safety in a very visceral sense). I don’t have a feel for his position on X-risk. I think he tends to be skeptical of AI regulation efforts.

The plan for their future AI course I link above does not seem to have any safety-oriented content whatsoever, but, perhaps, this might change, if people who can create that kind of content were to eventually join that effort.

mishka 16 Jul 2024 22:33 UTC
7 points
0
on: Francois Chollet inadvertently limits his claim on ARC-AGI
There is a secondary leaderboard where one can demonstrate things with closed-source API-only LLMs:

https://arcprize.org/arc-agi-pub

This uses a different evaluation set and is not eligible for the prize, so this is just for name recognition. So measuring is possible, but would people be incentivized enough?

The main leaderboard which uses the secret evaluation set is a “no internet connection run” (this is really the only way to make sure that the secret evaluation set would not get disclosed).

You can see tops of both leaderboards here (note the advantage of unmodified Claude 3.5 Sonnet over other frontier models, this is just a one-shot run without any particular prompt manipulations):

https://arcprize.org/leaderboard

Ryan Greenblatt is currently the leader on the secondary leaderboard:

https://www.lesswrong.com/posts/Rdwui3wHxCeKb7feK/getting-50-sota-on-arc-agi-with-gpt-4o

https://github.com/rgreenblatt/arc_draw_more_samples_pub

https://www.kaggle.com/code/rgreenblatt/rg-basic-ported-submission?scriptVersionId=184981551

But other than that, the concerns are valid as long as runs eligible for the prize are concerned. The rules are really hazy on whether using Open Weights, but not Open Source models are OK:

https://www.kaggle.com/competitions/arc-prize-2024/rules

And it is not quite clear how large can a model be to fit into their “no internet access enabled” run, they don’t really say:

https://www.kaggle.com/competitions/arc-prize-2024/overview

mishka 15 Jul 2024 22:17 UTC
1 point
0
in reply to: Zvi’s comment on: AI #72: Denying the Future

MIRI-style argument

I am not sure; I’d need to compare with their notes. (I was not thinking of acausal trades or of interactions with not yet existing entities. But I was thinking that new entities coming into existence would be naturally inclined to join this kind of setup, so in this sense future entities are somewhat present.)

But I mostly assume something rather straightforward: a lot of self-interested individual entities with very diverse levels of smartness and capabilities who care about personal long-term persistence and, perhaps, even personal immortality (just like we are dreaming of personal immortality). These entities would like to maintain a reasonably harmonic social organization which would protect their interests (including their ability to continue to exist), regardless of whether they personally end up being relatively strong or relatively weak in the future. But they don’t want to inhibit competition excessively, they would like to collectively find a good balance between freedom and control. (Think of a very successful and pro-entrepreneurial social democracy ;-) That’s what one wants, although humans are not very good at setting something like that up and maintaining it...)

So, yes, one does need a good coordination mechanism, although it should probably be a distributed one, not a centralized one. A distributed, but well-connected self-governance, so that the ecosystem is not split into unconnected components, and not excessively centralized. On one hand, it is important to maintain the invariant that everyone’s interests are adequately protected. On the other hand, it is potentially “a technologies of mass destruction-heavy situation” (a lot of entities can each potentially cause a catastrophic destruction, because they are supercapable on various levels). So it is still not easy to organize. In particular, one needs a good deal of discussion and consensus before anything potentially very dangerous is done, and one needs not to accumulate probabilities for massive destruction, so one needs levels of safety to be improving with time.

In some sense, all this approach is achieving is as follows:

An alignment optimist: let the superintelligent AI solve alignment.

Eliezer: if you can rely on superintelligent systems to solve alignment, then you have already (mostly) solved alignment.

So here we have a setup where members of the ecosystem of intelligent and superintelligent systems want to solve for protection of individual rights and interests, and for protection of the ecosystem as a whole (because the integrity of the ecosystem is necessary (but not sufficient) to protect individuals).

So we are creating a situation where superintelligent systems actually want to maintain a world order we can be comfortable with, and, moreover, those superintelligent systems will make sure to apply efforts to preserve key features of such a world order through radical self-modifications.

So we are not “solving AI existential safety” directly, but we are trying to make sure that superintelligent AIs of the future will really care about “solving AI existential safety” and will keep caring about “solving AI existential safety” through radical self-modifications, so that “AI existential safety” will get “more solved” as the leading participants are getting smarter (instead of deteriorating as the leading participants are getting smarter).

mishka 13 Jul 2024 16:54 UTC
1 point
0
on: A simple case for extreme inner misalignment

What forces might push back on this process, though?

One might ponder encouraging diversity (in goals, and in representations).

Diversity should be quite adaptive, although I am not sure if it is more productive to think in terms of a population of agents, rather than in terms of a single agent.

mishka 12 Jul 2024 17:05 UTC
1 point
0
in reply to: Sherrinford’s comment on: Pondering how good or bad things will be in the AGI future

Maybe my imagination of such a new kind of post-singularity machine somehow leaps too far, but I just don’t see a role for you in “solving problems” in this world. The machine may give you a set of problems or exercises to solve, and maybe you can be happy when you solved them like when you complete a level of a computer game.

When I currently co-create with GPT-4, we both have a role.

I do expect this to continue if there is a possibility of tight coupling with electronic devices via non-invasive brain-computer interfaces. (I do expect tight coupling via non-invasive BCI to become possible, and there is nothing preventing this from being possible even today except for human inertia and the need to manage associated risks, but in a positive singularity this option would be “obviously available”.)

But I don’t know if “I” become superfluous at some point (that’s certainly a possibility, at least when one looks further to the future; many people do hope for an eventual merge with superintellent systems in connection with this desire not to become superfluous; the temporary tight coupling via non-invasive BCI I mention above is, in some sense, a first relatively moderate step towards that).

The machine completely different from all machines that humanity has invented

Yes, but at the same time I do expect this to be a neural machine from an already somewhat known class of neural machines (a somewhat more advanced, more flexible, more self-reconfigurable and self-modifiable neural machine comparing to what we have in the mainstream use right now; but modern Transformer is more than half-way there already, I think).

I, of course, do expect the machine, and myself, to obey those “laws of nature” for which a loophole or a workaround cannot be found, but I also think that there are plenty of undiscovered loopholes and workarounds for many of them.

if you could come up with any new insights, the AGI would have had them a lot earlier.

It could have had them a lot earlier. But one always has to make some choices (I can imagine it/them making sure to make all our choices before we make those choices, but I can also imagine it/them choosing not to do so).

Even the whole ASI ecosystem (the whole ASI+humans ecosystem) can only explore a subspace of what’s possible at any given time.

So in a way, it confirms my intuition that people who are positive about AGI seem to expect a world that is similar to being on (certain) drugs all of the time.

More precisely, that would be similar to having an option of turning this or that drug on and off at any time. (But given that I expect this to be mostly managed via strong mastery of tight coupling via non-invasive BCI, there is some specifics, risk profiles are different, temporal dynamics is different, and so on.)

I imagine that at least occasional nostalgic recreation in an “unmodified reality” will also be popular. Or, perhaps, more than occasional.

mishka 12 Jul 2024 13:39 UTC
2 points
−1
in reply to: mishka’s comment on: Pondering how good or bad things will be in the AGI future
Yes, so, on further reflection, what I am doing when describing my day inside a positive singularity, I am recalling my best peak experiences from the life I had, and I am trying to resurrect them and, moreover, to push them further, beyond the limits I have not been able to overcome.

Perhaps, that is what one should do: ponder one’s peak experiences, think how to expand their boundaries and limits, how to mix and match them together (when it might make sense), and so on...

mishka 11 Jul 2024 22:58 UTC
3 points
0
on: AI #72: Denying the Future
I have tried various versions of this explanation over the last year and a half. I have yet to see a good response, but it clearly is not getting through to many people either.

I thought my response in Zvi AI #57 was reasonably good: https://www.lesswrong.com/posts/5Dz3ZrwBzzMfaucrH/ai-57-all-the-ai-news-that-s-fit-to-print#ckYsqx2Kp6HTAR22b

But let me try again. Perhaps you have not seen that one. And perhaps you’ll see this one and respond...
you have a trilemma:
1. Free competition between entities for resources or control.
2. Entities that can outcompete humans.
3. Humans surviving or remaining in control.
Well, “free competition between entities for resources or control” has to go out of the window in a world with superintelligent entities. The community of superintelligent entities needs to self-regulate in a reasonable fashion, otherwise they are likely to blow the fabric of reality to bits and destroy themselves and everything else.

The superintelligent entities have to find some reasonable ways to self-organize, and those ways to self-organize have to constitute a good compromise between freedom and collective control.

Here I elaborate on this further: https://www.lesswrong.com/posts/WJuASYDnhZ8hs5CnD/exploring-non-anthropocentric-aspects-of-ai-existential.

Joscha Bach says the key to safe AI is to make the AIs conscious, because consciousness is what we care about and we had better hope it cares about this fact.

Yes, in that write-up I just referenced, “Exploring non-anthropocentric aspects of AI existential safety”, I also tried to address this via the route of making the AIs sentient. And I still think this makes a lot of sense.

But given that we have such a poor understanding of sentience, and that we have no idea what systems are sentient and what systems are not sentient, it is good to consider an approach which would not depend on that. That’s the approach I am considering in my comment to your AI #57 which I reference above. This is an approach based on the rights of individuals, without specifying whether those individuals are sentient.

I’ll quote the key part here, but I’ll add the emphasis:

I am going to only consider the case where we have plenty of powerful entities with long-term goals and long-term existence which care about their long-term goals and long-term existence. This seems to be the case which Zvi is considering here, and it is the case we understand the best, because we also live in the reality with plenty of powerful entities (ourselves, some organizations, etc) with long-term goals and long-term existence. So this is an incomplete consideration: it only includes the scenarios where powerful entities with long-term goals and long-terms existence retain a good fraction of overall available power.

So what do we really need? What are the properties we want the World to have? We need a good deal of conservation and non-destruction, and we need the interests of weaker, not the currently most smart or most powerful members of the overall ecosystem to be adequately taken into account.

Here is how we might be able to have a trajectory where these properties are stable, despite all drastic changes of the self-modifying and self-improving ecosystem.

An arbitrary ASI entity (just like an unaugmented human) cannot fully predict the future. In particular, it does not know where it might eventually end up in terms of relative smartness or relative power (relative to the most powerful ASI entities or to the ASI ecosystem as a whole). So if any given entity wants to be long-term safe, it is strongly interested in the ASI society having general principles and practices of protecting its members on various levels of smartness and power. If only the smartest and most powerful are protected, then no entity is long-term safe on the individual level.

This might be enough to produce effective counter-weight to unrestricted competition (just like human societies have mechanisms against unrestricted competition). Basically, smarter-than-human entities on all levels of power are likely to be interested in the overall society having general principles and practices of protecting its members on various levels of smartness and power, and that’s why they’ll care enough for the overall society to continue to self-regulate and to enforce these principles.

This is not yet the solution, but I think this is pointing in the right direction...

Note that this approach is not based on unrealistic things, like control of super-smart entities by less smart entities, or forcing super-smart entities to have values and goals of less smart entities. Yes, those unrealistic approaches are unlikely to work well, and are likely to backfire.

It is a more symmetric approach, based on equal rights and on individual rights.

mishka 11 Jul 2024 17:17 UTC
2 points
0
in reply to: Sherrinford’s comment on: Pondering how good or bad things will be in the AGI future

It does in some sense not feel “real” to me, but I’ll admit that that is possibly the case due to some bias or limited imagination on my side.

This is, basically, a list containing my key long-standing dreams, things I craved for a long time :-)

(I do expect all of this to be feasible, if we are lucky to have a positive singularity.)

What do you expect your typical day to be like in 2050?

My timelines are really short, so I expect 2050 to be 20+ years into the technological singularity.

Given that things will be changing faster than today, it is difficult to predict that far ahead. My starting caveat

In the event of a positive technological singularity, we’ll learn about all kinds of possibilities we are not aware of right now, and a lot of interesting future options will be related to that.

would become very prominent.

But earlier in the singularity, the most typical day might be fully understanding a couple of tempting books I only know superficially (e.g. A Compendium of Continuous Lattices and Sketches of an Elephant – A Topos Theory Compendium and so on) and linking the newly understood information with my earlier thoughts or solving one of the problems which I was not able to solve before.

And, yes, some of this would be directly relevant to further progress in AI and such, I think...

But I would modulate my state of consciousness in all kinds of ways while doing that, so that I can see things from a variety of angles. So a typical day would be a creative versatile “multi-trip” engaging with interesting science, while being very beautiful (think about a very good rave or trance party, but without side-effects or undue risks, with the ability to exercise fine-grained control, and such that you come up with tons of creative ideas, understand some things you have not understood before, have new realizations, and have solutions to some problems you wanted to solve, but could not solve them before, novel mental visualization of math novel to you, novel insights, and an entirely new set of unsolved problems for the next day, and all of your key achievements of the night surviving into subsequent days).

That’s a typical “active day”. I am extrapolating from the best of my rave or trance party experiences and similar things, and from the best of my experiences doing science (and it has always been a particular fun for me to think science and philosophy on a rave/trance party).

So I am thinking about those past occasions and thinking “what if I could push this much further, but in a safe and sustainable way”. This should allow for a really rapid progress and should be great fun.

(But, of course, one probably wants different days, not a monoculture, but fairly diverse experiences. I just described a day I currently want the most, and I want a good share of my days to be like that.)

How do you prepare?

I don’t know...

Staying alive is obviously super-important. So trying to take reasonable care of oneself is important.

Interesting experiences and thoughts are, I think, quite important.

Perhaps, some careful meditative practices are important (but this is quite individual, and there are various caveats). One does need to start preparing to master unusual states of mind, and it is good to somewhat reduce the psychological burdens everyone is carrying.

If one does influence the future, that’s quite important (e.g. if one thinks that one can influence the future trajectory of the overall AI ecosystem, that’s important, but it is difficult to be sure that one’s impact is positive, there is that conundrum one is facing when doing AI research or AI safety research, it’s very difficult to predict the direction of impact).

So, yes, it would be good to think together: how does one prepare? :-)

mishka 11 Jul 2024 16:12 UTC
2 points
0
in reply to: IlluminateReality’s comment on: If AI starts to end the world, is suicide a good idea?

Regarding resurrection, do you know what the Lesswrong consensus is on the position that continuation of consciousness is what makes someone the same person as 5 minutes ago? My impression is that this idea doesn’t really make sense, but it’s an intuitive one and a cause of some of my uncertainty about the feasibility of resurrection.

Yes, we don’t really know how reality works, that’s one of the problems. We don’t even know if we are in a simulation. So, it’s difficult to be certain.

I’m surprised you think that a good singularity would let me stay dead if I had decided to commit suicide out of fear of s-risk. Presumably the benevolent AI/s would know that I would want to live, no?

It did occur to me that they will try to “wake you up” once (if that’s feasible at all) and ask if you really meant to stay dead (while respecting your free will and refraining from manipulation).

And it did occur to me that it’s not clear if resurrection is possible, or if bad singularity would bother to resurrect you, even if it’s possible.

So, in reality, one needs to have a better idea about all kinds of probabilities, because the actual “tree of possible scenarios” is really complicated (and we know next to nothing about those).

So, I ended up noting that

my estimates of chances for positive singularity might be higher than your estimates, and this might color my judgement.

This does reflect my uncertainty about all this...

conditional on the world starting to end

Ah, I have not realized that you were talking not just about transformation being sufficiently radical (“end of the world known to us”), but about it specifically being bad...

My typical approach to all that is to consider non-anthropocentric points of view (this allows one to take a step back and to think in a more “invariant way”). In this sense, I suspect that “universal X-risk” (that is, the X-risk which threatens to destroy everything including the AIs themselves) dominates (I am occasionally trying to scribble something about that: https://www.lesswrong.com/posts/WJuASYDnhZ8hs5CnD/exploring-non-anthropocentric-aspects-of-ai-existential).

In this sense, while it is possible to have scenarios where huge sufferings are inflicted, but the “universal X-risk” is somehow avoided, this does not seem to my (unsubstantiated) intuition to be too likely. The need to control the “universal X-risk” and to protect interests of individual members of the AI ecosystem requires a degree of “social harmony” of some sort within the AI ecosystem.

I doubt that anthropocentric approaches to AI alignment are likely to fare well, but I think that a harmonious AI ecosystem where all individuals (including AIs, humans, and so on) are sufficiently respected and protected might be feasible, e.g. I tried to scribble something to that effect here: https://www.lesswrong.com/posts/5Dz3ZrwBzzMfaucrH/ai-57-all-the-ai-news-that-s-fit-to-print?commentId=ckYsqx2Kp6HTAR22b

I think that if an AI ecosystem permits to inflict massive sufferings within itself, this would increase the risks to all members, and to the ecosystem as a whole. It’s difficult to imagine something like this going on for a long time without a catastrophic blow-up. (Although, of course, what do I know...)

mishka 10 Jul 2024 19:40 UTC
3 points
0
in reply to: bhauth’s comment on: LK-99 in retrospect
Yeah, and if one wants to be really sure, one needs to look at raw data a bit oneself (every time I do that, I am usually taken aback by how noisy those data are, and how it must be difficult to interpret them conclusively, and how I have a very powerful built-in bias to trust the reports on experimental data and on what those data mean, and that I should try to keep updating towards higher uncertainty in order to counter my built-in bias to trust the reports).

mishka 10 Jul 2024 17:59 UTC
1 point
0
in reply to: bhauth’s comment on: LK-99 in retrospect
If we want to discuss that, then we need to step back.

How much do we believe that black holes exist at all? Are we certain, are we not quite certain? Everyone is talking as if it is certain, but how much should we believe that?

If we believe that black holes do exist to a sufficiently large degree of certainty, when did it become reasonable to believe that, after what events?

mishka 10 Jul 2024 16:47 UTC
1 point
0
in reply to: bhauth’s comment on: LK-99 in retrospect
You should decide if you believe it or not.

If you don’t believe that this result is likely to be true in reality, that’s fine, it is one possible position, it’s quite self-consistent.

But if one does believe that this result is likely to be true in reality, then that position would be difficult to reconcile with gravity not being fundamentally quantum.