ChosunOne

Karma: 94

ChosunOne 13 Aug 2025 8:53 UTC
6 points
0
in reply to: ChosunOne’s comment on: Cryonics without standby services?
The main thing you want to prevent is large scale ice crystal formation. Dry ice sits just below the temperature needed for water to crystalize when freezing, so it delays the crystal formation by a substantial amount. However, for extended periods the ice can still crystalize so that’s why it’s important to get to cryogenic temperatures.

ChosunOne 13 Aug 2025 8:51 UTC
11 points
0
on: Cryonics without standby services?
I have some experience with handling a case where no standby service was available for my pet. I know pets are not humans, but I think there is some insight worth sharing.

If you can source dry ice, you can use this as a substitute for liquid nitrogen for about 6 months without suffering too much degradation compared to liquid nitrogen. Generally, you will want to opt for this if you cannot arrange for transport to a storage facility within 24 hours of clinical death. About half of all cryonics cases are “straight freeze” with no cryoprotectant, so there is a good chance methods will be invested in to revive those who undergo a straight freeze.

My pet passed away suddenly two years ago after a failed emergency surgery. I had her immediately transferred into ice-water bath after we could not restart her heartbeat. About 19 hours later I arrived with a source of dry ice to straight freeze her until I could arrange transport to Switzerland for storage in Tomorrow Bio’s facility. I held her in a dedicated freezer with dry ice (replenished every other day to every two days) for about a month before I could work out how to transport her to Switzerland.

From my conversations with Emil (CEO of Tomorrow Bio) this is very similar to their own procedure (as my pet was already under anesthesia from the surgery) minus the cryoprotectant. So if you can have someone act quickly to cool your body to close to freezing then get you in dry ice for extended pre-liquid nitrogen storage you won’t be much worse off than the other straight frozen cases.
For disclosure, I am signed up with Tomorrow Bio and this is my plan if my imminent cryopreservation is needed. I live in a country with no standby service but figure this is better than nothing.

ChosunOne 9 Jul 2024 7:39 UTC
1 point
0
in reply to: Ape in the coat’s comment on: Doomsday Argument and the False Dilemma of Anthropic Reasoning
I think the factors that determine your reference class can be related to changes over time in the environment you inhabit, not just how you are built. This is what I mean by not necessarily related to reproduction. If I cloned a billion people with the same cloning vat over 1000 years, how then would you determine reference class? But maybe something about the environment would narrow that reference class despite all the people being made by the same process (like the presence or absence of other things in the environment as they relate to the people coming out of the vat).

ChosunOne 8 Jul 2024 9:47 UTC
1 point
0
on: Doomsday Argument and the False Dilemma of Anthropic Reasoning
I think there is another assumption (or another way of framing your assumption, I’m not sure if it is really distinct), which is that all people throughout time are alike. It’s not necessarily groundless, but it is an assumption. There could be some symmetry breaking factor that determines your position that doesn’t simply align with “total humans that ever exist”, and thus renders using the “birth rank” inappropriate to determine what number you have. This thing doesn’t have to be related to reproduction though.

ChosunOne 28 Jun 2024 13:35 UTC
1 point
0
in reply to: eggsyntax’s comment on: LLM Generality is a Timeline Crux
It still seems slightly fuzzy in that other than check/mate situations no moves are fully mandatory and eg recaptures may occasionally turn out to be the wrong move?
Indeed it can be difficult to know when it is actually better not to continue the line vs when it is, but that is precisely what MCTS would help figure out. MCTS would do actual exploration of board states and the budget for which states it explores would be informed by the policy network. It’s usually better to continue a line vs not, so I would expect MCTS to spend most of its budget continuing the line, and the policy would be updated during training with whether or not the recommendation resulted in more wins. Ultimately though, the policy network is probably storing a fuzzy pattern matcher for good board states (perhaps encoding common lines or interpolations of lines encountered by the MCTS) that it can use to more effectively guide the search by giving it an appropriate score.

To be clear, I don’t think a transformer is completely incapable of doing any search, just that it is probably not learning to do it in this case and is probably pretty inefficient at doing it when prompted to.

ChosunOne 28 Jun 2024 6:17 UTC
1 point
0
in reply to: eggsyntax’s comment on: LLM Generality is a Timeline Crux
In chess, a “line” is sequence of moves that are hard to interrupt. There are kind of obvious moves you have to play or else you are just losing (such as recapturing a piece, moving king out of check, performing checkmate etc). Leela uses the neural network more for policy, which means giving a score to a given board position, which then the MCTS can use to determine whether or not to prune that direction or explore that section more. So it makes sense that Leela would have an embedding of powerful lines as part of its heuristic, since it isn’t doing to main work of search. It’s more pattern recognition on the board state, so it can learn to recognize the kinds of lines that are useful and whether or not they are “present” in the current board state. It gets this information from the MCTS system as it trains, and compresses the “triggers” into the earlier evaluations, which then this paper explores.

It’s very cool work and result, but I feel it’s too strong to say that the policy network is doing search as opposed to recognizing lines from its training at earlier board states.

ChosunOne 27 Jun 2024 15:08 UTC
1 point
0
in reply to: eggsyntax’s comment on: LLM Generality is a Timeline Crux
The cited paper in Section 5 (Conclusion-Limitations) states plainly:

(2) We focus on look-ahead along a single line of play; we do not test whether Leela compares multiple different lines of play (what one might call search). … (4) Chess as a domain might favor look-ahead to an unusually strong extent.
The paper is more just looking at how Leela evaluates a given line rather than doing any kind of search. And this makes sense. Pattern recognition is an extremely important part of playing chess (as a player myself), and it is embedded in another system doing the actual search, namely Monte Carlo Tree Search. So it isn’t surprising that it has learned to look ahead in a straight line since that’s what all of its training experience is going to entail. If transformers were any good at doing the search, I would expect a chess bot without employing something like MCTS.

ChosunOne 26 Jun 2024 4:33 UTC
11 points
10
in reply to: Nathan Helm-Burger’s comment on: LLM Generality is a Timeline Crux
My crux is that LLMs are inherently bad at search tasks over a new domain. Thus, I don’t expect LLMs to scale to improve search.

Anecdotal evidence: I’ve used LLMs extensively and my experience is that LLMs are great at retrieval but terrible at suggestion when it comes to ideas. You usually get something resembling an amalgamation of Google searches vs. suggestions from some kind of insight.

ChosunOne 11 Jun 2024 16:05 UTC
6 points
2
in reply to: sanxiyn’s comment on: On Dwarksh’s Podcast with Leopold Aschenbrenner
To your question of what to do if you are outmatched and you only have an ASI at your disposal, I think the most logical thing to do is “do what the ASI tells you to”. The problem is that we have no way of predicting the outcomes if there is truly an ASI in the room. If it’s a superintelligence it is going to have better suggestions than anything you can come up with.

ChosunOne 6 Jun 2024 16:49 UTC
3 points
0
in reply to: Nicolas Villarreal’s comment on: The Impossibility of a Rational Intelligence Optimizer
Then I wonder, at what point does that matter? Or more specifically, when does that matter in the context of ai-risk?

Clearly there is some relationship between something like “more compute” and “more intelligence” since something too simple cannot be intelligent, but I don’t know where that relationship breaks down. Evolution clearly found a path for optimizing intelligence via proxy in our brains, and I think the fear is that you may yet be able to go quite further than human-level intelligence before the extra compute fails to deliver more meaningful intelligence described in your post.

It seems premature to reject the orthogonality thesis of optimizing for things that “obviously bring more intelligence” before they start to break down.

ChosunOne 6 Jun 2024 16:26 UTC
3 points
1
on: The Impossibility of a Rational Intelligence Optimizer
So if I understand your point correctly, you expect something like “give me more compute” at some point fail to deliver more intelligence since intelligence isn’t just “more compute”?

ChosunOne 24 May 2024 10:45 UTC
1 point
2
on: Current LLMs are conscious and are AGI.
I don’t think you claim has support as presented. Part of the problem surrounding the question is that we still don’t really have any way of measuring how “conscious” something is. In order to claim that something is or isn’t conscious, you should have some working definition of what conscious means, and how it can be measured. If you want to have a serious discussion instead of competing emotional positions, you need to support the claim with points that can be confirmed or denied. Why doesn’t a goldfish have consciousness, or an earthworm? How can you know? You can borrow concepts from some theoretical concepts like Integrated Information Theory or Non-Trivial Information Closure etc. but you should make some effort to ground the claim.

As for my opinion, whatever your definition of consciousness is I think a digital entity can in principle be conscious since I think consciousness is closely related to certain types of information, but beyond that without a more rigorous perspective I think the conversation isn’t going anywhere fruitful.

ChosunOne 24 May 2024 9:52 UTC
2 points
0
in reply to: ProgramCrafter’s comment on: When is a mind me?
Well ultimately no information about the past is truly lost as far as we know. A hyper-advanced civilization could collect all the thermal radiation from earth reflected off of various celestial bodies and recover a near complete history, at least in principle. So I think the more you make it easy for yourself to be reconstructed/resurrected/what have you the sooner it would likely be, and the less alien of an environment you would find yourself in after the fact. Cryo is a good example of having a reasonable expectation of where to end up barring catastrophe since you are preserving a lot of you in good form.

ChosunOne 17 Apr 2024 17:08 UTC
8 points
3
on: When is a mind me?
An interesting consequence of your description is that resurrection is possible if you can manage to reconstruct the last brain state of someone who had died. If you go one one step further, then I think it is fairly likely that experience is eternal, since you don’t experience any of the intervening time (akin to your film reel analogy with adding extra frames in between) being dead and since there is no limit to how much intervening time can pass.

ChosunOne 17 Apr 2024 16:46 UTC
6 points
0
on: Transformers Represent Belief State Geometry in their Residual Stream
I’m curious how much space is left after learning the MSP in the network. Does representing the MSP take up the full bandwidth of the model (even if it is represented inefficiently)? Could you maintain performance of the model by subtracting out the contributions of anything else that isn’t part of the MSP?

ChosunOne 4 Apr 2023 20:39 UTC
6 points
−2
in reply to: Razied’s comment on: Steelman / Ideological Turing Test of Yann LeCun’s AI X-Risk argument?
I observe this behavior a lot when using GPT-4 to assist in code. The moment it starts spitting out code that has a bug, the likelihood of future code snippets having bugs grows very quickly.

ChosunOne 17 Mar 2023 5:27 UTC
3 points
2
on: Donation offsets for ChatGPT Plus subscriptions
I’ve found that using Bing/Chat-GPT has been enormously helpful in my own workflows. No need to have to carefully read documentation and tutorials just to get a starter template up and running. Sure it breaks here and there, but it seems way more efficient to look up stuff when it goes wrong vs. starting from scratch. Then, while my program is running, I can go back and try to understand what all the options do.

It’s also been very helpful for finding research on a given topic and answering basic questions about some of the main ideas.

ChosunOne 11 Mar 2023 16:27 UTC
1 point
0
in reply to: Noosphere89’s comment on: Why do we assume there is a “real” shoggoth behind the LLM? Why not masks all the way down?
I’m not sure how that makes the problem much easier? If you get the maligned superintelligence mask, it only needs to get out of the larger model/send instructions to the wrong people once to have game over scenario. You don’t necessarily get to change it after the fact. And changing it once doesn’t guarantee it doesn’t pop up again.

ChosunOne 11 Mar 2023 14:20 UTC
1 point
0
on: Why do we assume there is a “real” shoggoth behind the LLM? Why not masks all the way down?
This could be true, but then you still have the issue of there being “superintelligent malign AI” as one of the masks if your pile of masks is big enough.

ChosunOne 5 Mar 2023 12:11 UTC
2 points
4
on: Why kill everyone?
At a crude level, the earth represents a source of resources with which to colonize the universe shard and maximize its compute power (and thus find the optimal path to achieve its other goals). Simply utilizing all the available mass on earth to do that as quickly as possible hedges against the possibility of other entities in the universe shard from impeding progress toward its goals.

The question I’ve been contemplating is “Is it worth it to actually try to spend any resources dissassembling all matter on Earth given the cost of needing to devote resources to stop humans from interfering” when weighed against the lost time and energy from solar output given the resources needed to accomplish that. I don’t know, but it’s a reasonable way of getting “kill everyone” from an unaligned goal if it makes sense to pursue that strategy.