lukemarks

Karma: 381

lukemarks 2 Jul 2024 9:08 UTC
1 point
−2
in reply to: robo’s comment on: lukemarks’s Shortform
This should be an equivalent problem, yes.

lukemarks 2 Jul 2024 9:07 UTC
1 point
0
in reply to: robo’s comment on: lukemarks’s Shortform
Yes, your description of my hypothetical is correct. I think it’s plausible that approximating things that happened in the past is computationally easier than breaking some encryption, especially if the information about the past is valuable even if it’s noisy. I strongly doubt my hypothetical will materialize, but I think it is an interesting problem regardless.
My concern with approaches like the one you suggest is that they’re restricted to small parts of the universe, so with enough data it might be possible to fill in the gaps.

lukemarks 2 Jul 2024 6:56 UTC
3 points
−11
on: lukemarks’s Shortform
Present cryptography becomes redundant when the past can be approximated. Simulating the universe at an earlier point and running it forward to extract information before it’s encrypted is a basic, but difficult way to do this. For some information this approximation could even be fuzzy, and still cause damage if public. How can you protect information when your adversary can simulate the past?
The information must never exist as plaintext in the past. A bad way to do this is to make the information future-contingent. Perhaps it could be acausally inserted into the past by future agents, but probably you would not be able to act on future-contingent information in useful ways. A better method is to run many homomorphically encrypted instances of a random function that might output programs that do computations that yield sensitive information (e.g, an uploaded human). You would then give a plaintext description of the random function, including a proof that it probably output a program doing computations likely adversaries would want. This incentivizes the adversary to not destroy the program output by the random function, because it may not be worth the cost of destruction and replacement with something that is certainly doing better computations.
This method satisfies the following desiderata:
1. The adversary does not know the output of the encrypted random function, or the outputs of the program the random function output
2. There is an incentive to not destroy the program output by the random function
One problem with this is that your advserary might be superintelligent, and prove the assumptions that made your encryption appear strong to be incorrect. To avoid this, you could base your cryptography on something other than computational hardness.
My first thought was to necessitate computations that would make an adversary incur massive negative utility to verify the output of a random function. It’s hard to predict what an adversary’s preferences might be in advance, so the punishment for verifying the output of the random function would need to be generically bad, such as forcing the adversary to expend massive amounts of computation on useless problems. This idea is bad for obvious reasons, and will probably end up making the same or equally bad assumptions about the inseparability of the punishment and verification.

lukemarks’s Shortform

lukemarks2 Jul 2024 6:56 UTC

4 points

6 comments1 min readLW link

lukemarks 25 May 2024 12:36 UTC
1 point
0
in reply to: Chris_Leong’s comment on: Beta Tester Request: Rallypoint Bounties
We are recruiting people interested in using Rallypoint in any way. The form has an optional question for what you hope to get out of using Rallypoint. Even if you don’t plan on contributing to bounties or claiming them and just want to see how others use Rallypoint we are still interested in your feedback.

lukemarks 25 May 2024 9:53 UTC
2 points
0
in reply to: keltan’s comment on: Beta Tester Request: Rallypoint Bounties
Yes. If the feedback from the beta is that people find Rallypoint useful we will do a public release and development will continue. I want to focus on getting the core bounty infrastructure very refined before adding many extra features. Likely said infrastructure would be easily adapted to more typical crowdfunding and a few other applications, but that is lower on the priority list than getting bounties right.

Beta Tester Request: Rallypoint Bounties

lukemarks25 May 2024 9:11 UTC

25 points

4 comments1 min readLW link

lukemarks 28 Dec 2023 14:04 UTC
LW: 2 AF: 2
0
AF
on: Free agents
I don’t understand the distinction you draw between free agents and agents without freedom.
If I build an expected utility maximizer with a preference for the presence of some physical quantity, that surely is not a free agent. If I build some agent with the capacity to modify a program which is responsible for its conversion from states of the world to scalar utility values, I assume you would consider that a free agent.
I am reminded of E.T. Jaynes’ position on the notion of ‘randomization’, which I will summarize as “a term to describe a process we consider too hard to model, which we then consider a ‘thing’ because we named it.”
How is this agent any more free than the expected utility maximizer, other than for the reason that I can’t conveniently extrapolate the outcome of its modification of its utility function?
It seems to me that this only shifts the problem from “how do we find a safe utility function to maximize” to “how do we find a process by which a safe utility function is learned”, and I would argue the consideration of the latter is already a mainstream position in alignment.
If I have missed a key distinguishing property, I would be very interested to know.

lukemarks 8 Nov 2023 4:58 UTC
3 points
−7
on: Concrete positive visions for a future without AGI
I believe you misinterpreted the quote from disturbance. They were implying that they would bring about AGI at the moment before their brain would be unsalvageable by AGI such that they could be repaired, assumedly in expectation of immortality.

I also don’t think the perspective that we would likely fail as a civilization without AGI is common on LessWrong. I would guess that most of us would expect a smooth-ish transition to The Glorious Future in worlds where we coordinate around [as in don’t build] AI. In my opinion the post is good even without this claim however.

lukemarks 4 Nov 2023 7:55 UTC
4 points
−1
on: Untrusted smart models and trusted dumb models
models that are too incompetent to think through deceptive alignment are surely not deceptively aligned.
Is this true? In Thoughts On (Solving) Deep Deception, Jozdien gives the following example that suggests otherwise to me:
Back in 2000, a computer scientist named Charles Ofria was studying the evolution of simulated organisms. He wanted to limit their replication rate, so he programmed the simulation to pause after each mutation, measure the mutant’s replication rate in an isolated test environment, and delete the mutant if it replicated faster than its parent.
Eventually, organisms evolved to recognize and pause their replication in the test environment and resume normal replication when released. Ofria tried solving this by making it harder to detect the test environment, but the organisms simply started to randomly accelerate their replication so as to slip through the test environment some of the time.
I don’t see why a model would need to be cognitively able to process its own alignment for its alignment to be improper, and I think this assumption is quite key to the main claims of the post.

[Question] Shouldn’t we ‘Just’ Superimitate Low-Res Uploads?

lukemarks3 Nov 2023 7:42 UTC

15 points

2 comments2 min readLW link

lukemarks 26 Oct 2023 6:03 UTC
1 point
0
on: AI as a science, and three obstacles to alignment strategies

unless, by some feat of brilliance, this civilization pulls off some uncharacteristically impressive theoretical triumphs

Are you able to provide an example of the kind of thing that would constitute such a theoretical triumph? Or, if not; a maximally close approximation in the form of something that exists currently?

Early Experiments in Reward Model Interpretation Using Sparse Autoencoders

lukemarks, Amirali Abdullah, Rauno Arike, Fazl and nothoughtsheadempty

3 Oct 2023 7:45 UTC

16 points

0 comments5 min readLW link

lukemarks 11 Sep 2023 8:38 UTC
12 points
0
on: High school advice
I’m in high school myself and am quite invested in AI safety. I’m not sure whether you’re requesting advice for high school as someone interested in LW, or for LW and associated topics as someone attending high school. I will try to assemble a response to accommodate both possibilities.
Absorbing yourself in topics like x-risk can make school feel like a waste of time. This seems to me to be because school is mostly a waste of time (which is a position I held before becoming interested in AI safety,) but disengaging with the practice entirely also feels incorrect. I use school mostly as a place to relax. Those eight hours are time I usually have to write off as wasted in terms of producing a technical product, but value immensely as a source of enjoyment, socializing and relaxation. It’s hard for me to overstate just how pleasurable attending school can be when you optimize for enjoyment, and if permitted by your school’s environment; a suitable place for intellectual progress in an autodidactic sense also, presuming you aren’t being provided that in the classroom. If you do feel that the classroom is an optimal learning environment for you, I don’t see why you shouldn’t just maximize knowledge extraction.
For many of my peers, school is practically their life. I think that this is a shame, but social pressures don’t let them see otherwise, even when their actions are clearly value negative. Making school just one part of your life instead of having it consume you is probably the most critical thing to extract from this response. The next is to use its resources to your advantage. If you can network with driven friends or find staff willing to push you/find you interesting opportunities, you absolutely should. I would be shocked if there wasn’t at least one staff member at your school passionate about something you were too. Just asking can get you a long way, and shutting yourself off from that is another mistake I made in my first few years of high school, falsely assuming that school simply had nothing to offer me.
In terms of getting involved with LW/AI safety, the biggest mistake I made was being insular, assuming my age would get in the way of networking. There are hundreds of people available at any given time who probably share your interests but possess an entirely different perspective. Most people do not care about my age, and I find that phenomena especially prevalent in the rationality community. Just talk to people. Discord and Slack are the two biggest clusters for online spaces, and if you’re interested I can message you invites.
Another important point, particularly as a high school student is not falling victim to group think. It’s easy to be vulnerable to the failing in your formative years, but it can massively skew your perspective, even when your thinking seems unaffected. Don’t let LessWrong memetics propagate throughout your brain too strongly without good reason.
What links here?
- High Schooler getting involved in EA by GamerChess (EA Forum; 20 Apr 2024 11:19 UTC; 26 points)

lukemarks 8 Sep 2023 2:23 UTC
3 points
0
in reply to: Charlie Steiner’s comment on: The Löbian Obstacle, And Why You Should Care
I expect agentic simulacra to occur without intentionally simulating them, in that agents are just generally useful for solving prediction problems and that in conducting millions of predictions (as would be expected of a product on the order of ChatGPT, or future successors,) it’s probable for agentic simulacra to occur. Even if these agents are just approximations, in predicting the behaviors of approximated agents their preferences could still be satisfied in the real world (as described in the Hubinger post.)

The problem I’m interested in is how you ensure that all subsequent agentic simulacra (whether occurred intentionally or otherwise) are safe, which seems difficult to verify formally due to the Löbian Obstacle.

lukemarks 8 Sep 2023 1:31 UTC
6 points
1
in reply to: Charlie Steiner’s comment on: The Löbian Obstacle, And Why You Should Care
Which part specifically are you referring to as being overly complicated? What I take to be the primary assertions of the post to be are:
- Simulacra may themselves conduct simulation, and advanced simulators could produce vast webs of simulacra organized as a hierarchy.
- Simulating an agent is not fundamentally different to creating one in the real world.
- Due to instrumental convergence, agentic simulacra might be expected to engage in resource acquisition. This could take the shape of ‘complexity theft’ as described in the post.^[1]
- The Löbian Obstacle accurately describes why an agent cannot obtain a formal guarantee via design-inspection of its subsequent agent.
- For a simulator to be safe, all simulacra need to be aligned unless we figure some upper bound on “programs of this complexity are too simple to be dangerous,” at which point we would consider simulacra above that complexity only.
I’ll try to justify my approach with respect to one or more of these claims, and if I can’t, I suppose that would give me strong reason to believe the method is overly complicated.
1. ^
  This doesn’t have to be resource acquisition, just any negative action that we could reasonably expect a rational agent to pursue.

The Löbian Obstacle, And Why You Should Care

lukemarks7 Sep 2023 23:59 UTC

18 points

6 comments2 min readLW link

lukemarks 24 Aug 2023 22:05 UTC
4 points
2
on: AI Regulation May Be More Important Than AI Alignment For Existential Safety
The issue I have with pivotal act models is that they presume an aligned superintelligence would be capable of bootstrapping its capabilities in such a way that it could perform that act before the creation of the next superintelligence. Soft takeoff seems a very popular opinion now, and isn’t conducive to this kind of scheme.

Also, if a large org were planning a pivotal act I highly doubt they would do so publicly. I imagine subtly modifying every GPU on the planet, melting them or doing anything pivotal on a planetary scale such that the resulting world has only one or a select few superintelligences (at least until a better solution exists) would be very unpopular with the public and with any government.

I don’t think the post explicity argues against either of these points, and I agree with what you have written. I think these are useful things to bring up in such a discussion however.