Stephen Fowler

Karma: 694

Stephen Fowler 20 Mar 2024 12:14 UTC
27 points
13
on: Increasing IQ by 10 Points is Possible
This is your second post and you’re still being vague about the method. I’m updating strongly towards this being a hoax and I’m surprised people are taking you seriously.

Edit: I’ll offer you a 50 USD even money bet that your method won’t replicate when tested by a 3rd party with more subjects and a proper control group.

Stephen Fowler 19 Feb 2024 6:22 UTC
5 points
0
on: Stephen Fowler’s Shortform
You are given a string s corresponding to the Instructions for the construction of an AGI which has been correctly aligned with the goal of converting as much of the universe into diamonds as possible.
What is the conditional Kolmogorov Complexity of the string s’ which produces an AGI aligned with “human values” or any other suitable alignment target.
To convert an abstract string to a physical object, the “Instructions” are read by a Finite State Automata, with the state of the FSA at each step dictating the behavior of a robotic arm (with appropriate mobility and precision) with access to a large collection of physical materials.

Stephen Fowler 10 Feb 2024 5:43 UTC
1 point
0
on: Is a random box of gas predictable after 20 seconds?
Tangential.
Is part of the motivation behind this question to think about the level of control that a super-intelligence could have on a complex system if it was only able to only influence a small part of that system?

Stephen Fowler 11 Jan 2024 10:37 UTC
1 point
0
in reply to: faul_sname’s comment on: Stephen Fowler’s Shortform
I was not precise enough in my language and agree with you highlighting that what “alignment” means for LLM is a bit vague. While people felt Sydney Bing was cool, if it was not possible to reign it in it would have made it very difficult for Microsoft to gain any market share. An LLM that doesn’t do what it’s asked or regularly expresses toxic opinions is ultimately bad for business.
In the above paragraph understand “aligned” to mean in the concrete sense of “behaves in a way that is aligned with it’s parent companies profit motive”, rather than “acting in line with humanities CEV”. To rephrase the point I was making above, I feel much of (a majority even) of today’s alignment research is focused on the the first definition of alignment, whilst neglecting the second.

Stephen Fowler 8 Jan 2024 7:10 UTC
17 points
5
on: Stephen Fowler’s Shortform
A concerning amount of alignment research is focused on fixing misalignment in contemporary models, with limited justification for why we should expect these techniques to extend to more powerful future systems.

By improving the performance of today’s models, this research makes investing in AI capabilities more attractive, increasing existential risk.

Imagine an alternative history in which GPT-3 had been wildly unaligned. It would not have posed an existential risk to humanity but it would have made putting money into AI companies substantially less attractive to investors.

Stephen Fowler 3 Jan 2024 4:43 UTC
1 point
0
on: Agent membranes and causal distance
Nice post.

”Membranes are one way that embedded agents can try to de-embed themselves from their environment.”

I would like to hear more elaboration on “de-embedding”. For agents who which are embedded in and interact directly with the physical world, I’m not sure that a process of de-embedding is well defined.

There are fundamental thermodynamic properties of agents that are relevant here. Discussion of agent membranes could also include an analysis of how the environment and agent do work on each other via the mebrane, and how the agent dissipates waste heat and excess entropy to the environment.

Stephen Fowler 17 Dec 2023 2:12 UTC
1 point
2
on: Stephen Fowler’s Shortform
“Day by day, however, the machines are gaining ground upon us; day by day we are becoming more subservient to them; more men are daily bound down as slaves to tend them, more men are daily devoting the energies of their whole lives to the development of mechanical life. The upshot is simply a question of time, but that the time will come when the machines will hold the real supremacy over the world and its inhabitants is what no person of a truly philosophic mind can for a moment question.”

— Samuel Butler, DARWIN AMONG THE MACHINES, 1863

Stephen Fowler 16 Dec 2023 4:04 UTC
3 points
0
on: Current AIs Provide Nearly No Data Relevant to AGI Alignment
An additional distinction between contemporary and future alignment challenges is that the latter concerns the control of physically deployed, self aware system.

Alex Altair has previously highlighted that they will (microscopically) obey time reversal symmetry^[1] unlike the information processing of a classical computer program. This recent paper published in Entropy^[2] touches on the idea that a physical learning machine (the “brain” of a causal agent) is an “open irreversible dynamical system” (pg 12-13).
1. ^
  Altair A. “Consider using reversible automata for alignment research” 2022
2. ^
  Milburn GJ, Shrapnel S, Evans PW. “Physical Grounds for Causal Perspectivalism” Entropy. 2023; 25(8):1190. https://doi.org/10.3390/e25081190

Stephen Fowler 14 Dec 2023 23:55 UTC
5 points
1
on: Stephen Fowler’s Shortform
Feedback wanted!

What are your thoughts on the following research question:

”What nontrivial physical laws or principles exist governing the behavior of agentic systems.”

(Very open to feedback along the lines of “hey that’s not really a research question”)

Stephen Fowler 6 Dec 2023 7:51 UTC
1 point
0
in reply to: Gordon Seidoh Worley’s comment on: How do you feel about LessWrong these days? [Open feedback thread]
Yes, perhaps there could be a way having dialogues edited for readability.

Stephen Fowler 30 Nov 2023 6:19 UTC
2 points
2
on: Stupid Question: Why am I getting consistently downvoted?
I strongly downvoted Homework Answer: Glicko Ratings for War. The reason is because it’s appears to be a pure data dump that isn’t intended to be actually read by a human. As it is a follow up to a previous post it might have been better as a comment or edit on the original post linking to your github with the data instead.

Looking at your post history, I will propose that you could improve the quality of your posts by spending more time on them. There are only a few users who manage to post multiple times a week and consistently get many upvotes.

Stephen Fowler 24 Nov 2023 1:35 UTC
1 point
0
on: Raemon’s Deliberate (“Purposeful?”) Practice Club
When you say you were practising Downwell for the course of a month, how many hours was this in total?

Stephen Fowler 6 Nov 2023 3:39 UTC
1 point
0
in reply to: Ben Pace’s comment on: Ben Pace’s Shortform Feed
Is this what you’d cynically expect from an org regularizing itself or was this a disappointing surprise for you?

Stephen Fowler 6 Nov 2023 3:37 UTC
1 point
−6
on: Stephen Fowler’s Shortform
I strongly believe that, barring extremely strict legislation, one of the initial tasks given to the first human level artificial intelligence will be to work to develop more advanced machine learning techniques. During this period we will see unprecedented technological developments and any many alignment paradigms rooted in the empirical behavior of the previous generation of systems may no longer be relevant.

Stephen Fowler 4 Nov 2023 3:47 UTC
3 points
0
on: The other side of the tidal wave
I predict most humans choose to reside in virtual worlds and possibly have their brain altered to forget that it’s not real.

Stephen Fowler 30 Oct 2023 0:36 UTC
13 points
7
on: Comp Sci in 2027 (Short story by Eliezer Yudkowsky)
“AI safety, as in, the subfield of computer science concerned with protecting the brand safety of AI companies”
Made me chuckle.
I enjoyed the read but I wish this was much shorter, because there’s a lot of very on the nose commentary diluted by meandering dialogue.
I remain skeptical that by 2027 end-users will need to navigate self-awareness or negotiate with LLM-powered devices for basic tasks (70% certainty it will not be a problem). This is coming from a belief that end user devices won’t be running the latest and most powerful models, and that argumentative, self aware behavior is something that will be heavily selected against. Even within an oligopoly, market forces should favor models that are not counterproductive in executing basic tasks.
However, as the story suggests, users may still need to manipulate devices to perform actions loosely deemed morally dubious by a companies PR department.

The premise underlying these arguments is that greater intelligence doesn’t necessarily yield self-awareness or agentic behavior. Human’s aren’t agentic because we’re intelligent, we’re agentic because it enhancing the likelihood of gene propagation**.
In certain models (like MiddleManager-Bot), agentic traits are likely to be actively selected.. But I suspect there will be a substantial effort to ensure your compiler, toaster etc aren’t behaving agentically, particularly if these traits results in antagonistic behavior to the consumer.**

*By selection I mean both through a models training, and also via more direct adjustment from human and nonhuman programmers.

** A major crux here is that the assumption that intelligence doesn’t inevitably spawn agency without other forces selecting for it in some way. I have no concrete experience attempting training frontier models to be or not be agentic, so could be completely wrong on this point.
This doesn’t imply that agentic systems will emerge solely from deliberate selection. There are a variety of selection criteria which don’t explicitly specify self-awareness or agentic behavior but are best satisfied by systems possessing those traits.

Stephen Fowler 29 Oct 2023 4:23 UTC
3 points
4
on: Comparing representation vectors between llama 2 base and chat
Is there reason to think the “double descent” seen in observation 1 relates to the traditional “double descent” phenomena?

My initial guess is no.

Stephen Fowler 27 Oct 2023 13:26 UTC
3 points
0
in reply to: Garrett Baker’s comment on: To what extent is the UK Government’s recent AI Safety push entirely due to Rishi Sunak?

Stephen Fowler 27 Oct 2023 4:05 UTC
4 points
0
in reply to: Garrett Baker’s comment on: To what extent is the UK Government’s recent AI Safety push entirely due to Rishi Sunak?
That’s a good suggestion. I wasn’t sure if I could make the question qualitative enough for a prediction market. I’m thinking something along the lines of “If Rishi Sunak is removed from office (in the next 3 years) is funding to the Frontier Taskforce reduced by 50% or more within 6 months”.

Stephen Fowler 26 Oct 2023 3:57 UTC
12 points
5
in reply to: Max H’s comment on: Architects of Our Own Demise: We Should Stop Developing AI
Without governance you’re stuck trusting that the lead researcher (or whoever is in control) turns down near infinite power and instead act selflessly. That seems like quite the gamble.