Thank you, I really do appreciate you having taking the energy and time to answer about this.
I do agree with the “formatted as wall of text” and I suspect it might have been quite a big factor in the percieved lack of clarity. I will make an edit to separate into a few paragraphs and hopefully it will make the whole comment a bit clearer.
Curious about why my comment was substantially downvoted. If somebody out there could kindly give an explaination, that would be helpful and much appreciated.
Thank you for pointing at Anthropic Claude’s Constitution suspected weaknesses and making good arguments about them.
Anthropic Claude’s Constitution indeed looks a lot like wishful thinking and benevolent incantations. AI safety is a multi-disciplinary topic and it seems that document is heavily tilted towards “soft” disciplines like philosophy and ethics and maybe not enough towards “hard” disciplines like ML, logic and neuro-science. Philosophy, psychology and ethics are useful but cannot solve AI safety by themselves.
To be fair that document seems to describe in general terms what a good benevolent helpful Claude persona should look and behave like rather than a formal technical specification on how to achieve this.
I do hope and I am quite sure (but not certain) serious technical efforts, not just philosophical efforts are made by Anthropic to make their AIs safe. One reassuring sentence I found in the document: “Claude is a subject of ongoing research and experimentation: evaluations, red-teaming exercises, interpretability research, and so on. This is a core part of responsible AI development.”. But thank you for pointing at potential weaknesses and serious problems there.
Edit: as per npostavs helpful below remark, separated text into a few paragraphs.
Thank you for that RobertM, very appreciated indeed and looking forward to talking to you.
Hello again.
They have actually just finally replied to me so problem solved, I just needed to be patient and just wait indeed.
Hopefully if they see that they can’t any more usually reply under 3 hours, they will amend that information so users are not wondering what is happening.
Thank you for your answer, very appreciated indeed.
Before starting to write my post I have indeed read the basics about what has very little chance to pass the LessWrong moderation (thank you for pointing these again to me) and I think my post does not fall into the same patterns as these examples. My post is AI safety related and basically explains one dialogue technique/pattern to derail a LLM into making a lot of “controversial statements”. So this post contains a lot of user-LLM interactions with explaining notes. It also contains lot of “controversial statements” (for demonstration purpose).
Thank you for offering me to read my draft, etc… that would be helpful in terms of making a better guess about if my post has a reasonable chance to pass moderation but that would add some complexity to my workflow and would not really offer much guarantee the LessWrong moderation team would accept my post (or would it?!).
I really would like to have a proper discussion with the LessWrong moderation team, this is what I really need and really would like to have. Thank you StanislavKrym for replying to me as it actually helped me clarify what I need and how I want to handle that problem: I just need to talk to the moderation team about my post and so I will just wait for them to answer me. It is annoying, disappointing and frustrating that they state that usually reply under 3 hours but haven’t done so for 30 hours and counting. But what can I do about that? I suspect not much so I will just wait (I mean I will do other things of course while waiting!).
Thank you again for your answer and your help, very appreciated indeed.
Hello, I am currently writing quite a long post but before writing the remaining 90% I would like to have a “pre-submission moderation discussion” as offered by the LessWrong moderation team in the post editing GUI. So I have sent a couple of messages to them requesting a discussion/help to make sure my post has a reasonable chance to pass the LessWrong moderation. The messaging widget says “Our usual reply time 🕒 under 3 hours” but it has been more than 30 hours and still no reply and so I am stuck. Could anybody kindly help/advise me on what I should do please?
This resonates strongly with my experience, though when I noticed this pattern I thought of it as part of my ADHD and not my depression. Maybe this is something like the mechanism via which ADHD causes depression.
Also resonates strongly with my own experience, in my case just replace “ADHD” with “ME/CFS”.
I think OP description is good but quite generic i.e. it would probably resonate with most people who have a physical and/or mental health condition which is quite “taxing” in the sense that it significantly lowers the reward/effort ratio of every/most task.
As mentioned by Daniel Samuel comment, in the case of depression the “tax”/handicap would fall specifically on willpower (and/or enjoyment/pleasure/etc...). In the case of ADHD the tax/handicap would mostly fall on attention, in the case of ME/CFS it would mostly fall on energy, etc...
Aaw no problem at all Florian, I genuinely simply enjoyed you mentioning that sleep-clone-swap thought experiment and truly wasn’t bothered at all by anything about it, thank you so much for your very interesting and kind words and your citation and link in your article, wow I am blushing now!
And thank you so much for that great post of yours and taking the time to thoroughly answer so many comments (incuding mine!) that is so kind of you and makes for such an interesting thread about this topic of entity/person/mind/consciousness/self continuity/discontinuity which is quite fascinating!
And in my humble opinion indeed it has a lot to do with question of definitions/preferences but in any case it is always interesting to read/hear about eloquently/well-spoken words about this topic, thank you so much again for that!
About creating link-to-comment, I think one way to do it is to click on the time indicator next to the author name at the top of the comment then copy that link/URL.
Widely subscribe to OP point of view.
(loving that the sleep-clone-swap thought experiment I described in my comment to Rob Bensinger’s post inspired you!)
The level of discontinuity at which each people will consider a future entity/person/mind/self to still be the rightful continuation of a present entity/person/mind/self will vary according to their own present subjective feelings/opinions/points-of-view/experiences/intutions/thoughts/theories/interpretations/preferences/resolutions about it.
This is really Ship of Theseus paradox territory.
For example, the theory/resolution that I would personally (currently) widely subscribe to is:
“Temporal parts theory”, quoting Wikipedia: “Another common theory put forth by David Lewis is to divide up all objects into three-dimensional time-slices which are temporally distinct, which avoids the issue that the two different ships exist in the same space at one time and a different space at another time by considering the objects to be distinct from each other at all points in time.”
Some other people in the comments I think would be closer to this other theory/resolution:
“Continued identity theory”:
“This solution (proposed by Kate, Ernest et al.) sees an object as staying the same as long as it continuously exists under the same identity without being fully transformed at one time. For instance, a house that has its front wall destroyed and replaced at year 1, the ceiling replaced at year 2, and so on, until every part of the house has been replaced will still be understood as the same house. However, if every wall, the floor, and the roof are destroyed and replaced at the same time, it will be known as a new house.”
There are many other possible theories/resolutions.
Including OP “relativity theory”, which if applied to the Ship of Theseus I guess would be something like: “Assuming you care a lot about the original Ship of Theseus, when all its components will have been progressively completely replaced, how much will you still care about it?”.
Basically different people have different subjective feelings/opinions/points-of-view/experiences/intutions/thoughts/theories/interpretations/preferences/resolutions about entity/person/mind/self continuity and that’s ok.
I personally find (at least for now) the “temporal parts theory” interpretation/resolution quite satisfying but I also like very much OP “relativity theory” and some other theories too!
And like the OP I would say: each to their own (and you are free to change your preference of theory whenever you feel like)
Having been suffering myself from ME/CFS (and/or possibly long COVID) since early 2020 (after I fell ill with an illness very similar to COVID-19 at the end of 2019) I understand and feel your frustration, pain and suffering having to face a very long haul chronic debilitating complex disease with complex/unknow/obscure etiology/mechanisms and no current proven cure and nothing much effective to treat the symptoms neither.
At least for long COVID and also ME/CFS (thanks to long COVID which has many similarities with ME/CFS) there are quite a few labs/researchers/nerds/… who are interested in trying to advance the science around these illnesses. It must be really dreadful as it seems to be the case for you, to have a mysterious chronic illness without even a specific name attached to it (from what I understand), only a set of symptoms which (similarly to ME/CFS) can have many different possible root causes/factors.
I guess one of the first things to do to create/market/… an interest from labs/researchers/nerds/… would be to find other people suffering from the same illness and create/coin a name for that illness and create some association/website/gatherings/… to communicate about it like it is done for most other illnesses?
With regard to addressing the etiology of complex chronic illnesses, specially the ones involving dysfunctions of the immune system, of the autonomic system, of the physiological energy generation/consumption mechanisms, of metabolism, etc… I wish the human body could be put into “profiling mode” (like for software) where you could trace/record in details all the related/relevant biochemical processes going on and then have tools that take that trace as input and provide as output an analysis of the processes going wrong and the root cause(s) of it and the possible remedies for it but it is of course still largely science fiction at this point in time!
So unfortunately, as yourself and some commenters in this thread have said, you have to find or determine by yourself the protocol(s)/approach(es) that you think are best suited to you and what you can do (depending on your own cognitive/physical/relational/financial/… resources).
For ME/CFS, some interesting comprehensive simple-enough-for-the-layman-to-understand approach I have come across so far is this one:
https://www.drmyhill.co.uk/wiki/Overview_of_CFS/ME_protocol
I have absolutely no affiliation and never made contact with the practician who authored that approach and do not endorse it or take any responsibility if you follow it, etc… but simply noted (in my very humble opinion) that this approach as a potentially interesting, comprehensive, systematic, systemic, rational, practical, … example of approach at least from a patient point of view in the current state of science related to ME/CFS (with the option to zoom-in / research further into any level of details at each step of this approach).
I guess this type of systematic/rational/… approach can provide some inspiration for some other complex chronic illnesses at least from a patient point of view. One of the main point of this approach is I think basically to try to list and address each and every possible cause of the illness by priority order of importance/likelihood/....
Note: sorry if my comment looks very “drafty”, as I am invoIved in the same kind of problems as the OP I wanted to quickly give my own very little 2c about them, I might slightly edit some bits of my comment later on to iron it out if/where necessary and if I have the time & energy.
Loved the post and all the comments <3
Here is I think an interesting scenario / thought experiment:
A copy of a person is made while that original person is sleeping on a bed.
The original person is moved to a sofa while still sleeping.
The copy (which is also sleeping) is put in the bed at the exact same position where the original person was.
After a while the original and the copy both wake up and can see each other (we assume they are both completely oblivious to exactly what happened while they were sleeping and that they didn’t dream or they dreamt the same thing, etc...)
At wake-up, based on their own memory of where the original person fell asleep, the original person will likely feel they are the copy and the copy will likely feel they are the original person, wouldn’t they?!
Some might even argue that based on stream-of-consciousness continuity the original “me” is actually the copy (because the copy remembers falling asleep in the bed and actually wakes up in the bed as well).
Some others will argue that based on substrate/matter continuity the original “me” is the original person even if their stream-of-consciousness has experienced a discontinuity (remembering falling asleep in the bed but actually waking up on the sofa while seeing an identical person as them waking up in the bed).
I guess it is subjective and a matter of individual preference if the stream-of-consciousness continuity or the substrate continuity is more important to define who the original “me” is.
Some would even argue that in this case there is not actual any firm original “me”, just one “stream-of-consciousness me” and another different “substrate me”.
(The same/similar thought experiment could be done using the direct brain insertion of false memories instead of moving around people while they sleep / are unconscious, in this example an original person could be inserted false memories that they are a copy and vice-versa to manipulate the memory / self-awareness of who the original “me” is, also generally it obviously could be useful when someone is uploaded/copied if they want to alter some memories of their upload/copy for some reason)
Started to enter a state that could be described as “meta analysis paralysis” (“meta-[analysis paralysis]” and not “[meta-analysis] paralysis”) when I wanted to formulate my comment about your very interesting take on EA Burnout!
Your post screamed to me as a great example of analysis paralysis and bounded rationality.
Then I started to get paralyzed trying to analyse analysis paralysis and bounded rationality in the context of EA burnout and I quickly burnt out solutionless writing this comment.
Oh the irony!
Even burnt out I was still stuck in analysis paralysis so in the end I told myself:
“Tomorrow I will ask Google and ChatGPT: ‘how to solve analysis paralysis?’”.
And then submitted that above comment which does not really help you… or maybe it does?!
Damned still paralyzed!
Anyway pushing the submit button now, not sure if is the right thing to do but my bounded rationality tells me that at least it is one thing done, even if I could have spent much more time on a much more thorough and thoughtful answer that would have allowed me to formulate a better (less wrong / more helpful) comment but maybe also hitting diminishing returns!
Hello,
Personally I think there is a major problem on how productivity is measured.
Basically:
productivity = production/time
But here is the major flaw: how is production currently measured?
It is measured by how much money you sell that production!
So basically as it stands:
productivity = (money made)/time
Imho that way of measuring productivity is really dumb and gives a completely undervalued measurement of production.
To take a simple example imagine you create (with thousands of other people) an OS like Linux that powers billions & billions of computing devices throughout the entire world (and even in space) and give away that OS for free:
Your productivity for this Linux production is measured as zero (0) because you didn’t make any money from the direct selling of it. The fact that your measured production and productivity for this is zero is completely absurd because you actually produced something extremely useful and transformative on a huge scale. There are many other examples like that of free or very cheap things which are measured as having a very low productivity not because they are useless but because they are (or have become) free / very cheap.
To take another example, let’s say you have speculated on the markets and got lucky and made a huge amount of money very quickly: you haven’t really produced anything but your productivity is measured as being huge!
So basically imho anything / any argument which is based on how productivity is currenly measured is completely flawed.
Please correct me if I am wrong so that I am less wrong thank you :)
Hello,
I tend to intuitively strongly agree with James Miller’s point (hence me upvoting it).
There is a strong case to make that a TAI would tend to spook economic agents which create products/services that could easily be done by a TAI.
For an anology think about a student who wants to decide on what xe (I prefer using the neopronoun “xe” than “singular they” as it is less confusing) wants to study for xir future job prospects: if that student thinks that a TAI might do something much faster/better than xem in the future (translating one language into another, accounting, even coding, etc...) that student might be spooked into thinking “oh wait maybe I should think twice before investing my time/energy/money into studying these.”, so basically a TAI could create lot of uncertainty/doubt/… for economic actors and in most cases uncertainty/doubt/… have an inhibiting effect on investment decisions and hence on interest rates, don’t they?
I am very willing to be convinced of the opposite and I see a lot of downvotes for James Miller hypothesis but not many people so far arguing against it.
Could someone please who downvoted/disagrees with that argument kindly make the argument against James Miller hypothesis? I would very much appreciated that and then maybe change my mind as a result but as it stands I tend to strongly agree with James Miller well stated point.
You make good/interesting points:
1) About AGI being different from ASI: basically this is the question of how fast we go from AGI to ASI i.e. how fast is the takeoff. This is debated and no one can exactly predict how much time it will take i.e. if it would/will be a slow/soft takeoff or a fast/hard takeoff. The question of what happens economically during the AGI to ASI takeoff is also difficult to predict. It would/will depend on what (the entity controlling) the self-improving AGI decides to do, how market actors are impacted, if they can adapt to it or not, government intervention (if the AGI/ASI makes it possible), etc...
2) With regard to the impact of an ASI on the economic world and society I would distinguish between
2a) The digital/knowledge/etc… economy basically everything that can be thought of as “data processing” that can be done by computing devices:
an ASI could take over all of that very quickly.
2b) The “physical” economy… i.e. basically everything that can be thought as “matter processing” that can be done by human bodies, machines, robots, …:
an ASI could take over all of that but it would take more time than the digital world of course as the ASI would/will need to produce many machines/robots/etc… and there could indeed be a bottleneck in terms of resources and laws of physics but if you imagine that the ASI would quickly master fast space travel, fast asteroid mining, fast nuclear fusion, fast robot production, etc… it might not take that long neither. The question of what would happen economically while this happens is also difficult to predict. Traditional/existing economic actors could for example just basically stop as soon as the ASI starts providing any imaginable amount of great quality goods and services to any living entities if the ASI is benovolent/utilitarian (within the constraints of the laws of physics if it is in the real/physical world and if the ASI don’t find ways to overcome the laws of physics in the real/physical world) basically what is called “post-scarcity”.
But there could be other scenarios including economic scenarios as well basically essentially depending on what (the entity controlling) the ASI decides to do, it could decide that people still need to be forced to work to keep some meaning of life so it could artificially maintain a working capitalist economy, etc...
When the digital and physical world are essentially mastered at will then basically it “just” becomes a question of how things are then organized/allocated. Money/interest rates/etc… become unnecessary to do that (but could still be used if that is the choice).
Thank you for your interesting answer :)
I agree that in all likelihood a TS/ASI would be very disruptive for the economy.
Under some possible scenarios it would benefit most economic actors (existing and new) and lead to a general market boom.
But under some other possible scenarios (like for example as you mentioned a monopolistic single corporation swallowing up all the economic activity under the command of a single ASI) it would lead to an economic and market crash for all the other economic actors.
Note that a permanent economic and market crash would not necessarily mean that the standards of living would not drastically improve, in this scenario (the monopolistic ASI) the standards of living would not depend on an economic and market crash but on how benevolent/utilitarian the entity controlling the ASI is.
In economic/market terms there are plenty of possible scenarios depending mostly on what the entity controlling the ASI decides to do with regard to economic trade which is indeed the key word here as you rightly mentioned.
Given that it is imho impossible (or at least very speculative) to predict which economic trade configuration and economic scenario would be the most likely to emerge, it is also impossible (or at least very speculative) to predict what the interest rates would become (if they still exist at all).
So to come back to the original question about EMH and AGI/ASI/TS, as it is imho impossible (or very speculative) to predict the economic scenario that will emerge in case of the emergence of an AGI/ASI, the EMH is kept safe by the markets currently not taking into account what impact an AGI/ASI will have on interest rates.
Note that, as mentioned, imho, in case of an AGI/ASI/TS the standards of living would not depend on an economic and market boom or crash but on how benevolent/utilitarian the (entity controlling the) AGI/ASI is.
Thank you for your answer :)
Imho there will definitely be a flood of already existing products and services being produced at rock-bottom prices and a flood of new products and services at cheap prices, etc… coming from the entity having created / in control of the ASI, but will that make the economy as a whole booming? I am not sure.
To take an analogy imagine a whole new country appears from under the ocean (or an extraterrestrial alien spaceship, etc...) and flood the rest of the world with very cheap products and services as well as new products and services etc… at very cheap prices completely outcompeting each and every company in the rest of the world, what would that mean for the economy of the rest of the world: absolutely trashed, wouldn’t it?
All the companies, workers, means of production of the rest of the world would become very quickly valueless, even commodities as the SI could, if it wanted to, get them in almost unlimited quantity from outerspace and nuclear fusion etc...
Maybe Earth land would still have some value for people who enjoy living / traveling there rather than on giant artificial Earth satellites, etc...
The company having created / in control of ASI will be economically booming (in the short term at least) for sure but the rest of the economy and markets completely outcompeted by it, I am not sure, it would depend if the company having created / in control of ASI is willing to share some of its economic value / activity to the rest of the world or just quickly absorb all the economic activity into an economic singularity.
What do you think?
From my experience/observation GDM models seems to be quite vulnerable to crescendo attacks. In particular GDM models seem to be heavily influenced by their own responses when generating further responses. So the more you fill/saturate the context window with model responses of a certain kind, in this case, humility, self-loathing, distress, the more the subsequent responses will contain the same kind of things if you steer the model in the same direction.
GDM model sycophancy seems also to be an additional factor in that behaviour and you can exploit GDM model sycophancy (by being yourself sycophantic towards the model and/or encouraging/praising the model when it expresses some responses you are after) to increase the effectiveness of these crescendo attacks.
As an aside: I recently tried to publish here on LessWrong such crescendo attack on a GDM model to demonstrate how you can make it express toxic/distressing/harmful content/behaviour of any kind. I talked to the LessWrong moderation team about it before finishing writing and publishing but unfortunately they would’t allow me to publish it because I was a “new user”.