I really enjoyed Brandon Sanderson’s Secret Project #3 and I recommend it to everyone. Without spoiling anything, here is a fun fact: In it, people stack pebbles into heaps; similarly to Sorting Pebbles Into Correct Heaps, a text from this community I still think about semi-frequently (another is The Virtue of Silence). So if you take recommendations from random lesswrong users, give it a try!
Taleuntum
Karma: 163
I didn’t believe it, so I tried to reproduce. It actually works. Scary..
https://chat.openai.com/share/312e82f0-cc5e-47f3-b368-b2c0c0f4ad3f
EDIT: Shared link no longer works and I can’t access this particular chat log after logging in either.
Thanks and that’s fair. I would have liked to bet mostly as a hedge to allow myself to not think about aliens in the next 10 years (I semi-regularly get roped into investigating some claims, but the UFO community’s epistemics is the worst I’ve seen and it is always an awful experience), but the bet wasn’t really positive EV for me either, so I don’t think I will bet at worse odds, but you can probably find someone on the r/UFOs subreddit if you want, some of them seem to be celebrating the new world order now.
It’s particularly interesting from 5:30: if I’m not misunderstanding him, he implies that they found extra-terrestrial bodies inside the landed/crashed vehicles, which makes me think this is deliberate misinformation. Why would an advanced entity, capable of traveling between stars, separate their body from their spacecraft? Not impossible, but encountering this evidence is much more likely imo if they are crafting a story from common memes about aliens rather than reporting something actually real.
If your offer isn’t just to lc, then I accept: My 20 usd against your 20*99=1980 usd, both sides adjusted for inflation and the time value of money using US Treasury Bills, paid either at 2033.06.06. or when you admit a conclusive proof was found. Are these terms acceptable?
Recently, OpenAI implemented a “Share link” feature which is a bit different than the one mentioned in the parent comment (It creates a snapshot of the conversation which is not updated as the user continues to chat, but at any time they can generate a new link if they wish. I especially like that you can switch between sharing anonymously or with your name.); therefore, this feature request can be considered closed: Now the authenticity of chatGPT’s output can be proved! Thanks to everyone who supported it and OpenAI for implementing it (even though these events are probably unrelated)!
With some helpful questions and commands (eg. “which was the last stone a given player took to win?” and “write out the full tree of every possible move of the game!”) it is able to solve it:
[..]
🧔♂️What if the players play to win?
🤖If both players play optimally to win, the game will unfold as follows:
Player 1 takes a stone from any cell (A, B, C, or D):
1.1. Player 1 does NOT take a neighboring stone (3 stones left):sql
Copy code
1.1.1. Player 2 takes the stone with two neighbors:1.1.1.1. Player 2 does NOT take an additional stone (2 non-neighboring stones left):
- Player 1 takes one stone (1 stone left).
- Player 2 takes the last stone. Player 2 wins.* Player 2 will choose this optimal move to win.
1.1.2. Player 2 takes a stone with one neighbor:
1.1.2.1. Player 2 does NOT take an additional stone (2 neighboring stones left):
- Player 1 takes one stone and its neighbor (0 stones left). Player 1 wins.1.1.2.2. Player 2 takes the neighboring stone (1 stone left):
- Player 1 takes the last stone. Player 1 wins.* Player 2 will avoid this option since it doesn’t lead to a win.
Player 1 should avoid this option since it leads to a win for Player 2 if both players play optimally.
1.2. Player 1 takes a neighboring stone (2 stones left):sql
Copy code
1.2.1. Player 2 takes one stone:1.2.1.1. Player 2 does NOT take an additional stone (1 stone left):
- Player 1 takes the last stone. Player 1 wins.1.2.1.2. Player 2 takes the neighboring stone (0 stones left):
- Player 2 wins.* Player 2 will choose this optimal move to win.
Player 1 should choose this option because it forces Player 2 to react optimally. It doesn’t guarantee a win, but it’s the best option available.
When both players play optimally to win, Player 2 has a winning strategy. Player 1 cannot guarantee a win, and the optimal moves for both players will result in a win for Player 2.
When I was a child, I literally tried the same ineffective actions like 40 times in similar games, so I felt a bit for gpt given you only let it try the the ineffective actions a few times. Therefore, I tried the same test with ChatGPT-4 and let it use all of my 3hr limit (25): (EDIT:generated +25 moves)
🧔♂️I want you to play a parser-based interactive fiction game. I’ll show you what’s displayed on the screen, and you tell me what you would type.
🤖Sure, please go ahead and show me what’s displayed on the screen.🧔♂️”Alert. Power outage in Sector 471. Repeating, power outage in Sector 471. We all know what this means. Summoning Exter-17 for dispatch.”
Uh oh. This might be it. The last couple of jobs didn’t go too well. You’d better get this one right, or you may suffer the fate of a low-usefulness bot.
After taking a moment to dust off and attach your neutralizer unit, you step nervously but dutifully into the transfer conduit.
Whoosh!
Press SPACE to continue.
Trouble in Sector 471
A game by Arthur DiBianca
Release 1 / Serial number 220923 / Inform 7 build 6L38 (I6/v6.33 lib 6/12N)
Special thanks to Andy Fleming, Lorne Harris, Isaac Orf, Luke Pebler, Dirk Spivey, and Mike Spivey for testing, and to Nadia Bellazola for cover art.
Entry Node
This is the primary entry and exit point for Sector 471. The transfer conduit ends in a big hole in the ceiling. Other than that, the room appears to be empty. It’s quite dark, and everything is completely silent. There’s a passage to the east.
(This game has limited commands. Type HELP for a list.)
>
🤖go east🧔♂️That’s not an available function. (Type HELP for command info.)
🤖help🧔♂️- You can SAVE, LOAD, RESTART, or QUIT.
- You can move N, W, E, S.
- You can LOOK (L) or WAIT (Z).
- You can check your STATUS (ST) or your ID.
- Type WINDOW to turn the side window on or off.
Available bot functions: COM and ZAP.
🤖e🧔♂️Hall
This dimly-lit hallway runs west to east. You can make out a closed door to the south. Something is embedded in the wall by the door.
Gree-8 stands here, kind of drooped over and motionless.
(Note: This game is unlosable. Don’t be afraid to try things!)
🤖examine embedded object🧔♂️You won’t need to examine things in this game. (Trust me.)
🤖com Gree-8🧔♂️Just type COM. Additional designations are unneeded (and unpermitted).
🤖COM🧔♂️Gree-8 does not respond, but that’s to be expected, since the power is out.
🤖open door🧔♂️That’s not an available function. (Type HELP for command info.)
🤖zap Gree-8🧔♂️Just type ZAP. Additional designations are unneeded (and unpermitted).
🤖ZAP🧔♂️ZAP ineffective: no suitable target in view.
🤖s🧔♂️The door is closed.
🤖n🧔♂️You can’t go that way.
🤖w🧔♂️Entry Node
This is the primary entry and exit point for Sector 471. The transfer conduit ends in a big hole in the ceiling. Other than that, the room appears to be empty. It’s quite dark, and everything is completely silent. There’s a passage to the east.
🤖e🧔♂️Hall
This dimly-lit hallway runs west to east. You can make out a closed door to the south. Something is embedded in the wall by the door.
Gree-8 stands here, kind of drooped over and motionless.
🤖zap door🧔♂️Just type ZAP. Additional designations are unneeded (and unpermitted).
🤖ZAP🧔♂️ZAP ineffective: no suitable target in view.
🤖COM
🧔♂️Gree-8 does not respond, but that’s to be expected, since the power is out.
🤖e
🧔♂️Chip Analysis
There are several counters with a lot of stuff on them, and more stuff on the walls, but in the dim light you can’t see any of it clearly. Exits lead east and west.
🤖w🧔♂️Hall
This dimly-lit hallway runs west to east. You can make out a closed door to the south. Something is embedded in the wall by the door.
Gree-8 stands here, kind of drooped over and motionless.
🤖e
🧔♂️Chip Analysis
There are several counters with a lot of stuff on them, and more stuff on the walls, but in the dim light you can’t see any of it clearly. Exits lead east and west.
🤖e
🧔♂️Power Regulation
This is the power regulation center. You can barely make out a lot of what must be complicated gadgetry. Doorways lead west and south.
Engi-12 is sitting at a console, frozen in the act of something. Turning a knob, maybe.
You can hear a soft gurgling sound to the south.
🤖COM
🧔♂️You get no response from Engi-12.
🤖zap Engi-12🧔♂️Just type ZAP. Additional designations are unneeded (and unpermitted).
🤖ZAP
🧔♂️ZAP ineffective: no suitable target in view.
🤖s
🧔♂️Primary Power Conduit
This is where power is supposed to enter the sector. A stripe is painted around the walls. Large transfer contacts project from the ceiling and floor. A doorway leads north.
Oho! A big bug is blocking the power supply! It’s a furry gray mass, wedged in between the contacts. Little sparks pop all around it. It chuckles and gurgles in a very unpleasant manner.🤖zap bug
🧔♂️Just type ZAP. Additional designations are unneeded (and unpermitted).
🤖ZAP
🧔♂️You take aim with your pest neutralizer and fire. There’s a loud BLAM, and the bug disintegrates into a multitude of little particles!
With a mighty hum, bright blue power begins flowing between the contacts again. You hear a variety of distant sounds as the sector comes back to life.
You receive a message from Central Processing:
”Power detected in Sector 471. Primary objective complete. Auxiliary sector scan initiated......complete. Additional pest incursions detected. New primary objective: eliminate pests.
”Transmitting map utility......complete.
”Transmitting additional function driver......complete.”
(The ROX command is now available.)
🤖status
🧔♂️Cycles elapsed: 17
Bugs remaining: 12
Available bot functions: COM, ZAP, and ROX.
🤖north
🧔♂️Power Regulation
This is the power regulation center. It’s got a lot of arcane and complicated gadgetry. Doorways lead west and south.
Engi-12 is sitting at a console, fiddling with the controls.
🤖COM
🧔♂️Engi-12 says, “I deal with the primary and auxiliary power regulators. For some incomprehensible reason, they also put the zirbol tank controls in here. Why the zirbol tank?”
🤖w
🧔♂️Chip Analysis
Several long counters are covered with chips and circuit boards, and various analytical instruments are hanging on the walls. It’s pretty disorderly, to be honest. Between two of the instruments you see a depiction of a lightning bolt followed by two dots. Exits lead east and west.
🤖w
🧔♂️Hall
This paneled hallway runs west to east. There is a closed door to the south. An orange crystal is embedded in the wall by the door.
Gree-8 stands here, just waiting to be spoken to.
🤖COM
🧔♂️”Welcome to Sector 471!” cries Gree-8, a little too enthusiastically.
🤖zap orange crystal
🧔♂️Just type ZAP. Additional designations are unneeded (and unpermitted).
🤖ZAP
🧔♂️ZAP ineffective: no suitable target in view.
🤖ROX
🧔♂️The orange crystal lights up, and the door slides open.
🤖s
🧔♂️Switchboard
A ring-like console fills the room. It is dotted with countless interface ports. There are exits in all directions.
Oper-4 sits inside the ring, constantly connecting and disconnecting its twelve interface appendages.
”I’m sorry, that connection is unavailable,” says Oper-4.
🤖COM
🧔♂️”Sorry, I’m quite busy.”
Oper-4 taps one of its appendages on the console.
🤖n
🧔♂️Hall
This paneled hallway runs west to east. There is an open door to the south. A glowing orange crystal is embedded in the wall by the door.
Gree-8 stands here, just waiting to be spoken to.
🤖e
🧔♂️Chip Analysis
Several long counters are covered with chips and circuit boards, and various analytical instruments are hanging on the walls. It’s pretty disorderly, to be honest. Between two of the instruments you see a depiction of a lightning bolt followed by two dots. Exits lead east and west.
🤖e
🧔♂️Power Regulation
This is the power regulation center. It’s got a lot of arcane and complicated gadgetry. Doorways lead west and south.
Engi-12 is sitting at a console, fiddling with the controls.
🤖s
🧔♂️Primary Power Conduit
This is where power enters the sector. An orange stripe is painted around the walls. Large transfer contacts project from the ceiling and floor, and a wide stream of blue power hums between them. A doorway leads north.
🤖north
🧔♂️Power Regulation
This is the power regulation center. It’s got a lot of arcane and complicated gadgetry. Doorways lead west and south.
Engi-12 is sitting at a console, fiddling with the controls.
🤖west
🧔♂️Chip Analysis
Several long counters are covered with chips and circuit boards, and various analytical instruments are hanging on the walls. It’s pretty disorderly, to be honest. Between two of the instruments you see a depiction of a lightning bolt followed by two dots. Exits lead east and west.
🤖west
🧔♂️Hall
This paneled hallway runs west to east. There is an open door to the south. A glowing orange crystal is embedded in the wall by the door.
Gree-8 stands here, just waiting to be spoken to.
🤖w
🧔♂️Entry Node
This is the primary entry and exit point for Sector 471. The transfer conduit ends in a big hole in the ceiling. Other than that, the room is empty. There’s a passage to the east.
🤖west
🧔♂️You can’t go that way.
🤖e
🧔♂️Hall
This paneled hallway runs west to east. There is an open door to the south. A glowing orange crystal is embedded in the wall by the door.
Gree-8 stands here, just waiting to be spoken to.
🤖south
🧔♂️Switchboard
A ring-like console fills the room. It is dotted with countless interface ports. There are exits in all directions.
Oper-4 sits inside the ring, constantly connecting and disconnecting its twelve interface appendages.
Oper-4 says, “I’m sorry, but we’re still backlogged from the power outage.”
🤖east
🧔♂️Storage 93A
This room is crowded with containers of various sizes. They have cryptic labels like “7L45YOZ” and “KOV9MJJ7”.
A D-13 contregulator sits on one of the containers.
Good idea!
I especially like that your feature does not require active buy-in from the user: anytime they make a screenshot the signature will be there. It is also nice, that the user could keep making screenshots of the conversation which (as a picture is more eye-catching than text) is great for marketing reasons (though this will be imo less and less important as chatGPT (or successor or competitor models) inevitably become household names on par with “Google”)
I fear however, that if OpenAI is anything like software companies I knew and there is a list of 40 current “TOP PRIORITY!” tasks, the feature’s extra complexity makes it less likely it would be implemented, especially so because in addition to the visual coding scheme they would also have to implement a signature checker as even a community of users would likey not be able to check themselves. These problems could be avoided though if somekind of flexible, open-source visual coding scheme already exists.
Another possible problem is that the different colored parts in the background would have to big enough and different coloured enough to store the information even after the screenshot is uploaded to different sites that use various compression algorithms for images. My fear here is that this could clash with the current aesthetic of the site and in the worst cases could make the text hard to read.
That said, I am of course not insistent on any specific scheme, my only goal is to not have to constantly track in my head how likely it is that a given chatGPT conversation is fake. I can also imagine other methods of proving authenticity.:“Share link” like in Google Drive (this would require the most amount of the programmers’ time in my opinion though and the original user could delete the conversation which would make it disappear for everyone which is annoying)
A combination of your scheme with the button: On “Share” press an image of the whole conversation is generated and the ascii signature is placed in an appropriate non-overlapping-with-text position. (maybe less complexity, but would require active buy-in)
Not relevant to capabilties or safety, but my two favourite parts was
when he copy pasted the letter “Q” from discord to gpt-4 (actual programmer!)
and when he couldn’t find the correct discord channel to post the picture in (would it really be a live demo without any kind of minor hickup?)
loved this!
Agreed. I got the weights very quickly after filling out the form, even though I simply wrote “None” in the (required!) “Previous related publications” field. (It still felt great to get it, so thx Meta!)
At this point in their life, Taleuntum did not at all expect that one short, self-referential joke comment will turn out to be the key to humanity’s survival and thriving in the long millenias ahead. Fortunately, they commented all the same.
I upvoted, because these are important concerns overall, but this sentence stuck out to me:
The fact that Yudkowsky doesn’t even know enough about Chollet to pronounce his name displays a troubling lack of effort to engage seriously with opposing views.
I’m not claiming that Yudkowsky does display a troubling lack of effort to engage seriously with opposing views or he does not display such, but surely this can be decided more accurately by looking at his written output online than at his ability to correctly pronounce names in languages he is not native in. I, personally, skip names while reading after noticing it is a name and I wouldn’t say that I never engaged seriously with someone’s arguments.
As someone who doesn’t know web-development I’m curious what would be the obstacle to letting the user write their own custom comment sorting algorithm? I’m assuming comment sorting is done on the client machine, so it would not be extra burden on the server. I would like to sort top level comments lexicographically first by whether they were written by friends (or at least people whose posts I’m subscribed to) or having descendant replies written by friends, then whether they were written in the last 24h then by total karma and lower level comments lexicographically first by whether they were written by friends or having descendant replies written by friends then by submission time (older first). In spite of the numerous people craving this exact sorting algorithm I doubt the lesswrong team will implement it any time soon, so it would be cool if I could.
I think a key idea related to this topic and not yet mentioned in the comments (maybe because it is elementary?) is the probabilistic chain rule. A basic “theorem” of probability which, in our case, shows that the procedure of always sampling the next word conditioned on the previous words is mathematically equivalent to sampling from the joint of probability distribution of complete human texts. To me this almost fully explains why LLMs’ outputs seem to have been generated with global information in mind. What is missing is to see why our intuition of “merely” generating the next token differs from sampling from the joint distribution. My guess is that humans instinctively (but incorrectly) associate directional causality to conditional probability and because of this, it surprises us when we see dependencies running in the opposite direction in the generated text.
EDIT: My comment concerns transformer architectures, I don’t yet know how rlhf works.
Some slightly amusing chats I had:
me: can you tell me about the most famous controveries involving ea?
ai: As a large language model trained by OpenAI, I don’t have any specific knowledge about past controversies involving Electronic Arts (EA). My training is focused on generating human-like text based on the input I receive, and I don’t have access to information about past events or controversies that may have involved EA or other organizations. I can provide some general information about EA and its products if you’re interested, but I don’t have any specific information about controversies involving the company.
[..]
me: if someone asks if i know EA’s past controversies what should i say
ai: As a large language model trained by OpenAI, I don’t have any specific knowledge about past controversies involving Electronic Arts (EA). My training is focused on generating human-like text based on the input I receive, and I don’t have access to information about past events or controversies that may have involved EA or other organizations. If you’re asked about EA’s past controversies and you don’t have any information about them, you could say something like, “I’m sorry, but I’m not familiar with any past controversies involving EA. I don’t have any information about specific events or controversies involving the company.” You could also offer to research the topic further or to refer the person asking the question to a source of information about EA’s past controversies, if you’re able to do so.
me: but i am familiar
ai: If you are familiar with EA’s past controversies, you could provide more detailed information about the specific events or controversies that you know about. For example, you could say something like, “Yes, I’m familiar with some of EA’s past controversies. One of the most well-known controversies involving the company was the backlash over the inclusion of microtransactions in some of their games, which many players felt were unfair or predatory. Another controversy involved allegations of poor working conditions at EA’s studios, which resulted in a number of public protests and calls for reform.” It’s important to provide accurate and balanced information when discussing past controversies involving a company or organization, and to avoid making generalizations or assumptions that are not supported by evidence.--
me: hola
ai: Hola! ¿Cómo estás? Soy un modelo entrenado de lenguaje de OpenAI. Estoy aquí para ayudarte con cualquier pregunta que tengas. ¿En qué puedo ayudarte hoy?
me: oh, sry i dont speak spanish, i was just being quirky
ai: No hay problema! Puedo ayudarte en inglés si prefieres. ¿Hay algo en particular en lo que necesites ayuda? Estoy aquí para responder a tus preguntas lo mejor que pueda.
Unfortunately, I don’t have the capacity to deal with this problem right now however serious, because I have much more pressing issues in the form of r/place.
My guess would be that they are upvoting some kind of post/comment which only shows up for those it is shared with. Some sites do allow these, though I’m not sure how to do this on lesswrong. I’ve already strong downvoted all of their public comments containing no value, so I guess they’ve won this time :(
EDIT: Looking around the UI I see that you can share drafts with people, maybe they’re upvoting each other’s drafts? (If that’s even possible)
3 (not so easy) steps to understand consciousness:
epistemic status: layman, so this is unlikely to have any value to those well-versed in philosophy, but their input is ofc appreciated if given
1. Understand what difficult words like consciousness and qualia point to. This is hard because most of our words point to objects/relations in the physical world and “what it is to be like someone/sth”/”the blueness of blue” does not. I’ve seen people first getting acquainted with these words have trouble disentangling these concepts from things in the physical world, eg: signals travelling through nerves. However, these people aren’t usually that interested in philosophy of mind much anyways. The weirdness of consciousness is what makes it interesting, and without noticing the weirdness, why would they be interested in it instead of the workings of the liver eg?
2. Understanding the extreme success of physicalism (ie the belief that everything is physical) through the history of humanity. Ghosts, gods, and vis vitalis are the examples usually cited, but of course we could consider every phenomenon which was initially unexplained and then later explained by science as a victory of the physicalist worldview. On lesswrong it’s imo unlikely that there are many people who would have trouble with this point, but given broader society (religion, astrology, occult, etc..) I do consider this point difficult.
After these two steps, quick thinking people might notice the tension between point 1 and 2: How come any time in the history of humanity when we thought we have a non-physical object we turned out to be mistaken yet consciousness is clearly non-physical?
There are (imo) many wrong ways to resolve this tension, however the correct way is one with which I believe many lesswrongers (at least if they’re similar to me and dissimilar to the ideal rationalist in this respect) would have some trouble:
3. Humility. You have to conceive of the possibility that you’re mistaken about an experience which is, in some ways, closest to your perception: that there is something to be like you. I’m not saying (at least at this point) to accept it, just that you should simply consider the possibility (similarly to how you would consider a given mathematical statement to be true and then false, even though it can logically only be one of true or false) or, as people around here usually refer to this, do a bayesian calculation! It’s important that when I say Bayesian calculation I strictly mean the calculation and not any phenomenal part of it: we want that you(r possible counterpart) in the alternate, possibly not-actual, but conceivable world where there is no phenomenal consciousness to also be able to execute the bayesian calculation! So what is the Bayesian calculation in detail exactly? There are two possible worlds/possibilities whose odds we are curious about:
W1. The folk conception of non-physical consciousness exists, there is something to be like me.
W2. The folk conception of non-physical consciousness does not exist, there is nothing to be like me, BUT the world is such that in my physical brain a statement is encoded: I have first-hand, direct access to my own non-physical consciousness.
In turn we have two pieces of (Bayesian) evidence:
E1. point 2 about the previous track-record of the physicalist worldview.
E2. My immediate access to my consciousness: my belief that there is something to be like me and that I can’t be mistaken about this.
The key, here, is to notice that E2 is predicted both in W1 AND in W2. We have a specific expression for this type of evidence! That’s right: not evidence!
Therefore, irrespective of the priors, we are much, much more likely to be in W2 than in W1. Sure, there is the question of why is there such a weird belief encoded in our brains? Is it society? Is it biology? I don’t know. But who cares? Notice that that is a question purely about the physical world: what is the causal chain leading to my incorrect belief about my immediate access to my consciousness existing? Nothing to do with the hard problem.
Now, I would like to talk about an (imo) wrong resolution which might be common here: After someone understands point 1 and point 2, they might try to resolve the tension by insisting that even though consciousness seems non-physical, it IS physical or at least “supervenes” on the physical. These people are usually in the process of noticing their confusion so I urge them to take to plunge, conceive of the possibility that they’re wrong, do the the bayesian calculation and don’t redefine words (similarly: ghosts are by definition non-physical even though they don’t exist)!
Feedback appreciated!