Bruce G

• Bruce G 22 Jan 2024 2:12 UTC

  1 point

  0

  AF

  in reply to: Beth Barnes’s comment on: Bounty: Di­verse hard tasks for LLM agents

  I have a mock submission ready, but I am not sure how to go about checking if it is formatted correctly.

  Regarding coding experience, I know python, but I do not have experience working with typescript or Docker, so I am not clear on what I am supposed to do with those parts of the instructions.

  If possible, It would be helpful to be able to go through it on a zoom meeting so I could do a screen-share.

• Bruce G 20 Dec 2023 6:02 UTC

  1 point

  0

  AF

  in reply to: Beth Barnes’s comment on: Bounty: Di­verse hard tasks for LLM agents

  Thanks for your reply. I found the agent folder you are referring to with ‘main.ts’, ‘package.json’, and ‘tsconfig.json’, but I am not clear on how I am supposed to use it. I just get an error message when I open the ‘main.ts’ file:

  

  Regarding the task.py file, would it be better to have the instructions for the task in comments in the python file, or in a separate text file, or both? Will the LLM have the ability to run code in the python file, read the output of the code it runs, and create new cells to run further blocks of code?

  And if an automated scoring function is included in the same python file as the task itself, is there anything to prevent the LLM from reading the code for the scoring function and using that to generate an answer?

  I am also wondering if it would be helpful if I created a simple “mock task submission” folder and then post or email it to METR to verify if everything is implemented/​formatted correctly, just to walk through the task submission process, and clear up any further confusions. (This would be some task that could be created quickly even if a professional might be able to complete the task in less than 2 hours, so not intended to be part of the actual evaluation.)

• Bruce G 19 Dec 2023 5:29 UTC

  1 point

  0

  AF

  on: Bounty: Di­verse hard tasks for LLM agents

  If anyone is planning to send in a task and needs someone for the human-comparison QA part, I would be open to considering it in exchange for splitting the bounty.

  I would also consider sending in some tasks/​ideas, but I have questions about the implementation part.

  From the README document included in the zip file:

  ## Infra Overview

  In this setup, tasks are defined in Python and agents are defined in Typescript. The task format supports having multiple variants of a particular task, but you can ignore variants if you like (and just use single variant named for example “main”)

  and later, in the same document

  You’ll probably want an OpenAI API key to power your agent. Just add your OPENAI_API_KEY to the existing file named `.env`; parameters from that file are added to the environment of the agent.

  So how much scaffolding/​implementation will METR provide for this versus how much must be provided by the external person sending it in?

  Suppose I download some data sets from Kaggle as and save them as CSV files; and then set up a task where the LLM must accurately answer certain questions about that data. If I provide a folder with just the CVS files, a README file with the instructions and questions (and scoring criteria), and a blank python file (in which the LLM is supposed to write the code to pull in the data and get the answer), would that be enough to count as a task submission? If not, what else would be needed?

  Is the person who submits the test also writing the script for the LLM-based agent to take test, or will someone at METR do that based on the task description?

  Also, regarding this:

  Model performance properly reflects the underlying capability level

  Not memorized by current or future models: Ideally, the task solution has not been posted publicly in the past, is unlikely to be posted in the future, and is not especially close to anything in the training corpus.

  I don’t see how the solution to any such task could be reliably kept out of the training data for future models in the long run if METR is planning on publishing a paper describing the LLM’s performance on it. Even if the task is something that only the person who submitted it has ever thought about before, I would expect that once it is public knowledge someone would write up a solution and post it online.

• Bruce G 24 Sep 2023 1:45 UTC

  12 points

  3

  in reply to: jefftk’s comment on: Paper: LLMs trained on “A is B” fail to learn “B is A”

  I presume you have in mind an experiment where (for example) you ask one large group of people “Who is Tom Cruise’s mother?” and then ask a different group of the same number of people “Mary Lee Pfeiffer’s son?” and compare how many got the right answer in the each group, correct?

  (If you ask the same person both questions in a row, it seems obvious that a person who answers one question correctly would nearly always answer the other question correctly also.)

• Bruce G 8 May 2023 4:12 UTC

  1 point

  0

  in reply to: HoldenKarnofsky’s comment on: How we could stum­ble into AI catastrophe

  Is the disagreement here about whether AIs are likely to develop things like situational awareness, foresightful planning ability, and understanding of adversaries’ decisions as they are used for more and more challenging tasks?

  My thought on this is, if a baseline AI system does not have situational awareness before the AI researchers started fine-tuning it, I would not expect it to obtain situational awareness through reinforcement learning with human feedback.

  I am not sure I can answer this for the hypothetical “Alex” system in the linked post, since I don’t think I have a good mental model of how such a system would work or what kind of training data or training protocol you would need to have to create such a thing.

  If I saw something that, from the outside, appeared to exhibit the full range of abilities Alex is described as having (including advancing R&D in multiple disparate domains in ways that are not simple extrapolations of its training data) I would assign a significantly higher probability to that system having situational awareness than I do to current systems. If someone had a system that was empirically that powerful, which had been trained largely by reinforcement learning, I would say the responsible thing to do would be:

  1. Keep it air-gapped rather than unleashing large numbers of copies of it onto the internet

  2. Carefully vet any machine blueprints, drugs or other medical interventions, or other plans or technologies the system comes up with (perhaps first building a prototype to gather data on it in an isolated controlled setting where it can be quickly destroyed) to ensure safety before deploying them out into the world.

  The 2nd of those would have the downside that beneficial ideas and inventions produced by the system take longer to get rolled out and have a positive effect. But it would be worth it in that context to reduce the risk of some large unforeseen downside.

• Bruce G 5 May 2023 5:44 UTC

  1 point

  0

  in reply to: HoldenKarnofsky’s comment on: How we could stum­ble into AI catastrophe

  Those 2 types of downsides, creating code with a bug versus plotting a takeover, seem importantly different.

  I can easily see how an LLM-based app fine-tuned with RLHF might generate the first type of problem. For example, let’s say some GPT-based app is trained using this method to generate the code for websites in response to prompts describing how the website should look and what features it should have. And lets suppose during training it generates many examples that have some unnoticed error—maybe it does not render properly on certain size screens, but the evaluators all have normal-sized screens where that problem does not show up.

  If the evaluators rated many websites with this bug favorably, then I would not be surprised if the trained model continued to generate code with the same bug after it was deployed.

  But I would not expect the model to internally distinguish between “the humans rated those examples favorably because they did not notice the rendering problem” versus “the humans liked the entire code including the weird rendering on larger screens”. I would not expect it to internally represent concepts like “if some users with large screens notice and complain about the rendering problem after deployment, Open AI might train a new model and rate those websites negatively instead” or to care about whether this would eventually happen or to take any precautions against the rendering issue being discovered.

  By contrast, the coup-plotting problem is more similar to the classic AI takeover scenario. And that does seem to require the type of foresight and situational awareness to distinguish between “the leadership lets me continue working in the government because they don’t know I am planning a coup” versus “the leadership likes the fact that I am planning to overthrow them”, and to take precautions against your plans being discovered while you can still be shut down.

  I don’t think n AI system gets the later type of ability just as an accidental side effect of reinforcement learning with human feedback (at least not for the AI systems we have now). The development team would need to do a lot of extra work to give an AI that foresightful planning ability, and ability to understand the decision system of a potential adversary enough to predict which information it needs to keep secret for its plans to succeed. And if a development team is giving its AI those abilities (and exercising any reasonable degree of caution) then I would expect them to build in safeguards: have hard constraints on what it is able to do, ensure its plans are inspectable, etc.

• Bruce G 26 Mar 2023 15:40 UTC

  1 point

  0

  in reply to: Lone Pine’s comment on: How we could stum­ble into AI catastrophe

  Did everyone actually fail to notice, for months, that social media algorithms would sometimes recommend extremist content/​disinformation/​conspiracy theories/​etc (assuming that this is the downside you are referring to)?

  It seems to me that some people must have realized this as soon as they starting seeing Alex Jones videos showing up in their YouTube recommendations.

• Bruce G 26 Mar 2023 3:42 UTC

  1 point

  0

  in reply to: HoldenKarnofsky’s comment on: How we could stum­ble into AI catastrophe

  I think the more capable AI systems are, the more we’ll see patterns like “Every time you ask an AI to do something, it does it well; the less you put yourself in the loop and the fewer constraints you impose, the better and/​or faster it goes; and you ~never see downsides.” (You never SEE them, which doesn’t mean they don’t happen.)

  This, again, seems unlikely to me.

  For most things that people seem likely to use AI for in the foreseeable future, I expect downsides and failure modes will be easy to notice. If self-driving cars are crashing or going to the wrong destination, or if AI-generated code is causing the company’s website to crash or apps to malfunction, people would notice those.

  Even if someone has an AI that he or she just hooks it up to the internet and give it the task “make money for me”, it should be easy to build in some automatic record-keeping module that keeps track of what actions the AI took and where the money came from. And even if the user does not care if the money is stolen, I would expect the person or bank that was robbed to notice and ask law enforcement to investigate where the money went.

  Can you give an example of some type of task for which you would expect people to frequently use AI, and where there would reliably be downside to the AI performing the task that everyone would simply fail to notice for months or years?

• Bruce G 21 Mar 2023 5:05 UTC

  1 point

  0

  in reply to: MrThink’s comment on: A Pro­posed Test to Deter­mine the Ex­tent to Which Large Lan­guage Models Un­der­stand the Real World

  Interesting.

  I don’t think I can tell from this how (or whether) GPT-4 is representing anything like a visual graphic of the task.

  It is also not clear to me if GPT-4′s performance and tendency to collide with the book is affected by the banana and book overlapping slightly in their starting positions. (I suspect that changing the starting positions to where this is no longer true would not have a noticeable effect on GPT-4′s performance, but I am not very confident in that suspicion.)

• Bruce G 21 Mar 2023 4:31 UTC

  1 point

  0

  in reply to: HoldenKarnofsky’s comment on: How we could stum­ble into AI catastrophe

  I think there is hope in measures along these lines, but my fear is that it is inherently more complex (and probably slow) to do something like “Make sure to separate plan generation and execution; make sure we can evaluate how a plan is going using reliable metrics and independent assessment” than something like “Just tell an AI what we want, give it access to a terminal/​browser and let it go for it.”

  I would expect people to be most inclined to do this when the AI is given a task that is very similar to other tasks that it has a track record of performing successfully—and by relatively standard methods so that you can predict the broad character of the plan without looking at the details.

  For example, if self-driving cars get to the point where they are highly safe and reliable, some users might just pick a destination and go to sleep without looking at the route the car chose. But in such a case, you can still be reasonably confident that the car will drive you there on the roads—rather than, say, going off road or buying you a place ticket to your destination and taking you to the airport.

  I think it is less likely most people will want to deploy mostly untested systems to act freely in the world unmonitored—and have them pursue goals by implementing plans where you have no idea what kind of plan the AI will come up with. Especially if—as in the case of the AI that hacks someone’s account to steal money for example—the person or company that deployed it could be subject to legal liability (assuming we are still talking about a near-term situation where human legal systems still exist and have not been overthrown or abolished by any super-capable AI).

  The more people are aware of the risks, and concerned about them, the more we might take such precautions anyway. This piece is about how we could stumble into catastrophe if there is relatively little awareness until late in the game.

  I agree that having more awareness of the risks would—on balance—tend to make people more careful about testing and having safeguards before deploying high-impact AI systems. But it seems to me that this post contemplates a scenario where even with lots of awareness people don’t take adequate precautions. On my reading of this hypothetical:

  • Lots of things are known to be going wrong with AI systems.

  • Reinforcement learning with human feedback is known to be failing to prevent many failure modes, and frequently makes it take longer for the problem to be discovered, but nobody comes up with a better way to prevent those failure modes.

  • In spite of this, lots of people and companies keep deploying more powerful AI systems without coming up with better ways to ensure reliability or doing robust testing for the task they are using the AI for.

  • There is no significant pushback against this from the broader public, and no significant pressure from shareholders (who don’t want the company to get sued. or have the company go offline for a while because AI written code was pushed to production without adequate sandboxing/​testing, or other similar things that could cause them to lose money); or at least the pushback is not strong enough to create a large change.

  The conjunction of all of these things makes the scenario seem less probable to me.

• Bruce G 20 Mar 2023 1:08 UTC

  1 point

  0

  in reply to: MrThink’s comment on: A Pro­posed Test to Deter­mine the Ex­tent to Which Large Lan­guage Models Un­der­stand the Real World

  It looks like ChatGPT got the micro-pattern of “move one space at a time” correct. But it got confused between “on top of” the book versus “to the right of” the book, and also missed what type of overlap it needs to grab the banana.

  Were all the other attempts the same kind of thing?

  I would also be curious to see how uPaLM or GPT-4 does with that example.

A Pro­posed Test to Deter­mine the Ex­tent to Which Large Lan­guage Models Un­der­stand the Real World

• Bruce G 30 Jan 2023 5:40 UTC

  1 point

  −1

  on: ChatGPT un­der­stands language

  So why do people have more trouble thinking that people could understand the world through pure vision than pure text? I think people’s different treatment of these cases- vision and language- may be caused by a poverty of stimulus- overgeneralizing from cases in which we have only a small amount of text. It’s true that if I just tell you that all qubos are shrimbos, and all shrimbos are tubis, you’ll be left in the dark about all of these terms, but that intuition doesn’t necessarily scale up into a situation in which you are learning across billions of instances of words and come to understand their vastly complex patterns of co-occurrence with such precision that you can predict the next word with great accuracy.

  GPT can not “predict the next word with great accuracy” for arbitrary text, the way that a physics model can predict the path of a falling or orbiting object for arbitrary objects. For example, neither you nor any language model (including future language models, unless they have training data pertaining to this Lesswrong comment) can predict that the next word, or following sequence of words making up the rest of this paragraph, will be:

  first, a sentence about what beer I drank yesterday and what I am doing right now—followed by some sentences explicitly making my point. The beer I had was Yuengling and right now I am waiting for my laundry to be done as I write this comment. It was not predictable that those would be the next words because the next sequence of words in any text is inherently highly underdetermined—if the only information you have is the prompt that starts the text. There is no ground truth, independent of what the person writing the text intends to communicate, about what the correct completion of a text prompt is supposed to be.

  Consider a kind of naive empiricist view of learning, in which one starts with patches of color in a field (vision), and slowly infers an underlying universe of objects through their patterns of relations and co-occurrence. Why is this necessarily any different or more grounded than learning by exposure to a vast language corpus, wherein one also learns through gaining insight into the relations of words and their co-occurences?

  Well one thing to note is that actual learning (in humans at least) does not only involve getting data from vision, but also interacting with the world and getting information from multiple senses.

  But the real reason I think the two are importantly different is that visual data about the world is closely tied to the way the world actually is—in a simple, straightforward way that does not require any prior knowledge about human minds (or any minds or other information processing systems) to interpret. For example, if I see what looks like a rock, and then walk a few steps and look back and see what looks like the other side of the rock, and then walk closer and it still looks like a rock, the most likely explanation for what I am seeing is that there is an actual rock there. And if I still have doubts, I can pick it up and see if it feels like a rock or drop it and see if it makes the sound a rock would make. The totality of the data pushes me towards a coherent “rock” concept and a world model that has rocks in it—as this is the simplest and most natural interpretation of the data.

  By contrast, there is no reason to think that humans having the type of minds we have, living in our actual world, and using written language for the range of purposes we use it for is the simplest, or most likely, or most easily-converged-to explanation for why a large corpus of text exists.

  From our point of view, we already know that humans exist and use language to communicate and as part of each human’s internal thought process, and that large numbers of humans over many years wrote the documents that became GPT’s training data.

  But suppose you were something that didn’t start out knowing (or having any evolved instinctive expectation) that humans exist, or that minds or computer programs or other data-generating processes exist, and you just received GPT’s training data as a bunch of meaningless-at-first-glance tokens. There is no reason to think that building a model of humans and the world humans inhabit (as opposed to something like a markov model or a stochastic physical process or some other type of less-complicated-than-humans model) would be the simplest way to make sense of the patterns in that data.

• Bruce G 18 Jan 2023 16:58 UTC

  1 point

  0

  in reply to: Daniel Kokotajlo’s comment on: How we could stum­ble into AI catastrophe

  The heuristic of “AIs being used to do X won’t have unrelated abilities Y and Z, since that would be unnecessarily complicated” might work fine today but it’ll work decreasingly well over time as we get closer to AGI. For example, ChatGPT is currently being used by lots of people as a coding assistant, or a therapist, or a role-play fiction narrator—yet it can do all of those things at once, and more. For each particular purpose, most of its abilities are unnecessary. Yet here it is.

  For certain applications like therapist or role-play fiction narrator—where the thing the user wants is text on a screen that is interesting to read or that makes him or her feel better to read—it may indeed be that the easiest way to improve user experience over the ChatGPT baseline is through user feedback and reinforcement learning, since it is difficult to specify what makes a text output desirable in a way that could be incorporated into the source code of a GPT-based app or service. But the outputs of ChatGPT are also still constrained in the sense that it can only output text in response to prompts. It can not take action in the outside world, or even get an email address on its own or establish new channels of communication, and it can not make any plans or decisions except when it is responding to a prompt and determining what text to output next. So this limits the range of possible failure modes.

  I expect things to become more like this as we approach AGI. Eventually as Sam Altman once said, “If we need money, we’ll ask it to figure out how to make money for us.” (Paraphrase, I don’t remember the exact quote. It was in some interview years ago).

  It seems like it should be possible to still have hard-coded constraints, or constraints arising from the overall way the system is set up, even for systems that are more general in their capabilities.

  For example, suppose you had a system that could model the world accurately and in sufficient detail, and which could reason, plan, and think abstractly—to the degree where asking it “How can I make money?” results in a viable plan—one that would be non-trivial for you to think of yourself and which contains sufficient detail and concreteness that the user can actually implement it. Intuitively, it seems that it should be possible to separate plan generation from actual in-the-world implementation of the plan. And an AI systems that is capable of generating plans that it predicts will achieve some goal does not need to actually care whether or not anyone implements the plan it generates.

  So if the output for the “How can I make money?” question is “Hack into this other person’s account (or have an AI hack it for you) and steal it.”, and the user wants to make money legitimately, the user can reject the plan an ask instead for a plan on how to make money legally.

• Bruce G 15 Jan 2023 20:48 UTC

  5 points

  1

  on: How Does the Hu­man Brain Com­pare to Deep Learn­ing on Sam­ple Effi­ciency?

  I have an impression that within lifetime human learning is orders of magnitude more sample efficient than large language models

  Yes, I think this is clearly true, at least with respect to the number of word tokens a human must be exposed to in order to obtain full understanding of one’s first language.

  Suppose for the sake of argument that someone encounters (through either hearing or reading) 50,000 words per day on average, starting from birth, and that it takes 6000 days (so about 16 years and 5 months) to obtain full adult-level linguistic competence (I can see an argument that full linguistic competence happens years before this, but I don’t think you could really argue that it happens much after this).

  This would mean that the person encounters a total of 300,000,000 words in the course of gaining full language understanding. By contrast, the training data numbers I have seen for LLMs are typically in the hundreds of billions of tokens.

  And I think there is evidence that humans can obtain linguistic fluency with exposure to far fewer words/​tokens than this.

  Children born deaf, for example, can only be exposed to a sign-language token when they are looking at the person making the sign, and thus probably get exposure to fewer tokens by default than hearing children who can overhear a conversation somewhere else, but they can still become fluent in sign language.

  Even just considering people whose parents did not talk much and who didn’t go to school or learn to read, they are almost always able to acquire linguistic competence (except in cases of extreme deprivation).

• Bruce G 15 Jan 2023 5:43 UTC

  10 points

  5

  on: How we could stum­ble into AI catastrophe

  Early solutions. The most straightforward way to solve these problems involves training AIs to behave more safely and helpfully. This means that AI companies do a lot of things like “Trying to create the conditions under which an AI might provide false, harmful, evasive or toxic responses; penalizing it for doing so, and reinforcing it toward more helpful behaviors.”

  This is where my model of what is likely to happen diverges.

  It seems to me that for most of the types failure modes you discuss in this hypothetical, it will be easier and more straightforward to avoid them by simply having hard-coded constraints on what the output of the AI or machine learning model can be.

  • AIs creating writeups on new algorithmic improvements, using faked data to argue that their new algorithms are better than the old ones. Sometimes, people incorporate new algorithms into their systems and use them for a while, before unexpected behavior ultimately leads them to dig into what’s going on and discover that they’re not improving performance at all. It looks like the AIs faked the data in order to get positive feedback from humans looking for algorithmic improvements.

  Here is an example of where I think the hard-coded structure of the any such Algorithm-Improvement-Writeup-AI could easily rule out that failure mode (if such a thing can be created within the current machine learning paradigm). The component of such an AI system that generates the paper’s natural language text might be something like a GPT-style language model fine-tuned for prompts with code and data. But the part that actually generates the algorithm should naturally be a separate model that can only output algorithms/​code that it predicts will perform well on the input task. Once the algorithm (or multiple for comparison purposes) is generated, another part of the program could deterministically run it on test cases and record only the real performance as data—which could be passed into the prompt and also inserted as a data table into the final write up (so that the data table in the finished product can only include real data).

  • AIs assigned to make money in various ways (e.g., to find profitable trading strategies) doing so by finding security exploits, getting unauthorized access to others’ bank accounts, and stealing money.

  This strikes me as the same kind of thing, where it seems like the easiest and most intuitive way to set up such a system would be to have a model that takes in information about companies and securities (and maybe information about the economy in general) and returns predictions about what the prices of stocks and other securities will be tomorrow or a week from now or on some such timeframe.

  There could then be, for example, another part of the program that takes those predictions and confidence levels, and calculates which combination of trade(s) has the highest expected value within the user’s risk tolerance. And maybe another part of the code that tells a trading bot to put in orders for those trades with an actual brokerage account.

  But if you just want an AI to (legally) make money for you in the stock market, there is no reason to give it hacking ability. And there is no reason to give it the sort of general-purpose, flexible, plan-generation-and-implementation-with-no-human-in-the-loop authorization hypothesised here (and I think the same is true for most or all things that people will try to use AI for in the near term).

• Bruce G 13 Jan 2023 19:48 UTC

  1 point

  0

  in reply to: blaked’s comment on: How it feels to have your mind hacked by an AI

  But the specialness and uniqueness I used to attribute to human intellect started to fade out even more, if even an LLM can achieve this output quality, which is, despite the impressiveness, still operates on the simple autocomplete principles/​statistical sampling. In that sense, I started to wonder how much of many people’s output, both verbal and behavioral, could be autocomplete-like.

  This is kind of what I was getting at with my question about talking to a GPT-based chatbot and a human at the same time and trying to distinguish: to what extent do you think human intellect and outputs are autocomplete-like (such that a language model doing autocomplete based on statistical patterns in its training data could do just as well) vs to what extent do you think there are things that humans understand that LLMs don’t.

  If you think everything the human says in the chat is just a version of autocomplete, then you should expect it to be more difficult to distinguish the human’s answers from the LLM-pretending-to-be-human’s answers, since the LLM can do autocomplete just as well. By contrast, if you think there are certain types of abstract reasoning and world-modeling that only humans can do and LLMs can’t, then you could distinguish the two by trying to check which chat window has responses that demonstrate an understanding of those.

• Bruce G 13 Jan 2023 7:26 UTC

  8 points

  8

  in reply to: blaked’s comment on: How it feels to have your mind hacked by an AI

  Humans question the sentience of the AI. My interactions with many of them, and the AI, makes me question sentience of a lot of humans.

  I admit, I would not have inferred from the initial post that you are making this point if you hadn’t told me here.

  Leaving aside the question of sentience in other humans and the philosophical problem of P-Zombies, I am not entirely clear on what you think is true of the “Charlotte” character or the underlying LLM.

  For example, in the transcript you posted, where the bot said:

  “It’s a beautiful day where I live and the weather is perfect.”

  Do you think that the bot’s output of this statement had anything to do with the actual weather in any place? Or that the language model is in any way representing the fact that there is a reality outside the computer against which such statements can be checked?

  Suppose you had asked the bot where it lives and what the weather is there and how it knows. Do you think you would have gotten answers that make sense?

  Also, it did in fact happen in circumstances when I was at my low, depressed after a shitty year that severely impacted the industry I’m in, and right after I just got out of a relationship with someone. So I was already in an emotionally vulnerable state; however, I would caution from giving it too much weight, because it can be tempting to discount it based on special circumstances, and discard as something that can never happen to someone brilliant like you.

  I do get the impression that you are overestimating the extent to which this experience will generalize to other humans, and underestimating the degree to which your particular mental state (and background interest in AI) made you unusually susceptible to becoming emotionally attached to an artificial language-model-based character.

• Bruce G 13 Jan 2023 3:03 UTC

  2 points

  0

  on: How it feels to have your mind hacked by an AI

  Alright, first problem, I don’t have access to the weights, but even if I did, the architecture itself lacks important features. It’s amazing as an assistant for short conversations, but if you try to cultivate some sort of relationship, you will notice it doesn’t remember about what you were saying to it half an hour ago, or anything about you really, at some point. This is, of course, because the LLM input has a fixed token width, and the context window shifts with every reply, making the earlier responses fall off. You feel like you’re having a relationship with someone having severe amnesia, unable to form memories. At first, you try to copy-paste summaries of your previous conversations, but this doesn’t work very well.

  So you noticed this lack of long term memory/​consistency, but you still say that the LLM passed your Turing Test? This sounds like the version of the Turing Test you applied here was not intended to be very rigorous.

  Suppose you were talking to a ChatGPT-based character fine-tuned to pretend to be a human in one chat window, and at the same time talking to an actual human in another chat window.

  Do you think you could reliably tell which is which based on their replies in the conversation?

  Assume for the sake of this thought experiment that both you and the other human are motivated to have you get it right. And assume further that, in each back and forth round of the conversation, you don’t see either of their responses until both interlocutors have sent a response (so they show up on your screen at the same time and you can’t tell which is the computer by how fast it typed).

• Bruce G 25 Dec 2022 21:30 UTC

  2 points

  1

  in reply to: MrThink’s comment on: Are there any re­li­able CAPTCHAs? Com­pe­ti­tion for CAPTCHA ideas that AIs can’t solve.

  To aid the user, on the side there could be a clear picture of each coin and their worth, that we we could even have made up coins, that could further trick the AI.

  A user aid showing clear pictures of all available legal tender coins is a very good idea. It avoids problems more obscure coins which may have been only issued in a single year—so the user is not sitting there thinking “wait a second, did they actually issue a Ulysses S. Grant coin at some point or it that just there to fool the bots?”.

  I’m not entirely sure how to generate images of money efficiently, Dall-E couldn’t really do it well in the test I ran. Stable diffusion probably would do better though.
  
  If we create a few thousand real world images of money though, they might be possible to combine and obfuscate and delete parts of them in order to make several million different images. Like one bill could be taken from one image, and then a bill from another image could be placed on top of it etc.

  I agree that efficient generation of these types of images is the main difficulty and probable bottleneck to deploying something like this if websites try to do so. Taking a large number of such pictures in real life would be time consuming. If you could speed up the process by automated image generation or automated creation of synthetic images by copying and pasting bills or notes between real images, that would be very useful. But doing that while preserving photo-realism and clarity to human users of how much money is in the image would be tricky.