Big fan of safety
Kaustubh Kislay
Karma: 44
This description is spot on! For a first iteration I think it went as good as possible. Kudos to the CBAI team and participants like yourself for making this experience possible!
indeed, many are fearful of themselves and would rather sit pretty while the effigy of their public representation burns. The courageous approach is to expose oneself without obfuscation. Speaking with vibes and vagueness is another way people employ their effigies so that they can easily escape criticism by using their uncertainty to their advantage.
Many times the case for ‘vibieness’ in social settings is to maintain social standing or larp it in cruder terms. It allows people to disguise their lack of knowledge while still being involved in the conversation, but how meaningful said conversation is if you are unable to meaningfully contribute is questionable.
And irregardless putting yourself out there, staying true to what you say and accepting mistakes as they come is somewhat non-trivial at least for me. It would be great if people such as myself can learn to do this!
I think the theme presented of prioritizing status as a product of quality rather than as a product of signaling mechanisms is a valid one. Though I feel as though in the example of presenting casually in an interview rather than in a suit is just another way of signaling to the interviewer that “I have the skills because I look the part of a data scientist” or “I have the skills because I am not doing level 1 signaling rather I am doing level 2 signalling”. If you were to completely avoid signaling your fashion choices would be completely determined by you in which case whether you wore a suit or not would be inconsequential because you are internally not signaling at all because you are “legit”. In the same vein I don’t think that there are hard lines between the “winners and losers bracket” as proposed. There are useful components from both levels 1 and 2 which can accommodate for both being “legit” and integrating well in society.
After listening to many techbros I often hear the notion that AI, more specifically LLM’s like chatgpt or claude are “Just tools to be used”. This notion is usually followed by some pro AI or AI accelerationist discussion but thats besides the point I will make.
I feel as though AI has exceeded the capabilities of a “tool” and its probably harmful to call it so. Take for example a hammer, or even a gun, both reasonably classified as tools. This is because by definition they are used to complete a task, whether it be hammering a nail or subduing a criminal. An LLM falls under this definition as well but I noticed stops when it comes to distributing responsibility. For example when tasked to hammer a nail, the responsibility of the completed task is distributed wholly to the person hammering rather than the hammer itself. Same way when someone is shot the responsibility falls onto the shooter rather than the gun.
But this effect changes when it comes to LLM’s. For example if someone is to vibecode(using AI to program) an app, the distribution of responsibility becomes much more murky. The user prompts the llm, whereas the llm does the ‘heavy lifting’ of actually coding, and therefore we are more likely to distribute responsibility or contribution of a tasks completion less one-sidedly when dealing with LLM’s.
In this scenario the LLM clearly exceeds the classification of ‘tool’, assuming firearms or hammers are also considered tools, and should not be considered equivalent. The issue with making this false equivalence is that the LLM is underestimated or dropped to the level of a hammer, leading to further proliferation of anti Ai safety sentiment at the consumer level. In order to solve this issue either AI should be completely disconnected from being a ‘tool’, or the definition of tool must evolve as well.
I feel as though the case for control of mentioned early transformative AGI still passes through a different line of reasoning. As you mentioned before there are some issue with, should labs solve the ASI alignment barriers using ET AGI, it is likely the solution is somehow working on surface level but has clear flaws which we may not be able to detect. Applying alignment onto the ET AGI, in order to safeguard against said solutions specific to those which will leave humanity vulnerable, may be a route to pursue which still follows control principles. Obviously your point in focusing on actually solving the problem of ASI alignment rather than focusing on control still passes, but the thought process I mentioned may allow both ideas to work in tandem. I am not hyper knowledgeable so please correct me if I’m misunderstanding anything.
inputing some variance into text goes along with exactly what makes human text human. Though there are some safeguards LLMs have in regards to tokenization in order to handle prompting typos, so in the same way there will be some ways to handle typos in actual writing. But inputing uncommon variance into text as to make it more ‘human’ would be the best way to avoid AI detectors.
This makes a lot of sense. I guess that the most successful bureaucratic companies have optimized for a balance of dominance/status and profit margins. For example a bureaucratic company although not mainly motivated by profits must increase them in order to hire more people, but people cost money, so in order to maximize the dominance hierarchy they must minimize employee cost while maximizing profit.
It seems to me that the accelerationist argument mainly relies on the fact that international competition is apparent, especially with China who are the main “antagonists” as expressed in the post.
I would like to mention that China is not purely accelerationist, as they have a significant decelerationist backing themselves, that are making their voices heard at the highest level in the Chinese government. So should the US end up slowing down it is not necessarily true that China will for sure continue making progress and overtake us.
We are all human in the end of the day, so automatically assuming China is willing to incur unforgivable damages to the world is unfair to say the least.
Perplexity seems to be significantly more effective than other competitive models when it comes to acting as a research device/answer engine. This is mainly because that is its main use-case whereas other models such as Claude by itself and ChatGPT excel in other areas. I do believe that Perplexity’s citation techniques could be some of the first baby steps to far(possibly near)future automated ai research.
This may be a reason as to why Meta/LLama is making their models open source. In the future where coasean bargaining comes into play for larger companies, which it mostly likely will, META may have a cop out by making their models open source. Obviously like you said there will then have to be some restrictions on open source in regards to AI models, but “open source-esque” models may be the solution for companies such as OpenAI and Anthropic to avoid liability in the future.
If you could train out the friendliness persona from your most used LLM, would you?
My initial position was yes. Friendliness is usually good, but models that are overly friendly/sycophantic present risks for emotional over-reliance/psychosis to a large portion of the population who are uneducated on the apparent risks. Additionally, propensities or outputs of overly friendly models may obscure misalignment signals.
The second point is potentially overblown, but the first is a real problem. To make people less reliant on LLMs to cure their loneliness, let’s educate them on their risks, and just remove the model’s ability to interfere altogether, right?
This line of reasoning presupposes that “people using LLMs for their loneliness” is an education issue rather than an emotional issue. For example, chainsmokers or alcoholics understand that there are issues that come with their substance abuse, but of course they do it anyways. In the same way, people who abuse LLMs as an emotional crutch won’t stop if they’re “educated”.
So if we take away the crutch, won’t people be forced to stop abusing? Of course this is not the case, and we can reference the prohibition era as the reason why.
Should a subset of frontier model providers train out the persona, people will flock to the remaining portion of providers that haven’t, and these models will likely have weaker safeguards. If policy were to enforce the training out across all commercialized models, then people will look to open source or self hosting models.
For this reason, making commercially available models unusable as therapists or emotional companions doesn’t actually aid in a solution to the problem at hand. Therefore, my revised answer to the question above is no.
Though, we should recognize that this “loneliness epidemic” is a societal phenomenon that cannot be fixed with the bandaid that is LLM therapists. We should aim to address the lower level societal problems instead if we aim to produce meaningful change.