I think it would be incredibly unbecoming of the AI risk guys to conclude that this isn’t worth their time
I don’t think that’s what I’m saying. At least, it’s not what I’m trying to say. Rather, I’m saying that any given individual has finite time and needs to prioritize, the number of people in the “AI risk guys” reference class is small relative to need, and promoting, broad, basic understanding to facilitate safety in the context of mundane utility is not necessarily the highest priority for many or most of them.
Mundane utility seems to me to be the kind of risk that actually can, mostly, be handled by people with pretty basic understanding of AI (the kind you can actually get by trying things out and talking to people like you who know more than average) just adapting existing best practices and product development/implementation paths.
If you’re managing a hospital or nuclear power plant or airplane manufacturing plant, you don’t roll out a new technology until some company that has a highly credible claim to expertise demonstrates to a sufficiently conservative standards body that it’s a good idea and won’t get you sued or prosecuted into oblivion, and won’t result in you murdering huge numbers of people (on net, or at all). Even then you do a pilot first unless you’re sure there’s dozens of other successful implementations out there you can copy. If that takes another five years or ten or more, so be it. That’s already normal in your industry.
If you’re running a more typical company managing non-lethal or mostly-non-lethal-except-for-rare-edge-cases risks, management gets to decide how much to wager on how new and untested a technology, and if they bet the company and lose, well, so be it, that’s also normal. And if you want to adopt AI in some way and still be more sure you’re implementing it well, then you hire a professional—either a company, or a team, or a consultant, or something—just like you would if you were planning any other strategic change to how you do business. Someone in your position would not be that person, but I’d be confident you are able to spot when the situation calls for that kind of diligence, and could pretty reasonably evaluate which self-proclaimed professionals seem to have enough competence to give good advice.
In my personal example: I’m helping my employer decide what AI tools to use internally and how to roll them out and manage their use, but we are hiring an outside company to figure out how to incorporate AI into our actual product offering to clients. The latter is a higher business risk situation, and calls for deeper technical expertise. It does not call for an AI safety expert. In fact, I’d be disappointed if an AI safety expert were focused on something like this instead of focusing on either technical AI safety to try to ensure humanity’s survival, or policy advocacy to get people with real power to do useful things to promote the same.
Edit to add: I do also think you can get good mileage out of just having basic conversations with people you know in different positions, in the types of language they understand and care about. If you’re talking about AI to HR, then there’s headcount efficiency for some tasks more than others, the potential over the next couple of years for agents to automate some parts of some roles but less for others, and the need to start thinking about how you might need to reorganize roles iteratively over time. If you’re talking to an R&D director, then there’s all the different ways to use AI to act as a thought partner, improve understanding of the landscape or what’s known and available, or refine experimental design and improve data analysis. If you’re talking to sales, there’s ways to gain insight about what clients/customers want or respond to based on trends in a larger data set that a human would have a hard time finding. Sparking thought and curiosity while also highlighting general categories and intensities of risk is often enough to get the needed conversations going.
If you’re managing a hospital or nuclear power plant or airplane manufacturing plant, you don’t roll out a new technology until some company that has a highly credible claim to expertise demonstrates to a sufficiently conservative standards body that it’s a good idea and won’t get you sued or prosecuted into oblivion, and won’t result in you murdering huge numbers of people (on net, or at all). Even then you do a pilot first unless you’re sure there’s dozens of other successful implementations out there you can copy. If that takes another five years or ten or more, so be it. That’s already normal in your industry.
It would be really great if people were behaving about this in the way that is normal in my industry.
Unfortunately, they’re not. The hype has everyone bad, and in combination with the enormous pressure to cut costs in the NHS , the pressure on everyone’s time...
This isn’t like a new medical imaging device. A doctor can have it on their phone and use it without checking with anyone. Our ability to promote the stuff they’re supposed to be using for looking things up at point-of-care has been decimated by staff cuts, to add to the problem.
I am being put forward to produce training on what people should and shouldn’t be doing. If someone more qualified was coming along, that wouldn’t be happening. If everyone was just going to hang tight until the technology had matured and they got the official word to go ahead from someone who knew what they were about, I wouldn’t be worried.
Every country is different, but in the US, the natural course in those situations is that if it works, it gets normalized and brought into official channels, and if it doesn’t, a lot of people are going to get sued for malpractice. If you’re being put in that position, then you should insist on making sure the lawyers and compliance officers are consulted to refine whatever language the trainings use, or else refuse to put your name on any of it.
Some of this is pretty straightforward, if not simple to execute. For example, if you’re talking about LLMs, and you’re following the news, then you know that the US courts have ruled that the NY Times has the right to read every chat anyone, anywhere has with ChatGPT, no matter what OpenAI’s terms of service previously said, in order to look for evidence of copyright infringement. There seem (I think) to be exceptions for enterprise customers. But, this should be sufficient to credibly say, “If you’re using this through a personal account on an insecure device, you’re breaking the laws on patient confidentiality, and can be sanctioned for it in whatever ways just like you would if you got caught talking about a patient’s condition and identity in public.”
Good trainings build on what the audience already knows. Maybe start with something like, “Used well, you can consult LLMs with maybe about as much trust as you would a first-day new resident.” You need to be very careful how you frame your questions—good prompting strategy, good system prompts, consistent anonymization practices, asking for and confirming references to catch hallucinations or other errors. Those are all things you can look up in many places, here and elsewhere, but you’ll have to pick through a lot of inapplicable and poor advice because nothing is rock-solid or totally stable right now. Frame it as a gradual rollout requiring iteration on all of those, plan for updated trainings every quarter or two, provide a mechanism for giving feedback on how it’s going. You can use feedback to develop a system prompt people can use with their employer-provided accounts.
This may be harder, but you also need to make sure you’re able to collect accurate information on what people in various roles are actually, currently doing, and they need to be able to tell you that without fear of retroactive reprisal or social sanction, or you’re likely to drown in deception about the situation on the ground, and won’t know what goals or use cases you need to be thinking about.
One thing to add: if you find that the whole system is demanding you, an amateur, provide training while refusing to do the bare minimum necessary to even follow the law or maintain the standard of care, then think very carefully about what you want to do in response. Try to muddle through and do the best you can without incurring personal liability? Become a whistleblower? Find a new job? At that point you might well want to consult a lawyer yourself to help navigate the situation. And make sure you document, in writing, any requests made of you that you are uncomfortable with, and your objections to them.
I don’t think that’s what I’m saying. At least, it’s not what I’m trying to say. Rather, I’m saying that any given individual has finite time and needs to prioritize, the number of people in the “AI risk guys” reference class is small relative to need, and promoting, broad, basic understanding to facilitate safety in the context of mundane utility is not necessarily the highest priority for many or most of them.
Mundane utility seems to me to be the kind of risk that actually can, mostly, be handled by people with pretty basic understanding of AI (the kind you can actually get by trying things out and talking to people like you who know more than average) just adapting existing best practices and product development/implementation paths.
If you’re managing a hospital or nuclear power plant or airplane manufacturing plant, you don’t roll out a new technology until some company that has a highly credible claim to expertise demonstrates to a sufficiently conservative standards body that it’s a good idea and won’t get you sued or prosecuted into oblivion, and won’t result in you murdering huge numbers of people (on net, or at all). Even then you do a pilot first unless you’re sure there’s dozens of other successful implementations out there you can copy. If that takes another five years or ten or more, so be it. That’s already normal in your industry.
If you’re running a more typical company managing non-lethal or mostly-non-lethal-except-for-rare-edge-cases risks, management gets to decide how much to wager on how new and untested a technology, and if they bet the company and lose, well, so be it, that’s also normal. And if you want to adopt AI in some way and still be more sure you’re implementing it well, then you hire a professional—either a company, or a team, or a consultant, or something—just like you would if you were planning any other strategic change to how you do business. Someone in your position would not be that person, but I’d be confident you are able to spot when the situation calls for that kind of diligence, and could pretty reasonably evaluate which self-proclaimed professionals seem to have enough competence to give good advice.
In my personal example: I’m helping my employer decide what AI tools to use internally and how to roll them out and manage their use, but we are hiring an outside company to figure out how to incorporate AI into our actual product offering to clients. The latter is a higher business risk situation, and calls for deeper technical expertise. It does not call for an AI safety expert. In fact, I’d be disappointed if an AI safety expert were focused on something like this instead of focusing on either technical AI safety to try to ensure humanity’s survival, or policy advocacy to get people with real power to do useful things to promote the same.
Edit to add: I do also think you can get good mileage out of just having basic conversations with people you know in different positions, in the types of language they understand and care about. If you’re talking about AI to HR, then there’s headcount efficiency for some tasks more than others, the potential over the next couple of years for agents to automate some parts of some roles but less for others, and the need to start thinking about how you might need to reorganize roles iteratively over time. If you’re talking to an R&D director, then there’s all the different ways to use AI to act as a thought partner, improve understanding of the landscape or what’s known and available, or refine experimental design and improve data analysis. If you’re talking to sales, there’s ways to gain insight about what clients/customers want or respond to based on trends in a larger data set that a human would have a hard time finding. Sparking thought and curiosity while also highlighting general categories and intensities of risk is often enough to get the needed conversations going.
It would be really great if people were behaving about this in the way that is normal in my industry.
Unfortunately, they’re not. The hype has everyone bad, and in combination with the enormous pressure to cut costs in the NHS , the pressure on everyone’s time...
This isn’t like a new medical imaging device. A doctor can have it on their phone and use it without checking with anyone. Our ability to promote the stuff they’re supposed to be using for looking things up at point-of-care has been decimated by staff cuts, to add to the problem.
I am being put forward to produce training on what people should and shouldn’t be doing. If someone more qualified was coming along, that wouldn’t be happening. If everyone was just going to hang tight until the technology had matured and they got the official word to go ahead from someone who knew what they were about, I wouldn’t be worried.
Every country is different, but in the US, the natural course in those situations is that if it works, it gets normalized and brought into official channels, and if it doesn’t, a lot of people are going to get sued for malpractice. If you’re being put in that position, then you should insist on making sure the lawyers and compliance officers are consulted to refine whatever language the trainings use, or else refuse to put your name on any of it.
Some of this is pretty straightforward, if not simple to execute. For example, if you’re talking about LLMs, and you’re following the news, then you know that the US courts have ruled that the NY Times has the right to read every chat anyone, anywhere has with ChatGPT, no matter what OpenAI’s terms of service previously said, in order to look for evidence of copyright infringement. There seem (I think) to be exceptions for enterprise customers. But, this should be sufficient to credibly say, “If you’re using this through a personal account on an insecure device, you’re breaking the laws on patient confidentiality, and can be sanctioned for it in whatever ways just like you would if you got caught talking about a patient’s condition and identity in public.”
Good trainings build on what the audience already knows. Maybe start with something like, “Used well, you can consult LLMs with maybe about as much trust as you would a first-day new resident.” You need to be very careful how you frame your questions—good prompting strategy, good system prompts, consistent anonymization practices, asking for and confirming references to catch hallucinations or other errors. Those are all things you can look up in many places, here and elsewhere, but you’ll have to pick through a lot of inapplicable and poor advice because nothing is rock-solid or totally stable right now. Frame it as a gradual rollout requiring iteration on all of those, plan for updated trainings every quarter or two, provide a mechanism for giving feedback on how it’s going. You can use feedback to develop a system prompt people can use with their employer-provided accounts.
This may be harder, but you also need to make sure you’re able to collect accurate information on what people in various roles are actually, currently doing, and they need to be able to tell you that without fear of retroactive reprisal or social sanction, or you’re likely to drown in deception about the situation on the ground, and won’t know what goals or use cases you need to be thinking about.
One thing to add: if you find that the whole system is demanding you, an amateur, provide training while refusing to do the bare minimum necessary to even follow the law or maintain the standard of care, then think very carefully about what you want to do in response. Try to muddle through and do the best you can without incurring personal liability? Become a whistleblower? Find a new job? At that point you might well want to consult a lawyer yourself to help navigate the situation. And make sure you document, in writing, any requests made of you that you are uncomfortable with, and your objections to them.