This comment reflects those of me and not my employer (Cohere).
We are currently massively growing our safety team on both engineering and product sides and one of our major bottlenecks is the above technical talent. We are currently heavily focused on making our models in production as safe as possible during training and during production. One of the biggest projects to this extent is the safety harness project which should have more information coming out soon. https://docs.cohere.ai/safety-harness/. We are heavily focused on worse-case scenario’s especially as anyone can use our models relatively quickly. Here are 2 of the papers the safety team has worked on in the past. We have much more in the timeline.
Can someone briefly describe what empirical AI safety work Cohere is doing? I hadn’t heard of them until this post.
This comment reflects those of me and not my employer (Cohere).
We are currently massively growing our safety team on both engineering and product sides and one of our major bottlenecks is the above technical talent. We are currently heavily focused on making our models in production as safe as possible during training and during production. One of the biggest projects to this extent is the safety harness project which should have more information coming out soon. https://docs.cohere.ai/safety-harness/. We are heavily focused on worse-case scenario’s especially as anyone can use our models relatively quickly. Here are 2 of the papers the safety team has worked on in the past. We have much more in the timeline.
Mitigating harm in language models with conditional-likelihood filtration
No News is Good News: A Critique of the One Billion Word Benchmark
I am also interested in this.