Hello there! I’m Miguel, the Lead AI Safety Researcher at CirroLytix, an AI and social impact technology company based in the Philippines. We’re focused on alignment research and raising awareness about AI safety.
The opinions and projects I share here are my own.
This is a thought that frequently crosses my mind too, especially in the field of AI safety. Since no one really has definitive answers yet, my mantra is: keep asking the right questions, continue modeling those questions, and persist in researching.
I’ve had a similar experience with LTFF; I waited a long time without receiving any feedback. When I inquired, they told me they were overwhelmed and couldn’t provide specific feedback. I find the field to be highly competitive. Additionally, there’s a concerning trend I’ve noticed, where there seems to be growing pessimism about newcomers introducing untested theories. My perspective on this is that now is not the time to be passive about exploration. This is a crucial moment in history when we should remain open-minded to any approach that might work, even if the chances are slim.
This captures my perspective well: not everyone is suited to run organizations. I believe that AI safety organizations would benefit from the integration of established “AI safety standards,” similar to existing Engineering or Financial Reporting Standards. This would make maintenance easier. However, for the time being, the focus should be on independent researchers pursuing diverse projects to first identify those standards.
I think the Audio could be improved next time.
Hello there! What I meant as components in my comment are like the attention mechanism itself. For reference, here are the mean weights of two models I’m studying.
But empirically, the world doesn’t pursue every technology—it barely pursues any technologies.
While it’s true that not all technologies are pursued, history shows that humans have actively sought out transformative technologies, such as fire, swords, longbows, and nuclear energy. The driving factors have often been novelty and utility. I expect the same will happen with AI.
Hello. I noticed that your proposal for achieving truth in LLMs involves using debate as a method. My concern with this approach is that an AI consists of many small components that aggregate to produce text or outputs. These components simply operate based on what they’ve learned. Therefore, the idea of clarifying or “deconfusing” these components in the service of truth through debate seems not possible to me. But if have misunderstood the concept, let me know too thanks!
Another relevant material by Rob Miles here on why the shutdown problem is very difficult to solve.
We don’t know how to formally define corrigibility and this is part of the reason why we haven’t solved it so far. Corrigibility is defined in Arbital as:
[the agent] doesn’t interfere with what we would intuitively see as attempts to ‘correct’ the agent, or ‘correct’ our mistakes in building it
I think this is not the best definition of corrigibility, As defined in MIRI’s paper section 1.1:
We say that an agent is “corrigible” if it tolerates or assists many forms of outside correction, including at least the following: (1) A corrigible reasoner must at least tolerate and preferably assist the programmers in their attempts to alter or turn off the system. (2) It must not attempt to manipulate or deceive its programmers, despite the fact that most possible choices of utility functions would give it incentives to do so. (3) It should have a tendency to repair safety measures (such as shutdown buttons) if they break, or at least to notify programmers that this breakage has occurred. (4) It must preserve the programmers’ ability to correct or shut down the system (even as the system creates new subsystems or self-modifies). That is, corrigible reasoning should only allow an agent to create new agents if these new agents are also corrigible.
In theory, a corrigible agent should not be misaligned if we successfully integrate these four tolerance behaviors.
I think a better question is whether we have the environment that can cultivate the necessary behavior leading us to solve the most challenging aspects of AI alignment.
It does leave one question — how do we make a list of possible actions in the first place?
Suskever has an interesting theory on this that you might want to watch. he calls it the x and y data compression theory.
My current builds focuses on proving natural abstractions exists—but your idea is of course viable via distribution matching.
Yup, as long as there are similar patterns existing in both datasets (distribution matching) it can work—that is why my method works.
Most neurons dedicated to babies are focused on the mouth, I believe, up until around 7 months. Neuronal pathways evolve over time, and it seems that a developmental approach to AI presents its own set of challenges. When growing an AI that already possesses mature knowledge and has immediate access to it, traditional developmental caring mechanisms may not fully address the enormity of the control problem. However, if the AI gains knowledge through a gradual, developmental process, this approach could be effective in principle.
I’ve observed instances where the Pareto principle appears to apply, particularly in learning rates during unsupervised learning and in x and y dataset compression via distribution matching. For example, a small dataset that contains a story repeated 472 times (1MB) can significantly impact a model as large as 1.5 billion parameters (GPT2-xl, 6.3GB), enabling it to execute complex instructions like initiating a shutdown mechanism during an event that threatens intelligence safety. While I can’t disclose the specific methods (due to dual use nature), I’ve also managed to extract a natural abstraction. This suggests that a file with a sufficiently robust pattern can serve as a compass for a larger file (NN) following a compilation process.
“Democratic Fine-tuning” is an example of such a process. Our suggested implementation avoids ideological battles by eliciting the values underpinning our preferences, scales to millions of users, could be run continuously as new cases are encountered, and introduces a new aggregation mechanism (a moral graph) that can build trust, and inspire wiser values in participants.
This approach demands a well-informed population capable of setting aside their biases. People must be able to think both individually and collectively when deciding which values to prioritize. Overcoming this challenge will likely be a significant hurdle for this method.
What kind of training data would increase positive outcomes for superhuman AIs interacting with each other?
The training data should be systematically distributed, likely governed by the Pareto principle. This means it should encompass both positive and negative outcomes. If the goal is to instill moral decision-making, the dataset needs to cover a range of ethical scenarios, from the noblest to the most objectionable. Why is this necessary? Simply put, training an AI system solely on positive data is insufficient. To defend itself against malicious attacks and make morally sound decisions, the AI needs to understand the concept of malevolence in order to effectively counteract it.
Disclaimer: I’m Filipino, and I find this post intriguing.
The chart seems to contradict many of the compliments I’ve heard in general. While it’s true that some places are terrible for finding authentic Filipino cuisine, trying to encapsulate all food choices in a single chart is inherently difficult.
There are some tools that can be used to enhance audio from100% to 200%. via video editing softwares.