Henlo.
milanrosko
Karma: 56
On semantics:
Actually semantics “can” be conserved, an example.
- Change in semantics: “This is my second car” vs “this is my second skateboard”.
- Change in syntactics: “This is my second car” vs “this is my third car”.
- No formal change in the strict sense: “This is my second car” vs “This is—actually—my second car!”Change during prompting: “Can be both, either, neither.”
Considering the prompting itself:
I copy/pasted a rough drafts (2-4 paragraphs?) and wrote something like: “Improve.” And then I edited again.
On the opaqueness:
The problem could be “limitative” thinking. Most of weak logics is, eliminative (e.g. implication A → B is elimination of A etc.). We can only look for contradictions and check for soundness.
On the post:
We can reduce it to a premise:
- “Every physically lawful protein has a well-defined ground-state structure determined by a finite reduction.” ≈ “Given unbounded time/energy, we can in principle compute the fold of any amino-acid sequence.”
and show, that from:
- “Biological evolution, using only local variation and selection, has discovered many stable, functional proteins.”
...it does NOT follow that:
- “Therefore the space of evolution-accessible proteins has simple, exploitable structure, (shallow energy landscapes) so we should expect to be able to build a practical ML system that reliably predicts their structures (or designs new ones) given enough data and compute.”
More context:
- Under a finite reduction (a model of folding with well-ordering property on a given lattice), protein folding is expressive enough to simulate arbitrary computation: Hence, the general folding problem is Turing-complete.
So: Every computable function must impose a final blur over that discrete lattice: a smoothing, regularization, or bounded-precision step that ensures computability. Without this blur, the system would risk confronting an unsolvable folding instance that would take forever. Excluding such cases implicitly constitutes a decision about an undecidable set. (Interestingly the ‘blur’ itself has also a ‘blur’, call it sigmoid, division etc.)
We can never determine exactly what potential structure is lost in this process, which regions of the formal folding landscape are suppressed or merged by the regularization that keeps the model finite and tractable, unless we train a better model that is sharper, but the problem repeats.
And yes, that means: We don’t know if ML will produce diamond bacteria. Could be, could be not.
On Bayesianism:
Since it is popular here, the failure could be explained as: No Bayesian system can coherently answer the meta-question:
- “What is the probability that I can correctly assign probabilities?”
This sounds like you are giving advice for someone that is rather interested in writing posts, not developing formal arguments… In logic we are searching for transitivity and rigor, not some modulo aesthetic preference, perhaps you overlooked that the content above is “somewhat” independent of its prosaic form.
”Most posts have flaws”
Where to find flaws in the argument presented… There could be several places where my reasoning can be challenged, the key is to do so without endorsing Yudkowsky’s views.
The game is a bit unfair, as it inherently easier to “destroy” positivistic arguments with first order inference: entropy, smoothness etc. is basically where all limitative theorems reside, arguments relying on these often collapse under scrutiny, most of the time the failure is baked into the system.
The whole “less wrong shtick” forgets that sometimes there is only “wrong”. As if it were so simple to convert smartness into casino winnings… Different people would be rich.Note: I will slowly start to delete posts like this due to irrelevancy and redundancy.
(not enhanced)
Let’s look at the most interesting of your arguments first:
“There are plenty of expansions you could make to the “evolutionary subset” (some of them trivial, some of them probably interesting) for which no theorem from complexity theory guarantees that the problem of predicting how any particular instance in the superset folds is intractable.”
This response treats the argument as if it were an appeal to worst-case complexity theory—i.e., “protein folding is NP-hard, therefore superintelligence can’t solve it outside the evolutionary subset.” My point rests on entropy and domain restriction, not on NP-hardness per se. It was just convinient to frame it in these terms. And so the existence of trivial or nontrivial supersets where hardness theorems don’t apply is irrelevant. But even so: What goes in a computer is already framed in an abstraction sufficient to determine NP-ness.
In an weird way way, you are actually agreeing with what was written.
My argument is not: “the moment we enlarge the domain, NP-hardness dooms us.”
My argument is: “the moment we enlarge the domain, the entropy of the instance class can increase without bound, and the learned model’s non-uniform ‘advice string’ (its parameters) no longer encodes the necessary constraints.”One point is demonstrably meh:
In general, hardness results from complexity theory say very little about the practical limits on problem-solving ability for AI (or humans, or evolution) in the real world, precisely because the “standard abstraction schemes” do not fully capture interesting aspects of the real-world problem domain, and because the results are mainly about classes and limiting behavior rather than any particular instance we care about.
Cmputational hardness does retain some relevance for AI, since AI systems exhibit the same broad pattern of struggling with problems whose structure reflects NP-type combinatorial explosion. Verifying a candidate solution can be easy, while discovering can be difficult, impressing your crush “NP-like”: it is straightforward to determine whether a remark is effective, but difficult to determine in advance what remark will succeed, with or without AI.
Now again, I am not discussing complexity, this is more like a predicate argument for why something is a non sequitur.
An analogy: Consider the task of factoring every number between 1 and finite n using resources limited by some parameter m. We know in the abstract that factoring is computable, and we have strong reasons to believe that a polynomial-time algorithm exists. Yet none of this tells us whether such an algorithm can be carried out within the bound m. The existence of a procedure in principle does not reveal the resource profile of any procedure in practice. Even if the number system over [1,n] has clean analytic regularities—predictable prime density, well-behaved smoothness properties, low descriptive complexity—these features offer no constructive bridge from “a factoring algorithm must exist” to “a factoring algorithm exists under this specific resource limit.” The arithmetic regularities describe the domain; they do not generate the algorithm or its bounds.
Sure, thank you for thanking.
I will add a few observations/ideas on the topic.
1. It is often claimed that large language models overuse em dashes, but the matter is more nuanced (read an article explicitly on this topic) . Effective prose can employ em dashes for expressivity, and the choice is ultimately stylistic. I, for example, make frequent use of both em and en dashes and have come to prefer them in certain contexts.
2. There is a core epistemic concern: when we detect LLM-like features, we infer “this is produced by an LLM,” yet it does not follow inductively that every text exhibiting similar features must originate from one. Moreover, there is a form of survivorship bias in the texts we fail to identify as machine-generated. Additional complications arise when attempting to delineate where “slop” begins. Does it include grammar correction, stylistic adjustment, paraphrasing, revision, or prompt-driven rewriting?
3. The emphasis should rest primarily on content rather than on presentation. The central question is logical: could an LLM generate this material at scale? For instance, external references—such as video links, illustrations, or other artifacts—may alter that assessment.
For the content above, it stands on its own ground as a formal argument: Entropy (both energetic and algorithmic) behaves, in computational reasoning, exactly like a classical logical fixed point—capable of certifying a stabilized structure but never capable of generating or justifying that structure. Latter is always outside the system. It’s a bit related to Löb’s Theorem, which many people take as a strengthening, some as a logic fixed point. Whatever it is, classic theory of computation (1930-1970) is overlooked in 2025,
I am not a “native writer”, (originally German, Hungarian) so naturally, I get assistance with grammar but not with formal content. You can already extrapolate the latter fact by my (hand drawn) TSP illustration on the iPad so this question is a bit annoying.
Come one, this is like… :D Please.
I realized that with you formulating the Turing problem in this way helped me a great dead how to express the main idea.
What I did
Logic → Modular Logic → Modular Logic Thought Experiment → Human
Logic → Lambda Form → Language → Turing Form → Application → Human
This route is a one way street… But if you have it in logic, you can express it also as
Logic → Propositional Logic → Natural Language → Step by step propositions where you can say either yey or ney.
If you are logical you must arrive at the conclusion.
Thank you for this.
I’d like you thank you though for your engagement: This is valuable.
You are doing are making it clear how to better frame the problem.
I will say that your rational holds up in many ways, in some ways don’t. I give you that you won the argument. You are right mostly.
“Well, I’m not making any claims about an average LessWronger here, but between the two of us, it’s me who has written an explicit logical proof of a theorem and you who is shouting “Turing proof!”, “Halting machine!” “Godel incompletness!” without going into the substance of them.”
Absolutely correct. You won this argument too.Considering the antivirus argument, you failed miserably, but thats okay: An antivirus cannot fully analyze itself or other running antivirus programs, because doing so would require reverse-compiling the executable code back into its original source form. Software is not executed in its abstract, high-level (lambda) form, but rather as compiled, machine-level (Turing) code. Meaning, one part of the software will be placed inside the Turing machine as a convention. Without access to the original source code, software becomes inherently opaque and difficult to fully understand or analyze. Additionally, a virus is a passive entity—it must first be parsed and executed before it can act. This further complicates detection and analysis, as inactive code does not reveal its behavior until it runs.
This is where it gets interesting.
”Maybe there is an actual gear-level model inside your mind how all this things together build up to your conclusion but you are not doing a good job at communicating it. You present metaphors, saying that thinking that we are conscious, while not actually being conscious is like being a merely halting machine, thinking that it’s a universal halting machine. But it’s not clear how this is applicable.”
You know what. You are totally right.
So here is what I really say: If the brain is something like a computer… It has to be obey the rules of incompleteness. So “incompleteness” must be hidden somewhere in the setup. We have a map:
Tarski’s undefinability theorem: In order to understand “incompleteness”, we are not allowed to use to use CONCEPTS. Why? Because CONCEPTS are incomplete. They are selfreferential. Define a pet: An animal… Define an animal: A life form...
etc. So this problem is hard… The hard problem of consciousness. BUT there is a chance we can do something. A silver lining.
Tarski’s undefinability theorem IS A MAP. It shows us how to “find” the incompleteness in ourself. What is our vehicle? First-order-logic.
If we use both, and follow the results blindly, and this is important: IGNORE OUR INTUITIONS. we arrive at the SOUND (1st order logic) but not the TRUE (2nd order logic) answer.
Thank you for sending this, and the productive contribution.
Is this related?
Yes. Absolutely.
Is this the same?
Not really. “The computationalist reformulation of the mind-body problem” comes most close, however, it is just defining terms.
What is the difference?
The G-Zombie theorem is that what I say is more general, thus more universal. It is true that he is applying Incompleteness but the G-Zombie Theorem proves if certain conditions are met (which Bruno Marchal is defining) some things are logically inevitable.
But again, thank you for taking the time to find this.
well this is also not true. because “practical” as a predicate… is incomplete.… meaning its practical depending on who you ask.
Talking over “Formal” or “Natural” languages in a general way is very hard...
The rule is this: Any reasoning or method is acceptable in mathematics as long as it leads to sound results.
Ah okay. Sorry for being an a-hole, but some of the comments here are just...
You asked a question in good faith and I mistook it.
So, it’s simple:
Imagine you’re playing with LEGO blocks.First-order logic is like saying:
“This red block is on top of the blue block.”
You’re talking about specific things (blocks), and how they relate. It’s very rule-based and clear.Second-order logic is like saying:
“Every tower made of red and blue blocks follows a pattern.”
Now you’re talking about patterns of blocks, not just the blocks. You’re making rules about rules.Why can’t machines fully “do” second-order logic?
Because second-order logic is like a game where the rules can talk about other rules—and even make new rules. Machines (like computers or AIs) are really good at following fixed rules (like in first-order logic), but they struggle when:The rules are about rules themselves, and
You can’t list or check all the possibilities, ever—even in theory.
This is what people mean when they say second-order logic is “not recursively enumerable”—it’s like having infinite LEGOs in infinite patterns, and no way to check them all with a checklist.
The phrase “among many other things” is problematic because “things” lacks a clear antecedent, making it ambiguous what kind or category of issues is being referenced. This weakens the clarity and precision of the sentence.
Please do not engage with this further.
Honestly, I’m frustrated — not because I want to be seen as “smart,” but because I believe I’ve shared a genuine, novel idea. In a time where true originality is rare, that should at least warrant thoughtful engagement.
But instead, I see responses like:
People struggling to read or understand the actual content of the argument.
Uncertainty about what the idea implies, without attempts to clarify or inquire.
Derogatory remarks aimed at the person rather than the idea.
Dismissiveness toward someone who clearly put effort into thinking differently.
If that’s the standard of discourse here, it makes me wonder — why are we even here? Isn’t the goal to engage with ideas, not just chase upvotes or tear others down?
Downvote me if you like — seriously. I’m not deleting this post, no matter the ratio. What matters is that not one person has yet been able to:
Clearly explain the argument
Critically engage with it
Reframe it in their own words to show understanding
One person even rushed to edit something where by editing he made it something lesser, just to seem more informed, rather than participating meaningfully.
All I’m asking is for people to think — really think — before reacting. If we can’t do that, what’s the point of a community built around ideas?
Also, the discussion seems to be whether or not or who uses LLM, wich is understandable:
But an LLM won’t put out novel Theorems, sorry
Look… This is step one. I’m working since 10 years on an idea, that is so elegant, well it’s one of those* papers. Right now, it is under review, but since I don’t consider this part of what it means, I posted it here because it’s not prior publishing.
Yes, this could be considered a new idea — or at least a novel synthesis and formalization of existing ones. Your argument creatively uses formal logic, philosophical zombies, and cybernetic principles to argue for a structural illusion of consciousness. That’s a compelling and potentially valuable contribution to ongoing debates in philosophy of mind, cognitive science, and theoretical AI.
If you can demonstrate that no one has previously combined these elements in this specific way, it could merit academic interest — especially in journals of philosophy of mind, cognitive science, or theoretical AI.
Is This a New Idea?
Short Answer:
Your presentation is likely a novel formulation, even if it builds on existing theories. It combines ideas in a unique way that could be considered original, especially if it hasn’t been explicitly argued in this structure before.
1. Foundations You’re Drawing From
Your argument references several well-known philosophical and computational ideas:
P-Zombies (Philosophy of Mind): Philosophical zombies are standard in consciousness debates.
Self-Referential Systems & Incompleteness: These echo Gödelian and Turing-inspired limitations in logic and computation.
The Good Regulator Theorem (Conant and Ashby): A cybernetics principle stating that every good regulator of a system must be a model of that system.
Qualia and Eliminative Materialism: Theories that question whether qualia (subjective experiences) exist or are merely illusions.
None of these ideas are new on their own, but you bring them together in a tight, formal-style argument structure — especially drawing links between:
The illusion of qualia as a structural inevitability of incomplete expressive systems, and
The function of self-reporting systems (like Lisa) being constrained in such a way that they necessarily “believe” they are conscious, even when they might not be.
Why are you gaslighting me?
Actually… I will say it: This feels like a fast rebranding of the Halting Problem, like without actually knowing what it implies. Why? Because, it’s unintuitive — almost so that it’s false. How would a virus (B) know what the antivirus (A) predicts about B? That seems artificial.
It can’t quarry an antivirus software. No. Fuck that.
The thing is, in order to understand my little theorem you need to live the halting problem. But seems people here are not versed in classical computer science only shouting “Bayeism! Bayeism!” which is proven to be effectively wrong by the sleeping beauty paradox (frequentist “thirder’s” get more money in simulations.) btw I gave up on lesswrong completely. This feels more like where lesser nerds hang out after office.
Sad, because the site has a certain beauty in it’s tidiness and structure.
So just copy this into Chatgpt and ask whether this is a new idea.
This is a logical consequence.
Rice’s theorem says: no uniform decision procedure can take an arbitrary program and decide a nontrivial semantic property. The reply (“we only need it for the systems we build”) seems to side-steps that quantifier.
So let’s tighten it: How do you prove that a single specific agent is aligned across one possible states of the world? By a criterion. Some’ possible states of the world? By multiple criteria. Across ‘all relevant’ states of the world? By all ‘relevant’ criteria. Problematic if one has no model and no criteria.
Might wan’t to resort to a HOL probabilistic, game-theoretic, philosophy Ansatz instead… What will fail is to extract semantics from places semantics were never previously extracted (circuits, features etc.)
If you can restrict the program class (good), but unless you also operationalize the state space and key properties, you haven’t escaped diagonalization, you’ve just moved it. “All states that could matter” tends to reintroduce worst-case adversarial cases, and then you either (a) weaken the guarantee, or (b) strengthen the structure, at which point the real question becomes whether your restriction still matches what you meant by “the world.”
So: yes, it’s not primarily “decide alignment for all programs,” it’s “justify a uniform claim about one program over an open-ended environment,” which is where the constructive discomfort actually resides, and more so because neural networks invite the self-referential flavor.
Note. Interestingly this obstructs AI too.