A minor point, perhaps a nitpick: both biological systems and electronic ones depend on directed diffusion. In our bodies diffusion is often directed by chemical potentials, and in electronics it is directed by electric or vector potentials. It’s the strength of the ‘direction’ versus the strength of the diffusion that makes the difference. (See: https://en.m.wikipedia.org/wiki/Diffusion_current)
Except in superconductors, of course.
So the reason why the time value of money works, and it makes sense to say that we can say that the utility of $1000 today and $1050 in a year are about the same, is because of the existence of the wider financial system. In other words, this isn’t necessarily true in a vacuum; however if I wanted $1050 in a year, I can invest the $1000 I have right now into 1 year treasuries. The converse is more complex; if I am guaranteed $1050 in a year I may not be able to get a loan for $1000 right now from a bank because I’m not the fed and loans to me have a higher interest rate, but perhaps I can play some tricks on the options market to do this? At any rate, I can get pretty close if I were getting an asset-backed loan, such as a mortgage.
Note that I’m not saying that actors are indifferent to which option they get, but that it is viewed with equal utility (when discounted by your cost of financing, basically).
This is a bit of a cop-out, but I would say modelling the utility of money without considering the wider world is a bit silly anyway, because money only has value due to its use as a medium of exchange and as a store of value, both of which depend on the existence of the rest of the world. The utility of money thus cannot be truly divorced from the influence of eg. finance.
Is the fifth requirement not a little vague, in the context of agents with external memory and/or few-shot learning?
I haven’t heard of this, but I definitely do this.
I’m not sure why you keep bringing up social media, I haven’t so it’s quite irrelevant to my point.
Your specific point was that LW is better than predicting
96 of the last one civil wars and two depressions
I’m curious if you just think that, or if you actually have evidence demonstrating that LW as a community has a quantifiably better track record than social media. That’s completely beside my point though, since I was never talking about social media.
Regarding overconfidence, GPT-4 is actually very very well-calibrated before RLHF post-training (see paper Fig. 8). I would not be surprised if the RLHF processes imparted other biases too, perhaps even in the human direction.
How?
Edit:
Also, are you asking me for sources that people have been worried about democratic backsliding for over 5 years? I mean, sure, but I’m genuinely a little surprised that this isn’t common knowledge. https://scholar.google.com/scholar?hl=en&as_sdt=0%2C44&q=democratic+backsliding+united+states&btnG=&oq=democratic+ba
A few specific examples of both academic and non-academic articles:
How has the discourse on LW about democratic backsliding been better than these ~5 year old articles?
Remember, the “exception throwing” behavior involves taking the entire space of outcomes and splitting it into two things: “Normal” and “Error.” If we say this is what we ought to do in the general case, that’s basically saying this binary property is inherent in the structure of the universe.
I think it works in the specific context of programming because for a lot of functions (in the functional context for simplicity), behaviours are essentially bimodal distributions. They are rather well behaved for some inputs, and completely misbehaving (according to specification) for others. In the former category you still don’t have perfect performance; you could have quantisation/floating-point errors, for example, but it’s a tightly clustered region of performing mostly to-spec. In the second, the results would almost never be just a little wrong; instead, you’d often just get unspecified behaviour or results that aren’t even correlated to the correct one. Behaviours in between are quite rare.
I think you’re also saying that when you predict that people are limited or stunted in some capacity, that we have to intervene to limit them or stunt them even more, because there is some danger in letting them operate in their original capacity.
It’s like, “Well they could be useful, if they believed what I wanted them to. But they don’t, and so, it’s better to prevent them from working at all.”
If you were right, we’d all be hand-optimising assembly for perfect high performance in HPC. Ultimately, many people do minimal work to accomplish our task, sometimes to the detriment of the task at hand. I believe that I’m not alone in this thinking, and you’d need quite a lot of evidence to convince others. Look at the development of languages over the years, with newer languages (Rust, Julia, as examples) doing their best to leave less room for user errors and poor practices that impact both performance and security.
I’m mostly talking about academic discourse. Also, what a weird hollier than thou attitude; are you implying LW is better? In what way?
Yeah, I’m interested in why we need strong guarantees of correctness in some contexts but not others, especially if we have control over that aspect of the system we’re building as well. If we have choice over how much the system itself cares about errors, then I can design the system to be more robust to failure if I want it to be.
This would make sense if we are all great programmers who are perfect. In practice, that’s not the case, and from what I hear from others not even in FAANG. Because of that, it’s probably much better to give errors that will show up loudly in testing, than to rely on programmers to always handle silent failures or warnings on their own.
I think the crux for me here is how long it takes before people notice that the belief in a wrong result causes them to receive further wrong results, null results, or reach dead-ends, and then causes them to update their wrong belief. LK-99 is the most recent instance that I have in memory (there aren’t that many that I can recall, at least).
Sometimes years or decades. See the replicability crisis in psychology that’s decades in the making, and the Schron scandal that wasted years of some researchers time, just for the first two examples off the top of my head.
You have a cartoon picture of experimental science. LK-99 is quite unique in that it is easy to synthesise, and the properties being tested are easy to test. When you’re on the cutting edge, this is almost by necessity not the case, because most of the time the low-hanging fruit has been picked clean. Thus, experiments are messy and difficult, and when you fail to replicate, it is sometimes very hard to tell if it is due to your failure to reproduce the conditions (eg. synthesise a pure-enough material, have a clean enough experiment, etc.)
For a dark matter example, see DAMA/Libra. Few in the dark matter community take their result too seriously, but the attempts to reproduce this experiment has taken years and cost who knows how much, probably tens of millions.
I worked on dark matter experiments as an undergrad, and as far as I know, those experiments were built such that they were only really for testing the WIMP models, but also so that it would rule out the WIMP models if they were wrong (and it seems they did). But I don’t think they were necessarily a waste.
I am a dark matter experimentalist. This is not a good analogy. The issue is not replication, but that results get built on; when that result gets overturned, a whole bunch of scaffolding collapses. Ruling out parameter space is good, you’re searching for things like dark matter. Having to keep looking at old theories is quite different; what are you searching for?
I would posit that humans behave in a much more optimal manner in terms of long-run quality of life than are given credit for, excluding gambling addicts.
A lot of people who are willing to bet everything (ie. follow a linear utility function) are lower income. It is more that just that, however. Lower income people just by necessity have less savings relative to income, so losing all their savings isn’t a big deal compared to work-derived income. Losing a couple months of pay sucks, but eh.
People who like to think they’re being more rational by not betting the farm usually just have more to lose. If you’re a professional who accrued a few million over a few decades of work, you can’t make it back; you will invest prudently, with diversification across asset classes and markets; you might not even keep everything in one country.
What would the optimal utility function look like for someone who has a steady income? I would expect it to smoothly transition from a linear to a log regime, as the total winnings exceed the income per turn. Fun textbook exercise for stats undergrads, maybe I’ll use it sometime.
I’m not sure I’ve ever seen a treatment of utility functions that deals with this problem? (The problem being “what if your utility function is such that maximizing expected utility at time t1 doesn’t maximize expected utility at time t2?”) It’s no more a problem for Linda than for Logan, it’s just less obvious for Logan given this setup.
In economics, this is considered via the time-value of money. It is considered at the market level, however, not at the level of individuals so plausibly it could have individual variances.
As a bit of metacommentary, that your main thesis (the title) is presented and viewed as a significant insight in this year (2023), and is contested significantly in the comments, is to me a sign of how insular LW is.
Democratic backsliding isn’t exactly the same thing but it rhymes, and the democratic backsliding of the US has been discussed outside of LW for over half a decade at this point.
I come from science, so heavy scientific computing bias here.
I think you’re largely focusing on the wrong metric. Whether exceptions should be thrown has little to do with reliability (and indeed, exceptions can be detrimental to reliability), but instead is more related to correctness. They are not always the same thing. In a scientific computing context, for example, a program can be unreliable, with memory leaks resulting in processes often being killed by the OS, but still always give correct results when a computation actually manages to finish.
If you need a strong guarantee of correctness, then this is quite important. I’m not so sure that this is always the case in machine learning, since ML models by their nature can usually train around various deficiencies; with small implementation mistakes you might just be a little confused as to why your model performs worse than expected. In aerospace, correctness needs to balanced against aeroplanes suddenly losing power, so correctness always doesn’t always win. In scientific computing you might have the other extreme, where there’s very little riding on your program not exiting, since you can always do a bunch of test runs before sending your code off to a HPC cluster, but if you do run this thing and base a whole bunch of science off of it it better not be ruined by little insidious bugs. I can imagine correctness mattering a lot too in crypto and security contexts, where a bug might cause information to leak and it is probably better for your program to die from internal checks than for your private key to be leaked.
I’m not sure if I agree that a job poorly-done is worse than one not even started.
I think this is definitely highly context-dependent. A scientific result that is wrong is far worse than the lack of a result at all, because this gives a false sense of confidence, allowing for research to be built on wrong results, or for large amounts of research personpower to be wasted on research ideas/directions that depend on this wrong result. False confidence can be very detrimental in many cases.
As to why general purpose languages usually involve error handling and errors: they are general purpose languages and have to cater to use cases where you do care about errors. Built-in routines fail with exceptions rather than silently so that people building mission-critical code where correctness is the most important metric can at least kinda trust every language built-in routine to return correct results if it manages to return something successfully.
Edit: some grammatical stuff and clarity
How would you experimentally realise mechanism 1? It still feels like you need an additional mechanism to capture the energy, and it doesn’t necessarily seems easier to experimentally realise.
With regards to 2, you don’t necessarily need a thermal bath to jump states, right? You can just emit a photon or something. Even in the limit where you can fully harvest energy, thermodynamics is fully preserved. If all the energy is thermalised, you actually cannot necessarily recover Landauer’s principle; my understanding is that because of thermodynamics, even if you don’t thermalise all of that energy immediately and somehow harvest it, you still can’t exceed Landauer’s principle.
I don’t buy your ~kT argument. You can make the temperature ratio arbitrarily large, and hence the energy arbitrarily small, as far as I understand your argument.
With your model, I don’t understand why the energy ‘generated’ when swapping isn’t thermalised (lost to heat). When you drop the energy of the destination state and the particle moves from your origin to your destination state, the energy ‘generated’ seems analogous to that from bit erasure; after all, bit erasure is moving a particle between states (50% of the time). If you have a mechanism for harvesting energy, you can always use it.
I think there’s a more thermodynamically-sound ~kT argument. When you zero a bit by raising energy to cross a ~nkT barrier, if the bit is in 1, a ~nkT photon (or whatever other boson) is emitted. The carnot efficiency for harnessing this nkT energy source is (1-1/n), so only ~kT energy is lost.
You descale to prevent bits of scale from chipping off into your tea. That’s basically it.
The dictionary definition of consumerism is: https://www.merriam-webster.com/dictionary/consumerism
1: the theory that an increasing consumption of goods is economically desirable
also : a preoccupation with and an inclination toward the buying of consumer goods
2 : the promotion of the consumer’s interests
This is also definition 2.1 from wikipedia (https://en.wikipedia.org/wiki/Consumerism):
Consumerism is the selfish and frivolous collecting of products, or economic materialism. In this sense consumerism is negative and in opposition to positive lifestyles of anti-consumerism and simple living.[5]
Previously, from context, I believe it’s quite clear that we’re talking about definition 1 b (merriam webster) and 2.1 (wikipedia). The original post talks about how consumption is good even if frivolous, according to the OP; I believe this makes that quite clear. This is why the definitional issue of consumerism isn’t quite relevant, and the definitional issue that is relevant is regarding what’s frivolous. I see this a lot in internet discussion, where discussion revolves around a concept that is encapsulated by a word with multiple meanings, and a different-but-related meaning of the word keeps being brought up. It muddies the conversation. The discussion is about the concept, not the word; words are but the medium.
Regarding your more on-point criticism, I generally agree. I think the key, so to speak, is two-fold:
Sometimes things just can’t be equivalently-substituted not due to the goods/services, but due to the situation. That’s just life.
Sometimes the situation or one’s mindset, both of which are malleable, are the issue. The situation of amenities being too far away is one borne of bad urban planning. 2.5 mins, your benchmark, is quite short and good, however I do notice myself going out a lot less since I came to the US (almost a decade ago) because cities are extremely not walkable, so just going to the park is a whole thing. This is something you live with, but also fight to change. Thus, in the near-term, maybe consumption beats just utilising local amenities, but that is not necessarily the case, and once again is a semi-conscious choice made by the local communities and governments and can be changed. There is also a mindset aspect, which is that many things appear significantly less enjoyable than consumption, but that is something that we can change. For example, ‘sitting there with your thoughts for 15 minutes’ sounds quite fine to me! I strongly believe that isn’t because I’m special, it’s merely because many of my family who were part of my upbringing are buddhist and hence I was taught to find value in mindfulness. In other words, I think my rule-of-thumb holds, but one needs to look deeper, not at what is substitutable, but what could be, and what it would take to change that. That sounds like a lot, but a bit of incremental change every day or week adds up very quickly, and I think relaxing consumerist (by the contextual definition here) attitudes and stepping off of the treadmill a bit makes life a lot more fulfilling.
Sure, it could easily be that I’m used to it, and so it’s no problem for me. It’s hard to judge this kind of thing since at some level it’s very subjective and quite contingent on what kind of text you’re used to reading.
I genuinely don’t see a difference either way, except the second one takes up more space. This is because, like I said, the abstract is just a simple list of things that are covered, things they did, and things they found. You can put it in basically any format, and as long as it’s a field you’re familiar with so your eyes don’t glaze over from the jargon and acronyms, it really doesn’t make a difference.
Or, put differently, there’s essentially zero cognitive load to reading something like this because it just reads like a grocery list to me.
Regarding the latter:
I think it’s dumb for papers to only be legible to other specialists. Don’t dumb things down for the masses obviously, but, like, do some basic readability passes so that people trying to get up-to-speed on a field have an easier time
I generally agree. The problem isn’t so much that scientists aren’t trying. Science communication is quite hard, and to be quite honest scientists are often not great writers simply because it takes a lot of time and training to become a good writer, and a lifetime is only 80 years. You have to recognise that scientists generally try quite hard to make papers readable, they/we are just often shitty writers and often are even non-native speakers (I am a native speaker, though of course internationally most scientists aren’t). There are strong incentives to make papers readable since if they aren’t readable they won’t get, well, read, and you want those citations.
The reality I think is if you have a stronger focus on good writing, you end up with a reduced focus on science, because the incentives are already aligned quite strongly for good writing.
You would be deceiving someone regarding the strength of your belief. You know your belief is far weaker than can be supported by your statement, and in our general understanding of language a simple statement like ‘X is happening tonight’ is interpreted as having a strong degree of belief.
If you actually truly disagree with that, then it wouldn’t be deception, it would be miscommunication, but then again I don’t think someone who has trouble assessing approximate Bayesian belief from simple statements would be able to function in society at all.