Daniel V

Karma: 485

Daniel V May 17, 2025, 1:22 PM
2 points
1
on: P-Values Know When You’re Cheating
Yep, frequentist null hypothesis significance testing often gets critiqued for the epistemic failures of the people who use it, even when the core of the approach is fine. By the way, the term you’re describing is familywise error rate. The key questions are, do we care about that error rate, and who is in the family (answers depend on the research questions)?

Daniel V May 9, 2025, 7:15 PM
4 points
2
on: Cheaters Gonna Cheat Cheat Cheat Cheat Cheat
Alternatively or in addition to this, you can embrace AI and design new tasks and assignments that cause students to learn together with the AI.

This magical suggestion needs explication.

From what I’ve seen via Ethan Mollick, instead of copy-pasting, the new assignments that would be effective are the same as the usual—just “do the work,” but at the AI. Enter a simulation, but please don’t dual-screen the task. Teach the AI (I guess the benefit here is immediate feedback, as if you couldn’t use yourself or friend as a sounding board), but please don’t dual-screen the task. Have a conversation (again, not in class or on a discussion board or among friends), but please don’t dual-screen the task. Then show us you “did it.” You could of course do these things without AI, though maybe AI makes a better (and certainly faster) partner. But the crux is that you have to do the task yourself. Also note that this admits the pre-existing value of these kinds of tasks.

Students who will good-faith do the work and leverage AI for search, critique, and tutoring are...doing the work and getting the value, like (probably more efficiently than, possibly with higher returns than) those who do the work without AI. Students who won’t...are not doing the work and not getting the value, aside from the signaling value from passing the class. So there you have it—educators can be content that not doing the assignment delivers worse results for the student, but the student doesn’t mind as long as they get their grade, which is problematic. Thus, educators are not going quietly and are in fact very concerned about AI-proofing the work, including shifts to in-person testing and tasks.

However, that only preserves the benefit of the courses and in turn degree (I’m not saying pure signaling value doesn’t exist, I’m just saying human capital development value is non-zero and under threat). It does not insulate the college graduate from competition in knowledge work from AI (here’s the analogy: it would obviously be bad for the Ford brand to send lemons into the vehicle market, but even if they are sending decent cars out, they should still be worried about new entrants).

Daniel V Apr 7, 2025, 12:21 PM
1 point
0
on: How Gay is the Vatican?
Really enjoyed the post, but in the interest of rationality,
How many more older siblings should patrons of the Chelsea nightclub have than all other men in New York?
This question rests on the false premise(s) (i.e., model misspecification(s)) that homosexuality is only a function of birth order and that the Chelsea nightclub probability doesn’t stem from heavy selection. Relatedly, gwern notes that, “surely homosexuality is not the primary trait the Catholic Church hierarchy is trying to select for.” Maybe this was supposed to be more tongue-in-cheek. But identifying a cause does not require that it sufficiently explain something completely on its own.

Daniel V Feb 26, 2025, 2:02 AM
2 points
0
in reply to: Tim Garnett’s comment on: Economics Roundup #5
I agree with you. Unless the signal is so strong that people believe that their personal experience is not representative of the economy, it’s going to be overweighted. “I and half the people I know make less” will lead to discontent about the state of the economy. “I and half the people I know make less, but I am aware that GDP grew 40%, so the economy must be doing fine despite my personal experience” is possible, but let’s just say it’s not our prior.

Daniel V Feb 26, 2025, 1:54 AM
1 point
0
in reply to: JBlack’s comment on: Economics Roundup #5
Exactly, which is why the metric Mazlish prefers is so relevant and not bizarre, unless the premise that people judge the economy from their own experiences is incorrect.

Daniel V Feb 25, 2025, 7:09 PM
1 point
0
on: Economics Roundup #5
Why is this what matters? It’s a bizarre metric. Why should we care what the median change was, instead of some form of mean change, or change in the mean or median wage?

The critique that the justification wasn’t great because the mean wage dropped a lot in the example is fair. Yet, in the proposed alternative example it remains quite likely that people will perceive the economy as having gotten worse, even if the economy is objectively so much better − ²⁄₃ will say they’re personally worse off, insufficiently adjust for the impersonal ways of assessing the economy, and ultimately say the economy is worse.

Neither $Δ (M e d i a n I n c o m e)$ nor $M e d i a n (Δ I n c o m e)$ is a bizarre metric. $Δ (M e d i a n I n c o m e)$ may be great for observers to understand general trajectories of income when you lack panel data, but since people use their own lives to assess whether they are better off and in turn overweight that when they judge the economy, $M e d i a n (Δ I n c o m e)$ is actually more useful for understanding the translation from people’s lives into their perceptions.

Consider a different example (also in real terms):
T1: A makes $3, B makes $3, C makes $3 ,D makes $10, E makes $12
T2: A makes $2, B makes $2, C makes $3, D makes $9, E makes $16
The means show nice but not as crazy economic growth ($6.20 to $6.40), and the $Δ (M e d i a n I n c o m e)$ is $0 ($3 to $3) - “we’re not poorer!” However, the $M e d i a n (Δ I n c o m e)$ is -$1. And people at T2 will generally feel worse off (3/5 will say they can’t buy as much as they could before, so “this economy is tough”).
Contrast that with (still in real terms):
T1: A makes $2, B makes $2, C makes $4 ,D makes $10, E makes $12
T2: A makes $3, B makes $3, C makes $3, D makes $10, E makes $12
The means show nice but not as crazy economic growth ($6 to $6.20), and the $Δ (M e d i a n I n c o m e)$ is -$1 ($4 to $3) - “we’re poorer!” However, the $M e d i a n (Δ I n c o m e)$ is $0. And people at T2 will generally feel like things are going okay (only 1 person will feel worse off).

And these are comparing to 0. Mazlish’s post illustrates that people will probably not compare to 0 and instead to recent trajectories (“I got a 3% raise last year, what do you mean my raise this year is 2%?!”), so #1 means people will be dissatisfied. #2, as it bore out in the data, also means dissatisfaction. And #3, largely due to timing, means further dissatisfaction.

Then it is no surprise that exit polls show people who were most dissatisfied with the economy under Biden (and assumed Harris would be more of that) voted for Trump. Sure, there’s some political self-deception bias going on (see charts of economic sentiment vs. date by party affiliation), but note that the exit polls are correlational—they can indicate that partisanship is a hell of a drug or that people are rationally responding to their perceptions. It’s likely both. And if your model of those perceptions is inferior in the ways Mazlish notes, you’d wrongly think people would have been happy with the economy.

Daniel V Feb 25, 2025, 12:50 AM
7 points
4
in reply to: Bernhard’s comment on: Export Surplusses
Literally macroeconomics 101. Trade surpluses aren’t shipping goods for free. There is a whole balance of payments to consider. I’m shocked EY could get that so wrong, surprised that lsusr is so ready to agree, and confused because surely I missed something huge here, right?

Daniel V Feb 22, 2025, 2:51 PM
1 point
0
in reply to: dynomight’s comment on: The first RCT for GLP-1 drugs and alcoholism isn’t what we hoped
I guess I misunderstood you. I figured that without “regression coefficients,” the sentence would be a bit tautological: “the point of randomized controlled trial is to avoid [a] non-randomized sample,” and there were other bits that made me think you had an issue with both selection bias (agree) and regressions (disagree).
I share your overall takeaway, but at this point I am just genuinely curious why the self-selection is presumed to be such a threat to internal validity here. I think we need more attention to selection effects on the margin, but I also think there is a general tendency for people to believe that once they’ve identified a selection issue the results are totally undermined. What is the alternative explanation for why semaglutide would disincline people who would have had small change scores from participating or incline people who have large change scores to participate (remember, this is within-subjects) in the alcohol self-administration experiment? Maybe those who had the most reduced cravings wanted to see more of what these researchers could do? But that process would also occur among placebo, so it’d work via the share of people with large change scores being greater in the semaglutide group, which is...efficacy. There’s nuance there, but hard to square with lack of efficacy.
That said, still agree that the results are no slam dunk. Very specific population, very specific outcomes affected, and probably practically small effects too.

Daniel V Feb 21, 2025, 8:49 PM
13 points
10
on: The first RCT for GLP-1 drugs and alcoholism isn’t what we hoped
I appreciate this kind of detailed inspection and science writing, we need more of this in the world!
I’m writing this comment because of the expressed disdain for regressions. I do share the disappointment about how the randomization and results turned out. But for both, my refrain will be: “that’s what the regression’s for!”
This contains the same data, but stratified by if people were obese or not:
Now it looks like semaglutide isn’t doing anything.
The beauty of exploratory analyses like these is that you can find something interesting. The risk is that you can also read into noise. Unfortunately, all they did was plot these results, not report the regression, which could tell us whether there is any effect beyond the lower baseline. eTable3 confirms that the interaction between condition and week is non-significant for most outcomes, which the authors correctly characterized. That’s what the regression’s for!
This means the results are non-randomized.
Yes and no. People were still randomized to condition and it appears to be pretty even attrition. Yes, there is an element of self-selection, which can constrain the generalizability (i.e., external validity) of the results (I’d say most of the constraint is actually just due to studying people with AUD rather than the general population, but you can see why they’d do such a thing), but that does not necessarily mean it broke the randomization, which would reduce the ability to interpret differences as a result of the treatment (i.e., internal validity). To the extent that you want to control for differences that happen to occur or have been introduced between the conditions, you’ll need to run a model to covary those out. That’s what the regression’s for!
the point of RCTs is to avoid resorting to regression coefficients on non-randomized samples
My biggest critique is this. If you take condition A and B and compute/plot mean outcomes, you’d presumably be happy that it’s data. But computing/plotting predicted values from a regression of outcome on condition would directly recover those means. And from what we’ve seen above, adjustment is often desirable. Sometimes the raw means are not as useful as the adjusted/estimated means—to your worry about baseline differences, the regression allows us to adjust for that (i.e., provide statistical control where experimental control was not sufficient). And, instead of eyeballing plots, the regressions help tell you if something is reliable. The point of RCTs is not to avoid resorting to regression coefficients. You’ll run regressions in any case! The point of RCTs is to reduce the load your statistical controls will be expected to lift by utilizing experimental controls. You’ll still need to analyze the data and implement appropriate statistical controls. That’s what the regression’s for!

Daniel V Jan 21, 2025, 5:14 PM
4 points
0
on: Almost all growth is exponential growth
I really like this succinct post.

I intuitively want to endorse the two growth rates (if it “looks” linear right now, it might just be early exponential), but surely this is not that simple, right? My top question is “What are examples of linear growth in nature and what do they tell us about this perception that all growth is around zero or exponential?”

A separate thing that sticks out is that having two growth rates does not necessarily imply generally two subjective levels.

Daniel V Jan 18, 2025, 3:00 PM
1 point
0
in reply to: Dagon’s comment on: Tax Price Gouging?
This can be effectively implemented by the government accumulating tax revenues (largely from the rich) in good times and spending them on disaster relief (largely on the poor) in bad times. It lets price remain a signal while also expanding supply.

Daniel V Jan 18, 2025, 2:56 PM
5 points
2
on: Tax Price Gouging?
Taxation is better than a ban, but in this case it remains an attempt at price control. “Documented” cost increases is doing a lot of work. Better than “vibes about price,” but it is the same deal: the government “knows better” what prices should be than what is revealed by the market. I’d argue that if the government doesn’t like what the market is yielding, it can get involved in the market and help expand supply itself, which we see governments attempt during disaster relief already.

Daniel V Jan 18, 2025, 2:44 PM
3 points
0
on: Don’t ignore bad vibes you get from people
Agree, we’re not so shy about pursuing a good vibe, bad vibes are also informative.

Daniel V Jan 14, 2025, 6:33 PM
3 points
0
in reply to: Steven Byrnes’s comment on: Applying traditional economic thinking to AGI: a trilemma
Thanks, you had mentioned the short- vs. long-run before, but after this discussion it is more foregrounded and the “racing” explanation makes sense. :) Though I appreciated the references to marginal value and marginal cost.
You’re assuming that the economy will produce new jobs faster than the factories will produce new chips and robots to fill those jobs.
Well, the assumptions are primarily that the supply and demand for AI labor will vary across markets and secondarily that labor can flow across markets. This is an important layer separate from just seeing who (S or D) wins the race. If there is only one homogenous market, then the price trajectory for AI labor (produced through the racing dynamics) tells you all you’ll need to know about the price trajectory for its human substitute. So the question is just which is faster.

But if there are heterogenous markets, “which is faster” is informative only for that market and the price of human labor as a substitute in that market. The price trajectory for AI labor in other markets might be subject to different “which is faster” racing dynamics. Then, because of composition effects, the trajectory for the average price of AI labor that is performed may diverge from the trajectory for the average price of human labor that is performed.

This is true even if you assume the economy has no vacancies and will not produce new jobs (i.e., labor cannot flow across markets). For example, average hourly earnings spiked during COVID because the work that was being performed was high-cost/value labor, an increase seemingly entirely due to composition [BLS]. Although I am alleging that predicting the price trajectory remains difficult even if you take a stance on the racing dynamics because you need to know what the alternative human jobs are, in that world where jobs are simply destroyed, the total value accruing to human laborers certainly goes down. This is why I think the labor flows could be considered a secondary assumption for the left-side depending on how much you think that side would be arguing—they are not dispositive of what the price changes will be (the focus of the post was on price), but they definitely will affect whether human labor commands the same total value.

Daniel V Jan 13, 2025, 6:33 PM
7 points
0
on: Applying traditional economic thinking to AGI: a trilemma
I like that this post lays out the dilemma in principles A (marginal value dominates) and B (marginal cost dominates). One quibble is that the effects are on the supply and demand curves, not on the quantities supplied and demanded, i.e., it’s not about the slopes of the curves but the location of the new equilibrium as the curves shift left or right. It’s not about which part “equilibrates” faster (with what?) but about the relative strength of the shifts.

If AGI shifts the demand for AI labor to the right, under constant supply, we’d expect a price increase and more AI labor created and consumed. If AGI shifts the supply for AI labor to the right, under constant demand, we’d expect a price decrease and more AI labor created and consumed. Both of these things would happen, so there is a wide range of possible price changes (even no change in price) consistent with more AI labor created and consumed, but what happens to the price depends on which shift is “stronger.”

Still, with the quantity of AGI labor created and consumed increasing, you might wonder about how the experience curve impacts it—that’s just more right-shift in the supply curve, so maybe we don’t have to wonder after all. What about the effect on substitutes like human labor? Well, if the economy has a set number of jobs, you’d expect a lot of human labor displaced, but if the economy can find other useful work for those people, they will do those other jobs, which might be lower-paying (no more coding tasks for you—enjoy ⁷⁄₁₁), reducing the average price of human labor, or might be higher-paying (no more coding tasks for you—enjoy this support role for AGI that because of its importance requires, increasing the average price of human labor.

Can those niches exist? Yes, the supply and demand curves are curves of heterogeneous values and production functions. And markets are imperfect. Won’t those niches eventually disappear? Well, rinse and repeat. See ATMs and bank tellers, also see building luxury housing supply and the effects on rents throughout the housing supply.

I don’t think it’s only talking past each other—it’s a genuine ton of uncertainty.

Daniel V Nov 12, 2024, 10:28 PM
2 points
0
in reply to: Terence Coelho’s comment on: The lying p value
I’m here to say, this is not some property specific to p-values, just about the credibility of the communicator.

If $s c i e n t i s t s$ make a bunch of errors all the time, especially those that change their conclusions, indeed you can’t trust them. Turns out (BW11) that $s c i e n t i s t s_{p u b l i s h e d i n b e t t e r j o u r n a l s}$ are more credible than $s c i e n t i s t s_{p u b l i s h e d i n w o r s e j o u r n a l s}$ , the errors they make tend not to change the conclusions of the test (i.e., the chance of drawing a wrong conclusion from their data (“gross error” in BW11) was much lower than the headline rate), and (admittedly I’m going out on a limb here) it is very possible the errors that change the conclusion of a particular test do not change the overall conclusion about the general theory (e.g., if theory says X, Y, and Z should happen, and you find support for X and Y and marginal-support-now-not-significant-support-anymore for Z, the theory is still pretty intact unless you really care about using p-values in a binary fashion. If theory says X, Y, and Z should happen, and you find support for X and Y and now-not-significant-support-anymore for Z, that’s more of an issue. But given how many tests are in a paper, it’s also possible theory says X, Y, and Z should happen, and you find support for X and Y and Z, but turns out your conclusion about W reverses, which may or may not really have something to say about your theory).

I don’t think it is wise to throw the baby out with the bathwater.

Daniel V Oct 18, 2024, 12:43 AM
1 point
0
in reply to: Zac Hatfield-Dodds’s comment on: Monthly Roundup #23: October 2024
Supply side: It approaches the minimum average total, not marginal, cost. Maybe if people accounted for it finer (e.g., charging self “wages” and “rent”), cooking at home would be in the ballpark (assuming equal quality of inputs and outputs across venues..), but that just illustrates how real costs can explain a lot of the differential without having to jump to regulation and barriers to entry (yes, those are nonzero too!).

Demand side: Complaints in the OP about the uninformativeness of ratings also highlight how far we are from perfect competition (also, e.g., heterogeneous products), so you can expect nonzero markups. We aren’t in equilibrium and in the long run we’re all dead, etc.

I’m a big proponent of starting with the textbook economic analysis, but I was surprised by the surprise. Let’s even assume perfect accounting and competition:

Draw a restaurant supply curve in the middle of the graph. In the upper right corner, draw a restaurant demand curve (high demand given all the benefits I listed). Equilibrium price is P_r*. Now draw a home supply curve to the far left, indicating an inefficient supply relative to restaurants (for the same quantity, restaurants do it “cheaper”). In the bottom left corner, draw a home demand curve (again the point is I demand eating out more than eating at home). Equilibrium price for those is P_h*. It’s very easy to draw where P_h* < P_r*.

Daniel V Oct 16, 2024, 2:43 PM
3 points
−2
on: Monthly Roundup #23: October 2024
Cooking at Home Being Cheaper is Weird
I like the argument that the scaling should make the average marginal cost per plate lower in restaurants than at home, but I find cooking at home being cheaper not weird at all. First, there are also real fixed costs to account for, not just regulatory costs.
More importantly, the average price per plate is not just a function of costs, it’s a function of the value that people receive. Cooking at home does give some nice benefits, but eating out gives some huge ones: essentially leisure, time savings (a lot of things get prepped before service), no dishes, and possibly lower search costs (“what’s for dinner tonight?”).

Daniel V Oct 13, 2024, 1:50 PM
1 point
0
on: Prices are Bounties
A classic that seemingly will have to be reargued til the end of time. Other allocation methods are not clearly more egalitarian and are less efficient (depends on the correlation matrix of WTP, need, time budget, etc., plus one’s own judgment of fairness, but money prices come out looking great a lot of the time). In some cases, even prices don’t perform great (addressed in some comments on this post), but they’re better than the alternatives.

For more reading: https://www.lesswrong.com/posts/gNodQGNoPDjztasbh/lies-damn-lies-and-fabricated-options?commentId=nG2X7x3n55cb3p7yB

Daniel V Jul 13, 2024, 12:34 PM
3 points
0
on: Robin Hanson AI X-Risk Debate — Highlights and Analysis
To get Robin worried about AI doom, I’d need to convince him that there’s a different metric he needs to be tracking
That, or explain the factors/why the Robin should update his timeline for AI/computer automation taking “most” of the jobs.
AI Doom Scenario
Robin’s take here strikes me both as an uncooperative thought-experiment participant and as a decently considered position. It’s like he hasn’t actually skimmed the top doom scenarios discussed in this space (and that’s coming from me...someone who has probably thought less about this space than Robin) (also see his equating corporations with superintelligence—he’s not keyed into the doomer use of the term and not paying attention to the range of values it could take).
On the other hand, I find there is some affinity with my skepticism of AI doom, with my vibe being it’s in the notion that authorization lines will be important.
On the other other hand, once the authorization bailey is under siege by the superhuman intelligence aspect of the scenario, Robin retreats to the motte that there will be billions of AIs and (I guess unlike humans?) they can’t coordinate. Sure, corporations haven’t taken over the government and there isn’t one world government, but in many cases, tens of millions of people coordinate to form a polity, so why would we assume all AI agents will counteract each other?
It was definitely a fun section and I appreciate Robin making these points, but I’m finding myself about as unassuaged by Robin’s thoughts here as I am by my own.
Robin: We have this abstract conception of what it might eventually become, but we can’t use that abstract conception to do very much now about the problems that might arise. We’ll need to wait until they are realized more.
When talking about doom, I think a pretty natural comparison is nuclear weapon development. And I believe that analogy highlights how much more right Robin is here than doomers might give him credit for. Obviously a lot of abstract thinking and scenario consideration went into developing the atomic bomb, but also a lot of safeguards were developed as they built prototypes and encountered snags. If Robin is so correct that no prototype or abstraction will allow us address safety concerns, so we need to be dealing with the real thing to understand it, then I think a biosafety analogy still helps his point. If you’re dealing with GPT-10 before public release, train it, give it no authorization lines, and train people (plural) studying it to not follow its directions. In line with Robin’s competition views, use GPT-9 agents to help out on assessments if need be. But again, Robin’s perspective here falls flat and is of little assurance if it just devolves into “let it into the wild, then deal with it.”
A great debate and post, thanks!