I mean this is a summary of a talk I didn’t see so I want to reserve some judgement. But at the same time I just can’t imagine being in a situation where I don’t want to use the word signal because the other person might think I’m talking about cell signal, so instead I bust out “kodo”
In general, I think we should be pretty wary of taking basic ideas and dressing them up in fancy words. It serves no purpose other than in-group signaling.
That makes sense. However, I do see this as being meaningfully different from evals or mech interp in that to make progress you really kind of want access to frontier models and lots of compute/tooling, so for individuals who want to prioritize this approach it might make sense to try to join a safety team at a lab first.
How are these ideas different from “signal” and “noise”?
Do you think building a new organization around this work would be more effective than implementing these ideas at a major lab?
I think this might be a result of o-series being trained in a non-chat setup for most of the CoT RL phase and then being hamfistedly finetuned right at the end so it can go into ChatGPT, which just makes them kind of bad at chat and so o3 gets confused when the conversation has a lot of turns. Retraining it to be good at multi-turn chat with separate reasoning traces would probably just be super expensive and not worth the squeeze. (this is just a guess)
No, that is not the definition of a prior. There are priors which imply an expected number of buses, and priors that don’t. If you select a prior that doesn’t, you can still get a meaningful posterior distribution even if that posterior distribution doesn’t have a real-valued mean.
No, that’s not the posterior distribution—clearly, the number of buses cannot be lower than 1546, but that distribution has material probability mass on low integers. I’m not quite sure how you got that equation.
But regardless, I think this shows where we disagree. That prior has mean 2… that’s a pretty strong assumption about the distribution of n. If you want to avoid that kind of assumption, you can get posterior distributions but not a posterior expectation.
I’m not disagreeing with that categorically—for many priors the posterior distribution is well defined. But all of those priors carry information (in the information theoretical sense) about the number of buses. If you have an uninformative reference prior, your posterior distribution does not have a mean.
You can see the sketch of this proof if you consider the likelihoods of seeing the bus for any given n. If there are 1546 buses, there was a 1/1546 chance you saw this one. If there were 1547, there was a 1/1547 chance you saw this one. This is the harmonic series, which diverges. That divergence is the fundamental issue that’s going to cause the mean to be undefined.
You can’t make claims about the posterior without setting at least some conditions on what your prior is—obviously, for some priors the posterior expectation is well-defined. (Trivially, if I already think n=2000 with probability 1, I will still think that after seeing the bus.) But I claim that all such priors make assumptions about the distribution of the possible number of buses. In the uninformative case, your posterior distribution is well-defined (as I said, it’s a truncated zeta distribution) but it does not have a finite mean.
The intuition is that if we both saw bus 1546, and you guessed that there were 1546 buses and I guessed that there were 1547, you would be a little more likely to be correct but I would almost certainly be closer to the real number.
The Bayesian update isn’t generally well-defined because you get a divergent mean. Your implicit prior is 1/n which is an improper prior. This is fine for deriving a posterior median, which in this case happens to be about 3,100 buses, and a posterior distribution, which in this case is a truncated zeta distribution with s=2 and k=1546. But the posterior mean does not exist.
Yeah, I was a little hand wavy with that, sorry. You’re right about Sharpe scaling with (the square root of) number of trades, but that gets dampened a lot by pairwise correlation between trades and correlated leverage/financing costs. I worked at a hedge fund that had a lot of siloed strategies (“pods”) and each one targeted a Sharpe of ~2, but then they would get sliced and diced into end products for investors with different risk levels etc. (I never worked on that side of the firm and tbh do not know all the details of how that works).
In practice, you actually care about incremental Sharpe, which you get by subtracting the Sharpe of your strategy minus the covariance-adjusted Sharpe of your existing portfolio over the norm of the covariance matrix, and you want to keep that pro-forma portfolio Sharpe above 2. It would be awesome to have 100 uncorrelated bets each with a 2 Sharpe, but in practice our strategy was so specific that a lot of our trades were still pretty correlated and created incremental transaction and hedging costs. And beyond being directly correlated (if position A goes up position B will also go up), the structure of a pod means you introduce auto-correlation (if our financing rate goes up all of our positions will have reduced Sharpe, or if one position blows out we will have to exit multiple other trades suboptimally) so you end up getting pretty high covariance matrix norms. The cutoff for a standalone position is maybe like 1.2-1.5 Sharpe, if that position is small enough relative to the overall portfolio that you can just think about marginal Sharpe as a first-order effect. So a little lower but still pretty non-trivial for an individual investor picking single stocks to achieve, and in the case of this discussion—a pretty concentrated, highly auto-correlated, long-only equity portfolio—I don’t think it’s unreasonable to just round off individual thresholds to 2.0 even if it’s more like 1.8 or 1.9.
Just to clarify, guessing that there are 1546 buses maximizes the probability that you are exactly correct, but it does not minimize your expected error, since you are guessing close to many numbers (everything below 1546) that are impossible. This is known in statistics as the “German tank problem”[1] and the posterior distribution is actually not well-defined in many setups.
From WW2 soldiers trying to estimate enemies’ manufacturing capacity based on tank serial numbers
This is simply false. See e.g. https://en.wikipedia.org/wiki/List_of_countries_by_trade-to-GDP_ratio
The country with the highest GDP-adjusted export/import ratio is… Gabon. Then Qatar, Bermuda, Cambodia, Turkmenistan, and Libya. Norway is sandwiched between Congo and Azerbaijan. Zimbabwe ekes out a lead over the US. Luxembourg, the country with the highest GDP per capita, is right below Kuwait.
You are spreading misinformation. The impact of trade balances on GDP growth is an incredibly controversial topic among academic economists.
If you’re interested in this problem you should check out this preprint, written by a former colleague of mine:
https://bpb-us-e1.wpmucdn.com/sites.northwestern.edu/dist/1/2465/files/2025/02/Price-gouging.pdf
It gives a nice economic model of price gouging, elasticity, and inequality, and presents a rigorous argument for why the typical utilitarian welfare theorems do not hold in the price gouging setting.
This is incorrect. NOPAT refers to a company’s cash flows before mandatory debt repayments, recurring capex, and interest expenses. It’s used to compare companies without considering differences in capital structure; it is not a good measure of how much the company can afford to pay in dividends. Many companies that file for bankruptcy have positive NOPAT and yet strictly 0 equity value. You’re thinking of levered free cash flow.
We don’t know what the value/quantity curve looks like, or how it will change if demand changes.
This is untrue. Order book data is available for the right price (and although it doesn’t capture all capital flow, sophisticated algorithms can infer the full curves with high accuracy). For what it’s worth, many finance firms view the market price as basically noise and use order book curves as the “real” price input. Most contracts that depend on equity value will calculate it by VWAP over the last 30 or 90 trading days, which is a much better estimate of true value.
If you believe that market capitalization is meaningful, it is probably because of that conceptual error, combined with social validation. Since it is a common error, it seems correct. There are many other examples of common errors due to fallacies. People create language games that seem meaningful because people play them.
I think this post in general lacks a great deal of nuance and understanding, which would normally be fine except that you adopted such an arrogant tone that it was frustrating to read. I promise people in finance understand that the market cap isn’t literally how many dollars it would cost to buy all of the shares. Even Google’s AI summarization says that. That doesn’t make it a useless metric, and market cap remains a very important input for many processes in trading.
People think of the price metaphorically as a physical property, such as mass.
Who are these people, and where can I bet against them?
Because we don’t only care about the net effect of trade policy, we also care about other factors like the distribution of benefits and the unemployment rate. Especially during a recession or periods of high joblessness and income inequality, subsidizing exports may be necessary to stimulate economic activity among a particular demographic or in a specific region. And during economic downturns, a stronger domestic currency is not always a good thing. I’ll just quote Mankiw here, discussing export subsidies during a depression:
A weaker dollar means that our goods are cheaper relative to foreign goods. That stimulates our exports and reduces our imports. Higher net exports raise domestic production and employment. Foreign goods are more expensive, but more Americans are working. Given the desperate need for jobs, on net we are almost surely better off with a weaker dollar for a while.
This is simply incorrect, (both your and Eliezer’s explanations) and shows a misunderstanding of basic macroeconomics that would be taught at an intro-level university course.
When countries run a trade surplus—positive net exports—there mechanically must be a net inflow of capital. When I export goods and services, foreign buyers need to exchange their currency for the domestic currency to purchase those exports. This causes the domestic currency to appreciate, which puts downward pressure on domestic real interest rates relative to other countries. Businesses in the exporter’s country are thus able to invest more heavily in capital goods, raising their productivity and increasing long-term GDP growth.
For a thorough treatment, Mankiw’s textbook is a good resource: http://students.aiu.edu/submissions/profiles/resources/onlineBook/T9D9B4_Principles_of_Economics-_7th_Edition.pdf
This mechanism is covered in Chapter 31.
Are you including models that are only used by their creator firm? I work as an ML researcher in big tech (I want to keep this account anon, but it’s one of MSFT/OAI, DM, Meta, Anthropic, xAI) and have access to tooling substantially better than what’s commercially available (proof by existence?), but that’s not really what my post is about. My main model for this actually panning out is something like:
Big tech company has control over AI lab
AI lab makes cracked SWE agent
Big tech company notices that releasing that SWE agent will undermine the rest of their software development business, so instead of licensing it out they only make it available to their own staff and perhaps business allies
I’m just clarifying because it’s barely even confidential information that engineers at AI labs have better models than engineers at small or mid-size tech firms, and I want to check what you’re actually betting on.
I think even if the model wasn’t specifically trained for geolocation, it’s a reasonable assumption that metadata from photos (which often includes geo data) somehow gets passed to the models and this created a huge annotated dataset of (geo, photo) pairs during training for stuff like searching Google images.