Here are two stories about stocks that I find hard to reconcile:
Stock prices represent the market’s best guess at a stock’s future price.
Overall, stock prices tend to go up due to advances in technology.
But if stock prices tend to go up due to technology, why isn’t that already priced in?
This seems relevant to investment strategy:
The more true #1 is, the less you should expect to beat the market over the long term: lots of other people with skin in the game have worked hard to produce that best guess.
The more true #2 is, the more you might expect to beat the market over the long term: you only need to know which technologies are likely to grow, you don’t need to know better than everyone else.
As an example: if you think AI is going to accomplish big thing X, and you think the market already knows this, should you buy and hold relevant AI company stock? Or would you expect the anticipated growth to already be priced in?
Some potential answers I can think of:
The market is continually surprised by the pace of technology. (This seems unlikely to me.)
Significant technology gains are realized in private companies. Stock market indicators go up when these companies go public, but the gains are only realized by the private investors. (But we could then reframe the question as “why do index funds go up” and hit the same issues.)
Time-discounting: the market expects prices to go up, but they would rather have their money now than wait for the return and deal with volatility. As the technological gains are realized, the market expects the price to increase; individuals just can’t benefit from that given their (aggregate) preferences. Knowledge of which technologies are likely to grow is already factored in, to the point where expected growth is distributed roughly evenly across all public companies. So story #1 is false, but to beat the market, you still have to know better than everyone else. (My current leading candidate.)
Either story #1 or story #2 is false in some other way.
Question: What are good ways to think about this? What evidence do we have?
EDIT—Summary of things I got from answers/comments:
ike points out that “why stocks go up” is at least partially an open problem known as the “equity premium puzzle”. https://www.lesswrong.com/posts/4vcTYhA2X99aGaGHG/why-do-stocks-go-up?commentId=XtR6rrdTnXJ5aJyJ6
Vladimir_Nesov suggests that as the real cost of goods goes down, inflation targeting causes other things (including stock prices) to go up in nominal price. So technology uniformly increases stock prices by dropping the real cost of goods. https://www.lesswrong.com/posts/4vcTYhA2X99aGaGHG/why-do-stocks-go-up?commentId=92kj4rYCKgFWTG2G6
lsusr (and others) suggest that stocks going up has to do with risk. I’m still pretty confused about this one. If stock prices move to be what they need to be in order to match risk with returns, but they also move to match some measure of a stock’s actual value (in terms of dividends or otherwise), then how do those dynamics combine? https://www.lesswrong.com/posts/4vcTYhA2X99aGaGHG/why-do-stocks-go-up?commentId=BTQApZJJsGtBSHxfx
Dagon points out the “greater fool theory” that stock prices might not reflect real value, and go up due to the shared expectation that they’ll go up. https://www.lesswrong.com/posts/4vcTYhA2X99aGaGHG/why-do-stocks-go-up?commentId=GAzHsKrx49XXfB5gD
Stock prices represent the market’s best guess at a stock’s future time-discounted price. Stock prices go up even in the absence of technological advancement because stocks are tied to the bond market via arbitrage.
But that is a complicated abstract idea. Let’s examine things in simple Marxist terms.
A business is a machine that extracts rents from the proletariat. A stock is partial ownership of a business. Traditionally, the business extracted money and then distributed that money to shareholders in the form of dividends. This is called a dividend stock. In principle, a dividend stock does not go up in value. You just extract rents until the Revolution.
However, publicly-traded companies often reinvest profits into themselves instead of distributing revenues to shareholders as dividends. This is called a growth stock. If a company reinvests profits into itself then the total value of the company goes up because additional capital has been invested into it. Since you own an undiluted share of the company, the value of your stock goes up in equal proportion to the growth of the company.
For growth stocks, why is the expected future growth not already priced in? If I know the company will be re-investing into future growth later, why not invest now?
There may be uncertainty, but if stocks on average trend upwards, doesn’t it mean that the market continuously underestimates the amount that companies will re-invest?
If I’m understanding correctly, you’re suggesting they should go up at least at the bond market nominal rate—but they tend to go up much faster than that? I didn’t fully read through the argument, so I might be misunderstanding.
Yes. It is necessary to disentangle two separate ideas. The first idea is time-discounting. Time-discounting plus arbitrage means that securities grow at no less than the bond market risk-free rate. The price of a stock right now doesn’t reflected future growth. It represents future growth time-discounted by the bond market.
The second idea is risk-adjustment. A 100% chance at $1 million is more valuable than a 1% chance at $100 million. If two securities have equal risk-neural expected value then the security with lower volatility has higher risk-adjusted expected value. If stocks tend to have higher volatility than bonds then an efficient risk-adjusted market ought to price stocks lower than bonds for a given expected (average) growth rate.
The value of capital invested in the bond market goes up via reverse time-discounting. The value of capital invested in the stock market goes up via reverse time-discounting and because you have purchased additional average growth by tolerating higher volatility.
The opposite, right?
Fixed. Thanks.
Ok, so the argument would go:
Stocks’ expected values (in terms of time-discounted dividends or similar) have volatility, from things like business decisions, technology development, and capital re-investment.
A stocks expected value serves as an anchor point for its current price (kind of?)
A stock’s price will change around that anchor until it has expected returns that justify the volatility risk.
Thus a stock price will increase when either its expected value goes up, or its volatility goes down (without changing expected value).
In an efficient market, knowledge that changes your expected value of the stock is probably already priced in, but you can still capture the gains due to volatility.
Does that seem in line with what you’re saying?
I’m not certain I understand this bullet point. Can you explain what you mean?
[Fixed.]
Typo: “volatility goes up” → “volatility goes down”Fixed.
Rephrase attempt: If you were to buy and hold a company’s stock, and you don’t expect to know better than the market, then your anticipated gains over time are independent of what you think the company’s underlying value will do (since the market has already priced that in). But you should still anticipate gains over time due to volatility’s effect on pricing.
Yes. Your understanding is in line with the idea of risk-adjustment.
Question: say I have a company whose underlying value is volatile, but whose expected underlying value after any time span is the same as today’s value. Both of the above arguments seem to suggest I should expect the company’s price to increase over time, but wouldn’t this unanchor the company’s price from its underlying value?
Is the company a growth stock or a dividends stock?
What do you mean by “underlying value”?
Does this question concern risk-adjusted price or non-risk-adjusted price? Does it concern time-discounted price or non-time-discounted-price?
I’d be interested for both 100% growth and 100% dividend stocks—I’m not sure why they’d behave differently w.r.t. to this.
By underlying value, I’m not sure. Something like the real dollar value of all of its capital—or the real dollar price someone would pay to own it in its entirety?
By price, I mean what you can buy the stock for on the market.
Let me try clarifying: The volatility argument seems formal rather than empirical, so I’m wondering what we formally need to assume to make it go through.
I’d summarize the argument as “since stock prices are volatile, they’re expected to go up over time (more than the risk-free rate)”. But then why are stock prices volatile? I assumed that they’re volatile due to “underlying value” being volatile/hard to predict.
So my hypothetical is a company whose “underlying value” is hard to predict, but where the expectation of its “underlying value” is constant over time. To make it easier, assume the company is a magic vault that currently contains $10 million real dollars, and the money will undergo a stochastic process with the given property, and everyone knows this. Maybe it will disperse it’s full value as a dividend at some random future point.
It seems obvious to me I shouldn’t expect this company’s price to go up faster than the risk free rate, yet the volatility argument seems to apply to it. So I’m trying to identify what I’m missing.
You should, because the company’s current value will be lower than $10 million due to the risk. Your total return over time will be positive, while the return for a similar company that never varies will be 0 (or the interest rate if nonzero).
I agree the company’s current price should be lower than $10 million. But if it starts at price P and I expect it to go up at the risk free rate r, then at a time T later the company’s price should be PrT in expectation. At some point, that’ll be substantially more than the $10 million I expect it to pay out.
Well there’s some probability of it paying out before then.
If the magic value is a martingale, and the payout timing is given by a poisson process then the stock price should remain a constant discount off of the magic value. You will gain on average by holding the stock until the payout, but won’t gain in expectation by buying and selling the stock.
Let’s ignore risk. Suppose the company has a market value of P right now at time T0=0, you expect the vault to open at time ΔT, and the bond rate (which equals the equities rate because this is a risk-free thought experiment) equals r. Then the value of the company at time ΔT is 10 million dollars. If the value of the company at time ΔT is 10 million dollars then the market value (price) of the company right now is 10 million dollars times r−ΔT.
Suppose vaults are a fungible liquid securitized asset and that you can buy fractions of them. Suppose you invest in these vaults. Whenever a vault opens, you immediately invest your 10 million dollars cash in more vaults. Your investment grows at a rate r, exactly equal to the bond rate.
I think what you’re missing is that whenever a vault opens you immediately reinvest the cash. The vault has different time-adjusted value depending on when it opens. On a long enough time horizon, $10 million now is worth more than $10 billion later.
Yes, the volatility argument is formal rather than empirical. Whether it actually exists in practice is dubious. Fortunately, this does not affect the core issue of the subject at hand. This discussion concerns the theoretical market. For the volatility argument to apply, all we have to assume is an efficient market, rational actors and the Law of Diminishing Returns.
By “real” do you mean physical dollars or “real value”? In this answer, I ignore inflation and treat the vault as if it contains physical dollars.
We can build a physical system that replicates the effect of your magic vault. Suppose there is a vault tied whose lock mechanism is connected to a radioactive isotope. Each second there is a small chance the isotope will decay and the vault will open and the owner will receive $10 million cash. Each second, there is a large chance the isotope will decay and the vault will remain shut. Radioactive decay is a stocastic process. Therefore if the vault remains shut then the price of the vault remains at a constant price less than $10 million.
At every instant there is a small chance Schrödinger’s vault will open and a large chance the vault will stay shut. In the quantum future where the vault stays shut you are correct and the vault’s nominal market value stays constant. The time-discounted price of a closed vault actually goes down it it stays shut.
It’s not the price of the closed vault that goes up faster than time-discounted non-risk-adjusted value. It’s the average time-discounted risk-adjusted probability-weighted price of all possible future vault states (open and closed) that goes up faster than the time-discounted non-risk-adjusted value of the initial closed vault.
The equity premium puzzle is still unsolved. The answer to your question is that nobody knows the answer. Stocks shouldn’t have gone up historically, none of our current theories are capable of explaining why stocks did go up. Equivalently, stocks were massively underpriced over the last century or so and nobody knows why.
If you don’t know why something was mispriced in the past, you should be very careful about asserting that it will or won’t continue to be mispriced in the future.
The classic answer is risk. Stocks are riskier than bonds, so they should be underpriced (and therefore have higher returns) than bonds.
But we know how risky stocks have been, historically. We can calculate how much higher a return that level of risk should lead to, under plausible risk tolerances. The equity premium puzzle is that the observed returns on stocks is significantly higher than this.
Read through the wikipedia page on the equity premium puzzle. It’s good.
Why shouldn’t have stocks gone up historically? Isn’t there more real wealth today than during the days of the East India Company? If a stock represents a piece of a businesses, and those businesses now have more real wealth today than 300 years ago, why shouldn’t stock returns be quite positive? To be honest I’m bewildered by this perspective that stocks should never go up—I’ve never seen anyone in finance academically or professionally entertain this idea, so I’m surprised to hear you say “no one knows the answer,” as if it’s a common puzzle in the field. Why some stocks go up more than others is a good and open question, but that’s one of relative valuation, not whether market returns on an absolute are greater than zero.
Start with https://en.wikipedia.org/wiki/Equity_premium_puzzle. There’s plenty of academic sources there.
People have grown accustomed to there being an equity premium to the extent that there’s a default assumption that it’ll just continue forever despite nobody knowing why it existed in the past.
>Isn’t there more real wealth today than during the days of the East India Company? If a stock represents a piece of a businesses, and those businesses now have more real wealth today than 300 years ago, why shouldn’t stock returns be quite positive?
I simplified a bit above. What’s unexplained is the excess return of stocks over risk-free bonds. When there’s more real wealth in the future, the risk free rate is higher. Stock returns would end up slightly above the risk-free rate because they’re riskier. The puzzle is that stock returns are way, way higher than the risk-free rate and this isn’t plausibly explained by their riskiness.
The equity risk premium puzzle is about a very different question, no? The equity premium puzzle is a puzzle because of how large the equity premium is to bonds, but it is totally expected that there is an equity premium or that bond and equity returns are positive. Concretely, the puzzle asks, why do equities earn 8% if the risk-free rate is 2%, instead of 3%? The puzzle is that 6% spread is much larger than what standard financial theory predicts (although a lot of the dispute ends up being around the right parameterization and benchmarks). But standard financial does predict (a) an equity premium to the risk-free rate and (b) positive returns for both equities and bonds.
I think you get to that understanding in your last sentence (“stock returns are way, way higher than the risk-free rate”), but this is a much different statement than “stocks shouldn’t have gone up historically” or “people have grown accustomed to there being an equity premium to the extent that there’s a default assumption that it’ll just continue forever despite nobody knowing why it existed in the past.” I don’t want to be a pedant, but those are much different statements, because they state that the absolute return on stocks should be 0, and that the positive return of equities is some puzzle. Maybe I’m misunderstanding you, but the straightforward interpretation of what you were saying is quite bold, and not something I’ve heard of before or have seen on wikipedia (including what you linked).
The vast majority of the equity premium is unexplained. When people say “just buy stocks and hold for a long period and you’ll make 10% a year”, they’re asserting that the unexplained equity premium will persist, and I have a problem with that assumption.
I tried to clarify this in my first reply. You should interpret it as saying that stocks were massively undervalued and shouldn’t have gone up significantly more than bonds. I was trying to explain and didn’t want to include too many caveats, instead leaving them for the replies.
It’s interesting to note that several other replies gave the simplistic risk response without the caveat that risk can only explain a small minority of the premium.
>The vast majority of the equity premium is unexplained. When people say “just buy stocks and hold for a long period and you’ll make 10% a year”, they’re asserting that the unexplained equity premium will persist, and I have a problem with that assumption.
I completely agree with you here. My point is that this comment is different from the plain language interpretation of your top post. I know that is a seemingly small point, but I commented because I don’t want to leave future readers with the wrong impression. If we did not have this conversation, I think they may have been left thinking that it is a puzzle that stocks go up at all. I don’t view that as a simple caveat to the initial statement, but a different statement entirely.
Can we agree on the following statement? “While it is expected that stocks will go up, and go up more than bonds, it is yet to be explained why they have gone up so much more than bonds.”
Did you see my initial reply at https://www.lesswrong.com/posts/4vcTYhA2X99aGaGHG/why-do-stocks-go-up?commentId=wBEnBKqqB7TRXya8N which was left before you replied to me at all? I thought that added sufficient caveats.
>”While it is expected that stocks will go up, and go up more than bonds, it is yet to be explained why they have gone up so much more than bonds.”
Yeah, I’d emphasize slightly more in expectation.
Suppose at some point there is an announcement that in ten years Free Hardware Foundation will release a magical nanofactory that turns dirt into most things currently in the basket of goods used to calculate inflation. There is no doubt about truth of the announcement. No company directly profits from the machine, as it’s free (libre) hardware.
There’s some upheaval in the market, that eventually settles down. Yet real value of stock is predictably going to sharply go up after ten years, not (just) immediately, as that’s when the basket of goods actually becomes cheaper.
You may have to hold my hand on this one: I can agree the value of the stock (in the time-discounted future dividends sense) will go up after 10 years due to time-discounting—the technology would enable value production that “comes into scope” as it gets closer in time.
But is there any reason other than time-discounting that the PRICE won’t go up immediately? For instance, if I expect the time-discounted dividend value of the stock is $50 today and will be $5,000 ten years from now, and the rest of the market prices it at $50 today, then I could earn insane expected returns by investing at $50 today. Thus, I don’t think the market would price it at $50 today.
Everyone gets the insane nominal returns after ten years are up (assuming central banks target inflation), but after the initial upheaval at the time of the announcement there is no stock that gives more insane returns than other stocks, there are no arbitrage trades to drive the price up immediately. For nominal prices of stocks, what happens in ten years is going to look like significant devaluation of currency.
If a $5,000 free design car (that’s the only thing in our consumer busket) can suddenly be printed out of dirt for $50, and central banks target inflation, they are going to essentially redefine the old $50 to read “$5,000”, so that the car continues to cost $5,000 despite the nanofactory. At the same time, $5,000 in a stock becomes $500,000.
(Of course this is a hopeless caricature intended to highlight the argument, not even predict what happens in the ridiculous thought experiment. Things closer to reality involve much smaller gradual changes.)
Hmm, but what if everything gets easier to produce at a similar rate as the consumer basket? Won’t the prices remain unaffected then?
This makes sense!
Do you know anything about the state of evidence re: to what extent this is happening and/or driving stock returns? I’m not sure how you’d pick this apart from other causes of currency devaluation.
I’m not talking about time-discounting at all. The point is that real value of stock (and money) is defined with respect to a busket of consumer goods, and that’s the only thing that isn’t being priced-in in advance, it’s always recalculated at present time. As it becomes objectively easier to make the things people consume, real value of everything else (including total return indices of stocks) increases, by definition of real value. It doesn’t increase in advance, as valuation of the goods is not performed in advance to define consumer price index.
If I can summarize your question as something like “can I beat the returns on an index fund by only investing in companies with new/useful technologies”, I think you’ll find this question is similar to “which version of the EMH is true”, and you’ll also find a lot of good discussions about this, for example here: https://www.themoneyillusion.com/are-there-any-good-arguments-against-the-emh
For the two stories presented, I would say Story 1 is trivially true and Story 2 is probably false, although it’s not phrased super well (for example, is “advances in technology” company specific or general economic growth?). Trying to read between the lines, it seems like you’re wondering something like “if my choice to invest is between two companies, and Company 1 has current cash flow of $1-million and future cash flow of $1-million (no growth), and Company 2 has current cash flow of zero but future cash flow of $1-trillion (lots of growth), should I always invest in Company 2?” And my answer is “no, unless you think weak EMH is true and there’s a specific inefficiency that you can uncover through your research” ( and even then, your research might determine you should sell rather than buy Company 2).
To improve the framework, I suggest the following distinctions/assumptions:
Some form of EMH (weak, semi strong, strong) is true (I tend to agree with Sumner that anti-EMH models are not useful).
Value vs Price: Value is determined by things like future cash flows, risks, discounts and opportunity costs. Price is determined by supply and demand. I recommend Damodaran for discussions on this distinction, for example http://aswathdamodaran.blogspot.com/2020/03/a-viral-market-meltdown-iii-pricing-or.html
There’s a positive link/correlation between stock price and stock value in the long term. (In the short term price and value might diverge.)
Real vs Nominal: Real value/price is determined by things like goods produced, services provided, technology/productivity growth. Nominal value/price is determined by the above but also the supply and demand of money, inflation, exchange rates. I recommend Sumner for discussions on this distinction, he blogs at Econlog and TheMoneyIllusion.
Also, note that “technology” has a non-colloquial meaning under most economic models. For example, the Solow-Swan growth model says something like long term growth is determined by growth in the labor force and growth in “technology”, where “technology” is basically anything that’s not labor force and includes things like human capital/knowledge accumulation/diffusion and social/political institutions.
Using this framework, you could change your question to something like “assuming weak EMH is true, what sorts of public information about a company’s new/useful technologies would allow me to value a company more accurately than the average investor”. Then you could search for studies that try to answer this question or something similar.
The biggest reason stocks go up is pretty simple, in my view: a lot of very smart and hardworking people are working very hard to make stocks go up. In addition, lots of less smart and less hardworking people are also working to make stocks go up. In contrast, very few people are trying to make stocks go down.
While there are shortsellers, they are generally not trying to make stocks go down, i.e. by destroying value. Instead, shortsellers are simply saying that some companies are overvalued, and time and effort is better spent on other companies.
What do I mean by “make stocks go up”? I mean it in a fundamental sense: employees are trying to create value by making products better or adding new markets. Their rewards include stock compensation, bonuses, and promotions, so they are indeed incentivized to create value in whatever way possible. Meanwhile, financiers are trying to allocate capital (an abstract representation of time, effort, and value) in the best way, by optimizing society’s effort. These people are also rewarded with capital gains, bonuses, and promotions.
Given that there is so much effort and incentive to fundamentally increase the value of stocks, why should the market stay flat or neutral? I’m sure someone can try to find some elaborate corner case why this isn’t true, but mostly the reasons stocks wouldn’t go up is if no one is trying (no incentive) or if someone is trying to destroy asset value (expropriation, violent revolution, etc.). If people are trying to build stuff that makes stock prices go up, and there isn’t much effort to destroy them, it’s pretty straightforward that they go up.
For what it’s worth, this is basically Buffett’s view. Also note, the comments about equity risk premium, valuation puzzles, and the efficient market hypothesis miss the point: that’s only a matter of relative returns, not absolute returns. (If your question is about beating the market, then those points are more relevant, but it’s not clear anyone can beat the market today in developed economies outside narrow pockets of inefficiency.) As long as you can build new stuff, you’ll have a reason to invest, which demands a return, hence the positive returns we see. If there’s less “stuff” to build, you can expect lower prospective returns because capital competes for each opportunity, resulting in lower rates and a huge premium on growth opportunities, which is what we observe today.
One last way to think about it is in real terms. If you have a machine that can build more of itself, you will have more of those machines over time, which is a real return.
One word : risk. There is uncertainty as to the company’s ability to provide that future revenue and as That uncertainty reduces, the stock price goes up.
Is this the same as positing “the market is continually surprised by the pace of technology”?
E.g., say I value company X’s stock at $100. Then I learn a new fact that there’s a 50% independent chance the company will discover a technology that doubles its value by 1 year from today. If I ignore all other factors, my estimate of the company’s value 1 year from today should then be $150. If the company discovers the technology, I’ll value it at $200, and if not, I’ll value it at $100.
For the market to trend upward as it does, it seems like either:
Everyone’s consistently getting those bets wrong, and underestimating how often/how much technology will pay off, OR
There’s something else to the story, like time-discounting or a different way of thinking about risk.
Does this reduction come from seniority? Is the idea that older organizations are generally more reliable?
The reduction comes from the passage of time. Let’s say that a company predicts 10% growth over the year but 6 months into the year they have an equivalent annual growth rate of 2%. That doesn’t mean that they won’t make 10% at the end of the year, but it makes it less likely, so the value of the company changes to reflect that new reality. It’s important to define risk and uncertainty. Risk in this case means probability of winning or loosing something, and can be measured, whereas uncertainty is about the lack of information about a situation and can not be measured as it is unknown. Uncertainty is reduced as more knowledge is gained about reality and as long-term risks become short-term risks, due to the passage of time, only to be replaced with new longer term risks. There are some really interesting concepts around the future value of money, opportunity costs and how to value companies. I’d recommend coursera finance 101 courses.
Don’t forget the Greater Fool theory (https://www.investopedia.com/terms/g/greaterfooltheory.asp). Stocks go up because investors expect them to go up, and those investors’ purchases actually drive the price up.
There is some underlying truth in terms of dividends and stock buybacks (effectively a dividend, but paid by reverse-dilution), but the vast (VAST) majority of invested dollars are fully disconnected from any actual business outcomes, except by the publicity and expectations channel.
I think things become simpler when you look at the sum of all stocks, versus particular ones. Then, you only need to consider the market cap of the entire stock market, and what makes it change over time.
The economy is much bigger than the stock market. Money flows from small companies to larger one as the economy consolidates—since the former are more likely to be publicly traded than the latter, that makes the market become bigger.
It’s easier to invest in the stock market now than in the past. Since it’s accessible to more people, then more people’s money can be put in it. So, the market cap goes up.
Finally, as inequality increases, more of a fraction of wealth is disposable, and therefore can be invested. That makes the market grow as well.
I’m sure there are many other reasons along these lines.
Here’s some motions toward an answer. I’ll consider an informally specified market model, as opposed to a real market. Whether my reasoning applies in real life depends on how much real life resembles the model.
In particular, consider as model an efficient market. Assume the price of any stock X is precisely its expected utility according to all evidence available to the market. Then the only way for the price to go up is if new evidence arrives. This evidence could be the observation that the company associated with the stock continues to exist and produce income, that it continues to produce income, or that the income it produces is is going up.
Those bits of evidence remind me of the risk perspective. Sure, investors may believe that a company, if it continues to exist, will one day be worth more money than they could ever invest today. But if they think there’s a 5% chance each year that the company stops existing, then this can severely limit the expected utility of owning the stock right now. (You might ask, “What if I assign a nonzero probability to the class of futures where the company exists forever and the utility of holding its stock grows without bound?” which makes the expected utility infinite, and makes me suspect that defining utility over infinite spans of time is tricky.)
I feel like the time-discounting hypothesis makes a lot of sense and is probably part of the truth. To make sense of it within my toy model, I’d have to look at what time-discounting actually means in terms of utility. A reasonable assumption within this idealized model seems to be that the “utility” of anything you possess is equal to the maximum expected utility of anything you could do with it / exchange it for, including exchanges over time. (This is like assuming perfect knowledge of everything your could do with your possessions. Utility can never go up under this assumption, similar to how no legal move can improve a chess position in the eyes of a perfect player.) This means the utility of owning a stock is somewhere between the utility of its buy price and its sell price, as expected. And the utility of 1 dollar right now is no less than the expected utility of a dollar in 2030, given that you could just hold on to it. Then time-discounting is just the fact that the utility of 1 dollar right now is no less than the expected utility of buying one dollar’s worth of stock X and waiting a couple years. Let’s assume the stocks grow exponentially in price. That is to say, though neither the stocks nor the money increases in utility, stocks can be exchanged for more and more money over time. It seems converting money into stocks avoids futures where we lose utility. So how much should we pay for one stock? This is just determined by the ratio of the utility of the stock to the utility of a dollar. So the question becomes, why does money have any utility at all, if it is expected to fall in utility compared to stocks? And this must be because it can be exchanged for something of intrinsic utility, such as the enjoyment derived from eating a pizza. But why would someone sell you a pizza, knowing your money will decrease in utility? Because the pizza will decrease in utility even faster unless someone eats it, and the seller has too many pizzas to eat, or wants to buy other food for themself. (Plus, the money is backed by banks / governments / other systems.)
So if the average price of stocks tends to increase year by year, why are they not worth infinite money to begin with? Here’s another perspective. How is the average price calculated? Presumably we’re only averaging over stocks from active companies, ones that have not ceased to exist due to bankruptcy or the like. However, when evaluating the expected utility of a stock, we are averaging over all possibilities, including the possibilities where the company goes bankrupt. There is a nonzero chance that the company of a stock will cease to exist (unpredictably). So if you Kelly bet, you should not invest all your money into such a stock. As a consequence, successively lower prices are required to get you to buy each additional stock of the company.
Reflection: I imagine these concepts match with ideas from economic theory in quite a few places. I have a mathematical background myself, probably making the phrasing of this answer unusual. I’m not so sure about this whole informal, under-specified model I just made. It seems like the kind of thing that easily leads to pseudoscience, while at the same time playing around with inexact rules seems useful in early stages of getting less confused, as a sort of intuition pump. (Making the rules super strict immediately could get you stuck.)
I don’t know the full answer to why stocks go up, but I have a partial answer based on risk. Imagine there are only 2 products available in the market:
(1) A US government bond that pays $100 in 1 year.
(2) Ownership in a company that will dissolve in 1 year, and at the end either return to the owner $95 or $105 with equal probability.
Note that both have the same expected return in 1 year, $100. But people will prefer to buy the first product compared to the second one, since the first one is risk free. Say the current 1-year interest rate for risk free returns is 1%, then people will pay $99 for the first product. But since the second product is less desirable, they might only pay $98 for it. So the second product has greater expected profit, since you paid $98 for $100 of expected returns. If you only invest in products like (2), then in the long run you’ll make more money in expectation compared to investing in risk-free assets like (1).