FICO scores, by far the most popular credit score system, seem to be set up in a strange way. Their purpose is to measure the quality of a loan application, and yet their methodology seems quite suboptimal to do that.
From Patrick Mackenzie (Bits About Money):
FICO scores are unreasonably effective. Many, many, many teams have thought “I bet I can get better loss rates if I supplement FICO scores with another data source”, and just about the only data sources for which that is actually true are illegal to use.
And yet, if you look at the calculation of these scores, it basically is this:
Payment history (35%)
Amount owed (30%)
Length of credit history (15%)
New credit (10%)
Credit mix (10%)
Basically, this seems to just be
Have they paid loans back in the past (Payment History)
Will they pay loans back in the future
Will they be unable to pay their loans back in the future (Amount owed, Length of credit history)
Are they getting a lot of credit with the intention to not pay it back (Amount owed, Length of credit history, New credit, Credit mix)
1 is obvious, and likely why it is a ~third of the score. However, this is clearly not unreasonably effective.
2a seems less effective than a bank worker directly looking at a customer’s financial situation: these factors are basically a way to check someone’s financial health without access to their income, as that is much messier.
2b seems to not be important enough to make up a significant portion of the score. The risk from credit not intended to be repaid is separate from risk accounted for via past loan delinquency base rates and future changes in financial situations, mostly as a separate, rare-but-consequential event. I don’t think that adding the two tells you a lot about the person.
I think the most probable answers are the top-level bullets below, from most to least likely:
These factors are finding something else
The information about these scores is wrong
FICO scores are calculated in a way other than what you would expect (adding the factors together, weighted by percent above and mapping to values between 300-850)
FICO scores are less effective than I believe they are
FICO scores measure something more intrinsic about a person (like general trustworthiness)
Why does credit age go into account here?
Is this just a legal way to account for older people being more trustworthy/having more to lose?
What about people goodharting on these
Is doing that actually evidence that they are trustworthy/more likely to repay loans?
Is 2b a way to counteract goodharting?
People still seem to be able to goodhart credit scores, however.
The methods used for this might only work on some portions, and goodharting could prove that you aren’t untrustworthy because of a negative in this
Example: Credit Mix could mostly just be a way to measure how well the person understands credit, and goodharting your credit score shows you understand credit
FICO scores just work by getting easy access to a lot of data
Why are they useful? Creditors already use a lot of data to make decisions about lending.
Essentially, FICO scores do not seem to be made with a special process. How can they be especially good data?
You ask “Are FICO scores effective?” but to answer that you need to ask a further question, “Effective at what?”.
The purpose of a FICO score is not to tell you something about a person. The purpose of a FICO score is to not lose money.
If I’m considering lending you $100 for 1 year at 10% interest, the (simplified) outcomes are:
You pay me back and I make $10.
You don’t pay me back and I lose $100.
An important consequence of this is that I care a lot about the case where you don’t pay me back, even if it’s rare. If you pay me back 85% of the time, I still lose money.
So, I might use credit scores for less important things like determining interest rates, but the most important decision to make is “Do I offer you a loan at all?”.
With credit cards this is even harder since a typical customer doesn’t use their full balance (and may not pay interest at all), while nearly bankrupt customers will use as much of their balance as they can. If your average customer pays you 2% in interchange fees and uses 10% of their balance and your worst customers cost you 100% of their balance, even 1⁄500 customers not paying you back is a problem.
So, keeping that in mind, we can look at the pieces again:
Payment history is obvious. If you don’t pay other people, you probably won’t pay me. Yes, this is “only” 35% of the score, but 850 x (1 − 35%) = 550. No one will give you a credit card with a FICO score of 550.
Amount owed and new credit is a signal that you’re about to go bankrupt. Yes, this doesn’t tell me much about the person and whether they’re the kind of person who usually pays people back, but it does tell me that I shouldn’t lend them money right now.
Length of credit history matters because a short credit history prevents lenders from using any other metric to determine risk. Losing money is the default, so you’re guilty until proven innocent.
I’m not really sure on credit mix, but the fact that it’s only 10% means it will basically never be the reason you do or don’t get a loan (unless you’re already borderline for some other reason) but it might effect rates. I assume part of this is reducing fraud risk: If you’ve successfully convinced someone to give you a mortgage, you’re probably a real person).
One other part of this is that while the factors are weighted in the way you mention, the factors are not calculated in a straightforward way. For example, amount owed is 30% of your score, but that doesn’t mean that reducing the amount you owe from 50% to 0% improves your credit score by 15%. Each factor is calculated as “Looking at your X, how risky does that make you?”
For length of history, that means a history of <1 year is insanely risky[1], while any history above 5 years is basically the same. Or for amounts owed, anything under 30% is low risk, 80% is getting up there, and 99% is insanely risky.
Even for something that sounds straightforward like credit mix, it’s not necessarily the case that only having one credit account means you get a zero on that factor.
So all of that together:
Credit scores measure something less intrinsic about a person. Lenders care if they’re going to lose money, not if you’re a good person.
Losses per default are much higher than average profit per customer, so it’s expected that the filtering process will look “too strict”.
Credit age matters because a short credit age prevents lenders from using any other signals, and the people are high risk until proven otherwise.
You can try to game your credit score, but lenders don’t care about high credit scores (it doesn’t matter if your credit score is above 780), and gaming won’t save you if any individual metric is bad enough (no one will lend to you at all below 580).
A lot of the magic is how each category is calculated, not the high-level weights.
Source: I made it up.
I’m confused as to why you are confused.
You say “FICO scores do not seem to be made with a special process. How can they be especially good data?” The score was presumably designed for the purpose of assessing risk. Per Wikipedia, FICO scores were introduced for that purpose based on credit reports. I highly doubt that the Fair Issac company did that with “no special process.” Most likely, a data analysis was done where many different possibly relevant data points were pulled from credit reports and then analyzed to see which would be predictive. I imagine the weighting between the factors was optimized for that purpose.
Your explanation of how the various factors are relevant to creditworthiness mostly makes sense. The best predictor of future behavior is past behavior. I do think that for the majority of people a FICO score is getting at a mix of trustworthiness and stability. If someone has a long track record of paying their debts and nothing has changed in their financial situation, then there is a high probability they will continue to pay them.
Keep in mind also that credit cards and many loans also ask for your income with legal consequences if you lie. Combine FICO score with sufficient income and you have eliminated a large risk factor that may not show up yet on a credit report (i.e. loss of employment).
The only factor that has always seemed odd to me is credit mix. Indeed, Wikipedia has a section in its Criticism of credit scoring systems in the United States page about Poor predictor of risk, which is mostly about how a mix of credit can be misleading. For example, I have no installment loans because I don’t own a house, and my car is paid off. Does that make me a worse credit risk? No, in fact, it makes me a better risk because I have lower expenses and so much savings that I always buy cars with cash. The best theory I can come up with is that not having a mix of credit types is predictive of being low-income, since low-income people are less likely to own a home and more likely to own used cars that they don’t have a loan on, or to not even own a car.