“Ambitiously” tackling “the biggest problems”, like AI risk, is often actually easier and safer than the alternatives. You get to feel cool and important and you’re bolstered by your EA student group friends who are excited about what you’re doing, and the lack of good feedback loops means that you don’t fail hard and completely and visibly (but learn and grow more).
There’s a sense in which selfish or limited goals like “make a successful dating app” or “make 10 million dollars” or “get 10% more housing units built in my local city” are easier than “reduce the risk of human extinction”. But in practice, the sense in which they’re easier is often fake.
EAs would probably do better to mostly do narrow tractable things that feel real enough that you could __maybe__ actually concretely do them.
If when you consider items above, they feel extremely daunting—if you feel a twinge of fear or resistance to the thought of trying to solve them, if you feel like you wouldn’t even know where to start—consider that “reducing x-risk” is in fact a much thornier and harder problem, where you have much less leverage over the thing you’re trying to change, and dramatically worse ability to tell if you’re making progress.
Actually trying to make your startup succeed (or whatever), for real, feels real and scary, the in a way that it does not feel scary to be aiming for an abstraction like “having impact”.
“Impact” or “reducing x-risk” is too vague and too removed from feedback for anyone, including yourself, to ever really hold you accountable. So even if it’s extremely “ambitious”, it’s ultimately a safe thing to do.
Trying to “the most good you can do” means tackling the “most important problems”, which can allow you to hide from doing anything real or scary.
Actually I think this is somewhat of a myth / greatly exaggerated. You can get good feedback about ideas regarding alignment. It just takes certain efforts that people don’t do. For example, “do maker-breaker to alignment proposals and proofs of impossibility of alignment; generalize each breaking”. That’s good feedback. Philosophical arguments are also good feedback. The problem is hard for other reasons (e.g. we just have huge conceptual gaps). As a comparison, it would be only accurate in a confusing / contorted way, to say that Euclid “lacked good feedback loops” about algebraic topology or whatever. Algebraic topology has good feedback loops, math has good feedback loops, etc.; Euclid just lacks a bunch of concepts needed to even start with algebraic topology.
(If by good you meant highly legible, very inexpensive, very fast, etc., then sure.)
I agree with that, and also suspect that some alignment researchers are sufficiently bad at controlling their own confirmation bias that they never get a good look at the conceptual gaps. To those researchers, the lack of good feedback loops is the one central fact about alignment research. I.e., I suspect that some extremely bright people are 100x or 1000x better at avoiding confirmation bias than other extremely bright people are.
I think I agree in outline, though I’m unsure about describing that as confirmation bias. “Doesn’t try to counterargue one’s own hopes” is one of the main things; I guess that’s a kind of confirmation bias? I guess I’d rather describe it as a big missing skill or practice, but maybe I’m being too restrictive with the word “bias”.
This thought was inspired by reading this Ben Hoffman’s recent post, and more proximally by this Wei Dai shortfrom, which claims that approximately no one exhibits long horizon agency.
You might think that means “almost no one except these EAs and rationalists”. But, since it’s easy to use “optimizing for the long term future” as a cope to hide from accountability, I suggest that many (most?) of the people who appear to be exerting long-horizon agency are using that as a cover and less actually steering the future.
You might think that means “almost no one except these EAs and rationalists”.
I guess I didn’t express this in that shortform, but I meant to include EAs and rationalists in “approximately does not exist”. Examples: “philosophers don’t bother to think about long term implications of AI on philosophy production (positive or negative)” includes philosophers in EA who never talked about this until very recently (with founding of Forethought) and even now talk about this to a much lesser extent than I expect or think they should, and Eliezer making what I think are serious strategic mistakes earlier in his career.
About the “missing mood”, what do you think about this post of mine? I feel like the solution to the problem that we’re talking about isn’t to do less long-horizon agency/strategy, but to do more of it but more tentatively, slowly building up traditions/methodologies/foundations, kind of like how philosophy has progressed over time (but hopefully avoiding the destructive overconfidence of past philosophers). If we simply do it less, then how does civilization ever get to a state where we’re navigating the future with strategic competence?
I’ve been thinking lately about the attractors that develop when a person or community has spent some time thinking on something (enough to grow attached to their thoughts). It takes real effort to just waveat concepts you’ve seen before and then keep driving, whenever something you’re thinking about might roundto something more familiar.
Especially if the conceit of one’s inquiry is ‘I suspect we are very wrong about very fundamental things’, there are many alternatives and objections that lie along the path, which themselves dead-end in unsatisfying places. Walking down each once is fine, but I find it wearying to trod down the same handful of knee jerk off-ramps over and over again, every time I share an idea with a new interlocutor.
The slower feedback loops of academic life seem to help here; once you have some cache, you can more or less hole up and work on your maybe-insane idea for the rest of your life, with very limited accountability. This, of course, creates other problems, the enumeration of which is its own well-worn off-ramp.
I think that the rapid advance of AI is correctly prompting a taking of inventory, a moment of circum/introspection, to get a sense of whether or not we’re ‘ready’, and most people who’ve genuinely engaged with pre-existing literature that attempted to grapple with these possibilities answer that question with a resounding ‘NO’, but we’ve not yet seen a shift to a more butterfly-nurturing temperament, and we’ve not yet reached consensus on quite how far we ought to back up, which makes a lot of people who consider themselves to have backed up wince a bit when someone later says ‘no, further’.
It seems like you’re the person who’s over and over again saying ‘no, further’, and many people just hear the local ‘no’ to their particular idea, struggling to think that there’s any further to back up.
[fwiw, I currently think that we need to back up/zoom out/pass off-ramps, and then uh… keep doing that for a long while]
Humans are not automagically strategic. Failing at long timehorizon strategy is the default for humans because it requires thinking correctly which people fail at by default without feedback loops. No cope required. You can do real scarry things with feedback loops as your full-time job and do the long-term thinking as your hobby and side-hustle. Humans have timehorizons just like AI, but humans are trainable just like AI.
You can actually have a large impact over time by focusing on strategic moves, and also be running from tactical day to day operational accountability, and using the former to cover for the latter.
Should we stop attempting because it’s hard? Or notice cope and cover and correct it?
Not necessarily, but we should maybe be less gung ho about “trying to do the most good”.
It feels like there’s maybe a missing mood to a lot of EA and x-risk related efforts. Probably more people should be more paranoid about the impacts of their actions, and more willing to do ostensibly less ambitious things that are less fake to them?
I think you’re right locally, but I’d limit it to “easier and safer” in one way. I’d say the bigger overall risk by far is doing ambitious, harder to evaluate work.
Your point is valid, but it’s a one-sided argument. So I’m going to fill in a little of the other side.
Funding is a large and fairly obvious countervailing risk steering people away from doing more ambitious, broader work intended to address the whole difficult problem. Funding for safety work has been heavily to severely biased toward technical work in recent years, probably primarily because it’s easier to gauge success in that type of work.
I agree that there can be motivated reasoning for doing work that’s non-technical and therefore not vulnerable to obvious and outright failure. But there are strong arguments for not doing more legible things just because they’re more legible.
It’s not impossible to gauge the success of work aiming at solving big problems, it’s just a bit harder than judging success of narrow technical work.
To state the strong form: we will individually be taking on real, easily measurable challenges and making measurable progress if we all rearrange deck chairs under the streetlight. That doesn’t make it a good idea. Good work needs to have a good chance to actually shift the future. Evaluating whether it does is a large part of the challenge.
Thanks for spelling it out, I did feel smth like this, but only when I thought about “working on AI pause” vs. “working on technical alignment”. A pause is a very specific and real thing, and it requires talking and interacting with real people and existing systems, and we know from real life how easily those can go wrong. When you are “working on technical alignment”, the object you are working with is a future, nonexistent technology, and if you notice problems with your current best guess of how to make it safe, you can just go “huh, we should add this to our list of open problems to fix before we actually build the thing”, and this feels like making progress. But when working on a pause, you can’t just say “huh, our government isn’t optimal, I’ll add ‘fix democracy’ to the list of open problems”, you have to work with the real, imperfect thing.
I guess the question is, is there a class of things that do the most good that have tight feedback loops? Or ways to create such loops for things that don’t? It could be that we live in a world where it’s not so, and we must pick between unclear good + tight feedback, or the most good + poor feedback.
It could be that we live in a world where it’s not so, and we must pick between unclear good + tight feedback, or the most good + poor feedback.
I somewhat dispute the framing here.
If the “most important” problems are intractable because their feedback loops are so bad, then our standards for “most important” are themselves mistaken.
It’s like Richard Hamming saying that in some sense, developing anti-gravity is hugely important problem (if you solved that it would have enormous commercial and humanitarian benefit), it’s importance is a mirage, because there’s no-line of attack on it.
It could be case that there are a bunch of problems that will have huge impacts on many people through the whole future, and also, in practice, it’s a bad bet to work on them, because almost no one can do non-fake work in an unaccountable domain.
If when you consider items above, they feel extremely daunting—if you feel a twinge of fear or resistance to the thought of trying to solve them, if you feel like you wouldn’t even know where to start—consider that “reducing x-risk” is in fact a much thornier and harder problem, where you have much less leverage over the thing you’re trying to change, and dramatically worse ability to tell if you’re making progress.
What if they don’t feel daunting at all to me? They feel quite fun and nice. Getting real world feedback on failure is nice, not bad.
Agreed. Or at least, there are certain psychological factors[1] that make working on abstract alignment problems feel “safer” for some subset of people, myself very much included! On the other hand, these same factors can be “unsafe” to different people or in different contests, and there are plenty of features of other research styles that are psychologically attractive[2].
But I don’t really buy the implied add-on “this is psychologically safer for me, therefore it’s actually bad,” at least not as it applies to myself.
(e.g. lack of feedback loops can mean you don’t feel failure the same way, relatively low competition / professionalization, the real world doesn’t interrupt your research to tell you to change your mind very much)
e.g. you will naturally work with other people, you get to apply the neat math / CS you studied, you get to spend lots of money, your work fits in the normal paper format, etc.
First thought was that it’s something like applause lights. Upvoting your post feels like a really easy thing to do. Yay doing things that are real rather than fake! If you’re trying to challenge the fundamental way a bunch of people around you are approaching things, and the main reaction is people nodding along, that suggests that maybe you didn’t manage to articulate the core critique.
After sleeping on it: Something feels slippery about the relationship between the different parts of your post. Picking out a few particulars:
The things you’ve listed as alternatives aren’t the alternatives that EAs would be doing. They’re things you picked out to optimize for the best degree/flavor of dauntingness, for a subset of people who could do them
Some actual EA alternatives, like donating to AMF, feel very psychologically easy
Making progress on AI x-risk does feel very daunting to many people who consider it. Perhaps overly daunting?
Most EAs who are doing EA-related work are focused on something more specific than “the most good you can do”.
The one specific set of EAs you mention—“EA student group[s]”—are more in learning mode than in doing mode
People doing things in part so they can “feel cool and important” seems like a generic feature of any social group where people can be excited about what other people do. If founding new projects was the path that others saw as good & impressive, then people would be drawn to that to feel cool and important. (Perhaps that couldn’t last if the project didn’t work, at least locally, but the clout from doing vague big important things doesn’t last either, and people could be drawn to projects that do seem like they’re making some kind of local progress.)
The pattern of actions you recommend—choosing a narrower, more tractable target rather than the big ambitious one—resembles a common motivational pitfall that leads to doing faker things. Slipping off the ambitious aim that you really care about because it’s hard to think of what progress would even look like, and instead picking something that is locally easier to track. Perhaps that’s something kinda related to the original goal or it could be something completely different
So maybe the central complaint is more like Bulverism than applause lights. For any specific thing that people might do, there are some ways in which the motivations and social dynamics around it might be screwy.
If you’re just trying to say “here’s one of the ways that sus motivations can manifest when doing thing1” that seems potentially valuable. But if you’re trying to argue for doing thing2 instead of thing1, then it feels like you this post doesn’t show the cognitive work to think through that. Instead it picks out some of the sus motives for thing1 and papers together a narrative around them without thinking through most of the substance of the argument (possible sus motives for thing2, what thing2 might actually wind up looking like, for which specific set of people, other ways of approaching things like thing1 or thing2, the core reasons in favor of thing1 & thing2, etc.).
The closing “(This is mostly advice to myself.)” felt like the most grounding part of it, because it suggests that it’s pointed at a specific larger process of thinking things through.
Bananas implies oranges reasoning. It’s mostly a category error. You maybe have a goal that is based on flawed models of the world, or that isn’t directly actionable or something like that. That doesn’t mean you should pivot to something that is just a commodity/everyone else does it anyway type of thing.
“Ambitiously” tackling “the biggest problems”, like AI risk, is often actually easier and safer than the alternatives. You get to feel cool and important and you’re bolstered by your EA student group friends who are excited about what you’re doing, and the lack of good feedback loops means that you don’t fail hard and completely and visibly (but learn and grow more).
There’s a sense in which selfish or limited goals like “make a successful dating app” or “make 10 million dollars” or “get 10% more housing units built in my local city” are easier than “reduce the risk of human extinction”. But in practice, the sense in which they’re easier is often fake.
EAs would probably do better to mostly do narrow tractable things that feel real enough that you could __maybe__ actually concretely do them.
If when you consider items above, they feel extremely daunting—if you feel a twinge of fear or resistance to the thought of trying to solve them, if you feel like you wouldn’t even know where to start—consider that “reducing x-risk” is in fact a much thornier and harder problem, where you have much less leverage over the thing you’re trying to change, and dramatically worse ability to tell if you’re making progress.
Actually trying to make your startup succeed (or whatever), for real, feels real and scary, the in a way that it does not feel scary to be aiming for an abstraction like “having impact”.
“Impact” or “reducing x-risk” is too vague and too removed from feedback for anyone, including yourself, to ever really hold you accountable. So even if it’s extremely “ambitious”, it’s ultimately a safe thing to do.
Trying to “the most good you can do” means tackling the “most important problems”, which can allow you to hide from doing anything real or scary.
(This is mostly advice to myself.)
Actually I think this is somewhat of a myth / greatly exaggerated. You can get good feedback about ideas regarding alignment. It just takes certain efforts that people don’t do. For example, “do maker-breaker to alignment proposals and proofs of impossibility of alignment; generalize each breaking”. That’s good feedback. Philosophical arguments are also good feedback. The problem is hard for other reasons (e.g. we just have huge conceptual gaps). As a comparison, it would be only accurate in a confusing / contorted way, to say that Euclid “lacked good feedback loops” about algebraic topology or whatever. Algebraic topology has good feedback loops, math has good feedback loops, etc.; Euclid just lacks a bunch of concepts needed to even start with algebraic topology.
(If by good you meant highly legible, very inexpensive, very fast, etc., then sure.)
I agree with that, and also suspect that some alignment researchers are sufficiently bad at controlling their own confirmation bias that they never get a good look at the conceptual gaps. To those researchers, the lack of good feedback loops is the one central fact about alignment research. I.e., I suspect that some extremely bright people are 100x or 1000x better at avoiding confirmation bias than other extremely bright people are.
I think I agree in outline, though I’m unsure about describing that as confirmation bias. “Doesn’t try to counterargue one’s own hopes” is one of the main things; I guess that’s a kind of confirmation bias? I guess I’d rather describe it as a big missing skill or practice, but maybe I’m being too restrictive with the word “bias”.
I was uneasy about “confirmation bias” when I was writing my comment (and momentarily considered “motivated cognition”, which is not any better).
This thought was inspired by reading this Ben Hoffman’s recent post, and more proximally by this Wei Dai shortfrom, which claims that approximately no one exhibits long horizon agency.
You might think that means “almost no one except these EAs and rationalists”. But, since it’s easy to use “optimizing for the long term future” as a cope to hide from accountability, I suggest that many (most?) of the people who appear to be exerting long-horizon agency are using that as a cover and less actually steering the future.
I guess I didn’t express this in that shortform, but I meant to include EAs and rationalists in “approximately does not exist”. Examples: “philosophers don’t bother to think about long term implications of AI on philosophy production (positive or negative)” includes philosophers in EA who never talked about this until very recently (with founding of Forethought) and even now talk about this to a much lesser extent than I expect or think they should, and Eliezer making what I think are serious strategic mistakes earlier in his career.
About the “missing mood”, what do you think about this post of mine? I feel like the solution to the problem that we’re talking about isn’t to do less long-horizon agency/strategy, but to do more of it but more tentatively, slowly building up traditions/methodologies/foundations, kind of like how philosophy has progressed over time (but hopefully avoiding the destructive overconfidence of past philosophers). If we simply do it less, then how does civilization ever get to a state where we’re navigating the future with strategic competence?
I’ve been thinking lately about the attractors that develop when a person or community has spent some time thinking on something (enough to grow attached to their thoughts). It takes real effort to just wave at concepts you’ve seen before and then keep driving, whenever something you’re thinking about might round to something more familiar.
Especially if the conceit of one’s inquiry is ‘I suspect we are very wrong about very fundamental things’, there are many alternatives and objections that lie along the path, which themselves dead-end in unsatisfying places. Walking down each once is fine, but I find it wearying to trod down the same handful of knee jerk off-ramps over and over again, every time I share an idea with a new interlocutor.
The slower feedback loops of academic life seem to help here; once you have some cache, you can more or less hole up and work on your maybe-insane idea for the rest of your life, with very limited accountability. This, of course, creates other problems, the enumeration of which is its own well-worn off-ramp.
I think that the rapid advance of AI is correctly prompting a taking of inventory, a moment of circum/introspection, to get a sense of whether or not we’re ‘ready’, and most people who’ve genuinely engaged with pre-existing literature that attempted to grapple with these possibilities answer that question with a resounding ‘NO’, but we’ve not yet seen a shift to a more butterfly-nurturing temperament, and we’ve not yet reached consensus on quite how far we ought to back up, which makes a lot of people who consider themselves to have backed up wince a bit when someone later says ‘no, further’.
It seems like you’re the person who’s over and over again saying ‘no, further’, and many people just hear the local ‘no’ to their particular idea, struggling to think that there’s any further to back up.
[fwiw, I currently think that we need to back up/zoom out/pass off-ramps, and then uh… keep doing that for a long while]
Humans are not automagically strategic. Failing at long timehorizon strategy is the default for humans because it requires thinking correctly which people fail at by default without feedback loops. No cope required. You can do real scarry things with feedback loops as your full-time job and do the long-term thinking as your hobby and side-hustle. Humans have timehorizons just like AI, but humans are trainable just like AI.
Two things can be true.
You can actually have a large impact over time by focusing on strategic moves, and also be running from tactical day to day operational accountability, and using the former to cover for the latter.
Interesting. This I agree with. But it seems to apply to those doing more easily evaluate technical work at least as much.
Should we stop attempting because it’s hard? Or notice cope and cover and correct it?
Not necessarily, but we should maybe be less gung ho about “trying to do the most good”.
It feels like there’s maybe a missing mood to a lot of EA and x-risk related efforts. Probably more people should be more paranoid about the impacts of their actions, and more willing to do ostensibly less ambitious things that are less fake to them?
I guess I’m not understanding what mood you think is missing.
I’m not super clear on what the ideal response to this state of affairs is. I’m both confused and probably wrong.
But does “paranoia” not evoke the difference?
I think you’re right locally, but I’d limit it to “easier and safer” in one way. I’d say the bigger overall risk by far is doing ambitious, harder to evaluate work.
Your point is valid, but it’s a one-sided argument. So I’m going to fill in a little of the other side.
Funding is a large and fairly obvious countervailing risk steering people away from doing more ambitious, broader work intended to address the whole difficult problem. Funding for safety work has been heavily to severely biased toward technical work in recent years, probably primarily because it’s easier to gauge success in that type of work.
I agree that there can be motivated reasoning for doing work that’s non-technical and therefore not vulnerable to obvious and outright failure. But there are strong arguments for not doing more legible things just because they’re more legible.
It’s not impossible to gauge the success of work aiming at solving big problems, it’s just a bit harder than judging success of narrow technical work.
To state the strong form: we will individually be taking on real, easily measurable challenges and making measurable progress if we all rearrange deck chairs under the streetlight. That doesn’t make it a good idea. Good work needs to have a good chance to actually shift the future. Evaluating whether it does is a large part of the challenge.
Thanks for spelling it out, I did feel smth like this, but only when I thought about “working on AI pause” vs. “working on technical alignment”. A pause is a very specific and real thing, and it requires talking and interacting with real people and existing systems, and we know from real life how easily those can go wrong. When you are “working on technical alignment”, the object you are working with is a future, nonexistent technology, and if you notice problems with your current best guess of how to make it safe, you can just go “huh, we should add this to our list of open problems to fix before we actually build the thing”, and this feels like making progress. But when working on a pause, you can’t just say “huh, our government isn’t optimal, I’ll add ‘fix democracy’ to the list of open problems”, you have to work with the real, imperfect thing.
I guess the question is, is there a class of things that do the most good that have tight feedback loops? Or ways to create such loops for things that don’t? It could be that we live in a world where it’s not so, and we must pick between unclear good + tight feedback, or the most good + poor feedback.
I somewhat dispute the framing here.
If the “most important” problems are intractable because their feedback loops are so bad, then our standards for “most important” are themselves mistaken.
It’s like Richard Hamming saying that in some sense, developing anti-gravity is hugely important problem (if you solved that it would have enormous commercial and humanitarian benefit), it’s importance is a mirage, because there’s no-line of attack on it.
It could be case that there are a bunch of problems that will have huge impacts on many people through the whole future, and also, in practice, it’s a bad bet to work on them, because almost no one can do non-fake work in an unaccountable domain.
What if they don’t feel daunting at all to me? They feel quite fun and nice. Getting real world feedback on failure is nice, not bad.
have you therefore done them?
Agreed. Or at least, there are certain psychological factors[1] that make working on abstract alignment problems feel “safer” for some subset of people, myself very much included! On the other hand, these same factors can be “unsafe” to different people or in different contests, and there are plenty of features of other research styles that are psychologically attractive[2].
But I don’t really buy the implied add-on “this is psychologically safer for me, therefore it’s actually bad,” at least not as it applies to myself.
(e.g. lack of feedback loops can mean you don’t feel failure the same way, relatively low competition / professionalization, the real world doesn’t interrupt your research to tell you to change your mind very much)
e.g. you will naturally work with other people, you get to apply the neat math / CS you studied, you get to spend lots of money, your work fits in the normal paper format, etc.
I’m getting funny vibes from this post.
First thought was that it’s something like applause lights. Upvoting your post feels like a really easy thing to do. Yay doing things that are real rather than fake! If you’re trying to challenge the fundamental way a bunch of people around you are approaching things, and the main reaction is people nodding along, that suggests that maybe you didn’t manage to articulate the core critique.
After sleeping on it: Something feels slippery about the relationship between the different parts of your post. Picking out a few particulars:
The things you’ve listed as alternatives aren’t the alternatives that EAs would be doing. They’re things you picked out to optimize for the best degree/flavor of dauntingness, for a subset of people who could do them
Some actual EA alternatives, like donating to AMF, feel very psychologically easy
Making progress on AI x-risk does feel very daunting to many people who consider it. Perhaps overly daunting?
Most EAs who are doing EA-related work are focused on something more specific than “the most good you can do”.
The one specific set of EAs you mention—“EA student group[s]”—are more in learning mode than in doing mode
People doing things in part so they can “feel cool and important” seems like a generic feature of any social group where people can be excited about what other people do. If founding new projects was the path that others saw as good & impressive, then people would be drawn to that to feel cool and important. (Perhaps that couldn’t last if the project didn’t work, at least locally, but the clout from doing vague big important things doesn’t last either, and people could be drawn to projects that do seem like they’re making some kind of local progress.)
The pattern of actions you recommend—choosing a narrower, more tractable target rather than the big ambitious one—resembles a common motivational pitfall that leads to doing faker things. Slipping off the ambitious aim that you really care about because it’s hard to think of what progress would even look like, and instead picking something that is locally easier to track. Perhaps that’s something kinda related to the original goal or it could be something completely different
So maybe the central complaint is more like Bulverism than applause lights. For any specific thing that people might do, there are some ways in which the motivations and social dynamics around it might be screwy.
If you’re just trying to say “here’s one of the ways that sus motivations can manifest when doing thing1” that seems potentially valuable. But if you’re trying to argue for doing thing2 instead of thing1, then it feels like you this post doesn’t show the cognitive work to think through that. Instead it picks out some of the sus motives for thing1 and papers together a narrative around them without thinking through most of the substance of the argument (possible sus motives for thing2, what thing2 might actually wind up looking like, for which specific set of people, other ways of approaching things like thing1 or thing2, the core reasons in favor of thing1 & thing2, etc.).
The closing “(This is mostly advice to myself.)” felt like the most grounding part of it, because it suggests that it’s pointed at a specific larger process of thinking things through.
Bananas implies oranges reasoning. It’s mostly a category error. You maybe have a goal that is based on flawed models of the world, or that isn’t directly actionable or something like that. That doesn’t mean you should pivot to something that is just a commodity/everyone else does it anyway type of thing.