What you are saying is reasonable, but it feels to me like you put the burden of proof on the author of the article. The question is, why should we believe advertising works at all? So the way I see it, the burden of proof is on the people doing advertising, and the article is asserting that they have not met it.
I partly agree, but burden of proof is often the wrong framing for truth seeking.
The article provides strong evidence that ads are ineffective in certain classes of cases, and that fact in turn provides weaker evidence that ads are ineffective more generally. To support Akshat’s skepticism that the result generalizes, we’d need to evidence or priors that points towards ads being differentially effective depending on the type—targeted keywords vs. brand-ad keywords, and brand presence verus no brand presence.
In the first case, I’m somewhat skeptical that the difference between targeted and brand keywords will be large. My prior for the second difference is that there would be some difference, as Gordon argued in another comment. I don’t know of any evidence in either direction, but I haven’t looked. (The actual result doesn’t matter to me except as an exercise in Bayesian reasoning, but if it matters to you or others, it’s plausible high VoI to search a bit. )
What you are saying is reasonable, but it feels to me like you put the burden of proof on the author of the article. The question is, why should we believe advertising works at all?
It seems like a a reasonable prior would be that telling people about your product who didn’t already know about your product makes them more likely to buy your product.
I think you can certainly make the case against the above statement, but I don’t know why you wouldn’t start with that prior.
That prior of course, doesn’t make a case for brand advertising, which eBay was doing, but that’s not what David’s objection was about.
The question is, why should we believe advertising works at all?
This is a fair objection. I decided to look for a review paper summarizing the existing literature on the subject of advertising effectiveness.
Via Google Scholar, I was able to find a particularly useful review paper, summarizing both empirical effects and prior literature reviews for advertisements as well as political and health campaigns across multiple channels (print, TV, etc.). Overall, the literature paints a disjointed, inconclusive view of the value of advertising—there is insufficient data to conclude that advertising in general has no impact.
I invite you or anyone interested to read it in depth, but will, for the purpose of this discussion, summarize its concluding remarks (as available in the section “Behavioral Effects of Advertising or Commercial Campaigns”):
1. Advertising interventions appear to be correlated with short-term boosts in product sales.
A set of case studies has shown strong short-term effects of campaigns on sales (Jones 2002). In a recent study, the buying of a service (use of the weight room in a training facility) increased to almost five times the initial use after an outdoor advertising campaign (Bhargava and Donthu 1999). In another study, exposure to printed store sale flyers led to a doubling of the number of advertised products bought, and more than a doubling of the amount spent on items in ads (Burton, Lichtenstein et al. 1999).
2. There *is* disagreement on the long-term effects.
Ninety percent of advertising effects dissipate after three to fifteen months. The first response is most important; the share returns for advertising diminish fast. After the third exposure advertisers should focus on reach rather than frequency, according to research findings from advertising effects research (Vakratsas and Ambler 1999).
While some claim that advertising seems not to be important for sales in the short term, although more important in the longer term (Tellis 1994; see also Tellis, Chandy et al. 2000), others disagree. Jones found that advertisements must work in the short term to be able to have any medium or long-range effect on sales (Jones 2002).
3. Despite contributing to a short-term boost, advertising by itself is weaker compared to other kinds of promotional activities. Increased advertising spend yields diminishing results:
The influence of advertising has been estimated to be 9% of the variation in sales for consumer products. The effect of promotional activities – such as offers of reduced prices for shorter periods of time – was more than double that size (Jones 2002). In some studies price reductions have been found to be 20 times more effective for increasing sales than is advertising (Tellis 1994), a consequence being that since the late 1980s the industry has changed its emphasis from advertising to promotion (Turk and Katz 1992; Vakratsas and Ambler 1999; Jones 2002). The solution to the problem of small effects may be that most advertising research has not taken into consideration the fact that only a small amount of advertising seems to increase sales. Increased spending on advertising (increased number of exposures and increased gross rating points) has been found to induce larger sales when ads were persuasive, but not when they were not (Stewart, Paulos et al. 2002).
4. “Likeability”, medium, and what it’s selling matters a lot in the effectiveness of advertising:
The advertising copy and novelty in ads seemed more important than the amount of advertising itself (Tellis 1994). The two most important qualities of ads that sell products are likeability of the ad (Biel 1998) and its ability to make people believe that a company has an excellent product (Joyce 1998: 20). A study has shown that advertising likeability predicted sales winners 87% of the time (Biel 1998). It is no news that copy research works (Caples 1997; for a review, see Jeffres 1997: 252-263), but new data-processing techniques have made it possible to apply this knowledge almost instantly to TV advertising as well (Woodside 1996). Channel selection may also be an important influence on sales (Tellis, Chandy et al. 2000). For some groups of products (lower-priced daily consumer goods) the first exposure to advertising may contain most of the ad’s effect on behavior (Jones 1995; Jones 2002).
There are many other aspects of advertising influence that is covered in the conclusions which I have not summarized—I have selected the few that seem most salient here.
Overall, I think a reasonable prior is that advertising *has* an impact, but has strong situational limits to its effectiveness compared to other sales growth techniques. Since digital advertising is a specific case of advertising in general and there are some effects for advertising in general, it would be difficult to make the case that no digital advertising works at all—it is much safer to expect that digital advertising has *some* (albeit situational and weak) impact.
---
The other half of this comment is re:
the article is asserting that they have not met it
This is a reductive picture. It is true the article is setting out to check marketer’s claims of the effects of digital advertising. However, it is also setting out to provide an overview of the evidence for whether digital advertising works in general. This last aspect was the focus of my prior comment.
My comment was meant to highlight the flaws in their methodology for reviewing whether digital advertising does not work. Their review focus has been restricted to a very specific set of claims and cases targeted at larger advertising platforms, and one should not generalize early from those remarks. To do a better job of truth-seeking, articulating what specifically is currently not known after the analysis is necessary, I think—hence my last comment.
What you are saying is reasonable, but it feels to me like you put the burden of proof on the author of the article. The question is, why should we believe advertising works at all? So the way I see it, the burden of proof is on the people doing advertising, and the article is asserting that they have not met it.
I partly agree, but burden of proof is often the wrong framing for truth seeking.
The article provides strong evidence that ads are ineffective in certain classes of cases, and that fact in turn provides weaker evidence that ads are ineffective more generally. To support Akshat’s skepticism that the result generalizes, we’d need to evidence or priors that points towards ads being differentially effective depending on the type—targeted keywords vs. brand-ad keywords, and brand presence verus no brand presence.
In the first case, I’m somewhat skeptical that the difference between targeted and brand keywords will be large. My prior for the second difference is that there would be some difference, as Gordon argued in another comment. I don’t know of any evidence in either direction, but I haven’t looked. (The actual result doesn’t matter to me except as an exercise in Bayesian reasoning, but if it matters to you or others, it’s plausible high VoI to search a bit. )
It seems like a a reasonable prior would be that telling people about your product who didn’t already know about your product makes them more likely to buy your product.
I think you can certainly make the case against the above statement, but I don’t know why you wouldn’t start with that prior.
That prior of course, doesn’t make a case for brand advertising, which eBay was doing, but that’s not what David’s objection was about.
This is a fair objection. I decided to look for a review paper summarizing the existing literature on the subject of advertising effectiveness.
Via Google Scholar, I was able to find a particularly useful review paper, summarizing both empirical effects and prior literature reviews for advertisements as well as political and health campaigns across multiple channels (print, TV, etc.). Overall, the literature paints a disjointed, inconclusive view of the value of advertising—there is insufficient data to conclude that advertising in general has no impact.
I invite you or anyone interested to read it in depth, but will, for the purpose of this discussion, summarize its concluding remarks (as available in the section “Behavioral Effects of Advertising or Commercial Campaigns”):
1. Advertising interventions appear to be correlated with short-term boosts in product sales.
2. There *is* disagreement on the long-term effects.
3. Despite contributing to a short-term boost, advertising by itself is weaker compared to other kinds of promotional activities. Increased advertising spend yields diminishing results:
4. “Likeability”, medium, and what it’s selling matters a lot in the effectiveness of advertising:
There are many other aspects of advertising influence that is covered in the conclusions which I have not summarized—I have selected the few that seem most salient here.
Overall, I think a reasonable prior is that advertising *has* an impact, but has strong situational limits to its effectiveness compared to other sales growth techniques. Since digital advertising is a specific case of advertising in general and there are some effects for advertising in general, it would be difficult to make the case that no digital advertising works at all—it is much safer to expect that digital advertising has *some* (albeit situational and weak) impact.
---
The other half of this comment is re:
This is a reductive picture. It is true the article is setting out to check marketer’s claims of the effects of digital advertising. However, it is also setting out to provide an overview of the evidence for whether digital advertising works in general. This last aspect was the focus of my prior comment.
My comment was meant to highlight the flaws in their methodology for reviewing whether digital advertising does not work. Their review focus has been restricted to a very specific set of claims and cases targeted at larger advertising platforms, and one should not generalize early from those remarks. To do a better job of truth-seeking, articulating what specifically is currently not known after the analysis is necessary, I think—hence my last comment.