AI email writing statistics for sales contribution and production cost

Updated 11 source citationsSources checked 13 November 2026

On this page
  1. Quick answer
  2. Key Takeaways
  3. What did the randomized emails establish?
  4. Why did the preferred writing policy vary?
  5. Where do the cost savings enter?
  6. How can lower gross contribution still produce higher net profit?
  7. How much of the modeled cost saving must actually happen?
  8. Why is the break-even calculation not proof of a winning policy?
  9. What would a recovery team need to test?
  10. Which calculations can a reader reproduce?
  11. Methodology and Sources
Download study text

Quick answer

AI email writing statistics should distinguish the sales contribution of the message from the cost of creating it. In three Wine Access randomized newsletter trials, Dubé and Xu did not establish significant gross profit differences between human and AI creative policies at their stated significance levels. Their annual net profit projections nevertheless selected an AI or hybrid policy in each trial after labor and licensing costs were included. (Dubé and Xu, Jan 2026)

Changing how recurring emails are written affects both production costs and the contribution those messages earn. These experiments used newsletters, not abandoned cart sequences. They show how to evaluate creative economics without treating a failure to detect a sales difference as proof that every writing policy performs identically.

Key Takeaways

  • Only the first trial included a no newsletter holdout. (Dubé and Xu, Jan 2026).
  • Human and AI creative gross profit differences were not statistically significant at the stated significance levels. (Dubé and Xu, Jan 2026).
  • Annual net profit figures combine trial outcomes with policy cost assumptions. (Dubé and Xu, Jan 2026).

What did the randomized emails establish?

The first trial ran from January 15 to 28 2024 with 27,500 regular customers: 9,000 in each creative arm and 500 in a no newsletter comparison. Human, LLM and hybrid creative produced estimated incremental gross profit of USD 5.339, USD 5.064 and USD 4.789 per assigned customer over the two week period, respectively. (Dubé and Xu, Jan 2026)

Within each Wine Access trial, LLM and hybrid annual net profit point projections are compared with that trial's human policy. LLM is 2.05 percent above in trial one, 0.64 percent below in trial two and 8.48 percent above in trial three. The hybrid projection also varies.
Figure 1: Separate within-trial comparisons preserve the changing policies and audiences. The points are annual projections, not observed full-year lifts or a trend in model quality. Source: Dubé and Xu, Jan 2026. Original comparison from the cited study. Study period and calculation limits appear in the text.

Those figures compare each email policy with no newsletter. They do not identify how much of a particular customer's order was caused by one sentence. Assignment, rather than opening an email, defines the analysis. Shoppers who bought without engaging still belong to their assigned group.

The later trials compared creative policies without a no newsletter arm. A null creative difference in those trials cannot independently reestablish the entire newsletter program's incrementality. A store making two decisions, whether to send and how to write, needs evidence suitable for each.

Why did the preferred writing policy vary?

The source's annual net profit projections put Human, LLM and Hybrid at USD 7,129,844, USD 7,276,320 and USD 7,048,412 in the first trial. In the second trial, Hybrid was above Human while LLM was slightly below it. In the third, both AI policies were above Human in the projection. (Dubé and Xu, Jan 2026)

Our within trial calculation gives a projected LLM gain of 2.05% over Human in the first trial, a Hybrid gain of 4.11% in the second, and projected gains of 8.48% for LLM and 9.36% for Hybrid in the third. These are calculations from annual policy projections. They are not observed full year revenue lifts. (Dubé and Xu, Jan 2026)

The policies also changed. The hybrid editor and the AI approach were different across experiments. Wine offers, audiences and trial dates were not identical. A falling or rising annual projection across trials therefore cannot be read as a time trend in model quality or customer acceptance.

Where do the cost savings enter?

For the first trial's policies, the authors estimated annual cost of USD 375,000 for the Human writing approach and USD 32,250 for the LLM approach. The difference is USD 342,750. It combines labor assumptions and licensing, while retaining some human work. Those are policy cost inputs rather than randomized shopper outcomes. (Dubé and Xu, Jan 2026)

The arithmetic illustrates a decision that a sales only report can miss. A modest decrease in gross contribution can coexist with an increase in net contribution when a substantial avoidable production cost falls. The opposite is also possible when implementation, review, correction and oversight take more time than expected.

For a merchant with employees doing several jobs, saved writing hours are not automatically saved cash. Record whether capacity can be reassigned to useful work, staffing expense can actually be avoided or output can increase at unchanged cost. Include factual review, offer validation and corrections when assessing the writing policy.

How can lower gross contribution still produce higher net profit?

The first trial makes the trade-off unusually clear. The LLM policy's projected annual net profit was USD 146,476 above Human, while its modeled annual writing cost was USD 342,750 lower. Subtract the cost saving from the net advantage and the implied projected gross contribution difference is USD 196,274 against LLM. These are our calculations from Tables 6 and 7, using the source's annual point projections. (Dubé and Xu, Jan 2026)

That decomposition changes what a merchant should investigate. The favorable net projection does not mean AI creative produced more sales contribution. In this trial's projection, the production saving was large enough to cover a lower gross contribution estimate. A report that presents only the final net figure conceals the two opposing terms.

Trial-one LLM minus Human annual point projections: a USD342,750 modeled production cost saving offsets an implied USD196,274 lower gross contribution, leaving USD146,476 higher projected net profit.
Figure 2: The annual net advantage combines a cost assumption with a sales contribution projection. It is not evidence that the LLM creative generated more gross contribution. Source: Dubé and Xu, Tables 6 and 7, Jan 2026. Original comparison from the cited study. Study period and calculation limits appear in the text.

The reverse calculation is also useful as an audit. Human gross contribution implied by the projection is USD 7,129,844 plus USD 375,000, or USD 7,504,844. LLM implies USD 7,276,320 plus USD 32,250, or USD 7,308,570. Their difference is the same USD 196,274. This check uses the annual projection consistently. It does not multiply a rounded two-week per-customer estimate by an invented annual audience.

How much of the modeled cost saving must actually happen?

Holding that first-trial gross contribution point projection fixed, LLM breaks even with Human only when realised annual savings exceed USD 196,274. That is 57.3% of the authors' USD 342,750 modeled saving. This is our cost sensitivity calculation, not an observed adoption threshold or a second experiment. (Dubé and Xu, Tables 6 and 7, Jan 2026)

At 50% realisation, the saving is USD 171,375 and projected net profit is USD 24,899 below Human. At 75%, the saving is USD 257,062.50 and the net advantage is USD 60,788.50. Full realisation recovers the paper's USD 146,476 point-projection advantage. The same creative outcomes can therefore support different operating decisions when production costs do not fall as far as assumed.

Conditional sensitivity of trial-one annual net profit to realised cost savings. With gross contribution point projections fixed, 50 percent realisation leaves LLM USD24,899 below Human, 75 percent leaves it USD60,789 above, and break-even occurs at 57.3 percent.
Figure 3: Changing avoidable costs changes the decision even when the source's gross contribution point projection stays fixed. The line is a sensitivity, not a confidence band or a forecast. Source: Dubé and Xu, Tables 6 and 7, Jan 2026. Original comparison from the cited study. Study period and calculation limits appear in the text.

For a store, the practical question is where those saved hours go. If the editor remains on the same salary and no additional useful output is produced, a reduction in time spent drafting is not the modeled cash saving. If the released capacity produces other useful work, value that work explicitly rather than counting both the original salary reduction and the replacement output as separate benefits.

Additional review has a similarly direct effect. Under full modeled savings and the source's fixed gross point projection, another USD 146,476 of annual review, corrections or implementation expense would exhaust the LLM advantage. That figure is conditional headroom in this retailer's projection, not a suggested budget for another merchant. Costs already included in the source's LLM policy must not be charged twice.

Why is the break-even calculation not proof of a winning policy?

The cost frontier holds estimated gross contribution fixed to make one assumption visible. It does not remove sampling uncertainty from the trial. The paper's net projections have reported standard errors, and the separate treatment estimates are not independent pieces of data merely because they appear in different table columns. Without the covariance needed for a difference, combining arm standard errors as though the estimates were independent would create an unjustified precision claim.

A merchant can still make a decision before every uncertainty disappears. State the largest acceptable contribution loss, count only achievable cost savings and retain the ability to change the policy if monitored outcomes deteriorate. That is an operating decision under uncertainty. It is a different claim from demonstrating that two writing methods are equivalent or that one will increase profit at a new store.

The most transferable finding is consequently an accounting discipline: separate the message's incremental contribution, the cost of producing it and the uncertainty around both. Neither cheap drafting nor attributed email revenue alone answers whether a recovery programme is profitable.

What would a recovery team need to test?

A recovery message adds jobs absent from a generic newsletter: the current basket must still be valid, the shopper must remain eligible and the message must preserve product, price and checkout facts. A cheaper creative process cannot compensate for an incorrect offer or a reminder sent after purchase.

Use a comparison that matches the policy under consideration. Hold eligibility, timing, incentive and offer inventory constant when comparing writing approaches. Preserve a no contact group when measuring whether the recovery program itself creates contribution. Measure unsubscribe and complaint outcomes as well as order value over the chosen window.

The paper's non-significant creative differences do not prove equivalence. A store should decide what degree of underperformance it can tolerate and whether its sample can resolve that question. The useful lesson is to bring sales contribution and actual avoidable production cost into the same decision, while keeping the evidence for each visible.

Which calculations can a reader reproduce?

The download retains the published inputs, their units and their comparison populations. The following arithmetic uses those inputs. It does not add a new sample or a new causal comparison.

CalculationExpressionResult
RCT 1 LLM versus Human projected net gain, percent(7276320/7129844-1)*1002.054
RCT 2 Hybrid versus Human projected net gain, percent(6842378/6572564-1)*1004.105
RCT 3 LLM versus Human projected net gain, percent(5500457/5070486-1)*1008.480
RCT 3 Hybrid versus Human projected net gain, percent(5545229/5070486-1)*1009.363
RCT 1 LLM annual cost reduction, USD375000-32250342750.000
RCT 1 LLM minus Human incremental gross profit, USD per customer5.064-5.339-0.275

Methodology and Sources

We read the cited primary source and the decisive methods, results and exceptions on 13 November 2026, retaining its PDF and text under the source ledger. The download distinguishes reported aggregates from our arithmetic, with units and analysis populations preserved. The formulas shown here can be rerun from analyze.py. We do not reconstruct individual records from rounded aggregates or treat an implementation suggestion as a measured result. (Dubé and Xu, Jan 2026).

Newsletter creative, not an abandoned cart sequence. One opt in wine retailer. Short trials do not directly observe annual net profit. Different creative and editing policies across trials mean cross trial profit levels are not an estimate of model improvement over time. No significant difference in gross profit across Human and AI creative policies was established at the stated significance levels; this is not an equivalence proof. Labor savings require avoidable costs. The source reports the later two trial registrations after those experiments had begun, so not every analytical choice can be treated as prospectively registered.

Glossary

  • Gross profit: Revenue after the product costs included in the paper’s definition, before the additional writing-policy costs considered in its net projections.
  • Annual projection: A calculation extending trial inputs over an assumed year, rather than a directly observed annual outcome.
  • Avoidable cost: An expense that actually falls when the production process changes.
  • Hybrid policy: The specific combination of human and model contributions tested in a trial.
  • Holdout: An assigned comparison group that does not receive the newsletter.
  • Nonsignificant difference: A difference the test did not distinguish from zero at its stated level, without proving equal performance.

Sources

Cite this page

Use this citation when referring to the analysis. Retain the original study scope with any figures.

You Left Something. "AI email writing statistics for sales contribution and production cost." You Left Something, November 2026. https://youleftsomething.com/guides/ai-email-writing-profit-cost-statistics/
<a href="https://youleftsomething.com/guides/ai-email-writing-profit-cost-statistics/">AI email writing statistics for sales contribution and production cost</a> (You Left Something, November 2026)