Email channel allocation statistics from a beauty retail field experiment

Updated 10 source citationsSources checked 13 November 2026

On this page
  1. Quick answer
  2. Key Takeaways
  3. What did the field experiment compare?
  4. Which historical patterns held up?
  5. Was the reported allocation gain observed directly?
  6. How should a merchant use this countercase?
  7. Which calculations can a reader reproduce?
  8. How do the no-marketing and current-policy baselines connect?
  9. Methodology and Sources
Download study text

Quick answer

Email channel allocation statistics can change when a historical prediction is tested through random assignment. A beauty retailer’s historical data suggested email effectiveness for medium and high value customers. Its July to November 2017 Italy field experiment did not find a significant email effect in any value segment. Direct mail remained significant for prospects, with a reported elasticity of .132. (Valenti, Srinivasan, Yildirim and Pauwels, Aug 2023)

A customer’s past spending and their likely response to another message answer different questions. This analysis retains the difference between the historical studies, the randomized channel comparison and the later allocation calculations. It supplies a useful countercase to assuming that previously valuable customers will necessarily respond incrementally to more email.

Key Takeaways

What did the field experiment compare?

The four conditions were no marketing, email only, direct mail only and both channels. The experiment stratified six customer value segments and used customers who were contactable by both channels, respecting their expressed contact preferences. The field test was conducted with the beauty retailer in Italy from July through November 2017. (Valenti, Srinivasan, Yildirim and Pauwels, Aug 2023)

Email channel allocation statistics evidence ladder: actual no-marketing/email/mail/both cells supply response estimates; an estimated current-policy index 101.6 and computed 6.5% gain imply a proposed 108.204 index, not an observed policy result.
Figure 1: Randomized inputs do not turn a computed allocation into an observed result. Source: Valenti and colleagues, field design and policy computation, Aug 2023. Original comparison from the cited study. Study period and calculation limits appear in the text.

That eligibility matters. The source's international historical beauty data and separate US apparel data are broader descriptive inputs. They are not additional countries or product categories in the same randomized field comparison. A shopper who cannot be contacted by one of the channels also falls outside this particular channel choice experiment.

For a recovery team, the corresponding design begins with a common eligible population. Comparing email recipients with customers who have only a postal address would confound contact route with customer differences. Record the relevant permissions and current order state before assignment, then retain the assigned policy in the analysis.

Which historical patterns held up?

Direct mail remained effective for prospects in the field comparison, with a reported elasticity of .132 versus .164 in the historical main analysis. Email did not have a significant effect for any value segment in the field test, despite historical significance for medium and high value groups. The combined channel interaction was significant for the medium value group, at .011. (Valenti, Srinivasan, Yildirim and Pauwels, Aug 2023)

These are conditional study results, not a ranking of channels for every merchant. A non-significant estimate can be compatible with a small positive or negative effect. The source does not justify replacing every non-significant point estimate with zero in a campaign forecast.

The finding is a reason to test the current segmentation policy. High customer value describes purchasing, while responsiveness describes a change caused by contact. The two can differ. A valuable customer may purchase without another message, while a less established customer may respond to information the store has not previously delivered.

Was the reported allocation gain observed directly?

The authors calculated a 6.5% revenue lift potential from reallocating marketing using the field estimates, holding marketing costs constant. They explicitly explain that business as usual was not one of the experimental cells. The no marketing control therefore should not be renamed the current allocation policy. (Valenti, Srinivasan, Yildirim and Pauwels, Aug 2023)

The historical beauty model gave a 13.5% allocation projection, and the separate historical apparel model gave 9.3%. These are different analyses with different source populations. Our subtraction of 13.5 and 6.5 gives a 7.0 percentage point difference between two beauty allocation projections. It is not an estimate of revenue lost during the field test. (Valenti, Srinivasan, Yildirim and Pauwels, Aug 2023)

An experiment can identify the inputs to a policy calculation without directly testing the entire proposed policy. A store should retain that distinction when presenting a projected result to its team. A subsequent policy experiment would compare the proposed allocation with the actual current allocation, including contact constraints and costs.

How should a merchant use this countercase?

Keep prediction and intervention questions separate. Past order value, opens and clicks may help describe an audience, but they do not by themselves estimate the incremental response to another message. A field test should include the current policy and a valid holdout, with the customer value rules fixed before outcomes are observed.

Where multiple channels are eligible, evaluate the combined policy rather than adding standalone attributed revenue. One channel can reinforce another, substitute for it or reach the same natural purchaser. The source's significant medium value interaction is a reason to allow that possibility, not a transferable expectation for every combined campaign.

Finally, assess contribution after the real cost of contact. A low marginal email cost does not remove fatigue, unsubscribes or the opportunity cost of the shopper's attention. The study's contribution is a tested challenge to historical targeting assumptions. It does not prescribe a current contact cadence or a universal allocation between email and physical mail.

Which calculations can a reader reproduce?

The download retains the published inputs, their units and their comparison populations. The following arithmetic uses those inputs. It does not add a new sample or a new causal comparison.

CalculationExpressionResult
Beauty historical minus field based allocation projection, percentage points13.5-6.57.000
Field versus historical prospect direct mail elasticity difference0.132-0.164-0.032
Field based to historical beauty allocation projection, ratio6.5/13.50.481

How do the no-marketing and current-policy baselines connect?

Normalize the source's no-marketing comparison to an index of 100. The paper's estimated business-as-usual impact of 1.6% corresponds to 101.6 on that index. Applying the computed 6.5% allocation gain relative to business as usual gives approximately 108.204, or 8.204% above the no-marketing reference. This is our baseline reconciliation of rounded source projections, not an additional experimentally observed 8.204% gain. (Valenti and colleagues, managerial implications, Aug 2023).

Reference or policyReconciled indexEvidence status
No marketing100Actual field comparison condition, normalized
Business as usual101.6Authors' estimated impact, not an experimental cell
Proposed reallocationApproximately 108.204Calculation using field estimates and constant-cost policy assumptions

Adding 1.6 and 6.5 yields 8.1, which uses incompatible baselines. The correct composition is 1.016×1.065−1. Conversely, the relative gain from 101.6 to 108.204 is 6.5%, while their difference is 6.604 index points. Those quantities should not receive the same label in a budget proposal.

Why do randomized response inputs still need a policy validation?

The four field cells estimate outcomes for no contact, email, direct mail and both within the eligible population. A proposed allocation chooses how to distribute those actions across segments. The policy calculation therefore also depends on audience weights, contact costs, eligibility and the optimization constraints. Randomized inputs do not make the computed full allocation an observed treatment cell.

A policy selected using noisy point estimates can favor unusually favorable estimates from that same sample. Retain uncertainty around the inputs, inspect alternatives close to the selected allocation and test the chosen policy against the actual current policy in a new eligible audience or later randomized window. The source's nonsignificant email results should remain uncertain estimates; silently zeroing them can change the optimizer as much as blindly accepting them.

Which channel interaction can the four cells answer?

On an appropriate common outcome scale, the contrast both − email only − mail only + no marketing tests departure from the sum of the two standalone effects. The reported .011 medium-value interaction is a model coefficient, not a 1.1-point conversion lift or a standalone revenue elasticity. Keep its scale and segment attached to the claim. A significant interaction in one segment cannot be applied to everyone reached by both channels.

The original field sample requires contactability by both routes and respects customer preferences. A broader rollout that adds email-only or mail-only customers changes the feasible policy population. Report that coverage before transporting segment results or claiming the same constant-cost gain.

The projection-baseline reconciliation and calculation keep the observed field comparisons, estimated current-policy impact and computed allocation separate. They clarify what a subsequent policy test must establish rather than recasting a projection as realized revenue.

Methodology and Sources

We read the cited primary source and the decisive methods, results and exceptions on 13 November 2026, retaining its PDF and text under the source ledger. The download distinguishes reported aggregates from our arithmetic, with units and analysis populations preserved. The formulas shown here can be rerun from analyze.py. We do not reconstruct individual records from rounded aggregates or treat an implementation suggestion as a measured result. (Valenti, Srinivasan, Yildirim and Pauwels, Aug 2023).

Email effects were not significant in any field value segment, which does not prove zero effect. Combined channel effect was significant only for the medium value interaction. Field contact eligibility and geography are narrower than the historical analysis. The 6.5% allocation gain is computed using experimental estimates because business as usual was not an experimental arm; 13.5% and 9.3% are separate model based projections.

Glossary

  • Elasticity: A proportional response measure used by the source to describe how sales respond to a marketing input.
  • Interaction coefficient: A model term describing how the joint presence of two inputs differs from the model’s separate terms.
  • Business as usual: The retailer’s normal allocation of marketing, distinct from the experiment’s no-marketing control.
  • Field experiment: A comparison in the operating business using the stated treatment assignment.
  • Reallocation projection: A calculated outcome for moving contacts between segments using estimated responses and policy assumptions.
  • Contact eligibility: Whether a customer could be included under the permitted channels and study rules.

Sources

Cite this page

Use this citation when referring to the analysis. Retain the original study scope with any figures.

You Left Something. "Email channel allocation statistics from a beauty retail field experiment." You Left Something, November 2026. https://youleftsomething.com/guides/email-channel-allocation-retail-holdout-statistics/
<a href="https://youleftsomething.com/guides/email-channel-allocation-retail-holdout-statistics/">Email channel allocation statistics from a beauty retail field experiment</a> (You Left Something, November 2026)