Email conversion rate statistics by denominator

Updated 16 source citationsSources checked 13 November 2026

On this page
  1. Quick answer
  2. Key Takeaways
  3. What does an email conversion rate count?
  4. Which comparison estimates whether the email added purchases?
  5. Why does selecting visitors change the comparison?
  6. Did changing the denominator reverse the result?
  7. Can the visitor metric help improve the website?
  8. How far can these findings travel?
  9. Can the purchase change be reconciled into arrival and visitor-rate terms?
  10. Methodology and Sources
Download study text

Quick answer

Email conversion rate statistics describe different populations when their denominators differ. In Hillstrom's randomized retail experiment, the women’s merchandise email recorded purchases from 5.84% of visitors but 0.88% of all assigned customers. Against no email, the assignment based purchase difference was 3.11 additional purchasers per 1,000, with an approximate 95% interval from 1.50 to 4.72. The visitor only difference was 0.44 percentage points, with an interval from negative 0.79 to positive 1.68 (Hillstrom experiment, independent analysis, Nov 2026).

Both rates can be calculated correctly while answering different questions. This study reconstructs the counts for all three randomized arms, follows what changes when nonvisitors are removed, and distinguishes campaign incrementality from the experience of people who visited. The source was published in March 2008 and tracks two week outcomes for 64,000 previous customers. It does not supply a modern abandoned cart email benchmark, message exposure or an email click denominator.

Key Takeaways

  • The campaign denominator includes people who did not visit The no email arm had 21,306 assigned customers, 2,262 visitors and 122 purchasers over two weeks (Hillstrom experiment, independent analysis, Nov 2026).
  • Womens email added purchases on the assignment denominator Its 189 purchasers among 21,387 assigned customers implied 3.11 additional purchasers per 1,000 versus no email, with a 95% interval of 1.50 to 4.72 (Hillstrom experiment, independent analysis, Nov 2026).
  • The visitor comparison retained uncertainty Womens email visitors had a 5.84% recorded purchase rate versus 5.39% in the no email visitors. The 0.44 point difference had a 95% interval spanning zero (Hillstrom experiment, independent analysis, Nov 2026).
  • Mens email had a positive estimate on both denominators Its assignment based difference was 6.81 additional purchasers per 1,000, while its visitor only purchase rate difference was 1.46 percentage points (Hillstrom experiment, independent analysis, Nov 2026).
  • A website visit is not an observed email click The source visit field records whether a customer visited during the two week outcome period. No click, delivery or exposure field is supplied (Hillstrom experiment, independent analysis, Nov 2026).

What does an email conversion rate count?

A conversion rate needs a population as well as an outcome. Purchasers divided by assigned customers measures recorded purchasing across the campaign's randomized audience. Purchasers divided by visitors describes a selected group that was observed after assignment.

Email conversion denominator statistics accounting: the womens email total 3.111 purchasers per 1,000 decomposes into 2.540 visit-probability and 0.571 visitor-rate terms; mens email total 6.805 decomposes into 4.691 and 2.114.
Figure 1: An assignment gain can be large even when the visitor-rate change is small. Source: Hillstrom experiment, original product identity, Nov 2026. Original analysis of the cited public data. Historical periods and calculation limits appear in the text.

Hillstrom randomly assigned previous customers to a men’s merchandise email, a women’s merchandise email or no email. The campaign names describe the merchandise promoted. They do not identify the customer's gender. Outcomes were tracked during the following two weeks (Hillstrom experiment, independent analysis, Nov 2026).

Randomized armAssigned customersVisitorsPurchasersPurchase rate among all assignedPurchase rate among visitors
No email21,3062,2621220.573%5.39%
Mens merchandise email21,3073,8942671.253%6.86%
Women’s merchandise email21,3873,2381890.884%5.84%

In this file, every purchaser also has the visit indicator set to one. That makes the visitor rate a direct division of the same purchaser count by a smaller denominator. It does not make the two denominators interchangeable. The audit found no missing cells or nonbinary visit and conversion values across the 64,000 records (Hillstrom experiment, independent analysis, Nov 2026).

Which comparison estimates whether the email added purchases?

Retaining all assigned customers preserves the randomized campaign comparison. The men’s email recorded 12.53 purchasers per 1,000 assigned versus 5.73 without email, a difference of 6.81. The women’s email recorded 8.84 per 1,000, a difference of 3.11 versus the same no email arm (Hillstrom experiment, independent analysis, Nov 2026).

The approximate unadjusted 95% intervals were 5.00 to 8.61 additional purchasers per 1,000 for men’s email and 1.50 to 4.72 for women’s email. They are calculated from the two independent binomial arm proportions in each comparison. Both intervals are positive in this historical experiment (Hillstrom experiment, independent analysis, Nov 2026).

A stricter sensitivity accounts for the two reported treatment comparisons using Bonferroni adjusted normal intervals. Those ranges were 4.74 to 8.87 and 1.27 to 4.95 additional purchasers per 1,000, respectively. Their positive bounds survive that particular adjustment. The approximations do not remove the source's age, population or measurement limits (Hillstrom experiment, independent analysis, Nov 2026).

The no email arm is essential because previous customers sometimes return without receiving that campaign. Its 122 purchasers are observed background activity over the outcome window. The experiment estimates a group level difference. It does not identify which named treatment purchasers would have bought anyway.

Why does selecting visitors change the comparison?

The email changed the probability of visiting. Per 1,000 assigned customers, the men’s email added 76.59 visitors and the women’s email added 45.23 relative to no email. Removing nonvisitors therefore selects different proportions of each randomized group (Hillstrom experiment, independent analysis, Nov 2026).

A person in the no email visitor group visited despite not receiving that campaign. A person in an email visitor group may have visited with or without it. The data do not reveal those individual alternatives. Random assignment before sending does not imply that the remaining visitor groups are exchangeable after people have chosen whether to visit.

The women’s email visitor purchase rate was 5.84%, compared with 5.39% among no email visitors. The difference was 0.44 percentage points, with an approximate 95% interval from negative 0.79 to positive 1.68. For mens email, the corresponding difference was 1.46 points, with an interval from 0.24 to 2.69 (Hillstrom experiment, independent analysis, Nov 2026).

Those are descriptive comparisons among observed visitors. They cannot isolate what email did to an otherwise identical visitor's likelihood of purchasing. A change in who arrived can change the conditional rate even if the offer and website are unchanged.

Did changing the denominator reverse the result?

No. Both visitor only point differences were positive, as were both assignment based purchase differences. A claim that the campaign signs reversed would be false for these data (Hillstrom experiment, independent analysis, Nov 2026).

The more useful finding is that the womens campaign's visitor only comparison was statistically uncertain while its all assigned purchase difference was positive. That does not prove that two causal effects differ. The estimates concern different selected populations and use different units, and a significant result beside an insignificant result is not itself a test of their difference.

The 5.84% visitor purchase rate can still be a legitimate operational description. It says what fraction of those observed visitors purchased during the window. It does not say that 5.84% of email recipients purchased, that those purchases came through an email click or that the campaign caused all of them.

For a recovery report, define the unit in the metric name. Orders per delivered message, purchasers per assigned eligible customer, purchases per website visitor and checkout completions per checkout are separate measures. They can move in different directions because they describe different stages and different populations.

Can the visitor metric help improve the website?

A visitor based metric can help describe the experience after arrival. It should be interpreted alongside the source of the visitors, the offer they saw and any change in customer mix. It is not a substitute for an incrementality comparison at the campaign level.

If the decision concerns the landing page itself, a separate randomized page experiment can assign qualifying visitors after they arrive. That creates a new, explicit eligibility point. It tests the page intervention within that visiting population rather than pretending that a retrospective campaign visitor subset was randomized.

For conversational recovery, a useful report can retain the original contact assignment, the later return and the completed purchase as separate events. Current order status should suppress irrelevant messages. A customer who completes checkout before a proposed reminder arrives still belongs in the outcome group originally assigned to that reminder policy. Removing that customer afterwards would change the comparison.

How far can these findings travel?

The original challenge covers customers who had purchased during the prior year. It is a historical retail marketing experiment, not a sample of checkout abandoners and not a test of iMessage, SMS or modern automated cart flows. Its two week outcome field indicates whether a customer purchased; it does not count how many orders that customer placed (Hillstrom experiment, independent analysis, Nov 2026).

No delivered message, open, click or actual exposure field is available. Calling all observed visitors email clickers would add an event the dataset never recorded. Likewise, the no email group may have encountered other marketing, which remains part of the experiment's historical background rather than a measured absence of every channel.

The transferable lesson is about keeping the decision's denominator intact. The numerical rates remain attached to the original source population and period. They should not be used as a sales promise or as a target for a present day DTC store.

For a different preassignment question, Email recency statistics examines purchase history before campaign assignment. Selecting customers using information known beforehand is different from selecting them using whether they visited afterwards.

Can the purchase change be reconciled into arrival and visitor-rate terms?

Because every purchaser in this file also has a visit flag, assignment-based purchase probability equals the probability of visiting times the purchase rate among observed visitors. That exact identity explains how a small conditional-rate change can coexist with a meaningful assignment effect. It does not make either component a separately identified causal mechanism.

A symmetric arithmetic decomposition divides the product change into change in visit probability × average visitor purchase rate and change in visitor purchase rate × average visit probability. For womens email versus no email, those terms are 2.540 and 0.571 purchasers per 1,000 assigned, summing to 3.111. For mens email, they are 4.691 and 2.114, summing to 6.805. The average in each expression is the simple average of the two arm values. (Hillstrom experiment, original product decomposition, Nov 2026).

Assignment-based purchase differenceVisit-probability accounting termVisitor-rate accounting termTotal
Womens email versus no email2.5400.5713.111 per 1,000
Mens email versus no email4.6912.1146.805 per 1,000

Why are these not percentages of purchases caused by each stage?

The visitor rate uses a postassignment selected population. The campaign can change who visits, so its conditional-rate term mixes composition and behavior. Labeling the first term “purchases caused by traffic” and the second “purchases caused by the website” would add causal interpretations the randomization does not isolate. The decomposition is an accounting of two observed arm summaries, not a mediation experiment.

There are also several valid algebraic decompositions. Using the control visitor rate as a baseline assigns the product interaction differently from the symmetric formula. The total difference stays the same, while component shares can change. That dependence is another reason not to treat an accounting allocation as an observed mechanism share.

How does this improve a campaign report?

Keep the assignment purchase difference as the decision outcome, then show visits per assignment and purchases per observed visitor as separate descriptive diagnostics. A campaign that attracts more marginal visitors can improve total purchases while changing the visitor conversion rate little or even reducing it. A traffic or landing-page team should inspect the actual arrival mix before concluding that the page deteriorated.

If the next decision concerns a page change, randomize the page after an explicit arrival eligibility point. Keep that new experiment's population separate from the original campaign assignment. If the next decision concerns message policy, keep every originally assigned eligible customer, including nonvisitors and those who return without clicking an email.

The exact accounting decomposition and raw reconstruction verify the identity on all three arms. They export the terms in purchasers per 1,000 assigned customers, never relabel visits as email clicks, and do not supply new confidence intervals for causal stage effects.

Methodology and Sources

Kevin Hillstrom's primary challenge page, dated 20 March 2008, describes 64,000 previous customers, three randomized campaign arms and a two week outcome period. The page was opened on 13 November 2026. The analysis uses the retained public CSV archive and checks its exact compressed byte hash before reading it. The original page defines visit and conversion as separate binary outcomes. November 2026 citations marked independent analysis refer to the present reanalysis, while the experiment remains historical.

Counts are grouped by original assignment. Assignment based risk differences divide purchasers by all assigned customers. Visitor only differences divide purchasers with a visit by observed visitors. The audit verifies that all purchasers have a visit flag, all three arms partition the file and no records have missing cells. Individual difference intervals use an unpooled binomial standard error and 1.96 normal critical value. Arm proportion intervals in the data export use Wilson limits. A two comparison sensitivity uses the Bonferroni normal critical value 2.2414. The treatment comparisons share the same control, so they are not independent estimates to add together.

This is an exploratory reanalysis. A proposed sign reversal was checked and rejected. The visitor comparisons are explicitly descriptive because visitor status is measured after assignment. No causal landing page effect, individual persuasion label, click attribution or present day campaign benchmark is inferred.

Glossary

  • Assigned customer: A customer included in an original randomized campaign arm, whether or not a visit followed.
  • Visitor: A customer whose source visit indicator equals one during the outcome period.
  • Purchaser: A customer whose source conversion indicator equals one during the outcome period.
  • Risk difference: The treatment purchase proportion minus the comparison purchase proportion.
  • Percentage point: An absolute difference between percentages, distinct from a relative percentage change.
  • Conditional rate: A rate calculated within a selected subgroup, here observed visitors.
  • Post assignment selection: Choosing which records to compare using something observed after randomization.
  • Bonferroni adjustment: A way to use more conservative intervals when reporting several comparisons.

Sources

Cite this page

Use this citation for the original analysis, retaining its population, period and measurement definitions.

You Left Something. "Email conversion rate statistics by denominator." You Left Something, November 2026. https://youleftsomething.com/guides/email-conversion-denominator-statistics/
<a href="https://youleftsomething.com/guides/email-conversion-denominator-statistics/">Email conversion rate statistics by denominator</a> (You Left Something, November 2026)