How LeLoupe measures, and how sure it is.

Everyone sees everything. In a content or concept test, each respondent sees every piece and answers the same questions about each one. Comparing pieces within the same people is fairer and needs far fewer respondents than giving each piece its own group.

• Order is shuffled for each respondent, on our server, so no piece gains from going first and the order can’t be chosen from the browser. Each respondent’s order is recorded.
• The questions appear after a short pause (1 second), so people look at each piece before they answer.
• One question is on screen at a time, beside the piece. To fix a slip, people can go back one question within a piece, but no further, and never to a piece they’ve finished.
• The final ranking starts in its own random order, not the viewing order, so someone who doesn’t move anything doesn’t simply rank in the order they saw. People tap the pieces in order, favorite first, or drag them.
• Nobody is asked to critique your work. There are no open comment boxes.

The questions come from an established advertising copy test, with LeLoupe’s own opening question and pricing questions added. In order:

1. How does this make you feel? Five faces, from very negative to very positive. Then, revealed beneath it: how strongly do you feel that way? Five figures, from mildly to very strongly. Anyone who picks the strongest is asked, beneath that, why do you feel so strongly? Answering is optional, and it isn’t scored.
2. Three word questions. Each asks them to pick the word that best fits their reaction (in tests launched before 30 October 2026, how the piece makes them feel) and offers four feeling words, shown in an order of their own for each respondent: Excited, Content, Annoyed or Bored; Curious, Relaxed, Frustrated or Indifferent; Inspired, Calm, Uncomfortable or Lukewarm. The structure comes from James Russell’s circumplex model of affect (Journal of Personality and Social Psychology, 1980): in each question, one word is pleasant and stirred, one pleasant and quiet, one unpleasant and stirred, and one unpleasant and quiet. Each word’s place rests on published ratings of how pleasant and how stirring it is (Warriner, Kuperman and Brysbaert, Behavior Research Methods, 2013). Tests launched on 20 October 2026 offered Astonished, Tired, Delighted, Droopy and Alarmed in place of five of these. Tests launched before then offered the copy test’s own words (Interesting, Boring, Nice or Irritating; Soothing, Unpleasant, Unique or Ordinary; Mellow, Disturbing, Involving or Weak), scored the same way. Results from different sets of words aren’t like for like.
3. Brand recall: would they know it came from your brand, from “it could be for anything” to “it couldn’t be anyone else”. Tests launched before 30 October 2026 asked which phrase applies, from “could have been for almost anything” to “couldn’t help but remember it was for” your brand, on the same five-point scale. Asked with Content effectiveness in tests launched since 21 October 2026, and part of the Effectiveness score.
4. Something new (how much new information it gives), relevance, brand appeal (how it made them feel about your brand) and use (whether it makes them more or less likely to use you).
5. Price: an acceptable price (tests launched before mid-October 2026 asked what they’d be willing to pay), the most they’d find acceptable, and a price so expensive they wouldn’t consider it (a prohibitively expensive price, in tests launched before 30 October 2026). If a price runs backwards against one they gave before (the most below the acceptable price, say), the question says so quietly; they can keep it. Tests launched since October 2026 also ask the price that would be so cheap they’d doubt its quality. If you set prices to try, two “Would you buy it at $X?” questions follow.

After every piece, respondents rank them all from most to least favorite. Questions about your respondents, if you add any, come before any piece.

Value rank tests work differently. Respondents see sets of four or five features, add-ons or plan options and pick which is most, then least, worth paying for. Each option appears about three times per respondent. At the end, they mark any option that would put them off and, in tests launched from 25 October 2026, any they’d be disappointed not to get.

Every score is an average over respondents. The Effectiveness score, its parts and the First impression score run 20–100 with 60 as neutral: on the copy test’s questions, the lowest answer scores 20 (25 for Use), not 0.

Effectiveness score

The average of seven diagnostics: Feeling, Feeling intensity, Something new, Relevance, Brand appeal, Would use more and Brand recall. Each runs 20–100 with 60 as the middle answer, so the score does too.

Feeling intensity follows the feeling. How strongly someone felt counts in a piece’s favour when their face was positive, against it when it was negative, and not at all on the middle face. It sits on the same footing as the other parts: 100 if everyone felt very strongly positive, 20 very strongly negative, 60 for the middle face or an even mix. So a piece people strongly dislike loses points, and Feeling intensity counts no more and no less than any other part.

First impression score

Combines which way each person felt with how strongly they felt it, so a loud reaction moves it more than a mild one. It runs 20–100 like the Effectiveness score: 100 if everyone chose the most positive face very strongly, 20 the most negative face very strongly, and 60 is neutral. A clear winner is 68 or more and a clear loser 52 or less, with the range of error clear of 60.

Reaction strength

The share of all word picks that were strong words (Excited, Curious, Inspired, or Annoyed, Frustrated, Uncomfortable; on 20 October 2026 Excited, Astonished, Delighted, or Annoyed, Frustrated, Alarmed; in earlier tests Interesting, Unique, Involving, or Irritating, Unpleasant, Disturbing) rather than mild ones. It measures how strongly people reacted. The bar beneath it shows which way: the negative share in red, the positive in green. Each word pair also shows which of its two words people leaned toward.

Ranking

Where people placed a piece when they had to put them all in order. A piece everyone ranked first scores 100; one everyone ranked last scores lowest. Beside it, we show the share who put it in each place.

Brand fingerprint

Whether an idea leaves a mark that’s yours. It combines three things: remembered as your brand, warmer toward you, and unlike what people have seen. It is in the results export; each of its three parts is shown on the results page.

Price range

• Each price figure is the middle answer (the median), not the average, so one person typing $5,000 can’t drag it.
• Acceptable prices: from what people would happily pay to the price that puts them off. With the “too cheap” question, we use Van Westendorp’s range and optimal price (shown as the balance point) instead, once 10 people have answered all four.
• Best-revenue price: among what people said is the most they’d pay, the price that would bring in the most if people paid what they say they would.
• Premium over the others: how much more or less people say they’d pay for this piece than their own average across the pieces, then the middle of those, so someone who prices everything high doesn’t count for more.
• Would they buy: at each of your prices, the share answering Definitely or Probably would. Each respondent is asked about two of your prices, drawn at random on our server.

These are what people say they’d pay. People say yes more readily than they buy, so treat them as a range to start from, not a price tag.

Value rank tests

Each option scores by how often it was picked as most worth paying for against how often it was picked as least, out of the times it was shown. It runs from −100 to +100, and the average option sits near 0. A leader is clearly above average. A killer puts off 20% or more of respondents, whatever its score, and clearly more than 15%. One at 20% that isn’t clearly past 15% yet is a possible killer. Where the disappointed question was asked, an option that isn’t a leader or a killer but that half or more would be disappointed not to get, clearly more than 40%, is table stakes: people expect it rather than pay extra for it. One at half that isn’t clearly past 40% yet is a possible table stakes. Leaders, killers and fillers follow Monetizing Innovation; it gives no number for table stakes, so ours is a majority, with the floor set the way the killer’s is. The rest are fillers. “Clearly” here means what it means below, under How we decide a result is clear.

We don’t show p-values or significance stars. They mean little to most people, and they invite checking again and again until something looks like a win. Instead, every result says in plain words how sure it is, and those words come from a real calculation.

A range around every number

Each score is worked out one respondent at a time, then averaged. The spread of those per-respondent scores gives a 95% range (a t-interval) around the average. That’s the “±2.5” beneath a score: with few respondents it’s wide, and it narrows as more come in.

Comparisons are paired

Because every respondent sees every piece, we compare two pieces by each respondent’s own difference between them, then take the 95% range of the average difference. That cancels out people who score everything high or low. It is tighter, and fairer, than comparing two averages.

Looking as often as you like

You can open results while a test is still running, and come back as often as you want. An ordinary 95% range isn’t built for that: check it after every new respondent and, sooner or later, a gap that is only chance will look clear. So whether a piece leads is judged with a range built to be checked any number of times (an anytime-valid confidence sequence, from Howard and colleagues, 2021, and Waudby-Smith and colleagues, 2023). It is wider than an ordinary range, so it takes a little longer to call a lead, but a lead it calls stays honest however often you look.

Each test you pick is a separate chance to name a winner, so the 5% chance of a false lead is split evenly across the tests you picked that can name one. Picking more tests doesn’t make a false “Signal found” more likely.

The ± beneath a score is still the ordinary 95% range: it describes the number, it doesn’t declare a winner.

What the words mean

• Clear winner: the piece is ahead of every other piece, and each paired difference’s range, as above, is above zero.
• Too close to call: it’s ahead, but the gap is within the range of error for this many respondents. No winner is named.
• Price range uses a sign test, because prices are compared by their middle answer. Among respondents who priced two pieces differently, the share who priced the winner higher must be clearly above half.
• Value rank is judged the same way: each respondent’s own score for the top option against every other one.
• First impression shades a piece as a clear winner or loser only at 68 or more, or 52 or less, with its 95% range clear of 60. Whether one piece is the clear winner on it is judged like the rest.
• Colors under Relative grade stay pale when a score’s gap from the middle is inside its own 95% range, so a tiny gap never shows as deep green against deep red.
• Too few to read: with fewer than 10 respondents behind any piece in view, the results say so instead of showing scores.
• Signal found: at least 10 respondents and a piece clearly ahead on one of the tests you picked: First impression, Effectiveness score, positive reaction, Ranking, Price range or Value rank.

How many respondents you need

The builder works out Target respondents with a standard power calculation on the same rule: enough respondents that a real lead shows as clear about four times in five. How big a lead counts is set by how different you say your pieces are, measured against how much people’s answers vary, so one answer works for every test: a large lead for clearly different pieces, a medium one for somewhat alike, a small one for variations of one idea (0.8, 0.5 and 0.3 standard deviations, the usual large, medium and small sizes). A clear order of every piece checks each one against its neighbour, so it needs more. So does each test you add, and looking as results come in. For 4 pieces and two tests that can name a winner:

There is no ceiling: pieces that are very alike can need hundreds of people, and the builder says so. These sizes are a starting point. No LeLoupe test has real results yet, so once enough real tests finish we will check them against what those tests found.

These checks run when results are read, so every test, old ones included, is judged by the same rules. Flagged respondents stay counted unless you choose Hide flagged respondents.

• Speeders: finished in under a third of the test’s median time, judged once 5 people have finished.
• Straight-liners: gave identical answers to every piece, or one answer down every 1–5 question of every piece.
• Possible repeats: completes from a network that had already completed the test. We store only a hashed (pseudonymised) form of the network address, never the address itself. Repeats are counted, never blocked, because offices share a network.
• Price answers that can’t be right are left out of every price figure: a ladder that runs backwards (willing above the most they’d pay, say), or a price over 10 times that question’s middle answer. The number left out is shown.

• The scoring is tested. Every score is checked against a fixed set of made-up answers (seven pieces, 200 respondents, two markets), so a change that moves any number is caught, and a small case is worked out by hand. LeLoupe’s own scoring (signed Feeling intensity, the seven-part Effectiveness score) and every newer measure have their own tests: Brand fingerprint, the ranges and paired comparisons, the data-quality rules, the pricing analyses and Value rank scores. We run them all before changes ship.
• Scores are worked out from the answers each time, never stored, so a rule fixed once applies to every test, and nothing drifts.
• One set of calculations. The results page, each piece’s page and the export all use the same scoring code.
• Guides are labelled as guides. The cut-offs behind the LeLoupe grade (strong from 68, weak below 62) are LeLoupe’s reading of the copy test’s results, not an industry standard, and we say so.
• No single “LeLoupe score”. Very different results can average to the same number, so we show Effectiveness score, Reaction strength and Ranking side by side instead of folding them into one.

• It doesn’t predict sales. It reads stated reactions from one viewing, in a test.
• Your own audience is friendly. Customers and followers tend to be kind, so scores can run high. Compare pieces against each other, and include a piece you already know as a baseline.
• Stated prices run high. People overstate what they’d pay. Use the range, not a single figure.
• Answer scales are averaged. As is standard in copy testing, 1–5 answers are treated as evenly spaced. We show the full answer breakdown beside each average, so you can see the shape behind it.
• One reference test isn’t a benchmark. The copy test is where the questions come from, not a norm your scores are judged against.

See the method on example data in the sample tests.