Sample Size Matters

How reliable is my test mailing response?

Enter the test mailing size and the response rate achieved.


The smaller the size of your test mailing, the less reliable it is to forecast a particular response rate in the future.  
This calculator suggests that, at the 95% confidence level, given the test sample size and the response rate achieved to the test (and all other factors being equal!), a rollout mailing will achieve a response rate within a range plus or minus a standard “limit of error” either side of the test response.
Again, the larger the sample size on which we are making our inference and the higher the response rate achieved in the test, the smaller will be our “limit of error” and therefore the smaller the variation in expected rollout response.
Thus, we come back to our first question “What sample size should I use?” and the answer is “as big as possible”, “as large as you can afford”!


Base Data
Direct Marketing & Data Specialists


If you achieved a test response of 1.80% on a mailing of 8,400 records, your “limit of error” is 0.28% - believe me, it is - and you can therefore expect a rollout response of between 1.52% and 2.08%.  This is the case no matter how big your rollout – if you can accept at least the lower predicted level of response (95 times out of a hundred and all other things being equal) you can continue to as large a rollout as you have available!  It is as well to read that caveat in brackets again!