If you’ve ever attended a seminar, or read a book or website post about nonprofit email communications, no doubt someone’s told you you should be testing and optimizing your emails. For small nonprofits, figuring out how to do testing can be a daunting task, even more so in the case of year-end email fundraising appeals, which hit in the midst of an already overwhelming fundraising season.
So why should you make the time to design, plan, and implement a test of your email fundraising appeals?
Why bother testing?
When it comes to a year-end email campaign, your goal is to remind and inspire your past donors to renew their gift, whether online or through another channel. Testing will help you figure out how to create an effective campaign and continually improve the process over the years.
For example, you might test the effectiveness of one landing page or donate button over another. You could also test whether personalizing your subject line increases response rates. The options are effectively endless.
If you are just beginning to incorporate email into your fundraising strategy, you’ll need to know (and often prove to your board) that your emails actually raise additional money for your organization. A simple test can evaluate this and determine if it is worth the effort.
Also, since emails are much easier and cheaper to test than a hard copy direct mail appeal, testing your emails can be a good way to get your feet wet, learn how to do it, and demonstrate its value. Then you’ll be in a good position to incorporate testing into other areas of your fundraising.
Designing your test
Once you’ve decided to take the plunge, how do you get started?
Let’s say you want to determine whether your final email appeal gets a better response delivered on December 28th or December 31st. Here’s how to set up your test:
Step 1: Make a Plan
Before you jump right in take some time make a plan and write it down. Start by clarifying the one question you are trying to answer. That is the only variable you’ll use – everything else about your email will stay the same. In this particular case, we’ll only change the date of delivery. Make a timeline including every action you’ll need to take and determine when you’ll have enough results to start collecting and analyzing your data.
Step 2: Identify test donor pool
Pull a segment of donors (around December 27th) who haven’t yet responded with their annual gift. Use a minimum of 100 donors in order to get a significant results.
Step 3: Separate your donors into 2 random groups
Divide that group (of non-responding donors) into two separate lists split alphabetically by last name. This is an easy random division and a clear way to know at a glance to which group a particular donor belongs.
Many email programs will do randomized split testing for you, so that you won’t have to manually divide your list by last name. You’ll need to create a new list for your segment of non-responsive donors within your email program. Make sure your email provider lets you see which donors belong to which group. You’ll need to know this to measure giving.
Schedule email delivery to Group 1 on December 28th.
Schedule email delivery to Group 2 on December 31st.
Step 4: Tracking
A week or so after delivery, you’ll need to track responses (clicks and gifts) from both groups. Hopefully your donor database or email program will allow you to identify those donors as members of either group 1 or 2. If not, create a simple spreadsheet for each group where you can manually note when someone on the list makes a gift. This will obviously be more manageable for a relatively small list. Make sure to record when gifts are made through other channels besides just online.
Step 5: Data Collection
For each group, you’ll want to gather the following stats and calculations:
- Click-through rate (from your email provider)
- Percentage making an online gift
- Percentage making a gift through another channel (such as mail)
- Overall response rates (through any channel)
- Total number of gifts
- Total amount of gifts
- Average gift
- Response rate
Step 6: Analyze your results
Determine the % change in responses between groups. Here’s a mathematical reminder of how to calculate % change:
Where “group x” is the group with the largest response rate (or whatever else you’re measuring):
[(% response rate of group x – % response rate of group y) / % response of group y] x 100 = % increase in response rate.
Step 7: Determine Outcomes
Document everything you did and include any conclusions and recommendations for future appeals based on your results.
If you are prepared with a plan once December rolls around, and assuming you have a pretty good understanding of your email software and donor database, setting up a test of your year-end email fundraising communications can be easier than you might think!