There’s an old saying in annual giving. Test, don’t guess.

Testing has become increasingly popular in recent years. In fact, according to an AGN poll, 80 percent of annual giving professionals indicated they plan to do more testing in the next six months. It’s viewed by many programs as the simplest and low-risk way to see how potential changes will perform before rolling them out to the entire audience or implementing them permanently.

In direct appeals, the practice of A/B testing involves sending two almost identical packages out at the same time to similar audiences (in terms of size and makeup) with a single variable changed in only one of the packages. Everything else remains the same. Examples of variables that can be tested are the color of an envelope, the length of a letter, or the name of the signatory. It could also involve adding an element to one of the packages like a teaser line on the outer envelope, a brochure, or a premium offer. The list of possible things to test is almost endless. Then comparing the results of the two packages can help determine which one is more effective.

Sometimes, however, more can be learned when elements are removed from a control, rather than added. This technique is often referred to as “testing backward”. Here’s how it works.

Take an existing package and consider what’s essential vs what could be eliminated without causing the results to be any less. Could a two-page letter be consolidated into one page? Could a long letter be condensed into a simple postcard? Could the reply device be a tear-off instead of a separate card? What costly elements could we take away from this piece and still be successful?

Saving money (and time!) are great reasons for testing backward. Perhaps an invoice-style letter appeal will bring in more money than an expertly designed and beautifully printed (and expensive!) mail package. If so, the appeal’s cost to raise a dollar will decrease accordingly. The other reason to test backward is to determine if there is any positive significance to taking a simpler, more streamlined approach. Since cost savings are at the core of testing backward, a response that is the same or even close is a successful test.

James Madison University is currently testing backward to see if it can reduce the cost of its fall renewal appeal that goes out with a copy of its annual report. Although already a successful appeal, they are looking to increase their return on investment by reducing costs. Segment A is receiving a package with a printed cover letter and a printed report. Segment B is receiving a package with a printed cover letter and a link to view the report digitally online. If it turns out that segment B (no printed report) generates the same return in terms of donors and dollars then it could reveal a way to save thousands of dollars in printing and postage costs next time they do the mailing.

Karen Risch Mott, Director of Annual Giving at JMU said, “It doesn’t make sense sending this expensive piece for our renewal appeal if the digital version gets a similar response. We can save that money and use it somewhere else in our program.”