I Reveal The Ugly Truth About Email Popup Subscription Forms

I added an email popup subscription form to this blog back in February.

I wanted to find out what effect an email subscription popup had on the blog.


I find them annoying and interfere with the user experience, but that’s just an opinion.

 Let’s settle this with cold hard data!

Creating My Email Subscription Popup

I took one of the templates that came with the plugin and modified it slightly.

I updated the font colours to match my branding and added a couple of award images.

my email popup

I think its looks pretty good, spells out the deal clearly with a clear call to action.

It is setup to pop after 7 seconds which gives readers time to fully load the page and begin reading.

Closing the email popup will prevent it from opening for another 24 hours. If you subscribe you never see it again.

How To Setup Your Own Email Popup

At first I tried to use the premium WPSubscribers email subscription popup plugin for wordpress and was shocked to find it was not compatible with any caching plugins.

When I get in touch with the developer he told me it wasn’t possible with this kind of plugin – I knew he was lying.

Eventually I came across NinjaPopups which costs just $18, is compatible with caching plugins, is bundled with lots of themes and has some cool features.

ninja popups wordpress plugin

It can be used to create opt in popups, opt in lockers, social lockers, exit popups, page specific popups and a bunch of other stuff.

Certainly get a great amount of bang for your buck! The developer is super helpful as well.

Setup Instructions

It takes minutes to setup your first email popup form-

  1. Upload the plugin & activate it
  2. Go to Ninja Popups > Settings
  3. Configure to your liking – I disable it on mobile devices
  4. Make sure you connect Aweber or whichever service under the Mailing List Manager
  5. Go to Ninja Popups > Add New Popup
  6. Select a template & theme
  7. Configure the other settings (see below)

Here are the exact settings I use for the email popup form on this blog.

Click on the image to get the full version so you can see them all!

email subscription popup settings

But Is It Actually Worth It?

So what measurable effect has adding an email subscription popup actually had on the blog?

I deployed the email popup at 9pm on the 28th February 2013.

I’m going to compare before (Jan 9th-Feb 27th) statistics with after (Mar 1st-Apr 19th) which are 50 day periods.

I’m also going to segment this based on new & returning traffic.

New traffic has technically never seen the site before – this is their first impression.

Returning traffic – I love you guys, you keep coming back for more – I better look after you!

The Engagement Effect

First of all let’s take a look at how it affected engagement statistics like pages per visit, visit duration and bounce rate.

New Before New After Returning Before Returning After Total +/-
Pages/visit 2.42 2.15 3.28 3.02 -9.29%
Avg Duration 00:03:46 00:03:08 00:06:11 00:05:48 -10.20%
Bounce Rate 59.08% 64.51% 44.58% 48.39% -9.02

traffic engagement

Well thats not great now is it!

Does that email subscription popup put people off to the tune of 9.5% across the critical engagement metrics?

Perhaps the increase in conversions will make up for it.

The Conversion Effect

So if its doing that much damage to the user experience, what is it doing for conversions?

Lets find out!

Before The Email Popup

Week Subscribed Visits Conversion
Jan 7th 23 3,586 0.64%
Jan 14th 30 4,259 0.70%
Jan 21st 24 3,711 0.65%
Jan 28th 32 4,872 0.66%
Feb 4th 36 4,203 0.86%
Feb 11th 39 3,975 0.98%
Feb 18th 69 5,101 1.35%
Total 253 29,707 0.85%

Note: On the 7th February I activated a plugin that added every comment submission to the email list. That ended up with a bunch of junk emails on the list from spam submissions.

I had to clean up the data from Feb 7th-12th – luckily I setup ad tracking so know precisely where each subscriber comes from or this test would have being up a creek without a paddle.

After The Email Popup

Week Subscribed Visits Conversion
Mar 4th 105 6,334 1.66%
Mar 11th 108 6,502 1.66%
Mar 18th 60 6,029 1.00%
Mar 25th 82 5,936 1.38%
Apr 1st 73 5,268 1.39%
Apr 8th 79 8,804 0.90%
Apr 15th 54 6,873 0.79%
Total 561 45,746 1.23%

What Does It All Mean?

So before the email popup I had a conversion rate of 0.85%

After the email subscription popup I had a conversion rate of 1.23%

That’s a 44.71% increase in conversion – pretty good right?

But what does that actually mean in cold hard subscriber numbers-

Visitors # Subs Before # Subs After + / –
100 0.85 1.23 +0.38
1,000 8.5 12.3 +3.8
10,000 85 123 +38
50,000 425 615 +190
100,000 850 1,230 +380

In Summary

Adding the email subscription opt in popup had a noticeable effect in a few areas.

Last month my blog got 13,487 unique visitors, based on the numbers above I would have got 114 subscribers without the popup and 166 with it.

That’s an extra 52 subscribers from the same traffic.

However that comes at the cost of you guys dropping 9.29% pages/visit, 10.2% visit duration and 9.02% off the bounce rate.

That is certainly not in line with the conversion increase Darren Rowse saw way back in 2008 but I suspect that is because over that time we have become ‘popup blind’ and they have lost the impact they once had.

Plus his stats were from a photography blog which is much more likely to be receptive to the popup when compared to the seasoned internet marketing niche who see them on every other site every day of the week.

I also have to factor in how the other opt in boxes perform across the blog-

aweber statistics

Just for reference, About is from the about page, EndPost is at the end of every post, IncomeReport is the income report page, Inpost are the inline optin boxes, Sidebar well you can figure that one out.

The pop up is by far the best performing site wide email subscription option at 1.23% – but I don’t think the increase in conversion is worth the trade off in engagement metrics.

I’m going to spend some time tweaking the settings of the popup as well as the design & copy to see what can be done to increase conversion.

At 60-70% it would become worthwhile based on my site and my traffic and sometimes changing just 1 word can have that impact.

Should You Add One To Your Site?

Ninja Popups allows you to quickly and easily increase subscriber conversion on your blog.

When you’re starting out every subscriber counts, so that 44% increase in subscriber conversion is important.

Note: When someone has joined your email list you have earnt their trust and respect – don’t abuse that!

I believe this technique is much more effective in any niche that isn’t related to internet marketing or making money online.

Especially if you have a great proposition! Which I don’t really.

There are also a lot more things you can do with NinjaPopups over and above what I have shown you here.

As I try to fine tune the subscriber conversion number (even with the popup at 1.23% its poor) I’m going to try things like-

  • Add an email locker to part 2 of the tiered link building tutorial series
  • Add an email locker to the income reports with a 60 second delay to get the hook first
  • Tweak the forum to include inline opt in forms after the opening post
  • Tweak the copy & design of the existing email subscription popup to boost conversion further
  • Offering an exclusive resource of some type

If you do add an email popup to your blog, make sure you add a note/annotation in Analytics so you can easily compare the before/after yourself!

Test, test, test!

What are your thoughts on how the increase in conversion caused a drop in engagement?

 Is the 44% increase in conversion worth the 9.5% drop in engagement?