Welcome to Press This, the WordPress community podcast from WMR. Each episode features guests from around the community and discussions of the largest issues facing WordPress developers. The following is a transcription of the original recording.
Powered by RedCircle
Doc Pop: You’re listening to Press This, a WordPress Community Podcast on WMR. Each week we spotlight members of the WordPress community. I’m your host, Doc Pop. I support the WordPress community through my role at WP Engine, and my contributions over on TorqueMag.Io where I get to do podcasts and draw cartoons and tutorial videos. Check that out.
You can subscribe to Press This on Red Circle, iTunes, Spotify, your favorite podcasting app or you can download episodes directly at wmr.fm.
Now in the realm of marketing and web development, I think we might sometimes fall victim to superstitions. I’m sure I do. For example, when designing a contact form on a site, we might feel that certain decisions might cost us thousands of customers, but we might not always have the data to back up those decisions.
Instead, sometimes we might rely on previous experiences in pushing those superstitions forward, or just hunches, or even things that we heard someone said once that we just took to heart. Today I’m going to talk to Viola Gruner, the head of marketing at Inpsyde, the biggest WordPress agency in Europe.
Viola recently gave a talk at WordCamp Europe 2023 about “Testing Instead of Guessing: Generate More Leads Through Growth Hacking.” We are going to talk today about how to do that, how to test and confirm things and not just rely on our hunches. Viola, welcome to the show. I want to start off by just asking, what is your WordPress origin story? How did you join our community?
Viola Gruner: Hello. Thank you very much for the invitation. So for my story of coming to WordPress, I haven’t been in the WordPress community very long. It’s now just a year. I worked for a startup and we scaled up and we were building a website that used WordPress. One day this website crashed because we didn’t update it.
I wasn’t involved in this whole topic, but I was there when it broke. So, I met with Inpsyde at this moment. They tried to help me fix the website. And when we were growing with the startup, the founder said, “Hey, we need to relaunch.” So we decided to relaunch. And then the founder said to me, we will now use Type3 instead of WordPress. And that was the day when I quit. And then I came to Inpsyde and they showed me the WordPress community. So yeah, it was amazing seeing this community and being there. And at my first WordCamp US, that was my first big experience with it.
Doc Pop: What year was that?
Viola Gruner: It was last year.
Doc Pop: San Diego?
Viola Gruner: Yes, San Diego, it was amazing. Great people, a great community, amazing spirit. Before I worked for a logistics company and it was always a little bit about competition and when I came to WordPress, I saw, wow, it’s one big community. And we are one big team and we want to make WordPress bigger. It’s amazing with amazing people.
Doc Pop: So congratulations on your talk at WordCamp Europe 2023. You gave a talk, titled “Testing Instead of Guessing.” And you were talking, I believe to marketers, was sort of your audience, about the importance of testing ideas rather than running on assumptions. What inspired that talk?
Viola Gruner: When I started working in marketing, I was very inexperienced and young. I still studied, and I was working for the startup and they told me, “Hey, you are now making the whole decisions for this company.” And I was like how do I make decisions? And so instead of saying, “My gut feeling is saying something,” I started to test. I started to test everything or every bigger decision. And through this process, I saw, very often, all books are saying this kind of thing, but if you test it for your target audience, there are other results sometimes. I tested a lot and I tested a lot with my team and we scaled everything up and it was amazing seeing this and it was amazing to learn.
To see that the target group is always a little bit different. And sometimes we expected that this test will be a success, and sometimes it was, and sometimes it wasn’t. We tried to test significantly. And it was also great for my team because often I heard marketers with a team saying “Hey, you need to do this and that, because of my gut feeling and I’m experienced.”
But during my experience, I saw if someone says this, there is a risk. My experience showed me, you need to test often to really say that will work or not.
Doc Pop: I mentioned marketers being superstitious. I think I definitely come from that school. I was in print before WordPress and in print and marketing, we always had these beliefs, like the customer has to see something three times before they actually act on it. This rule of three.
And that was partially because we didn’t have the ability really to test from doing a weekly newspaper. It was kind of hard to quantify how many people saw something and how many times it took them to act on it. But then I also think beyond the fact that was before this technology allowed us to test, I think even now there’s still some things in marketing. Like we should sponsor this event, but maybe it’s hard to quantify it. You have to rely on it should do well. It felt like it did well and it feels like something we need to do.
So I feel like marketers are kind of balancing that. But you were talking specifically in your talk about maybe somebody wants to add on a contact form, one of the most important parts of your site. Getting people to convert, to sign up for what you need or sign up for the newsletter. And somebody on the team might want to add an extra dropdown bar for some reason. And it’s very important to them.
And it feels to you, maybe that this is a terrible thing. We’ve optimized this flow. And so you’ve got these two people who are working on the same team, but they have these different priorities and different hunches. How would you solve that conflict? What would be the first thing that your team would do to figure out what the right decision is?
Viola Gruner: That’s funny that you say that because we had exactly this kind of small conflict. It was not really a conflict but the sales team told us, “Hey, we really need one more form field.” And as you said, for a marketer adding one more form field means, our conversion rate will drop. If you research it, they always say, take care of your form field because the conversion rate will drop. So the first thing I did was ask the sales team, “Why do you want us to add one more form field and what form field do you need?” And they told us, “Because of quality, we want to save time. We need one more form field about volume so we can qualify them better and we can save time.”
So for me, the process was writing their ideas and goals down in a one pager. And what do I think happened? I created a hypothesis, always numbers based, what does it mean adding one more form field? If you research this kind of question, you’ll find a lot of data and the data normally says, by adding one more form field, the conversion rate will drop. And so we added numbers based information that said, the conversion rate will drop from 7% to 6%.
I added the experiment design, what will it look like, required duration, how long we need to run this to get a significant result. I presented it to my team and after pitching an idea, the team decided to go with the ICE model, which means Impact, Confidence, and Ease.
We rated this idea on how big is the impact for our conversion rate? If it drops, for example, from 0 to 10, oh my god that’s a big impact. We rated on confidence, how much do we think this test will work or will fail? And my whole team said, yeah the conversion rate will drop. And how easy is it to implement this test? And out of these results, we decided on the best test to start with. So we often collected a lot of different tests.
Every other week we had a growth hacking meeting, and everyone came up with an idea, always in the same structure. We rated it and decided afterwards where we were going to start. So the higher the number was at the end, out of the three numbers from the ICE model, the sooner we ran that test.
And for example, with the contact form, we agreed the conversion rate would drop, so we knew we needed to start with that test. And we started this test and we got the results and we were very surprised about it.
Doc Pop: Let’s take a quick break here. And when we come back, you can tell us about what surprised you and what the results were and get into a little bit about some of the tools that you use for testing. So stay tuned for just a quick commercial break. And when we come back, we’re going to keep talking to Viola Gruner, Head of Marketing at Inpsyde about testing instead of guessing.
Doc Pop: And welcome back to Press This a WordPress community podcast on WMR. I’m your host Doc Pop talking today to Viola Gruner, Head of Marketing at Inpsyde and Viola was just telling us a story about adding a modification to a contact form. And there was a hypothesis that this would maybe result in fewer signups, which is, I think, what a lot of marketers feel.
Instead of just going on that hunch, Viola, you and your team did a whole bunch of testing on this. And right before the break, you said you were surprised. What surprised you about the results here?
Viola Gruner: We were all surprised, the whole team, because we thought, if we added one more form field, our conversion rate will drop from 7% to nearly 6%. And the results were that it didn’t drop. So, we were so surprised that the conversion rate didn’t change significantly. We were running this A/B test for 12 weeks and around 45,000 people visited the two different pages. And we saw there was no difference between the conversion rates. This means we can implement the form field on the new form. That was a big surprise for us.
Doc Pop: And I think that’s a good example of exactly what you’re trying to say, nobody should be just relying on their hunches. Doing testing is easy these days, much easier than back when I was working in weekly newspapers.
Kind of specific to WordPress, what are some of the tools that your team uses for testing or, what do you recommend for people who want to do, A/B testing would be the main form of testing these days.
Viola Gruner: So I personally was using two different A/B testing tools. So there is one free tool called Google Optimize. And there’s another tool from HubSpot and it’s already included but you pay for it. So with these two tools we were testing. But I also created our own workflow with different sheets and I structured everything. It’s simple to have this kind of workflow where you collect everything, and then you have your pitch card where you talk to your team and you write everything down there.
It’s also important that you document which tests were successful and which aren’t running like you expected. So you should celebrate the test if you finished it and document it well.
Also if it fails for example, you can check it out again in a year and you can think about how you can make this test better for the next time? And could we test this again? And of course, for mathematics, there are different online tools, for example, an A/B test guide that helps you to see how many visitors do I have? How many unique visitors do I have? And it’s calculating for you, how long do you need to run a test to get a significant result?
Doc Pop: I’m kind of wondering, are there some mistakes marketers might be making, is it possible to over test? Or are there any other kind of common mistakes that you’ve kind of come across that you just maybe wanted to talk about here?
Viola Gruner: I’m a big fan of testing [laughs] but I think you also need to be careful to not over test. Sometimes it’s okay listening also to the gut feeling like you said before, if you are going to a big event, and it’s harder to test here and to get significant data. That’s one thing I want to say.
The other thing I want to say is also, that it’s very important that if you run a test, for example, if you run a test on your contact form, it’s going to influence your conversion rate. So if you are going to test something else at the same time, it can also influence conversion rate. So you need to be always looking at your key metric.
For example, if my key metric is the conversion rate, I cannot say spontaneously, I’m running two different tests because they can influence each other. It can influence your conversion rate. So you could say, I will run a social test on social media, and I will run a test on my website. That’s a possibility you can do but be careful that you’re not running two different tests for the same key metric.
Doc Pop: Yeah, that makes a lot of sense. Definitely don’t cross the streams, and mess up your data. Prioritize one test at a time, or at least have different groups that aren’t being tested on at the same time.
Well one thing that you mentioned earlier in the show is you have a growth hacking meeting at Inpsyde, like a kind of weekly growth hacking meeting. What is growth hacking? How do you describe that?
Viola Gruner: So there are a lot of different descriptions for it and many people are using it already. They’re growth hacking this and that. But for me and my team, making number based decisions and changing things without investing a lot of money.
So it means to come up with new ideas with clever ideas without really having that much money for it and test it. In the previous company I worked for, we were growth hacking a lot. And we had very successful tests with some landing pages and we could improve the conversion rate a lot without really paying a lot. It was our time and that’s it. And we were growing a lot.
Doc Pop: I think that’s a great spot for us to take our final break. And when we come back, we are going to talk to Viola Gruner about the one final message I got from the talk that you gave, don’t be afraid to fail test. I want to hear more about that. So stay tuned after this short commercial break for more Press This.
Doc Pop: Welcome back to Press This. I’m your host Doc Pop talking to Viola Gruner today about testing instead of guessing, and we’ve talked about what is growth hacking. We’ve talked about how and why you should be testing your ideas rather than just sticking to the hunches or the previous experiences you’ve had. And I guess it should also be mentioned, and you mentioned this at the end of your WordCamp Europe talk, you said every target group is different. And I feel like that’s also maybe a reminder that just because you ran this test once at a different company or once even at the same company, but two years ago, that doesn’t mean never test again.
Like this is probably a chance to reevaluate things because you’re not always going to have the same results each time you test it. Also at the end of your talk, you said, don’t be afraid to fail test. And I thought that was really interesting. Can you tell us what you mean by that?
Viola Gruner: There is no failing for me if you test. Every finished test, like I said is a success because you get to know your target group better. And if you have a team, the nice side effect also is that you made something new with them. You gave them the chance that their voice gets heard and it’s indescribable motivation for them to see if this test works or not. And you learn a lot.
You should see every test you did as a learning opportunity and it brings you further. It makes you understand your target group better, step by step. And like you said, if you change to a different company, it doesn’t mean that this test will work or not. So many people came to me asking me, what would be the perfect test for my company? And I said, there is no perfect test for your company, or perfect growth hack. You need to test.
Doc Pop: Well, I really enjoyed your WordCamp Europe talk. Congratulations again. You mentioned in the talk that you have a workflow that people could download. People can follow along with that.
I didn’t make it to WordCamp Europe this year. Did you have a good time there?
Viola Gruner: Wow, yes, it was amazing. It was great. It’s a great community and people are so kind and open. And it was also very well organized. And I feel so thankful that I was allowed to be a part of this, and that I had the chance to speak about this very important topic for me.
Doc Pop: That’s great. Thank you so much for joining us today, Viola. If people want to follow you, what’s a good way to keep track of what you’re working on.
Viola Gruner: Viola Gruner that’s my name on LinkedIn. Or Twitter at GrunerViola that’s it.
Doc Pop: Thanks for listening to Press This, a WordPress community podcast on WMR. Once again, my name’s Doc and you can follow my adventures with Torque magazine over on Twitter @thetorquemag or you can go to torquemag.io where we contribute tutorials and videos and interviews like this every day. So check out torquemag.io or follow us on Twitter. You can subscribe to Press This on Red Circle, iTunes, Spotify, or you can download it directly at wmr.fm each week. I’m your host Doctor Popular I support the WordPress community through my role at WP Engine. And I love to spotlight members of the community each and every week on Press This.
