What comes first: Quantitative or qualitative data analysis?
15 Feb 2022

What came first, the chicken or the egg? This debate has been going on for hundreds of years, and scientific results are still somewhat inconclusive. For us in marketing, there are other heated debates. For example, what data is more important when trying to increase conversion rates and improve customer experience? Or what data comes first when analysing website or mobile app performance, quantitative or qualitative data? This article will explain what these two data flows entail, how to collect this data, and how the data should be used together to maximize website and app performance.

As a marketer, UX researcher, or designer, you want your website and app to offer a great experience to your audience. The goal is often to generate conversions, for example with sign-ups or purchases. To achieve these goals, it is essential to make use of all the breadcrumbs your visitors leave behind through their behaviour, actions, and feedback. Especially with the growing trends of using human interaction and empathy, it is time to understand your visitor’s thoughts and feelings to help them achieve their goals, and not just your goals. How? By analysing the entire digital customer journey with quantitative and qualitative website and app data.

We’re focusing on both App and Website analytics in this article as nowadays, over 50% of internet users are on mobile devices. That means you can’t improve the entire customer journey if you aren’t analysing app data as well.

What is quantitative and qualitative website and app data?

Quantitative data, or hard data, is numerical, measurable, and objective data. It tells you about the actions your visitors take and the frequency of these actions. It is about answering the ‘What?’ question:

• What are visitors doing on my website?
• What page has the highest conversion rate?
• Which button in my app is clicked most often?

On the other hand, qualitative data, or soft data, is non-numerical and more descriptive and subjective. It gives you more information about the context and what visitors and users think about your product. Examples are customer sentiment and customer loyalty. It helps to answer the ‘Why?’ question:

• Why are users uninstalling my app?
• Why are visitors not converting?
• Why are visitors not coming back to my website?

What is the benefit of quantitative data?

So quantitative data is numerical and can be measured, which helps you set specific goals and assess performance. Analysing quantitative data is often the starting point for companies looking to improve website and app performance. However, nowadays, there is an overload of quantitative data that can be collected and measured. The question is not, do I have enough data? It’s: Am I analysing the right data? The challenge is making sense of all the data and using the most important and essential information. It is about focusing on the data that is directly related to your targets and the overall business goals.

If your goal is to have more newsletter sign-ups, or to get more in-app purchases, you have to improve each step your customer takes to convert. How do visitors that convert find you in the first place? What pages do they visit? What buttons do they click? How far do they scroll? And how often do they leave and come back before making a purchase? Assessing the entire digital customer journey is what will help you understand where to make improvements.

The basic quantitative web and app data

We would consider some quantitative data more basic than others like number of visitors, bounce rate and page visits. This is data that you can get from some of the most known tools out there, such as Google Analytics.
But there are tools that help you map out the entire customer journey way more in-depth. These tools help you understand exactly how individual visitors and users get to convert. We’re talking about user recordings, heatmaps, and funnel analytics.

More advanced quantitative data

Heatmaps show you a summary of where users click most often, and what buttons are used to further explore your website or app, and which lead to the most conversions. User recordings show you exactly what a particular visitor is doing on your website or app, so you can see where individual users click, how they navigate and where they drop off or become idle. Watching recordings will show you where a user gets stuck or drops off, or how they get to the end of the funnel where they convert. Funnel analytics shows you how many users are at each stage of your funnel, and you can see where most people drop off before converting. This is the bottleneck where you need to focus improvements on.

You can even go further, for example with a software like Smartlook. It shows you the entire digital customer journey with both app and website analytics. When specific visitors come to your website or app multiple times, you can see their behaviour in their first and subsequent visits. It lets you identify your customers, so that you can watch website recordings and analyse the customer journey for that specific customer on both your app and website. This will help you see how converting customers get to the bottom of your funnel, and where and how others drop off before converting. This directs you to the exact place where you need to make improvements. If you’re going to invest time and money in making your website or app better for the user, you’re investing in the right place, making you save valuable resources.


Source: Smartlook

Quantitative feedback data

Although many people consider customer loyalty and customer satisfaction qualitative data, as it is based on subjectiveness of the person, it can be measured in quantitative data. A Net Promoter Score (NPS) or Customer Satisfaction Score (CSAT) tells you how you’re doing on a scale, and it can be measured over time and compared to different segments. You can also compare it with industry benchmarks. This data is therefore measurable. But what is important to consider, is that it does not give you insights into why they are satisfied, or why they would or would not promote you to their friends or family.

Is that it?

So, quantitative data can tell you a lot about your visitor’s behaviour and where to make improvements to help your customers achieve their goals both faster and easier. Using an in-depth and user-friendly tool like Smartlook for this can be more handy than using a tool such as Google Analytics, where marketers often need help from developers for event tracking and funnels.

Now you know where improvements are needed, hooray! That’s great news, but you’re missing half the story. Because… What’s next after you know where the bottleneck is in your funnel? What kind of improvements are needed? You might just make a change for the worse… Which is the last thing you want. So in the next step, you need to walk in the shoes of your customer and figure out exactly what they require. And that is where qualitative data comes in.

What is the benefit of qualitative data?

Qualitative data helps you gain an understanding of the reasoning and motivation behind why your customer has provided negative or positive (quantitative) feedback, why they get stuck on certain pages, or why they drop out of the funnel. We like to refer to it as the ‘why?’ as it exposes the root causes behind the behaviour we previously analysed. Examples of how you can collect qualitative data are:

• Feedback forms on your app and website (or even email)
• Surveys
• Usability tests
• User interviews

How to collect qualitative data?

With long surveys to your email lists, you can ask multiple specific questions about the ‘bottleneck’. However, this might not be the right audience to ask, they might not be the ones dropping out of the funnel or uninstalling your app. And you might not get enough response to actually get the answer you need.

Usability testing is yet another way to ask your customer the “why” question. You let them use your app or website, and while they are browsing through, they will tell you what they think and why they do what they do. This is very specific information and is commonly used during the development of a product. However, it is time-consuming and again, this might not be the right audience.

User interviews come quite close to usability testing. Instead of watching and listening to the user while they are engaging with your app or website, you ask them specific questions directly in an interview. User interviews can also be done in focus groups. But the questions might be too specific or not relevant to the person you’re interviewing.

Collect qualitative web and app data with feedback

Mopinion’s solution is feedback forms, providing that you use them correctly. With our feedback forms, you get to ask your visitors and users questions in real-time while they are in the customer journey. So it’s guaranteed that you’re asking the right person for feedback at the right time. For example, you can ask why visitors leave a certain page with an exit-intent trigger. You can ask them why they came to your website in the first place with a pop-up survey, or why they came back for a second or third visit. You can ask them how the customer journey went directly after they made a purchase with embedded forms. Or ask them whether they are missing something with a feedback button on all your pages or a dedicated place within your app.

Okay, you get the point, you can ask your customers very specific questions wherever and whenever. That of course does not mean that you should, you don’t want to cause ‘feedback fatigue’.

How to ask for qualitative feedback?

Asking your visitors or users the right questions at the right moment is key here. Luckily, there are many ways to do this in a subtle way, such as with embedded questions or small feedback buttons on your website or app. And you can make questions based on user behaviour, conditions, and much more. It does not have to be limited to the customer journey or conversions. It can be related to the experience, to the sentiment, and audience characteristics. If you ask them whether they achieved their goal on a scale, ask them what their actual goal is, or perhaps inquire why they achieved their goal or not. Or, you can find out whether the information provision on your website is sufficient, and ask what their age or function title is, so you can link the feedback you’re getting to a characteristic.

Don’t forget to give your visitors the option to leave their contact information to follow up after feedback. If there is a bug preventing them from converting, this will give you the opportunity to get them back into the funnel after fixing the issue. This is something quantitative data alone can not do for you.

How to analyse qualitative feedback data?

There are different ways to analyse incoming feedback data. When it’s quantitative feedback data, such as the NPS or CSAT, you can easily follow trends. Are these scores improving over time? Then plot the data against weeks. Are these scores different based on the user’s device? Then compare the data per device. Go ahead and put the data in a graph and upload it to your dashboard. In a blink of an eye, you will know how you’re performing compared to a week earlier and whether this is different per device.

If you see that a certain page gets a lower CSAT than other pages, you can look at individual feedback items. Here we really dive into the qualitative data. You can look at each comment to see what the issue is. Here you can also filter out open comments. At Mopinion, we have automated categorisation of open comments with our Artificial Intelligence, so that when an open comment meets a certain condition, it will receive an automated tag. For example, when open comments contain the word pricing, the comment will automatically receive the tag ‘pricing’, making it easier to go through and analyse open comments.

Or you can use our text analytics. Not only will this show the most frequent words, it will also show you the sentiment of the feedback. Perhaps the buttons on the page with a low CSAT aren’t working properly. The most frequent words might be: bug, broken, or problem. This will show up in your text analytics and show a negative sentiment. And in the open comments, you will see whether that is what actually happened to visitors. You can then fix the issue right away.

Empower your quantitative data with qualitative data.

Put the data together, and you’ll have loads of insights behind the ‘what?‘ and ‘why?‘ of your research. Combining numbers, ratings, and observations with qualitative and subjective feedback empowers you to contextualise your app and website performance.

The quantitative data shows you where your bottleneck is and how much it’s hurting your conversions. For example, when you set up your funnel in Smartlook, it will show you where most visitors drop off. With heatmaps, you can see where in that stage of the funnel users move, click and scroll, telling you whether they are scrolling far enough to read all the relevant information, or whether certain buttons are not clicked. The session recording will show you exactly where and when visitors decide to drop off, while others do convert. Launching a Mopinion feedback form on that page or in that step of the funnel will tell you the root cause of the bottleneck. In the open comments, your visitor can tell you why they are dropping out, or aren’t satisfied. This directs you to the right improvements and helps you save money, time, and other valuable resources which would otherwise be invested in the wrong “improvements”.

A two-way street

Now we looked at using quantitative data before the qualitative data. But this doesn’t have to be a one-way street. We can also start with qualitative data collection and analysis.

When you have a subtle feedback option on your app or website, users can give you feedback whenever they want and feel the need to do so. It can be completely embedded onto your website or app in your own design, so it’s part of the customer experience. Anytime visitors encounter an issue, bug, or are missing something, then they can let you know straight away. They can easily hit the feedback button and tell you what is happening. Alternatively, you might notice the NPS dropping from a week earlier in your dashboard graphs, you can go into the filters and see whether it’s based on location or device. You can go into individual feedback items to see whether there is an issue, or whether they are missing information, or just aren’t satisfied with what they encountered. If you know what the issue is, you can look at your quantitative data in Smartlook with heatmaps and session recordings to see if other visitors had similar issues. This will help you to prioritize your improvements.

Example time

Let’s take a more practical example. You’re an online retailer, and you’ve launched a campaign for a new product. You also offer a 10% discount-code to visitors who sign up for your newsletter.

Start by looking at your quantitative data. Unfortunately, you see a large bounce rate on your cart confirmation page. This is the second last step in your funnel, and you see that the largest drop-off happens right there. This is where people confirm their order before payment. You quickly look at the heatmap of the page and see that many visitors did scroll down to the confirmation button. But you see a large red bubble around the area where visitors can enter their coupon code. You start watching some user recordings and see that visitors are spending a lot of time in the coupon-code area before dropping off. So it’s clear there’s something up with the place where the coupon-code needs to be filled in.

Luckily, you also have a feedback button on the side of the page. You start looking at the feedback with open comments on that page. And you instantly see that the tag ‘bug’ has been added to the feedback items. The comments tell you: “I can not fill in the coupon code, there seems to be a bug, it won’t accept my code as it only has 11 characters instead of 12”. Of course, you instantly contact your web developer and tell him to fix the bug and make it possible to use 11 characters. And the best part is, you’ve added the option ‘leave your email address’ on the feedback form. Meaning, you can contact the visitors that dropped out of the funnel and get them back to actually convert.

As both Mopinion and Smartlook help you collect website and app data, together they will tell you the full customer journey story. Don’t leave anything to assumption and chances, but get to the bottom of the customer journey.

Conclusion

So, we won’t have a clear answer to what comes first, quantitative or qualitative data. Other than the chicken before the egg debate, it doesn’t really matter for data analyses. It’s a two-way street, in which they are strongest when they work together. We recommend collecting, analysing and making improvements based on both data flows. Only then will you be able to tell the full customer journey story, and use the power of data to achieve your business goals.