Apps You Need (And Don’t Need) For Evaluation Research

Teams, Slack, Zoom… despite being random words, I’m sure these three words bring an image to your mind.

That is because these random words have very specific meanings during the 9 to 5 workday. After all, they are the names of popular business applications. We use software and applications, known as "apps," increasingly frequently to make our work lives simpler and more effective. Apps are computer programs created to help complete a certain activity or task. The capabilities of an app can range from managing tasks for project management to creating beautiful presentations and reports. What gets complicated with apps is that there are a lot of them! You can quickly get bogged down in finding and testing them out.

A subset of research called evaluation can also benefit from apps. The use of apps can facilitate several evaluation-related activities. But I've discovered that some apps are more useful than others in my role as an evaluator at the research firm Rathbone Falvey Research. 

In addition to discovering apps useful for evaluation research, I’ve also discovered when free apps meet my needs, when I need to upgrade to a paid version, and which ones are a waste of time and make the work more difficult. In this blog post, I’ve outlined some of my essential apps for evaluation research, as well as a few I could live without or want to upgrade in the future. 

Apps for Collecting and Analyzing Data in Evaluation Research

Evaluations typically require data collection. Whether it is qualitative (like interviews and focus groups) or quantitative data (like surveys), apps can be extremely useful for this process.

For quantitative data collection, there are a few popular survey apps out there, like Qualtrics and SurveyMonkey that we will go into more detail in a future post.

For qualitative data collection, I’ve found some useful apps specifically for evaluation research that put them in my “need” category. Two of them have become very important in an ever-increasing online world: Zoom and XMind.

My favorite interview app is Zoom. Zoom has a lot of great features for conducting interviews and is useful in-person or for online interviews. Let’s break down the pros and cons of Zoom:

Zoom pros

• You can change your settings to record meetings automatically and never forget to record again
• It has the ability for Closed-Captioning, making virtual interviews more accessible
• It has a ‘Focus Mode’ feature, which eliminates distractions by showing only the videos that need to be shown
• You can mute participants upon entry, eliminating distractions if observers join late

Zoom cons

• You need the paid version to be in meetings with more than two people longer than 45 minutes
• It has many features so you need to spend some time familiarizing yourself with the settings to optimize usage

Another useful app for data collection is XMind. I use XMind to create mind maps (e.g., Ripple Effects Map) for evaluation. Ripple Effects Maps are an interesting technique used in evaluation research to capture the ripple effects of a complex community event or program. They are typically done in large groups. XMind allows for both in-person and virtual mapping sessions, and at the end, you have a digital version of the map to share or edit for coding and analysis.

I’ve only found one pro and con for XMind, which are related to each other. The pro is that you can create mind maps fast – like really fast. The con is there is a learning curve to it. They have a nice tutorial when you first download the app that helps, but the speed comes from keyboard shortcuts. It could take some time to get used to this app if you aren’t familiar with using keyboard shortcuts.

Once qualitative data has been collected, I’ve used Otter as a transcription tool for qualitative data, but after transcribing over 500 hours in Otter.ai, I wouldn’t recommend it. Here’s why:

Otter.ai pros

• Lower cost when it comes to transcription software
• Has a free version for short projects or one-time use

Otter.ai cons

• Prioritizes speed over transcription quality
• Clunky and you need to refresh your browser pretty frequently
• Has hard time identifying speakers which can require you to spend extra time assigning the correct speakers

Apps for Analyzing Evaluation Research

The next step for interviews and qualitative data is theming and coding. Dovetail is advertised as a tool for customer research, but it also has great capabilities for general theming and coding of interviews. It’s a level up from an excel or word doc and two levels up from printing and highlighting. Here’s why I like it:

Dovetail pros: 

• Coding is easy with the highlight feature
• You can see your codes in different views, from boards to lists
• You can view all of one code at once

Dovetail cons: 

• As a customer research platform, it has more features than I need, making it overwhelming to use at first
• It isn’t great for multiple levels (e.g. themes and then codes for those themes)

Some other useful tools for analysis include Mural and Miro. Both are whiteboard-type applications great for collaborating with others or for getting a bunch of floating ideas and topics in one space. They both prove useful when you are conducting evaluation research. They are nearly identical tools so you can get away with using the free version of both to maximize your number of boards. Let's look at the pros and cons of Miro and Mural:

Miro and Mural pros

• Useful for finding patterns and themes
• Can be used for creating drafts of SWOT analysis
• Useful for multiple uses, including presenting findings

Miro and Mural cons

• On the free version you are limited so opt to use your entire board for a single project rather than creating multiple ones

Apps for Reporting Evaluation Research

Now we’re in the final stretch of evaluations: visualizing and reporting the results. When it comes to visualization, there are a lot of apps, but some are more effective than others. For basic charts and analysis, excel or google sheets does a great job. However, when you need to step up your visualization for reporting, Infogram proves to be a useful tool. There are higher-level, more expensive apps but the cost-value of Infogram has served me well for several projects. 

Infogram pros

• Easy to use
• Can duplicate charts to quickly create a lot of visualizations
• Easy to change the colors and fonts
• A level-up from excel charts

Infogram cons

• Very limited in its features
• Need to copy and paste the data
• Can’t change raw data to percentages or vice versa
• Manual process to change colors or fonts

Three other tools I plan to use in the future for more advanced visualization are Tableau, Rawgraphs.io, and Figma. 

Tableau is the ultimate visualization tool and is especially useful for creating maps. However, Tableau functionality comes with a larger price tag that ultimately hasn’t proven to have a cost-benefit analysis worthy of that high price. Tableau Public is a free alternative but all data is publicly available with this version. 

Rawgraphs.io can make some really interesting data visualizations but is pretty advanced, resulting in a high learning curve. I hope to use this in the future but haven’t had the need just yet. Bonus: it’s free!

I could see Figma being useful for creating beautiful visualizations, but you really need to use it with other graphing and charting apps and then use Figma for design purposes only. 

The last part of an evaluation is reporting. I have tried to use Canva for an evaluation report but ultimately found Adobe InDesign to be the best tool for me.

InDesign was created for page layout and publishing and it shows. I personally have over 1,000 hours of experience in InDesign, so I am familiar and comfortable with it. If you aren’t familiar with InDesign, hiring a graphic designer could be the easiest and most time-efficient way to have a high-quality report. 

Canva pros

• Built-in icons and templates
• You can quickly change a lot of colors
• Easy to learn

Canva cons

• Page numbering is nearly impossible
• The tables are limited

InDesign pros

• A lot of design freedom
• Seamlessly change page numbers and fonts
• Setup document templates with company branding

InDesign cons

• Big learning curve – it takes a while to understand the application and how to use it
• Only available by a paid subscription (Plans start at $21 per month)

Overall, apps prove to be pretty useful in conducting evaluation research. Without them, I would have spent more hours on each research phase. Apps can quickly add up in monthly fees though, so make sure you are finding the ones that work for you and that level up your research. 

Know of any useful apps for evaluation research? I’d love to hear from you! Contact me at tasha@rathbonefalvey.com. 

Note: This post is not sponsored. All opinions expressed are solely my own.

‍