Service Feedback, how to ‘git gud’
Running a service like Noteable comes with its own particular set of problems. Academic colleagues who have used the service in their teaching always say that the service is great, which itself is very good, the problem comes when trying to solidify this into something that you can take to others.
I’ve realised that it sounds like I’m complaining that people like our service, let me clarify, the problem is getting other people to believe me and get on board as well. Within academia it can be difficult to find time to experiment with something new, one of the best parts of being a Learning Technologist is to make it easier for colleagues to find tools and services that suit their needs and help them to deliver teaching.
This is how I meander my way to talking about feedback, the process of documenting how people found using your service; turning this into things that you can work on and then using it to (identify and then) shout about what your service is good at.
So, you’ve decided to do some service feedback. Here’s what you should know…
I spent a lot of time trying to find some sort of template for what questions to ask but I ran up a blank, I also found a collection of incredibly strange/weird questions which I had to fight the urge to include purely to indulge myself: “If our service were a car, what brand of car would it be?”
Since I couldn’t find anything I decided to try and make my own and then share the template out for others to improve on. I’ll break this down into two components, questionnaire feedback and User Stories/Use Cases (call them what you want but in the end it’s a first-hand account of what your users do with the service and how they found it). Today I’m going to focus on the first.
Questionnaire Feedback:
Questionnaires are usually a bit of a go to, they don’t take as long and you can normally get so numbers which look good on a page at the end of it. They (can be) a quick way of finding things out and the information can be useful, if you have the right questions that is…
In thinking about what the right questions are you really need to think about; what information do you want to know and do these questions allow your users to give you that information. For the vast majority of you reading this that will be very obvious but it still bears thinking about. Below, i’ll list all of the questions that I eventually created alongside the rationale.
How have you used the Noteable service in your teaching? (Free text answer)
A basic opener to get a bit more information about how your users have been engaging with your service. This sounds like a fairly bland opener but it does a number of things; gives you context information and also gives you a handy list of “this service can be used for…”
Do you intend to use the Noteable service again? (Free text answer)
Very straight to the point, this service was offered in a trial phase so it’s important to know how many people thought it was good enough to want to use again. This gives you direct retention numbers and helps you to justify that you are fulfilling a continuing need.
How likely are you to recommend the Noteable service to a colleague who has similar teaching needs? (Free text answer)
For a service that wants to grow this is key. People might like your service but if they really like it then they will suggest it to their colleagues. It’s a good metric as it’s a great thing to brag about, essentially it’s not just you that is saying that the service is great but also the users. Users will always be more open to hearing how good a service is from their colleagues so there is also the implication that the service will continue to grow naturally.
How would you rate the ease of use of the Noteable service? (Scale Question 1Low-5High)
Ease of use means how easily is it for you to use each time you use it in your teaching. You want to know if your users constantly feel like they are straining to use your service. Making the question more open like this means you aren’t highlighting what you think are barriers but looking out for anything that could be. A caveat with these questions is that any score under 3 will open a follow up question asking about specifics.
How would you rate the ease of adoption? (Scale Question 1Low-5High)
This is in relation to how much effort it took to introduce your service into their teaching. You obviously want to pay attention to this as it is going to impact the amount of new users and the early attrition rate. Again, this has a follow up if it gets a low mark to get more useful information on what you can improve on. Removing or lowering as many barriers to adoption is going to be the biggest thing to help you grow a service.
What would you do without the Noteable service? (Free text answer)
This might be one that is more suited to our position with the Noteable service. We are offering a centralised service so it’s good to know if this is reducing a burden or making it easier for people to use this specific tool – Jupyter notebooks. As a pilot service we need to be able to say there is a justifiable need for providing this service, that without this service users would be inconvenienced.
What is the most important feature that you would like to see? (Free text answer)
Nothing like crowd-sourcing your new features….
To what extent do you agree with the following statement: “Using Noteable helped me to deliver my teaching/training” (Scale Question 1Low-5High)
Directly asking if introducing this tool has any discernible affect on the ability of our colleagues to actually deliver their teaching. It’s often very difficult to quantify where something has a pedagogic impact but by asking this directly we are getting the answer from the people who are most likely to know.
To what extent do you agree with the following statement: “Using Noteable helped my students to interact with the material” (Scale Question 1Low-5High)
Again, this is something that is difficult to quantify but can have a massive impact. If your students can more easily engage with the material it means you can get further into more complex subject, spend more time in theory and less time getting over barriers. This will be followed up with student questionnaires at a later date.
It’s important to not lose sight of how valuable this information can be, whilst you will want to shout about good feedback make sure you spend the time fixing what users have reported and working on what they have said they want in the future. Few things can improve your relationship with your users more than “you said, we did”.
This questionnaire is currently live, I will share the feedback when it is finally complete. Please let me know if you have any improvements or insights into this, I’m always trying to make this better.
p.s. if you are going to steal these questions, at least replace Noteable with [your service name]. (I have actually seen this before and it is amazing/terrible)
