Usability Testing
Live streaming challenges for next gen sports fans
Fostering safe virtual spaces using focus groups to discuss live streaming behaviors, habits & challenges with gen z
Year
2021
Client
Buzzer
Role
Facilitator, UX Researcher
Methods
Focus Groups, Surveys
Live streaming challenges for next gen sports fans
Fostering safe virtual spaces using focus groups to discuss live streaming behaviors, habits & challenges with gen z
Year
2021
Client
Buzzer
Role
Facilitator, UX Researcher
Methods
Focus Groups, Surveys
Approach
Focus groups within user research are typically a tricky method. The risk of the bandwagon effect arises and most people don’t want to be the odd one out, so naturally they assimilate. However if group interviews are run correctly and paired with other research methods to gather a more holistic view about user behavior, feelings and attitudes then they can be a very powerful tool.
In our situation we were looking to gather insight on sports fans' willingness to pay for $.99 Buzzer Moments, which can be defined as the most exciting part of a game that you never want to miss. To balance out the assumed bias we would receive from focus groups we also surveyed participants, which in this case were basketball fans who’ve never used our product yet consume live sports regularly.
Pairing together focus groups and surveys helped us better understand the perception and value of our product with our target audience. We discussed their willingness to pay for moments, compared the Buzzer product to our competitive landscape and gathered suggestions on product opportunities that could solve other common challenges with our users.
Background
I like to view focus groups as a way of gaining clarity on potential hypotheses. After a full NBA season of gathering user data and developing assumptions on how users perceive our product, this felt like the perfect time to reevaluate our target audiences’ beliefs and opinions.
There were many questions that our internal data analytics couldn't answer for us. What access do fans have to currently stream live sports? Would they pay .$99 to watch their favorite players? How many users watch live sports illegally? Post-onboarding what are first impressions from users? In what areas can we compete in the sports media landscape? What are the most frustrating aspects of live sports streaming that we could build solutions to help?
I proposed we conduct “next-gen” fan focus groups as an opportunity to update our understanding of gen-z sports fans’ motivations and challenges regarding live sports. Along with identifying high priority next steps on our roadmap and clarify hypotheses during this early stage of our product development.
Goals
Gather quantitative & qualitative data on users’ willingness to pay for Buzzer moments based on various pricing options.
Uncover the motivations and challenges for younger sports fans who consume live sports.
Discover new product opportunities by understanding what participants want from a next-gen consumer live sports app and prioritize potential new features
Research Themes
Primary rationale for those who didn't want to use Buzzer is because they were not getting enough value for their microtransactions
At the time our only payment option to watch live sports was microtransactions. This payment model did not resonate well with fans who currently watch tons of sports for free. If fans are required to pay for live sports access, behind the scenes footage, championship talk and interviews, and exclusive access are all types of content fans would be willing to pay for.
“Buzzer has to be offering something that I'm unable to do myself, because for almost any game I can definitely find a free stream link. I don't know about paying the .99 when you can find something for free even though it may be a little annoying to find.”
Integrating stats and scores into the app was a common product suggestion from participants. Fans feel that stats and scores are a vital part of their sports watching experience and encourage them to watch/buy
Data, stats & scores provide storyline context and set the stage for big time moments
Fans don't necessarily want to necessarily be nerdy and dig deep into data but users want to understand more game context after receiving a notification. Sports fans have this ability to jump straight in and understand what's happening in the game just by viewing stats, shooting percentages, shot volume etc.
The younger audience didn't feel that our language and messaging is tailored for them
“I feel like it needs some type of extra gen z elements. It sounds like its not coming from or said by a person from our generation”
“I don't know if I would actually say something such as lightning in a bottle, it feels a little too antiquated. I don't know if there is a similar term but I don't really resonate with this term”
Following the focus groups, we transcribed the audio and identified insights that help us generate an affinity map. This map was used to prioritize common themes from users across all 4 sessions and integrated feature suggestions and major challenges into our product roadmap.
Project wins
Integrated product feature suggestions from focus group participants onto product roadmap and began design and development of app redesign centered around personalized sports timeline.
Designed an intuitive credits based viewing system for the app, resulting in a 250% increase in moments viewed per week based on price feedback gained from this research study.
Overhauled paid marketing strategy to include language and insights gathered from gen-z participant audience.
Reflections
While I did notice a lot of groupthink within the study, pairing surveys with focus groups was a powerful combo of triangulation as we worked to limit bias. Obviously focus groups tell you what people say, not what they do so it's a bit lacking compared to usability testing however I felt confident in the data we gathered regarding what users actually want from a next gen live sports app. After conducting four 90 minute sessions, we left with ample amounts of user observation data and insights which led to affinity diagramming.
We made sure to remove any Buzzer or company messaging during recruiting to limit any bias from participants. It was important that they felt free to provide any harsh feedback on the product that was difficult to avoid during regular user interviews.
Although I've conducted 100+ 1:1 user interviews and facilitated multiple design sprints, this was my first opportunity moderating a virtual focus group. During the group session, my main areas of focus were focus and collaboration. Ensuring the discussion stays on track and that all group members contribute to the discussion was top of mind. Because focus groups can easily produce faulty data since participants might believe they want one feature/capability when they actually need another. I worked to minimize this issue by being explicit when displaying and describing concrete examples of how the Buzzer app functions.
Planning, let alone moderating a successful focus group is an art itself. Props to anyone else who has conducted one. While focus groups in ux research are often frowned upon, if you need to clarify your research hypotheses and gather data regarding how people discuss a potential problem/product I’d say it’s worth a shot.
Approach
Focus groups within user research are typically a tricky method. The risk of the bandwagon effect arises and most people don’t want to be the odd one out, so naturally they assimilate. However if group interviews are run correctly and paired with other research methods to gather a more holistic view about user behavior, feelings and attitudes then they can be a very powerful tool.
In our situation we were looking to gather insight on sports fans' willingness to pay for $.99 Buzzer Moments, which can be defined as the most exciting part of a game that you never want to miss. To balance out the assumed bias we would receive from focus groups we also surveyed participants, which in this case were basketball fans who’ve never used our product yet consume live sports regularly.
Pairing together focus groups and surveys helped us better understand the perception and value of our product with our target audience. We discussed their willingness to pay for moments, compared the Buzzer product to our competitive landscape and gathered suggestions on product opportunities that could solve other common challenges with our users.
Background
I like to view focus groups as a way of gaining clarity on potential hypotheses. After a full NBA season of gathering user data and developing assumptions on how users perceive our product, this felt like the perfect time to reevaluate our target audiences’ beliefs and opinions.
There were many questions that our internal data analytics couldn't answer for us. What access do fans have to currently stream live sports? Would they pay .$99 to watch their favorite players? How many users watch live sports illegally? Post-onboarding what are first impressions from users? In what areas can we compete in the sports media landscape? What are the most frustrating aspects of live sports streaming that we could build solutions to help?
I proposed we conduct “next-gen” fan focus groups as an opportunity to update our understanding of gen-z sports fans’ motivations and challenges regarding live sports. Along with identifying high priority next steps on our roadmap and clarify hypotheses during this early stage of our product development.
Goals
Gather quantitative & qualitative data on users’ willingness to pay for Buzzer moments based on various pricing options.
Uncover the motivations and challenges for younger sports fans who consume live sports.
Discover new product opportunities by understanding what participants want from a next-gen consumer live sports app and prioritize potential new features
Research Themes
Primary rationale for those who didn't want to use Buzzer is because they were not getting enough value for their microtransactions
At the time our only payment option to watch live sports was microtransactions. This payment model did not resonate well with fans who currently watch tons of sports for free. If fans are required to pay for live sports access, behind the scenes footage, championship talk and interviews, and exclusive access are all types of content fans would be willing to pay for.
“Buzzer has to be offering something that I'm unable to do myself, because for almost any game I can definitely find a free stream link. I don't know about paying the .99 when you can find something for free even though it may be a little annoying to find.”
Integrating stats and scores into the app was a common product suggestion from participants. Fans feel that stats and scores are a vital part of their sports watching experience and encourage them to watch/buy
Data, stats & scores provide storyline context and set the stage for big time moments
Fans don't necessarily want to necessarily be nerdy and dig deep into data but users want to understand more game context after receiving a notification. Sports fans have this ability to jump straight in and understand what's happening in the game just by viewing stats, shooting percentages, shot volume etc.
The younger audience didn't feel that our language and messaging is tailored for them
“I feel like it needs some type of extra gen z elements. It sounds like its not coming from or said by a person from our generation”
“I don't know if I would actually say something such as lightning in a bottle, it feels a little too antiquated. I don't know if there is a similar term but I don't really resonate with this term”
Following the focus groups, we transcribed the audio and identified insights that help us generate an affinity map. This map was used to prioritize common themes from users across all 4 sessions and integrated feature suggestions and major challenges into our product roadmap.
Project wins
Integrated product feature suggestions from focus group participants onto product roadmap and began design and development of app redesign centered around personalized sports timeline.
Designed an intuitive credits based viewing system for the app, resulting in a 250% increase in moments viewed per week based on price feedback gained from this research study.
Overhauled paid marketing strategy to include language and insights gathered from gen-z participant audience.
Reflections
While I did notice a lot of groupthink within the study, pairing surveys with focus groups was a powerful combo of triangulation as we worked to limit bias. Obviously focus groups tell you what people say, not what they do so it's a bit lacking compared to usability testing however I felt confident in the data we gathered regarding what users actually want from a next gen live sports app. After conducting four 90 minute sessions, we left with ample amounts of user observation data and insights which led to affinity diagramming.
We made sure to remove any Buzzer or company messaging during recruiting to limit any bias from participants. It was important that they felt free to provide any harsh feedback on the product that was difficult to avoid during regular user interviews.
Although I've conducted 100+ 1:1 user interviews and facilitated multiple design sprints, this was my first opportunity moderating a virtual focus group. During the group session, my main areas of focus were focus and collaboration. Ensuring the discussion stays on track and that all group members contribute to the discussion was top of mind. Because focus groups can easily produce faulty data since participants might believe they want one feature/capability when they actually need another. I worked to minimize this issue by being explicit when displaying and describing concrete examples of how the Buzzer app functions.
Planning, let alone moderating a successful focus group is an art itself. Props to anyone else who has conducted one. While focus groups in ux research are often frowned upon, if you need to clarify your research hypotheses and gather data regarding how people discuss a potential problem/product I’d say it’s worth a shot.
