AI clipping for streamers
Powder • 2021
During my tenure at Powder, an iOS and Android app positioned as a social media hub for gamers, we embarked on a project aimed at streamers and content creators. The objective was to develop an app leveraging machine learning for automatic facial expression and loud voice spike detection during Twitch and YouTube streams.
Community Manager
• Community building
• Feedback and Product implementation
• Victor Pillard, Content Manager
• Yannis Mangematin, Head of Content, CEO
• Aneta Ozierańska, Product Owner
• Kevin Cathaly, Lead Mobile Engineer
Our journey was not without hurdles, grappling with a scarcity of suitable data for machine learning and a deficiency in direct user feedback. In response, we devised a holistic strategy focused on creating a vibrant community of streamers actively engaged in testing and providing feedback.

In collaboration with Victor and Aneta, we crafted a solution to our challenges: the creation of a dynamic community comprising streamers and content creators. This community would serve as an invaluable testing ground for our product, with members providing crucial feedback. Moreover, their diverse stream clips would become the fuel for training our AI models, elevating our innovation.

I assumed the responsibility of steering the community, orchestrating the feedback flow, and curating stream clips for AI model training. This role, undertaken in close collaboration with the growth, content, and product teams, ensured a seamless and strategic approach to our mission.

  • Growth:
  • Collaborating with the growth team, we invited all the members who were signing up for the app thorugh social ads. It resulted in about 100 streamers joining our designated Discord community.
  • Platform:
  • I set up a dedicated Discord server and implemented bots to gather pertinent data on streamers' streaming behavior, platform they stream on, frequency of their streams, country they're from and gender. Then other bots for streamline bug reporting where members can create tickets and report issues they're facing. Then I created comprehensive video and image tutorials to inform the community members on how the AI actually works.
  • Feedback wave 1:
  • As soon as the members started joining the community, I contacted them for personalized one-on-one interviews to gather feedback. Partnering with Aneta I prepared a targeted set of questions aligning with development needs. In total I interviewed 70 streamers.

    Initial Feedback Report

    One of the most important questions were the core features of the app. The chart below shows the percentage of streamers in need for Gameplay detection, Facial expression detection, Voice detection, a hotkey to clip from streams and a PC app instead of mobile.

    From all the streamers I interviewed, 60% of them were Competitive game streamers, which explains the need for gameplay detection. Then the remaining 26% and 14% were casual and IRL streamers who wanted more facial and voice detection.

    Out of all the clips the AI detected, only 20% of them where good clips. Others where irrelevant.



    Analyzing the data, it became apparent that the technology resonated more with IRL and casual game streamers than their competitive counterparts. This realization prompted a strategic shift in our community outreach approach. The clip rating provoked the need for curating clips for machine learning.

    Shift in Outreach Strategy

    With the strategic shift in the community outreach approach, I decided to contact handpicked IRL and casual streamers to join our community. In total, i managed to invite 55 streamers. I started with feedback interviews as soon as new members started joining.

    Feedback report post outreach

    After 30 interviews, I created a report. This time the revelation was interesting, 65% of them wanted facial expression and voice detection. 19% wanted a feature where streamers would say a word like "clip it" and it would create a clip. The remaining 16% wanted facial expression, voice detection and gameplay in addition to it.

  • Curating Clips for Machine Learning:
  • The developers required high-quality clips to train the AI model, acknowledging that only 20% of the generated clips were deemed "clip-worthy." To address this, I sat with Aneta and Kevin to plan a manual clip curation process. It involved scanning through the backend data for clips of streamers in our community. We had approximately 250 members at this point, with data on each one of them thanks to the discord bots I configured. I made a list of streamers who streamed on certain days and were of certain genres, certain ethnicities and gender.

    For a clip to be good, it must either be funny or trigger emotion within viewers. In order to expand facial expression detection the AI needed faces of different ethnicity and gender. The more diverse the better. videos with different resolutions because not all streams are of the same quality. As for the voice detection, a good mix of male and female streamers' clips were required.

  • Second Wave Feedback Report:
  • Post-implementation of insights gained from the initial feedback and curated clips, we sought a second wave of feedback. For this one, I contacted all the IRL and casual game streamers, the new and the old ones. I interviewed 80 streamers in total.

The feedback reports played a pivotal role in steering product decisions. The project's direction shifted towards providing more value to IRL and casual game streamers. This research laid the groundwork for a new project focusing on a PC application for gameplay detection, acknowledging the significance of competitive gaming in content creation. Read case study Reviving the community.
In conclusion, our collaborative efforts at Powder have fueled innovation through a community-centric approach. The establishment of a dedicated streamer community not only provided a valuable testing ground but also yielded essential feedback, guiding strategic decisions and prompting an adaptive shift towards better meeting the preferences of IRL and casual game streamers. The manual curation of high-quality clips, shaped by community insights, has not only enriched AI training but set the stage for future developments, including a tailored PC application for gameplay detection. As Powder continues to pioneer in the gaming and streaming landscape, this project underscores our commitment to responsive innovation and user-centric solutions, ensuring Powder's sustained leadership in reshaping the social media experience for gamers and content creators.