Curio is a system that enhance video learning experience through integrated, just-in-time help seeking with AI.
Client
Open Learning Initiative
Feb 2023 - Aug 2023
Role
UX Design
User Research
Team
Ben Tseng (Designer & Project Lead)
Yu-Hsin Lin (Software Engineer)
Problem
Diverse Learners, Fixed Content
The rise of online video learning platforms has marked a transformative era in education, extending knowledge access on an unprecedented scale. Nevertheless, despite their many contributions, MOOCs and similar platforms have introduced long-lasting challenges in providing personalized scaffolding for diverse learners.
For instance, a user encountering difficulty in comprehending a specific line of code or formula during a course video may find it hard to progress through the material.
Current Online Help Seeking Mechanisms
Although multiple solutions have been proposed and examined, including discussion forums and external resources such as search engines, these methods often leave room for improvement, as they result in delayed assistance and introduce additional cognitive load through the process of help-seeking.
Recent strides in Language Learning Models (LLMs) present a hopeful path for tackling these obstacles. Nonetheless, users continue to grapple with issues such as the generation of "hallucinated'' content and the formulation of effective questions.
User Study
Evaluate Curio's effectiveness in help-seeking with mixed method research
To evaluate the effectiveness of Curio in enhancing the help-seeking process, I developed two research questions:
Research Questions:
Compared to other help-seeking tools, does Curio enhance the efficiency of the help-seeking process?
Does Curio lead to better learning outcome for video learners compared to other help-seeking tools?
In-Widget Video Recommendation
Upon acquiring an on-screen element, the system will utulize a relevancy algorithm to surface targeted learning reosurce recommendations directly within the learning context.
User Interface
Just-in-time, Contextualized Support
The primary interaction with the Curio system occurs through an intuitive search interface, which overlays video players within an online learning platform.
For platform users, this facilitates convenient access to learning resources, enhances interactivity, and provides personalized explanations to address their misconceptions.
Interaction Design
Reduce Frictions in Help Seeking by OCR
Recognizing that external help-seeking solutions outside of video learning platforms generate friction—due to switching between tabs, formulating and typing queries, and filtering out non-educational content (such as ads)—I initiated the design process with the following HMW statement:
"How might we provide help to learners directly within the video player?"
To address this challenge, I came up with a interaction design interface leveraging OCR (Optical Character Recognition) technology. This would allow learners to use a selection tool within the video player to “capture and search” any text-based element displayed on the screen, including terminology, bullet points, codes, and formulas.
Overview
Online video learning platforms have democratized access to learning but often struggle to offer personalized, context-sensitive support when learners encounter challenging concepts.
Therefore, I designed Curio, a help-seeking system capable of being integrated into existing video learning platforms.
When confronted with complex content, learners can "capture'" any on-screen text-based elements to instantly receive targeted assistance and continue their educational journey without diverting their focus to external resources.
Currently, Curio has been adopted by multiple learning platforms, including Open Learning Initiative (OLI), to offer personalized scaffolding for numerous online video learners.
Contextualized Concept Explanation with LLM
Curio integrates GPT-4 to provide instant explanations, even when platform-specific resources are unavailable The context of the current video is transmitted as variables in the main prompt during API calls.
Supporting Diverse Learning Resources
Since the relevancy algorithm could be generalized to any text-based educational content, Curio can support content recommendation in a diverse modality to fulfill different learner's need and different platform's context.
Overview
Online video learning platforms have democratized access to learning but often struggle to offer personalized, context-sensitive support when learners encounter challenging concepts.
Therefore, I designed Curio, a help-seeking system capable of being integrated into existing video learning platforms.
When confronted with complex content, learners can "capture'" any on-screen text-based elements to instantly receive targeted assistance and continue their educational journey without diverting their focus to external resources.
Currently, Curio has been adopted by multiple learning platforms, including Open Learning Initiative (OLI), to offer personalized scaffolding for numerous online video learners.
Problem
Diverse Learners, Fixed Content
The rise of online video learning platforms has marked a transformative era in education, extending knowledge access on an unprecedented scale. Nevertheless, despite their many contributions, MOOCs and similar platforms have introduced long-lasting challenges in providing personalized scaffolding for diverse learners.
For instance, a user encountering difficulty in comprehending a specific line of code or formula during a course video may find it hard to progress through the material.
Current Online Help Seeking Mechanisms
Although multiple solutions have been proposed and examined, including discussion forums and external resources such as search engines, these methods often leave room for improvement, as they result in delayed assistance and introduce additional cognitive load through the process of help-seeking.
Research Findings
Curio reduced friction in help-seeking process while maintaining effectiveness
Comparing Curio with other help seeking methods, participants reported a statistically significant reduction in time demand (3.364 v.s. 4.000, p=0.047) and help-seeking task performance. (3.318 v.s. 4.182, p=0.034). Other dimensions and overall workload were also reduced (p >0.05).
In-Widget Video Recommendation
Contextualized Concept Explanation with LLM
Upon acquiring an on-screen element, the system will utulize a relevancy algorithm to surface targeted learning reosurce recommendations directly within the learning context.
Supporting Diverse Learning Resources
Since the relevancy algorithm could be generalized to any text-based educational content, Curio can support content recommendation in a diverse modality to fulfill different learner's need and different platform's context.
Curio integrates GPT-4 to provide instant explanations, even when platform-specific resources are unavailable The context of the current video is transmitted as variables in the main prompt during API calls.
Insight
Current design minimize cognitive burden for novice but constraint learner with more prior knowledge
In terms of qualitative feedback, participants described 4 supporting roles t Curio serves in streamlining the help-seeking process during video learning.
Simplified typing, focused learning
“I can spend more time on understanding the material than on typing or capturing information manually” -U5
Eliminate the workload for “constructing query”
“I don’t have to summarize myself” -U12
Curating relevant educational content
“Using Google is like finding a needle in a haystack when I am new to a topic” - U3
Minimize context-switching
“This keep me in a single workflow so I don’t need to jump between different tabs or platforms”- U17
Streamlining Help Seeking Process
However, while Curio effectively reduced cognitive load for novice learners, it did not demonstrate a statistically significant advantage over traditional help-seeking methods in terms of learning outcomes.
To address this incongruity, I conducted an exploratory analysis of the connection between the users’ backgrounds and their eventual learning gains and observed medium negative correlations on Prior Knowledge / Learning Gain using Curio, indicating learners with less prior knowledge tended to benefit more from Curio, whereas those with more prior knowledge did not experience the same level of benefit.
Design Iteration
Strike a Balance between efficiency and customization
To address this challenge, I introduced AI-enhanced query templates powered by advanced language models. These templates allow experienced learners to refine or customize queries effortlessly, preserving Curio’s simplicity for novices while enhancing flexibility for experts. This balanced design significantly broadens Curio’s applicability, ensuring effective, personalized support across diverse learner expertise levels.
Refine Query Formulation Process: Query Template
Interaction Design
Reduce Frictions in Help Seeking by OCR
Recognizing that external help-seeking solutions outside of video learning platforms generate friction—due to switching between tabs, formulating and typing queries, and filtering out non-educational content (such as ads)—I initiated the design process with the following HMW statement:
"How might we provide help to learners directly within the video player?"
To address this challenge, I came up with a interaction design interface leveraging OCR (Optical Character Recognition) technology. This would allow learners to use a selection tool within the video player to “capture and search” any text-based element displayed on the screen, including terminology, bullet points, codes, and formulas.
User Interface
Just-in-time, Contextualized Support
The primary interaction with the Curio system occurs through an intuitive search interface, which overlays video players within an online learning platform.
For platform users, this facilitates convenient access to learning resources, enhances interactivity, and provides personalized explanations to address their misconceptions.
In-Widget Video Recommendation
Upon acquiring an on-screen element, the system will utulize a relevancy algorithm to surface targeted learning reosurce recommendations directly within the learning context.
Contextualized Concept Explanation with LLM
Curio integrates GPT-4 to provide instant explanations, even when platform-specific resources are unavailable The context of the current video is transmitted as variables in the main prompt during API calls.
Flexible Content Integration
Since the relevancy algorithm could be generalized to any text-based educational content, Curio can support content recommendation in a diverse modality to fulfill different learner's need and different platform's context.
For learners with low prior knowledge, a recurring positive feedback theme was Curio’s ability to alleviate the mental workload associated with crafting a query.
“I don’t have to put my thoughts into words, which is a relief”
- U16 ( Low Prior Knowledge Learner )
Conversely, those with high prior knowledge reported limitations with Curio’s ability to meet more specialized help-seeking needs, which could be better addressed by customized queries in other platforms.
“If it could more accurately answer my questions,... or maybe because I’ve actually never tried what would happen if I input sentences, then it could answer my questions much better”
- U17 ( Low Prior Knowledge Learner )
The Query Paradox: Simplicity vs. Precision
Conclusion
Enable Personalized Scaffolding in Video Learning at Scale
This project was published and presented at top academic conferences, including AIED 2023 and ECTEL 2024, and has been adopted by several learning platforms, such as the Open Learning Initiative (OLI).
Curio’s success has drawn attention from leading educational organizations. I was invited by Scholastic—the largest publisher in the U.S.—and Aestheticell, the creator of Taiwan’s first digital textbook, to integrate Curio into their intelligent textbook platforms. These collaborations have extended Curio’s reach to millions of learners globally, demonstrating its potential to shape the future of AI-powered education.
User Study
Evaluate Curio's effectiveness in help-seeking with mixed method research
To evaluate the effectiveness of Curio in enhancing the help-seeking process, I developed two research questions:
Research Questions:
Compared to other help-seeking tools, does Curio enhance the efficiency of the help-seeking process?
Does Curio lead to better learning outcome for video learners compared to other help-seeking tools?
In terms of qualitative feedback, participants described 4 supporting roles Curio serves in streamlining the help-seeking process during video learning.
Simplified typing, focused learning
“I can spend more time on understanding the material than on typing or capturing information manually” -U5
Eliminate the workload for “constructing query”
“I don’t have to summarize myself” -U12
Curating relevant educational content
“Using Google is like finding a needle in a haystack when I am new to a topic” - U3
Minimize context-switching
“This keep me in a single workflow so I don’t need to jump between different tabs or platforms”- U17
Streamlining Help Seeking Process
Streamlining Help Seeking Process
Streamlining Help Seeking Process
Research Findings
Curio reduced friction in help-seeking process while maintaining effectiveness
Comparing Curio with other help seeking methods, participants reported a statistically significant reduction in time demand (3.364 v.s. 4.000, p=0.047) and help-seeking task performance. (3.318 v.s. 4.182, p=0.034). Other dimensions and overall workload were also reduced (p >0.05).
In terms of qualitative feedback, participants described 4 supporting roles Curio serves in streamlining the help-seeking process during video learning.
Simplified typing, focused learning
“I can spend more time on understanding the material than on typing or capturing information manually” -U5
Eliminate the workload for “constructing query”
“I don’t have to summarize myself” -U12
Curating relevant educational content
“Using Google is like finding a needle in a haystack when I am new to a topic” - U3
Minimize context-switching
“This keep me in a single workflow so I don’t need to jump between different tabs or platforms”- U17
Insight
Current design minimize cognitive burden for novice but constraint learner with more prior knowledge
Influence of Prior Knowledge
However, while Curio effectively reduced cognitive load for novice learners, it did not demonstrate a statistically significant advantage over traditional help-seeking methods in terms of learning outcomes.
To address this incongruity, I conducted an exploratory analysis of the connection between the users’ backgrounds and their eventual learning gains and observed medium negative correlations on Prior Knowledge / Learning Gain using Curio, indicating learners with less prior knowledge tended to benefit more from Curio, whereas those with more prior knowledge did not experience the same level of benefit.
The Query Paradox: Simplicity vs. Precision
For learners with low prior knowledge, a recurring positive feedback theme was Curio’s ability to alleviate the mental workload associated with crafting a query.
“I don’t have to put my thoughts into words, which is a relief”
- U16 ( Low Prior Knowledge Learner )
Conversely, those with high prior knowledge reported limitations with Curio’s ability to meet more specialized help-seeking needs, which could be better addressed by customized queries in other platforms.
“If it could more accurately answer my questions,... or maybe because I’ve actually never tried what would happen if I input sentences, then it could answer my questions much better”
- U17 ( High Prior Knowledge Learner )
For learners with low prior knowledge, a recurring positive feedback theme was Curio’s ability to alleviate the mental workload associated with crafting a query.
“I don’t have to put my thoughts into words, which is a relief”
- U16 ( Low Prior Knowledge Learner )
Conversely, those with high prior knowledge reported limitations with Curio’s ability to meet more specialized help-seeking needs, which could be better addressed by customized queries in other platforms.
“If it could more accurately answer my questions,... or maybe because I’ve actually never tried what would happen if I input sentences, then it could answer my questions much better”
- U17 ( High Prior Knowledge Learner )
The Query Paradox: Simplicity vs. Precision
Design Iteration
Strike a Balance between efficiency and customization
Refine Query Formulation Process: Query Template
To address this challenge, I introduced AI-enhanced query templates powered by advanced language models. These templates allow experienced learners to refine or customize queries effortlessly, preserving Curio’s simplicity for novices while enhancing flexibility for experts. This balanced design significantly broadens Curio’s applicability, ensuring effective, personalized support across diverse learner expertise levels.
Refine Query Formulation Process: Query Template
Project Impact
Enable Personalized Scaffolding in Video Learning at Scale
This project was published and presented at top academic conferences, including AIED 2023 and ECTEL 2024, and has been adopted by several learning platforms, such as the Open Learning Initiative (OLI).
Curio’s success has drawn attention from leading educational organizations. I was invited by Scholastic—the largest publisher in the U.S.—and Aestheticell, the creator of Taiwan’s first digital textbook, to integrate Curio into their intelligent textbook platforms. These collaborations have extended Curio’s reach to millions of learners globally, demonstrating its potential to shape the future of AI-powered education.
This project was published and presented at top academic conferences, including AIED 2023 and ECTEL 2024, and has been adopted by several learning platforms, such as the Open Learning Initiative (OLI).
Curio’s success has drawn attention from leading educational organizations. I was invited by Scholastic—the largest publisher in the U.S.—and Aestheticell, the creator of Taiwan’s first digital textbook, to integrate Curio into their intelligent textbook platforms. These collaborations have extended Curio’s reach to millions of learners globally, demonstrating its potential to shape the future of AI-powered education.
