ActiveAI: Reimagine K-12 AI Literacy with Interactive Tutoring System. A scalable approach to prepare the next generation for a future increasingly influenced by AI.
ActiveAI: Reimagine K-12 AI Literacy with interactive tutoring system. A scalable approach to prepare the next generation for a future increasingly influenced by AI.
Client
PaGamO
2023
Role
User Experience
User Research
Team
Ben Tseng (Design & Project Lead)
Gautam Yadav (Learning Engineer)
Overview
The rapid advancements in Artificial Intelligence (AI), especially in generative AI, underscore the pressing need for comprehensive AI Literacy Education. This entails the ability to interact seamlessly with AI, ensuring effective and ethical problem-solving across diverse sociocultural landscapes.
In collaboration with world-class AI experts and learning engineers, I led the design and development of ActiveAI, a series of bite-size, interactive AI Literacy modules for K-12 learners. Our goal is to develop a scalable, evidence-based AI Literacy Education product to prepare the upcoming generation to navigate an AI-driven future with confidence.
Design Challenge
Current Barriers in AI Literacy Education
Complexity and inherent randomness of AI outputs
The intrinsic complexity of AI concepts and the required understanding of computational principles often pose a significant challenge for young learners due to the opaque nature of these advanced AI algorithms, which operate like a 'black box'. Moreover, the inherent randomness and uncertainty of AI outputs intensify the difficulty of crafting authentic and meaningful learning experience.
Design Challenge
Current Barriers in AI Literacy Education
The intrinsic complexity of AI concepts and the required understanding of computational principles often pose a significant challenge for young learners due to the opaque nature of these advanced AI algorithms, which operate like a 'black box'. Moreover, the inherent randomness and uncertainty of AI outputs intensify the difficulty of crafting authentic and meaningful learning experience.
Complexity and inherent randomness of AI outputs
Solution
Bring Active Learning Experience with Intelligent Agents
To bring scalable and pedagogically sound AI literacy Education for K-12 learners, I led the design of ActiveAI, an interactive tutoring system that enable learners to directly interact with AI through intelligent agents and receive immediate targeted feedback to ensure effective and engaging learning experience.
Intelligent Agents in Tutoring System
Intelligent agents is a way to teach AI by performing real-time computation with AI Algorithm based on learner input without coding.
Using AI as medium to reach AI is curial to introduce the inherent randomness and uncertainty of AI outputs to learners. Embedded them into the tutoring system also minimize the reliance on multiple tools and reduce teacher training requirements.
High Investment Barriers Limiting Access to AI Literacy
While there are some effective AI Literacy curriculums, most of them rely on various tools, platforms, and teaching plans that necessitate significant investments in teacher training and resources. A
Although there's readings and online videos for K-12 AI education are readily available and easily distributed, they lack the active learning elements essential for effective learning.
After a thorough literature review and discussion with learning scientists, I strategically designed a set of learner input interactions to facilitate student interaction with the intelligent agent.
By limiting the interactions to these three types, the modules ensures a consistent and familiar interaction across the system, reducing cognitive load for learners and allowing students to focus on mastering AI Literacy concepts. Additionally, this approach ensures manageable system design and development as the product expands.
Interaction Design
Three Types of Learner Interaction for AI Literacy
For instance, to help learners grasp the influence of "temperature" on an LLM Chatbot's sentence completion, we integrate an intelligent agent that completes the same sentence using two distinct temperature setting . Learners can directly observe that a "low" temperature setting yields precise and predictable outcomes, while a higher temperature results in more unexpected word choices.
This experience equips them to understand the trade-offs between different temperature settings when using tools like ChatGPT or Bing Chat. For example, a low-temperature chatbot is ideal for factual tasks like gathering information for a history report, whereas a high-temperature setting is better suited for creative endeavors like brainstorming for creative writing.
High Investment Barriers Limiting Access to AI Literacy
While there are some effective AI Literacy curriculums, most of them rely on various tools, platforms, and teaching plans that necessitate significant investments in teacher training and resources. A
Although there's readings and online videos for K-12 AI education are readily available and easily distributed, they lack the active learning elements essential for effective learning.
Solution
Bring Interactive Learning Experience at Scale with Intelligent Agents
To bring scalable and pedagogically sound AI literacy Education for K-12 learners, I led the design of ActiveAI, an interactive tutoring system that enable learners to directly interact with AI through intelligent agents and receive immediate targeted feedback to ensure effective and engaging learning experience.
Intelligent Agents in Tutoring System
Intelligent agents is a way to teach AI by performing real-time computation with AI Algorithm based on learner input without coding.
Using AI as medium to reach AI is curial to introduce the inherent randomness and uncertainty of AI outputs to learners. Embedded them into the tutoring system also minimize the reliance on multiple tools and reduce teacher training requirements.
Rapid Iteration with Learners in the Loop
Mid-Fi Prototype and Pilot User Testing
To ensure my design not only deliver satisfying user experiences but also enhance learning outcomes, I developed mid-fi prototypes, conducted user tests with learners, and iteratively refine the interface in a weekly basis for various modules.
Reach Out for Additional Details
yingjuit@andrew.cmu.edu
For instance, to help learners grasp the influence of "temperature" on an LLM Chatbot's sentence completion, we integrate an intelligent agent that completes the same sentence using two distinct temperature setting . Learners can directly observe that a "low" temperature setting yields precise and predictable outcomes, while a higher temperature results in more unexpected word choices.
This experience equips them to understand the trade-offs between different temperature settings when using tools like ChatGPT or Bing Chat. For example, a high-temperature setting is better suited for creative endeavors like brainstorming for creative writing.
Steppers are designed to provide learners with step-by-step explanations or control time-related variables. By offering a guided approach to navigating complex AI processes, steppers help students grasp concepts sequentially and methodically.
Steppers
Navigating Complex AI Processes with Segmentation Principle
Sliders lets learners adjust variables and receive immediate feedback. Sliders can be employed to modify various quantitative variables, such as the threshold for a classification model or the amount of training data used. This real-time feedback fosters deeper understanding and experimentation as students explore AI literacy concepts.
Sliders
Discover Causal Relationship between Variables
Collectors
Observe how AI Learn from Patterns in Data
Collector lets learners capture their own datasets or label provided dataset into different classes to train the intelligent agent. By participating actively in training data collection, learners gain a deeper understanding of the process and a stronger connection to the problem.
Interaction Design
Three Types of Learner Interaction for AI Literacy
After a thorough literature review and discussion with learning scientists, I strategically designed a set of learner input interactions to facilitate student interaction with the intelligent agent.
By limiting the interactions to these three types, the modules ensures a consistent and familiar interaction across the system, reducing cognitive load for learners and allowing students to focus on mastering AI Literacy concepts. Additionally, this approach ensures manageable system design and development as the product expands.
Steppers
Navigating Complex AI Processes with Segmentation Principle
Steppers are designed to provide learners with step-by-step explanations or control time-related variables. By offering a guided approach to navigating complex AI processes, steppers help students grasp concepts sequentially and methodically.
Sliders
Discover Causal Relationship between Variables
Sliders lets learners adjust variables and receive immediate feedback. Sliders can be employed to modify various quantitative variables, such as the threshold for a classification model or the amount of training data used. This real-time feedback fosters deeper understanding and experimentation as students explore AI literacy concepts.
Collectors
Observe how AI Learn from Patterns in Data
Collector lets learners capture their own datasets or label provided dataset into different classes to train the intelligent agent. By participating actively in training data collection, learners gain a deeper understanding of the process and a stronger connection to the problem.
Personalized Help throughout Learning Journey with Hint and Targeted Feedback
To provide personalized scaffolding and alleviate the workload on teachers when students get stuck, each learning activity comes with on-demand hints and targeted feedback.
Personalized Help throughout Learning Journey with Hint and Targeted Feedback
To provide personalized scaffolding and alleviate the workload on teachers when students get stuck, each learning activity comes with on-demand hints and targeted feedback.
Rapid Iteration with Learners in the Loop
Mid-Fi Prototype and Pilot User Testing
To ensure my design not only deliver satisfying user experiences but also enhance learning outcomes, I developed mid-fi prototypes, conducted user tests with learners, and iteratively refine the interface in a weekly basis for various modules.
Certain interface elements can sometimes yield unintended learning outcomes. For example, user testing revealed that varying facial expressions on agents led learners to misconceive that low and high temperature settings in chatbots result in more 'negative' word outputs. This issue was resolved through subsequent iteration.
Evaluation Research
Measuring Learning Experience and System Usability in the Field
Evaluate ActiveAI in Real Classroom Environment with 285 Middle School Students
I led the controlled experiment with 285 seventh-grade students at our partner school to assess the impact of ActiveAI on AI literacy learning outcome comparing to the traditional 'tell and practice' method commonly used in AI education.
Significant AI Literacy Learning Gains
Our pre-test and post-test results revealed that ActiveAI significantly outperformed the control condition, achieving four times the learning gains and markedly higher self-reported competence in using AI technology.
The rapid advancements in Artificial Intelligence (AI), especially in generative AI, underscore the pressing need for comprehensive AI Literacy Education. This entails the ability to interact seamlessly with AI, ensuring effective and ethical problem-solving across diverse sociocultural landscapes.
In collaboration with world-class AI experts and learning engineers, I led the design and development of ActiveAI, a series of bite-size, interactive AI Literacy modules for K-12 learners. Our goal is to provide a scalable, evidence-based AI Literacy product to prepare the upcoming generation to navigate an AI-driven future with confidence.
Overview
High Engagement and Usability Satisfaction Among Students
Additionally, qualitative interviews reflected a high level of user satisfaction, with participants unanimously expressing satisfaction with the system's usability and overall experience.
"The system's interactive learning experience is really engaging. I found the hands-on classification task particularly memorable, especially when I encountered a little failure. It made a lasting impression and was the most interesting part for me."
- Grade 8 Middle School Participant
The rapid advancements in Artificial Intelligence (AI), especially in generative AI, underscore the pressing need for comprehensive AI Literacy Education. This entails the ability to interact seamlessly with AI, ensuring effective and ethical problem-solving across diverse sociocultural landscapes.
In collaboration with world-class AI experts and learning engineers, I led the design and development of ActiveAI, a series of bite-size, interactive AI Literacy modules for K-12 learners. Our goal is to provide a scalable, evidence-based AI Literacy product to prepare the upcoming generation to navigate an AI-driven future with confidence.
Overview
Certain interface elements can sometimes yield unintended learning outcomes. For example, user testing revealed that varying facial expressions on agents led learners to misconceive that low and high temperature settings in chatbots result in more 'negative' word outputs. This issue was resolved through subsequent iteration.
Significant AI Literacy Learning Gains comparing to Control Condition
Our pre-test and post-test results revealed that ActiveAI significantly outperformed the control condition, achieving four times the learning gains and markedly higher self-reported competence in using AI technology.
Significant AI Literacy Learning Gains comparing to Control Condition
Our pre-test and post-test results revealed that ActiveAI significantly outperformed the control condition, achieving four times the learning gains and markedly higher self-reported competence in using AI technology.
Evaluation Research
Measuring Learning Experience and System Usability in the Field
I led the controlled experiment with 285 seventh-grade students at our partner school to assess the impact of ActiveAI on AI literacy learning outcome comparing to the traditional 'tell and practice' method commonly used in AI education.
Evaluate ActiveAI in Real Classroom Environment with 285 Middle School Students
Project Impact
Establish Product Direction and Cultivate Learner-Centered Design Culture
To ensure my design not only deliver satisfying user experiences but also enhance learning outcomes, I developed mid-fi prototypes, conducted user tests with learners, and iteratively refine the interface in a weekly basis for various modules.
Rapid Iteration with Learners in the Loop
Mid-Fi Prototype and Pilot User Testing
The research findings validated the company's product direction and were published in learning science conferences. Furthermore, the learner-in-the-loop design cycles and evidence-based approach were recognized by the organization and incorporated into future product design processes.
Evaluation Research
Measuring Learning Experience and System Usability in the Field
Evaluate ActiveAI in Real Classroom Environment with 285 Middle School Students
I led the controlled experiment with 285 seventh-grade students at our partner school to assess the impact of ActiveAI on AI literacy learning outcome compared to the traditional 'tell and practice' method commonly used in AI education.
Summary
Prepare Today's Youth for a Future Where AI is a Fundamental Aspect of Daily Life and Decision-Making
In this project, I proposed and designed an AI Literacy tutoring system targeting the 5 Big Idea in Artificial Intelligence for K-12 learners as an answer the the challenges posed by complexity of AI concepts and resource-intensive initiatives. We believe ActiveAI has the potential to democratize AI Literacy education and cultivate efficient and ethical AI Users at scale.
Certain interface elements can sometimes yield unintended learning outcomes. For example, user testing revealed that varying facial expressions on agents led learners to misconceive that low and high temperature settings in chatbots result in more 'negative' word outputs. This issue was resolved through subsequent iteration.
High Engagement and Usability Satisfaction Among Students
Additionally, qualitative interviews reflected a high level of user satisfaction, with participants unanimously expressing satisfaction with the system's usability and overall experience.
"The system's interactive learning experience is really engaging. I found the hands-on classification task particularly memorable, especially when I encountered a little failure. It made a lasting impression and was the most interesting part for me."
- Grade 8 Middle School Participant
High Engagement and Usability Satisfaction Among Students
Additionally, qualitative interviews reflected a high level of user satisfaction, with participants unanimously expressing satisfaction with the system's usability and overall experience.
"The system's interactive learning experience is really engaging. I found the hands-on classification task particularly memorable, especially when I encountered a little failure. It made a lasting impression and was the most interesting part for me."
- Grade 8 Middle School Participant
Project Impact
Establish Product Direction and Cultivate Learner-Centered Design Culture
The research findings validated the company's product direction and were published in learning science conferences. Furthermore, the learner-in-the-loop design cycles and evidence-based approach were recognized by the organization and incorporated into future product design processes.
In this project, I proposed and designed an scalable AI Literacy tutoring system targeting the 5 Big Idea in Artificial Intelligence for K-12 learners as an answer the the challenges posed by complexity of AI concepts and resource-intensive initiatives. We believe ActiveAI has the potential to democratize AI Literacy education and cultivate efficient and ethical AI Users at scale.
Summary
Prepare Today's Youth for a Future Where AI is a Fundamental Aspect of Daily Life
Project Impact
Establish Product Direction and Cultivate Learner-Centered Design Culture
Project Impact
Establish Product Direction and Cultivate Learner-Centered Design Culture
The research findings validated the company's product direction and were published in learning science conferences. Furthermore, the learner-in-the-loop design cycles and evidence-based approach were recognized by the organization and incorporated into future product design processes.
Summary
Prepare Today's Youth for a Future Where AI is a Fundamental Aspect of Daily Life
In this project, I proposed and designed an AI Literacy tutoring system targeting the 5 Big Idea in Artificial Intelligence for K-12 learners as an answer the the challenges posed by complexity of AI concepts and resource-intensive initiatives. We believe ActiveAI has the potential to democratize AI Literacy education and cultivate efficient and ethical AI Users at scale.