Emotion AI Research
Research · Affective Computing · 2021 — Present

Rethinking how machines read human emotion.

A four-year research initiative investigating why current Emotion AI fails — and how a personalized, culturally-adaptive alternative could work. Two peer-reviewed papers, one Master's thesis, and a working prototype that questions the foundational assumptions of the field.

4
Years of Research
3
Published Papers
1,094
Self-labelled Events
72.5%
Model Accuracy
The Central Question

What if we've been training machines to read emotion the wrong way?

Most Emotion AI today is built on Paul Ekman's 1978 model — assuming seven universal emotions visible through facial expressions, generalized across all humans.

But what happens when these systems meet a country of 1.4 billion people, 22 official languages, and thousands of cultural display rules? What happens when context — the cognitive layer that gives emotion its meaning — is completely absent from the dataset?

"A webcam cannot objectively read a person's true inner emotion. The face is behavioural evidence — not emotional truth."
— Personalized Emotion AI, 2025

This research argues for a fundamental shift: from universal classification to personalized calibration.

The Research Arc

Four stages of inquiry — from theoretical critique to working prototype.

2021
M.Des Thesis · IIT Guwahati
Foundation Research
Deconstructed emotion across its transitional subfields — primary emotion, feelings, mood, temperament, and personality. Established that current Emotion AI captures only the brief, spontaneous primary state.
2023
ICoRD '23 · IISc Bangalore
Emotion AI in India
Two user studies demonstrated that context — not the face — carries the real emotional signal. 89.4% of participants correctly identified emotion when context was given.
2025
ICoRD '25 · IIT Hyderabad
Culturally-Adaptive Emotion AI
Proposed an ethical framework using IDEO's AI Ethics Cards. Four studies mapping data parameters, minimum viable data, visible "seams" for transparency, and a GUI based on Russell's circumplex model.
2025
Working Prototype
AffectLab — Personalized Emotion AI
54 sessions, 1,094 self-labelled events. 72.5% accuracy and 0.7259 macro F1 — proving bottom-up, personalized models can outperform universal ones.
The Core Problem

Three structural failures of current Emotion AI.

01
Only Primary Emotions
FACS-based systems capture short-term reactions lasting seconds. They miss mood, temperament, and the long-term affective state that defines a person.
02
No Cultural Awareness
Ekman's seven universal emotions ignore cultural display rules — the unwritten guidelines about when, how, and with what intensity to express emotion.
03
Context-Blind by Design
A wink and a blink look identical. A cringing smile and a happy smile look identical. AI trained only on faces cannot tell the difference.
Paper 01 · 2023

Emotion AI in India: Why Western models fail in a culturally diverse nation.

Madhukaillya, M., & Kapoor, G.S. · ICoRD '23 · IISc Bangalore

This paper ran two studies to test the limits of facial-only emotion recognition.

Study 01
The Context Test
15 participants, image pairs with contradictory captions. With context, 89.4% labeled correctly. The same facial expression could carry two opposite meanings.
Study 02
The Cultural Test
Culturally-loaded scenes: Muharram mourning, Kathakali dance, India vs Pakistan cricket. Emotional responses varied dramatically by personal experience and cultural memory.
Figure 01 · Map of Cognition Factors
"Emotion AI right now has a weak foundation. The skewed datasets cannot account for thousands of emotions beyond Ekman's universal seven."
— Emotion AI in India, 2023
Interactive · Recreation of Study 01
Take the Context Test
You'll see an image, read a sentence with a missing word, and choose between two emotions that produce the same facial expression.
Paper 02 · 2025

Culturally-Adaptive Emotion AI: An ethical framework for individualistic models.

Kapoor, G.S., & Madhukaillya, M. · ICoRD '25 · IIT Hyderabad

Four studies using IDEO's AI Ethics Cards as design principles.

Study 01
Blind Spot Check
Mapped every data parameter — automatic detection and manual feedback.
Study 02
Mapping the System
Traced emotion data through the pipeline. Two stages where bias enters.
Study 03
Minimum Viable Data
Maximum vs minimum viable data for every parameter. Privacy by design.
Study 04
Designing the Seams
"System Knows Best" vs "Person Knows Best". Visible thresholds for user control.
Figure 02 · The Journey Map
Figure 03 · GUI Wireframe
Paper 03 · 2025 · Working Prototype

AffectLab: From universal labels to individual models.

Self-calibrated facial-affect collection, training, and live prediction system

Start with one individual, train deeply, then look for transferable patterns.

Figure 04 · Conceptual Shift

The System

Built in Python using OpenCV, MediaPipe Face Landmarker, scikit-learn, and Random Forest. Captures dense facial landmarks paired with self-labelled affect tags.

Figure 05 · AffectLab Interface

Six Affect-Family Buckets

Positive Expressive Affect
Joy · Happiness · Laughter · Amusement · Pride · Relief
Negative Tension / Threat
Anxiety · Fear · Frustration · Anger · Disgust · Contempt
Cognitive Engagement
Focused · Reflective · Interest · Curiosity · Confusion
Low-Valence Withdrawal
Sadness · Boredom · Fatigue · Sleepiness · Yawn
Surprise / Startle
Surprise
Neutral / Calm Baseline
Neutral · Calm

Results

54
Sessions
1,094
Events
72.5%
Accuracy
0.7259
Macro F1
"Affect recognition is not just a classification problem — it is a person-specific interpretation problem."
— AffectLab, 2025
Key Concepts

Three ideas from four years of research.

01
Universal → Personalized
Many models, each trained per person, with patterns generalized only where they statistically hold.
02
Face is Behaviour, Not Truth
A smile can mean joy, politeness, embarrassment, sarcasm, or masking. Interpretation requires context.
03
Person Knows Best vs System
The best models honour both human context and machine measurement, with visible seams.
Ethics & Concerns

A system that asks how you feel could become an affective surveillance layer.

Four non-negotiables: Transparency at every extraction point. Consent-based collection. Minimum viable data. Individual ownership of personal models.

Publications

Peer-reviewed work and ongoing research.

2025
From Universal Labels to Individual Models — A Self-Calibrated Approach
Kapoor, G.S., & Madhukaillya, M.
Under Review
2025
Culturally-Adaptive Emotion AI: Unravelling the Fabric of Diversity
Kapoor, G.S., & Madhukaillya, M.
ICoRD '25 · IIT Hyderabad
2023
Emotion AI in India
Madhukaillya, M., & Kapoor, G.S.
ICoRD '23 · IISc Bangalore
2021
M.Des Thesis — Foundation Research on Emotion AI
Kapoor, G.S.
IIT Guwahati

The future of Emotion AI should be honest, personalized, and culturally aware.

If you're researching affective computing or thinking about the ethics of these systems — I'd love to talk.

Start a conversation →
🔧

Page Under Maintenance

We're currently updating this page. Please check back soon!

Back to Home