A product already live. A value gap no one had named yet.

The brief

SecureLeap came to IVI with a gamified cybersecurity learning app already in market. The product had a mascot, a lesson system, a points economy, and a mobile-first experience. What it didn't have was validated evidence that any of it was working for learners. The brief was to find out: who the learners were, what they actually needed, and whether the product as designed could serve them.

My team for this engagement was a project manager and a principal investigator from the Design Innovation department at Maynooth. The research design, interview facilitation, analysis, and synthesis were mine.

Where I started: learning from the inside

Before interviewing anyone, I used the app myself. Auto-ethnography, for a digital product, means becoming a first-time user with deliberate attention: noticing every friction point, every moment of confusion, every reward that lands flat. I went through full onboarding, completed the introductory cybersecurity course (8 modules), and documented every step.

What I found wasn't catastrophic. The install was fast, navigation was broadly clear, and the bite-sized modules (~3 minutes each) had genuine potential. But the reward system was unexplained. Diamonds accumulated with no stated purpose. Health points depleted on wrong answers with no tooltip about refills. After finishing an 8-module course: no badge, no signal of completion, no win. The app had gamification mechanics but no gamification logic.

The bigger structural problem was an information architecture mismatch. Tapping "Intro to Cybersecurity" on the home screen routed to a "Quest" screen, a different section entirely. The mental model broke before learning even began.

Auto-ethnography: First-session walkthrough findings

The user research

With the auto-ethnography complete, I ran 4 semi-structured user interviews and 1 SME interview. The user interviews covered learners at different stages: a security professional upskilling for career relevance, two postgraduate research assistants using the app for their own development, and a participant who had tried multiple cybersecurity platforms and disengaged from all of them. The SME brought 15 years of e-learning experience including deep knowledge of instructional design theory, assessment standards, and cognitive load research.

Interviews were structured around motivation, existing learning habits, platform experience, gamification attitudes, pricing logic, and what success looked like to each participant. Synthesis followed thematic coding across all five transcripts, with the auto-ethnography findings used as a lens throughout.

The value gap

Across the interviews, a single pattern emerged more clearly than any usability problem. Learners were not struggling to use the app. They were struggling to justify paying for it. The calculation was precise and unsentimental.

This wasn't a pricing objection. It was a product architecture question. The competitive landscape made it concrete: TryHackMe at $14/month, Hack The Box Academy at $18/month, both offering browser-based labs and structured career paths. Against free YouTube and generous free tiers on major platforms, any paid product has to offer something structurally unavailable for free: guided lab environments, role-based career tracks, credentialed assessment, or a community that peer-validates progress. The product had none of these with clarity or depth.

The assessment problem

The SME interview opened a thread the brief hadn't anticipated: what does assessment actually mean in cybersecurity education? Most platforms issue a certificate when a learner scores above a threshold percentage. But a threshold score on multiple-choice questions does not tell you whether someone can do the job.

The research drew on Criterion Referenced Test Development by Sharon A. Shrock and William C. Coscarelli, a foundational text in instructional design. Criterion-referenced assessment defines a minimum performance standard for a specific job task and tests directly against that standard: you either meet the threshold or you don't. Norm-referenced assessment ranks learners against each other. For cybersecurity, the distinction matters: you either know how to respond to an incident, or you don't. Percentile ranking is operationally irrelevant. A score of 70% on an incident response module says nothing about whether a learner can actually contain a breach.

For SecureLeap, this had direct product implications. If the platform wanted to issue certification that employers would take seriously, the assessment design had to start from job task analysis: defining minimum competency for each skill area before writing a single test item. That work hadn't been done. The certificates the app was capable of issuing would carry no credibility until it had.

SME Insights & Competitive Landscape

What the research produced

The synthesis produced four findings, each with a direct design implication for the next phase.

Research Synthesis & Insights Deck

The result

The research phase is complete. The insights deck and user journey documentation are in final synthesis ahead of handover. The project is on track to convert to a second IVI Innovation Voucher, moving from research into design: translating what the research found into a product direction the client can act on.

The most useful thing the research produced was not a list of UX fixes. It was a reframe of the core problem. The app's usability was serviceable. The value architecture was not. A second phase that addresses the value structure, the assessment model, and the web experience will have a clear research foundation to build from.