G / C
ASSISTIVE MAGIC
GENRE
Accessibility, augmented reality, companion app
ENGINE
Unity — EasyAR, Vosk, Wit.ai
ROLE
Research, UX/interaction design, development (AR tracking, voice pipeline, data integration)
TEAM SIZE
1 (individual thesis project, academic supervision)
GAME STATUS
Validated prototype — tested with 20 users
GOALS
-
Reduce physical card handling during a game turn, without removing the cards, the table, or the opponent from the experience.
-
Offer a genuinely multimodal interaction — touch and voice, redundant with each other — that stays usable even when one channel degrades.
-
Preserve the authenticity, the player's control, and the social dimension of the traditional physical game intact.
Assistive Magic is an augmented-reality mobile app designed for Magic: The Gathering players with upper-limb motor impairments, developed as a Master's thesis project in Computer Science. Its distinctive mechanic combines AR recognition of physical cards with voice commands, letting the player draw, move cards between zones, and look up card text without having to repeatedly lift or handle them — keeping the cards, the table, and the opponent physically present, instead of replacing them with a screen.
APP
MY CONTRIBUTION
Being an individual thesis project, the research, design, and implementation are entirely my own.
-
Two-stage voice pipeline (Vosk + Wit.ai)
-
Result: reliable voice commands even under offline activation, with a command vocabulary that can grow without retraining a local model.
-
Problem: recognizing speech and interpreting it as a structured game command, while staying usable on an unstable connection and without perceptible latency at activation.
-
Solution: on-device local transcription with Vosk (no network round-trip for the wake word), semantic interpretation via a Wit.ai REST request, with controlled network-error recovery and coroutine-based sequencing to eliminate race conditions between the two stages.
-
-
Redundant multimodal interaction (touch + voice)
-
Result: every game function stays accessible even when one of the two channels degrades — later confirmed in testing with real users.
-
Problem: ensuring a player with reduced upper-limb mobility could always complete an action, even when voice recognition failed or precise touch wasn't practical.
-
Solution: a dual-channel architecture where touch and voice converge on the same game state with no dependency on each other — every button-accessible function has an equivalent voice command, and vice versa.
-
-
AR card tracking and zone management
-
Result: drawing, moving, and looking up cards without lifting them from the table, with game state (hand, battlefield, graveyard) synced automatically.
-
Problem: reliably recognizing physical cards under the variable lighting, angles, and occlusion typical of a real play surface, without replacing the physical experience with a purely digital one.
-
Solution: image tracking (EasyAR) on a preloaded deck with QR markers on the card backs, feeding a query to the Scryfall API for text, keywords, and metadata — keeping the physical card as a tangible "anchor" for the interaction.
-
VALIDATION: User Testing
Test goal: to verify whether multimodal interaction actually reduces physical card handling, stays effective when one channel degrades, and preserves the authenticity and social dimension of the physical game — without sacrificing the game experience.
Framework and references: a single questionnaire administered at the end of the session, built on validated instruments — the System Usability Scale, the Raw NASA-TLX, an adapted Game Experience Questionnaire — plus original items mapped to 4 research questions, with the design of the multimodal items informed by the Customizable Interactions Questionnaire.
Key results:
-
20 participants: 8 with upper-limb motor impairments and 12 controls.
-
System Usability Scale: 84.0 for the control group, 75.0 for the impairment group — both in the "good usability" range.
-
Perceived workload (NASA-TLX) stayed low for both groups.
-
Enjoyment of the game (Positive Affect, GEQ) was nearly identical between the two groups (4.62 vs 4.50) — the app reduces physical handling without affecting how much fun the game is.