Is there an alternative to how punctuation has always been taught?
Designing a web app to help students learn intuitive punctuation, not grammar rules.
As a writing tutor, when students want help with grammar and punctuation, I always offer:
"Can I read your text aloud and see what trips me up?"
I couldn't always explain why this worked, but it always does. Somehow, hearing the writing spoken aloud highlights what silent reading misses. Polly is my attempt to understand why, and build something from it.
Polly was developed in a team of 5 (myself, Nathan Hadjiyski, Emily Lin, Isabelle Tang, Alishba Gul) within 3 weeks for Dr. Yukang Yan's HCI course.
The decade-long problem
For the last decade, punctuation misuse has consistently ranked top #20 most common issues in college students' writing (Lunsford & Lunsford 2008). Grammar rules, when offered, are pedantic and hard to apply. And the tools students turn to don't help: Grammarly corrects without explaining, ChatGPT rewrites without teaching. Students aren't building revision skills, but dependency.
"I rely too much on auto-correct features and ChatGPT.[a]"
"My methods using built-in autocorrect features can get repetitive and also feel meaningless, so I don't retain what I learn.[a]"
"Because there's a lot of grammar rules, it gets confusing on how to properly identify which rule applies to which.[a]"
[a] students' responses to our user survey
Yet the deeper problem isn't the tools, rather, it's the underlying assumption that punctuation is a matter of rules to memorize and apply -- an incorrect assumption.
"The punctuation of a 'good' writer is meaning-dependent, not grammar-dependent." (Dawkins, 1995)
I believe this from my firsthand experience tutoring: the right tool doesn't enforce rules, but helps students hear what their writing means.
Students know what they want. Nothing delivers it.
We surveyed 27 university students about their revision habits and challenges. Two findings defined the project. First, the dependency problem is real:
85% of students rely on writing assistants, including AI tools.
However, most find them ineffective for actually learning. Students know they're outsourcing their revision. They don't want to, but nothing better exists.
"AI-based tools are not as effective as TAs or professors at pointing out where I went wrong. Instead, they rearrange the sentence and make it correct. It makes me more reliant on the tool, not a better writer."
Second, students know what they want, they just can't get it:
74% value being able to revise independently, and 60% want support that helps them learn how to revise better, not just get corrections.
The gap between what students want and what tools offer is the design opportunity. From these findings, we defined three principles for Polly:
Hands-On Learning.
Trivial or one-click interactions don't lead to lasting change. This is why students turn to tutors, TAs, and professors for support.
No Prescriptive Rules.
Students find memorizing and following grammar rules ineffective, tedious, and unsustainable.
Repeating is OK.
Students do not mind meaningful repetition as they shift their revision habits.
When a tutor's hunch meets linguistic research
My tutoring hunch needed validating. Through synthesizing 10 research works in linguistics and writing pedagogy, I found that there is a well-documented link between effective punctuation and speech patterns, particularly intonation.
Three findings shaped Polly's design:
Meaning-based grammar is more reliable than rule-based grammar.
Even "foolproof" rules break down in practice, since their application is always context-dependent. For example, the common rule “do not put a comma between a Subject and Verb” can become complicated, as a pair of 2 commas can be valid.
Intonation signals sentence structure.
Significant changes in intonation mark the boundaries between information units (the chunks of meaning a sentence is organized around). If you can hear where your intonation shifts, you can hear where your sentence structure changes. (Halliday, 1989)
Intonation patterns are consistent enough across English speakers to be useful.
Despite regional variation, the fundamental intonational habits of English speakers overlap significantly across the US, UK, Canada, Australia, and South Africa. (Elbow, 247)
My tutoring hunch was correct; I just didn't have the language for it. Now I have a framework to formalize what I'd been doing for students and represent decades of linguistics research with design for the first time.
Intonation signals sentence structure, so if students can hear intonation, they can design the sentence structure better; punctuation is merely the visual notations for that structure.
A tool that makes this connection clear will teach punctuation the way a tutor does: not through correction but through discovery.
Step 1: Read aloud. The user performs their sentence.
Step 2: Review intonation graph. Polly generates a graph of the user's intonation pattern. The user reviews significant changes in intonation, which denote potential information unit boundaries.
Step 3: Revise & repunctuate. Polly guides users to revise their sentence using meaning-based grammar, with their intonation pattern as cues.
Polly's simple 3-step flow is designed to be easily repeated by users, regardless of how large the text they are revising is. It mimics the tried-and-true process writing tutors use to help students with grammar concerns. The activity loop honors the ACT-R theory of learning: skills are built through repeated, applied practice, not through memorizing declarative facts (like grammar rules) and hoping they transfer (Anderson 1996).
Each time a student reads aloud, reviews their intonation, and revises, they're practicing a procedure. Unlike rules, procedures get faster and more automatic the more they're repeated. This is why Polly is content-agnostic: it doesn't penalize students for repeating on the same sentence, or for using it on "easy" ones (not that there is actually such a thing), because repetition on any instance of the skill strengthens it.
Research changed a technical decision!
Early in development, we assumed pauses in speech would be a meaningful signal for punctuation placement. When my research revealed otherwise (pauses are too variable and context-dependent to be reliable), I worked with our back-end developer to revise our approach, ultimately switching to pitch and volume (intonation) as the primary signal. This changed which speech-to-text and visualization packages we used.
We designed an experiment, not usability tests.
Most product evaluations ask: can users complete tasks? That wasn't the right question for Polly. We needed to ask:
Does Polly change how students revise?
That required a behavioral experiment, not a usability test.
Participants
9 undergraduate students from the University of Rochester and RIT:
- 5/9 ESL students (10–20+ years of English education)
- 4/9 native English speakers
- Range of class years and prior writing instruction
Experiment structure
We used a within-subjects design: every participant experienced both conditions (baseline and Polly). This reduced individual variability and allowed direct performance comparison without needing a separate control group.
Each participant revised three texts (Text A, B, and C) which were controlled for content and complexity (academic paragraphs on US History events). All three contained the same five punctuation issue types, drawn from the top self-reported errors in our user survey:
- Missing hyphens
- Unnecessary hyphens
- Comma splices
- Missing commas in compound sentences
- Missing commas with nonrestrictive elements
Participants rotated through different texts in each condition so prior exposure to content wouldn't inflate results:
- Text A: initial baseline interaction, participants freely revise the text however they'd like
- Text B: participants use Polly to revise text B
- Text C: participants freely revise the text however they'd like
What we measured
- Punctuation accuracy: how many issues participants identified and correctly revised, measured using a rubric on the texts.
- Confidence: self-rated before and after, for both identifying and correcting issues. Participants self-report on a scale of 1–5 in a survey administered after revising text C.
- Revision strategy: what approach participants described using to identify punctuation issues. Participants self-report in a qualitative question on the survey.
Punctuation accuracy and Confidence tell us whether Polly was helpful. Revision strategy tells us whether it compels participants to change their behavior.
How did Polly affect participants?
Accuracy improved
Participants are able to identify more punctuation issues (+26%) and of more types.
Every attempt to
correct an
identified issue was accurate (+25%).
Participants who revised more than 4 sentences with Polly showed the
clearest gains.
Confidence improved
Self-rated confidence in identifying issues rose to 3.9/5 (+0.5).
Confidence in correcting rose to 4.1/5
(+0.5).
Revision strategies now include reading aloud
Before using Polly, 5/9 participants relied on intuition and 4/9 on grammar rules.
After using Polly,
7/9 read
aloud to revise, integrating speech patterns with punctuation placement across all punctuation types,
not just
commas.
"I didn't consider reading aloud before because I only do it for very complicated texts, but turns out it helps."
"Incorporating more vocal patterns made revising easier the second time around."
These results are not statistically significant at this sample size. But a behavioral shift in 7 of 9 participants after a single session suggests the underlying approach is promising.
If I had more time…
I'd like to explore a baseline approach, where users record a speech sample before starting, so Polly can give feedback calibrated to each speaker's natural intonation. I believe this can improve feedback accuracy for speakers with non-standard intonation patterns, like ESL speakers.
I would also like to conduct longitudinal testing to assess whether Polly effectively supports and sustains the desired writing behaviors over time.