One of the more unique challenges presented by a game where the player can form their own sentences is not only how easy it is for the player to mess up their sentence, but how many ways there are to do it! Everything from wrong-gendered pronouns to disagreeing pronouns and verbs to incorrect word order to valid sentences but wrong content…
The way we handle this is with language parsing. We actually break down the player’s answer into its words, look at their parts of speech, and figure out what properties they have (gender, number, and person are the properties we care about in our prototype). We do the same for the known correct answer – we now have syntax token lists.
From there we launch into a series of structure comparisons, gates by which we determine what kind of wrong the player’s answer is. Depending on how their answer is syntactically incorrect or differs from the expected answer, we’re able to deliver a clear and targeted pre-authored response, which is customized based on the words and sentence in question. A diagram of the process (for the sample incorrect answer of “Ella la cocinan”, where “cocinan” does not agree with “Ella”):
The way that we encode the property information is completely extensible – it’s a seamless process adding new properties such as tense, voice, etc., because part-of-speech analysis and agreement is completely data-driven. The next step for this parsing system to make it applicable to hypothetical future exercises that delve into more advanced Spanish territory would be to encode grammar in a more formal way and represent syntax as a tree to account for clauses, but that’s getting into territory where 3rd party frameworks would become worth investigating.