According to a recent study performed by a graduate student in Sweden, most educational software isn’t actually helping students. Björn Sjöden, who led the study, claims that only about 17% of math and Swedish teaching software, out of the top 100 apps in those categories, actually offer informative feedback. Most just test students and then provide the correct answer but nothing else. That, he says, isn’t helpful.

An app developed to teach parts of speech was actually more successfully used by illiterate children than their literate peers, not because it taught them anything, but because they simply guessed well. Literate children who took the time to answer questions got more wrong.

The problem as Sjöden sees it is that in order to actually be educational, apps must give students helpful feedback and not just tell them if they got something wrong. At the least they have to explain what they got wrong, and help them understand the correct answer.

He found that apps which engaged students were also better. Engagement can be hard, and can vary from student to student, but part of the study had students use a math learning app for eight weeks in which they helped a character learn math. When tested after using the app, half the students took a test featuring that same character, while half took the same test without the character. Unsurprisingly, the students tested with the character performed better, because they remained engaged in the process of helping this character, a digital pupil, learn math.

Interactive spaces, like the Internet and app stores, are overrun with “teaching apps,” most of which would likely fail to meet Sjöden’s standards. Sjöden’s findings probably wouldn’t come as much of a surprise to many educators, but lots of parents might be shocked to learn that the apps they downloaded to help their kids learn probably aren’t having any benefit.