Aug. 27, 2015 “I can make or break you” may sound more like a line from Good Fellas than a children’s app review site, but the skullduggery and reticent storytelling seeping out of the industry cracks since I wrote “Part One: Children’s Apps: Does Money Talk?” has nothing on Scorsese.
The last few weeks have been a sophisticated sift and sort game of “perception vs reality” as I try to matrix the players responding to an ‘open call for feedback’ on paid app reviews, sorting out serious dirty laundry that has me covering my ears like the three wise monkeys from my own childhood stint in Japan, ‘Hear no evil, see no evil, speak no evil.’
Being outside of the kids apps sphere, my wide-eyed “say whaaa?” incredulity of “off the record” tidbits has ranged from trying to determine what constitutes disgruntled former employees, favoritism and influence peddling, to stories of lawsuits, IP theft, giveaway fraud, tainted research dollars with corporate tethers and graft and corruption that rivals the robber barons of the wild west frontier…21st century digital edition.
…Like the perception that app reviewers are gleefully rolling in money gleaned from fee-based app reviews, when most indie sites doing thorough assessments barely make enough cash to put in a ‘tip jar,’ once site upkeep and costs of doing business are extracted.
…Or the naïve notion by app reviewers that their self-selection process of determining what’s review-worthy (with pay for play predominance) is NOT blatantly perpetuating an unlevel playing field already dinged with divots and sinkholes that can cripple a talented team that missteps and put them out of the game permanently before even getting to show they’re stuff.
Then there’s the “corporate tech/media doesn’t influence ME” balderdash, which is laughable given the fear-based ‘disease to please’ by all parties, along with new partnerships and mentoring models like apps hooking up with big industry players in early cycles of app creation, or transitions of ad hoc, grassroots collectives like Moms with Apps into formalized policy alliances, since they’re now ACT/The App Association…It brings a whole new meaning to the “Know What’s Inside ® ” transparency motto.
Let’s all just acknowledge bias exists in many shapes and forms and use critical thinking skills to make sense of it, shall we?
For the “TLDR” Crowd, I’ll Start with the Finish
Are paid app reviews happening? Unequivocally.
Becoming normative? Yes, much more so.
Openly disclosed? Rarely per review.
Are paid app reviews ethical? Objective? Effective? Depends on who you ask. (not to mention site reputation, criteria, and content quality)
Do “expedited reviews” increase sales and visibility? Not necessarily. That’s a summary overview from some of the feedback we’ll dig into with voices/views from all lenses in part two.
Tipsters asking to be cloaked in anonymity with some of the vested interests in play leave me open to libel if I ‘print what I now know’ on some of the backstories, which stirs the queasies in me, but there are enough repeated patterns in profound commentary to report the basics and enlighten while giving a sweeping brushstroke to a canvas of ‘too fast, too soon’ galactic startup growth in an industry comprised of flame outs and shooting stars.
The good news is, “on the record” names and backgrounds have helped me deconstruct business models to gain a better understanding of both the value and the systemic problems of kids app review money matters that shift faster than you can say ‘expedited review.’
In sum, we’re only scraping the surface of the problematic underbelly of money machinations and media giants dominating the landscape, as fledgling apps struggle to ‘get discovered’ in the current review process which is lopsided in favor of the almighty greenback, even for the best of reasons, like site sustainability. In today’s marketplace of over a million children’s apps in the iTunes store alone, the opportunities and challenges for app creators astound.
Carisa Kluver of Digital Storytime offers a unique, hybrid perspective and context, given her five years of ongoing insights as a reviewer since the iPad’s very inception, the app developers in her family, and her own current career segue back into teaching, given the massive influx of voices and media and monetization shifts. She summarily cautioned,
“The app market is very crowded today and getting noticed is more of a needle in the haystack issue than it is like getting noticed at a school dance. Chances are, even with a paid review, if your app isn’t exceptional it simply won’t break even, period. This isn’t your grandmother’s app store anymore … it’s hard core…
…For app developers, I recommend paying for expedited reviews on ‘trusted’ sites with solid traffic if they want coverage, but not to expect direct sales from reviews. For parents, my tips are in the article “The Dirty Little Secret About App Review Sites” and my favorite sites are here: (full Q&A interview with Carisa Kluver’s candor about the industry forthcoming)
“Here’s what I would say to a new app developer: There is no magic bullet. Even getting featured by Apple doesn’t guarantee continued success. A good review on a respected site won’t shift the needle much. Four years ago an innovative app could rise to the top. Not anymore. You probably shouldn’t quit your day job.
It is very hard to make a living as a children’s app developer these days. Even some of the leading app companies are struggling. Discoverability, the culture of free apps, the overwhelming number of apps in the App Store all add to problems. Lack of direct contact with customers is a big issue. Also, Apple does not count ‘educational sales’ towards ranking, so for someone targeting that audience like me in special education, those sales don’t help an app get ranked and discovered…” —Libby Curran (@readingbuds K-1 Special Educator, Grand Prize Winner, Technology and Reading Award, ILA, 2015, a People Magazine Teacher of the Year 2012, Christa McAuliffe Sabbatical Fellow, ’06-07 etc.)
Apps Environs and Tech Titans
Clearly the universal problem of discovery is readily apparent and agreed upon…the solutions and best practices? Not so much.
In part one we discussed the difficulty for parents and educators trying to discern what’s good out there beyond word of mouth and buzz from pals since the FTC interpretation of what constitutes ‘disclosures’ for paid reviews is squishy not to mention loaded with inherent challenges like bias, monitoring, enforcement, fines and big tech, big budget domination.
Today, we’re talking to the creators and reviewers themselves…trying to discern exactly ‘what’s changed and why’ in the short amount of time apps have been in existence and help parents and educators navigate through the hype of ‘content marketing’ and help FIX the flaws in the very foundation of this nascent process to build a better system for all.
Market research firm IDC reports a $7 billion K-12 and higher ed school sector, so it’s no wonder some of the blood baths among ed tech creators are messy as they joust for position like a Game of Thrones script!
The Bill and Melinda Gates Foundation spending alone brings a whole new meaning to “EdSurge,” as makers of apps vie for attention of teachers to be used (or even swallowed by) tech titans as research and funders co-mingle with opportunities in the education realm, and winners and losers become entangled in the siege.
How do larger hub sites that do reviews keep from becoming ‘lobbyists with apps’ impacted by influential media tethers and tech platforms that can influence, especially in the education sphere?
As it is, Google Play and Apple continue their App Store wars in the media and now the platforms are fueling their fires targeting schools with kids no doubt caught in the crossfire for mindshare.
Beyond the popularity of education apps, the free apps and “freemium” focus dominating the kids app sphere has led to modifications of the review platforms themselves…
Some review hubs have shifted focus from more robust, detailed coverage to watered down “click-throughs for page views” to feature “free app” banners and tons of contests, code giveaways, and app chat noise that ultimately shifts toward quantity over quality when the sameness of stars and graphics trump helpful analysis and critical thinking with nuanced commentary.
The more in-depth the coverage, the more one has to ask, “and who’s going to pay for that writing” since we all know those of us with indie sites who do NOT have staffers and are NOT paid via ad revenue are writing our hearts out in a public service labor of love, in-depth research, and personal analysis that is NOT sustainable. (myself included…things have to shift…ideas?)
We’re seeing seismic quakes and shake outs that may signal it’s time for rebuilding a whole new structural foundation that’s stronger and reinforced with logic, fairness, value and disclosure criteria. So, parents and educators:
What are ‘paid app reviews’ and why should we even care?
What’s the difference between an expedited review and a “sponsored” review?
When did this become ‘a thing?’ Here are a few voices and views for better understanding:
The App Developer Lens:
An “expedited review” claims to fast track an app’s presence into the process of discovery, but that can get pricey fast for small developers trying to debut a worthy app without breaking the bank.
More importantly, this means well-funded, mediocre apps can get a boost to hopscotch over exceptional ones that are languishing “in the tall grass awaiting discovery” (as one creative app developer phrased it!) simply due to their fledgling status. That’s a profound problem for innovation, and has rubbed many startups the wrong way as evidenced by this Facebook dialogue screenshot (closed group/names removed) discussing my last post in part one.
One app creator lamented to me,
“Reviewers who offer an “expedited” review (and there are MANY) are biased toward developers who pay. Even if the reviewer only takes money to review “good quality” apps, they are ignoring other “good quality” apps by developers who weren’t prepared to pay their expedited review fee. In my experience, a “free” review either takes months or, more likely, never materializes at all.
Do the readers know this? I don’t think so.
…My experience has been that a noticeable majority of websites/blogs exclusively doing kids app reviews are asking for some sort of a fee for reviews. I would estimate 70% of the reviewers that got back to me were in this category.”
Mei of Mosster Toys validated a similar experience in her comment on part one, “After our experience with trying to get reviews on our kids apps, where we went down the route of paying and non-paying – I am starting to think that a high percentage of the reviews on these sites are paid…They didn’t have much time to look at apps (where) we didn’t pay for reviews”
Jenna Hoffstein of Little Worlds Interactive said, “…It’s not enough to just be good at making games. To give yourself a real chance of financial success you also need to invest significant amounts of time into business and PR/marketing. In a hit-driven business like games there’s never a silver bullet for success, and all you can do it give yourself the best chance you can. As it becomes easier to make games and most of the marketplaces like the App Store become more and more crowded, discoverability becomes a bigger and bigger concern. It’s entirely too easy for an app to be lost among the sea of hundreds of other apps released on the same day.”
Most sites put a caveat in place to pass their own self-ascribed standards to even be considered for a review, free or paid, yet there’s a notable lack of outreach for high quality apps who clearly make the grade, which smacks of skewed objectivity…
Reviewers should be upfront if a “fee for service” review is their “policy” and drop the illusion there’s any ‘unpaid’ option, as this was a MAJOR time sink for indie app developers wasting valuable hours awaiting word and following up while their app collected dust bunnies in the review queue with months and months of lag…IF a “we’re too busy” response was even given at all.
As one award-winning app creator summed:
“ALL of our coverage came from unpaid sites. Interestingly, NOT ONE of the websites that offered an expedited review option has reviewed our app. This would lead me to believe that either these websites are exclusively doing app reviews that have been paid for or they are so inundated with paid review requests that they are unable to get around to doing the free reviews in a reasonable amount of time. ”
Cash vs Cache: Costs, Credibility and Context
To give a general idea of how much review sites are charging, this 2013 AppyNation starter compilation offers an interesting thread of comments I found more helpful than their brief list itself, as it’s important to uncork a living lab (a ‘collaboratory’) to work together on the policies and practices of children’s media to raise the bar for all.
TeachersWithApps.com – $200 accelerated review; one week turn around for a homepage featured review with links to app/site/and promoted on TWA social media channels/over 100K followers, $55 sponsored post or press release w/media blast
Digital Storytime $150 Premium/full review, $100 Basic/brief review both within 10 business days
BalefireLabs.com $250 + $125 follow up of ‘pre-market and app testing’ and ‘fast tracked reviews’ of $25 per app review, plus the cost of your app (Update: note from founder in our comments section)
theimum.com $75 expedited review within 10-14 days, $100 sponsored post
iphonemom.com – $125 expedited review within one week
Fun2Tap.com (this site had pop-ups specifying ETAs for review time) $45 for expedited review within 3-10 working days, $25 for indie package 10-30 working days, not accepting free reviews because of volume
FunEducationalApps.com $90 fixed rate priority service/social media promotion within 15 days
bestappsforkids.com – $125 priority review service within 14days will not publish apps under 4 stars; free reviews 3-6 month wait
bestkidsapps.com (their site says they use ads vs paid reviews, but according to one app developer, they were told they “review very few submissions – to improve odds – For $200, we’ll describe your app, link to it on iTunes, and feature it in our email newsletter”)
Anyway, you get the gist. It REALLY adds up cost-wise for an app developer and becomes frustratingly vapid from a valuation standpoint for a parent or educator when sites have vastly different depth in ratings/analysis/graphics/seals and certifications for any form of ‘at a glance’ depiction of what’s worthy.
The App Reviewer Lens:
App reviewers have been very forthcoming (and sometimes unnervingly candid) in discussing just how much things are changing in the apps world, and how we’ve ended up with so many expedited reviews with sheer volume impacting quality of critical content.
They emphasize the imperative of using only sites with vetted, experienced reviewers and remind parents to read the bios/backgrounds, depth of critique and time-stamps of each review, since any one of those elements can skew the focus and validity. As a whole, many I spoke with seem to regard app reviews as a business more than a journalistic endeavor…voicing bias in today’s mainstream media editorial lens as well.
Jill Goodman, adjunct faculty at University of Central Florida Department of Education where her duties include serving on the dissertation committees of students doing their theses on apps in early childhood education, happens to write reviews for AppoLearning. She also has written over 1000 app reviews at other major sites like Smart Apps for Kids where she previously served as Editor.
Jill Goodman shared:
“Most sites with 10,000+ followers or 100,000 or less Alexa scores DO use expedited reviews.
During my tenure with one of these sites, I never saw instances of ratings inflation due to being an advertiser or “friend.” As editor I bumped down by 1/2 point far more scores than I bumped up. In some ways there’s a bigger conflict when an app is reviewed as a favor to a developer.
Paying for an expedited review makes it business and is about more than the review. It comes with substantial visibility and social media promotion.”
Jayne Clare of the Teachers with Apps site said:
“Reviewing any app is time consuming especially if you want to do field-testing with kids as TWA does. And field-testing is only part of the overall process, after gathering data a review must be written and then it is no small feat posting reviews into the website with images, links, configuring SEO and other tedious tasks… I don’t feel the necessity to state we were given a promo code for an app we review, many are free and the cost is usually minimal.
As for Best Practices, it is most important to be true to one’s standards, we do not review mediocre apps – even if they offer to pay. We only want to represent apps that we feel have educational merit of fine quality.
…TWA is a small operation, we would like to become a non-profit and have corporate sponsors so that we could promote the indie developers and help get some of the underdogs recognition without charging developers. Personally, I do not feel that sponsors should impose their own products on reviewers and reviewers should be strong enough to stand by what they believe is quality.”
Carisa Kluver of Digital Storytime said:
“I do think expedited reviews are now the norm for any decent sized site that has been around for more than a couple years. To get real recognition, an app either needs to be break-out exceptional (like the 1% or for apps the .01%) or it needs a real marketing budget that includes paying for ‘expedited’ reviews.
Many sites have found a fine-line to straddle that lets them continue to review with an unflinching eye, but others have succumbed to pressure to make paid reviews less negative or to gloss over or omit issues that consumers would otherwise want to know about. We get literally HUNDREDS of requests every month, beyond what anyone could review or even sample.”
I hope they tackle this topic with their academic research fellows, meanwhile, parents seeking more context on how to find good content might want to peruse two fabulous Facebook groups useful for anyone interested in the children’s media sphere, Dust or Magic is an open group discussing children’s apps and interactive media (shown here) and Children’s Media Professionals is a closed/professional group by invitation/referral with larger topicality.
Both are invaluable resources for opening up dialogue on important topics like this, though I’ve noticed in keeping with the cliques and tribes of industry dynamics there’s a propensity to ‘lurk’ instead of leap into the conversations, unless you’re ancillary and unaligned as I am. (I’ll post this to both groups and see if any sparks fly)
One app reviewer summed the general industry consensus,
“Anyone who reads more than five app reviews can easily recognize honest opinion from promotion. And anyway, after reading a review, users land on the App Store page where they will often find more feedback from users to help them make their mind.”
That said, I’m not personally seeing that level of media literacy discernment for those outside of the industry, especially among parents searching for apps.
Of the dozens of parents I interviewed most didn’t have ANY idea kids app reviews were often paid for, and since many parents also equate ‘education’ with institutional neutrality, many were surprised to hear paid reviews or “fee for service” arrangements are becoming a norm, especially sites involving teachers and research among educators.
Again, payment may be gaining ground and normative for the “ed tech” industry wide, but for many parents there are assumptions made when using the word “education”…Many have come to free associate the word with…well…being free, as in public education.
These days the private sector is becoming more enmeshed with student media in multiple ways, from data privacy and records beyond, so parents need to amp their media literacy, monitor and manage industry transparency (as well as their own bias) and reframe that conversation altogether with more accuracy and savvy.
So Who DOES Offer Unpaid Reviews?
It takes a lot of hustle to keep knocking on the door and filter out which sites offer unpaid reviews in a sea of expedited fees. Larger, mainstream press coverage often comes from staffed or syndicated sites like Common Sense Media (with its entire Graphite apps and ed tech review division) Tech with Kids, or subscription offerings like Children’s Technology Review.
If you follow the media literacy money trail you’ll also find the quality review sites without fee for service arrangements are simply monetized differently, giving indie developers much needed lift without perceived conflict of interest.
For example, Appysmarts uses their YouTube video channel clout and GeeksWithJuniors.com breaks even with affiliate marketing and sidebar ads (policy screenshot from GWJ below) KinderTown (now owned by Demme Learning) offers free reviews in appx 8 weeks, is age specific geared only from ages 3-8, reducing volume.
Globally, Pappasappar.se in Sweden is a heavily trafficked kids app review site using affiliate marketing commissions and Google AdSense (with a policy that developers may not advertise their apps) to keep their extensive collection of kids app reviews free, and clear of conflict of interest. And app developers have also said Turkish site OnlineAnne.com is a reliable resource without review fees. Point is, unpaid reviews ARE still out there, it just takes time to wade through the tsunami of so many review sites…
Ironically, just a few years ago, expedited reviews were called out as contaminated content with lengthy commentary…
Today, I find it’s harder to discover sites who do NOT charge outright with rate cards and cash for cache.
As for bias? We’ve already talked about ed tech tethers a bit with the Gates Foundation, and of course Edutopia is aligned with the George Lucas Educational Foundation, sister company to Lucas Education Research. Ed tech mergers and acquisitions are frankly a revolving door of business interests…
One seasoned reviewer spoke off the record in full candor,
“I don’t know of any review sites completely free of any ties to anybody. Maybe Kindertown but with their former director now at Apple even that could be bias. Educade is AT&T, AppoLearning is owned by a guy from Yahoo News. In the end I’m not sure it matters. There are few outliers. Most apps get consistent ratings across sites. Balefire might be the exception but I find some of those ratings ridiculous. Giving a Dr. Panda app an F is crazy. They are just silly harmless fun.”
Reviewing the Reviewers: Quality, Bias, Insights
Representatives from Common Sense Media, Teachers with Apps and the Children’s Technology Review convened at the Dust or Magic Children’s AppFest (the day before their AppCamp) an insightful albeit pricey “pay to play” endeavor in itself where startups, leaders and reviewers share best practices and pitch their goods for feedback and collaborative support from colleagues within the industry.
If you’re a fellow media infovore (intrigued but not involved with apps on a granular level) also check out the free offerings of EdCamp a free hub for educators, homeschoolers, reviewers and developers that’s been described as “loaded with armies of sponsors but helpful and free” with giveaways like classroom licenses for BrainPop and other fun finds.
Clearly invaluable conversations are happening here on ethics, privacy, design, best practices, what works and what doesn’t, and ways to be ‘discovered’ in the sea of selections. (Special thanks to industry good guy Scott Traylor of 360Kid recording these video clips to enlighten/inform and raise the bar in the kids media sphere; he’s woofed as a watchdog from both sides of the media fence and has provided very helpful video documentation for newbies like me)
Listen closely to the discussion around the 17:20-21:00 mark about the ethics behind expedited reviews and the ‘contaminants of objectivity’ cited by Dust or Magic/Children Technology Review’s Warren Buckleitner as they all discuss the app environments on iTunes, Google Play and review hubs…
It’s a very helpful 4-minute clip within the abbreviated video, worth your time to see the elephant in the living room very fast.
Expedited Reviews & Ethics: At appx 17:20-21:00 mark
I’ll leave it there for now, as the video speaks volumes.
Do Expedited Reviews Even Work?
Why Have Kids App Review Sites Changed So Much Recently?
How Can We Build a Better System with Fairness & Frankness?
What Tips Can Industry Pros Give Parents to Find Their Apps?
In Part 1 we focused on expedited review disclosures (as opposed to ‘in-app’ upsell wristslaps by ASA watchdogs this week) and in Part 2 we’ve heard pros/cons from some developers and reviewers directly…
As we segue to Part 3 we’ll be looking at solutions-based ideas, design thinking improvements, and talking tips for PARENTS from industry pros to help navigate the app quagmire. Sound off and we’ll keep compiling the commentary…
Do you feel ‘big media’ will be ‘owning’ the app space as an inevitable outcome? What will that mean for creativity at the onset? Will new models emerge? How can parents figure out who’s playing in who’s sandbox as they try to make sense of paid and unpaid recommendations?
Stay Tuned for more in Part 3
Visual Credits: Jeff Parker, cartoonist 2009 Florida Today/Cloak of Secrecy, screenshot of photo by Gabriella Demczuk for The New York Times, Money talks graphic ’08 anunciosgratis.cl; Journalist for sale via Tineye at horamadeira.blogs.sapo.pt, reused in this eye-popping PK blog post; Little Red Hen clip art-FramingTheDialogue.com.