Kids have been augmenting their realities since the beginning of time. A flowing beach towel or tablecloth make for an excellent superhero cape, and any sport-loving child knows the words “He shoots, he scores!”—even if he or she never scores a goal.
Fast-forward to 2016, and a whole new kind of augmentation has taken the kids space by storm. While a whole host of interesting virtual reality experiences offer us a completely immersed world that replaces all daylight or ambient sound with a 360-degree view, the key distinguishing feature of augmented reality is that a created world is superimposed on the real one.
Personalized soccer commentary and superhero tablecloths may be a challenge for the brightest of smartphones to provide, but a typical mobile device can offer a live-camera view that can superimpose images, animations and text. This technology was not developed for AR mobile juggernaut Pokémon GO, but it may be the clearest and most recent example.
My team at WGBH has been mixing human-visible and camera-rendered worlds for about a decade. Before smartphones, we harnessed laptop webcams and desktop computers to let users to jump in order to make Curious George do the same on-screen, to tell stories to make animations activate for Martha Speaks, and to dance alongside Arthur’s friend Prunella. But our first foray into what we now label as AR was with the Lunch Rush mobile game in 2011, and it remains our most popular app to date.
With funding from the US Department of Education, through a Ready to Learn grant to PBS and CPB, we developed the AR app that allows first and second graders to help Ruff Ruffman fill sushi lunch orders by solving challenges created from national math standards. Players can print markers with custom codes and place them around their real world—whether it’s the living room, school library or backyard—and each code represents a number that could solve each challenge. Magical 3D sushi would appear on-screen as the kids worked their way through a busy lunchtime of orders.
The fun is there, and so too the math, but given the limitations of the technology back then, the act of printing markers can be seen as a hassle.
More recently, we have switched this kind of camera-view work away from markers, allowing kids to create mythical creatures to populate their backyards. They’ve taken over two million photos with our Plum’s Creaturizer app (pictured), which lets them create in-app animals to suit their varying environments. These help them hunt, nest, drink and protect their young.
As with Lunch Rush, we’re combining STEM learning goals (in this case an environmental science curriculum) with the fun of looking through the camera at the real world. In the Creaturizer app, kids see their backyards augmented both with digital images and their own imagination as they undertake a fairly serious exploration of how animals have evolved and adapted to their habitats.
Our current AR work in development for launch in 2017, again with the support of Ready to Learn and the popular Ruff character, takes advantage of tech improvements that have been made since 2011. These days, we can harness facial and color recognition to create an in-app view of the world in which real textures and materials provide a costume for Ruff. The app also places masks digitally over kids’ faces in real-time.
We know from user testing that there are challenges in AR. For example, kids can stop noticing the real world if the balance is skewed too heavily towards activity on the screen. And game playing can be harder still if the player has a tablet, since those can be difficult to hold up as a camera for more than a few moments at a time.
But we continue to see in AR opportunities at the intersection of learning and engagement. AR offers more chances for kids to undertake social and learning-based activities with their friends and classmates using devices that many of them already own.
Bill Shribman is an ex-pat Brit, Emmy-winning senior executive producer, and director of digital partnerships at WGBH in Boston. He is a producer of apps, websites and digital products for many signature TV brands. Visit wgbhkids.com for more.