How AI Is Taking Over Our Gadgets
AI is moving from data centres to devices, impacting everything from phones to tractors.
5 min
If you think of AI as something futuristic and abstract, start thinking different.
We’re now witnessing a turning point for artificial intelligence, as more of it comes down from the clouds and into our smartphones and automobiles. While it’s fair to say that AI that lives on the “edge”—where you and I are—is still far less powerful than its datacentre-based counterpart, it’s potentially far more meaningful to our everyday lives.
One key example: This fall, Apple’s Siri assistant will start processing voice on iPhones. Right now, even your request to set a timer is sent as an audio recording to the cloud, where it is processed, triggering a response that’s sent back to the phone. By processing voice on the phone, says Apple, Siri will respond more quickly. This will only work on the iPhone XS and newer models, which have a compatible built-for-AI processor Apple calls a “neural engine.” People might also feel more secure knowing that their voice recordings aren’t being sent to unseen computers in faraway places.
Google actually led the way with on-phone processing: In 2019, it introduced a Pixel phone that could transcribe speech to text and perform other tasks without any connection to the cloud. One reason Google decided to build its own phones was that the company saw potential in creating custom hardware tailor-made to run AI, says Brian Rakowski, product manager of the Pixel group at Google.
These so-called edge devices can be pretty much anything with a microchip and some memory, but they tend to be the newest and most sophisticated of smartphones, automobiles, drones, home appliances, and industrial sensors and actuators. Edge AI has the potential to deliver on some of the long-delayed promises of AI, like more responsive smart assistants, better automotive safety systems, new kinds of robots, even autonomous military machines.
The challenges of making AI work at the edge—that is, making it reliable enough to do its job and then justifying the additional complexity and expense of putting it in our devices—are monumental. Existing AI can be inflexible, easily fooled, unreliable and biased. In the cloud, it can be trained on the fly to get better—think about how Alexa improves over time. When it’s in a device, it must come pre-trained, and be updated periodically. Yet the improvements in chip technology in recent years have made it possible for real breakthroughs in how we experience AI, and the commercial demand for this sort of functionality is high.
From swords to plowshares
Shield AI, a contractor for the Department of Defense, has put a great deal of AI into quadcopter-style drones which have already carried out—and continue to be used in—real-world combat missions. One mission is to help soldiers scan for enemy combatants in buildings that must be cleared. The DoD has been eager to use the company’s drones, says Shield AI’s co-founder, Brandon Tseng, because even if they fail, they can be used to reduce human casualties.
“In 2016 and early 2017, we had early prototypes with something like 75% reliability, something you would never take to market, and the DoD were saying, ‘We’ll take that overseas and use that in combat right now,’” Mr. Tseng says. When he protested that the system wasn’t ready, the response from within the military was that anything was better than soldiers going through a door and being shot.
In a combat zone, you can’t count on a fast, robust, wireless cloud connection, especially now that enemies often jam wireless communication and GPS signals. When on a mission, processing and image recognition must occur on the company’s drones themselves.
Shield AI uses a small, efficient computer made by Nvidia, designed for running AI on devices, to create a quadcopter drone no bigger than a typical camera-wielding consumer model. The Nova 2 can fly long enough to enter a building, and use AI to recognize and examine dozens of hallways, stairwells and rooms, cataloging objects and people it sees along its way.
Meanwhile, in the town of Salinas, Calif., birthplace of “Grapes of Wrath” author John Steinbeck and an agricultural center to this day, a robot the size of an SUV is spending this year’s growing season raking the earth with its 12 robotic arms. Made by FarmWise Labs Inc., the robot trundles along fields of celery as if it were any other tractor. Underneath its metal shroud, it uses computer vision and an edge AI system to decide, in less than a second, whether a plant is a food crop or a weed, and directs its plow-like claws to avoid or eradicate the plant accordingly.
FarmWise’s huge, diesel robo-weeder can generate its own electricity, enabling it to carry a veritable supercomputer’s worth of processing power—four GPUs and 16 CPUs which together draw 500 watts of electricity.
In our everyday lives, things like voice transcription that work whether or not we have a connection, or how good it is, could mean shifts in how we prefer to interact with our mobile devices. Getting always-available voice transcription to work on Google’s Pixel phone “required a lot of breakthroughs to run on the phone as well as it runs on a remote server,” says Mr. Rakowski.
Google has almost unlimited resources to experiment with AI in the cloud, but getting those same algorithms, for everything from voice transcription and power management to real-time translation and image processing, to work on phones required the introduction of custom microprocessors like the Pixel Neural Core, adds Mr. Rakowski.
Turning cats into pure math
What nearly all edge AI systems have in common is that, as pre-trained AI, they are only performing “inference,” says Dennis Laudick, vice president of marketing for AI and machine learning at Arm Holdings, which licenses chip designs and instructions to companies such as Apple, Samsung, Qualcomm, Nvidia and others.
Generally speaking, machine-learning AI consists of four phases:
- Data is captured or collected: Say, for example, in the form of millions of cat pictures.
- Humans label the data: Yes, these are cat photos.
- AI is trained with the labelled data: This process selects for models that identify cats.
- Then the resulting pile of code is turned into an algorithm and implemented in software: Here’s a camera app for cat lovers!
(Note: If this doesn’t exist yet, consider it your million-dollar idea of the day.)
The last bit of the process—something like that cat-identifying software—is the inference phase. The software on many smart surveillance cameras, for example, is performing inference, says Eric Goodness, a research vice president at technology-consulting firm Gartner. These systems can already identify how many patrons are in the restaurant, if any are engaging in undesirable behaviour, or if the fries have been in the fryer too long.
It’s all just mathematical functions, ones so complicated that it would take a monumental effort by humans to write them, but which machine-learning systems can create when trained on enough data.
Robot pratfalls
While all of this technology has enormous promise, making AI work on individual devices, whether or not they can connect to the cloud, comes with a daunting set of challenges, says Elisa Bertino, a professor of computer science at Purdue University.
Modern AI, which is primarily used to recognize patterns, can have difficulty coping with inputs outside of the data it was trained on. Operating in the real world only makes it tougher—just consider the classic example of a Tesla that brakes when it sees a stop sign on a billboard.
To make edge AI systems more competent, one edge device might gather some data but then pair with another, more powerful device, which can integrate data from a variety of sensors, says Dr. Bertino. If you’re wearing a smartwatch with a heart-rate monitor, you’re already witnessing this: The watch’s edge AI pre-processes the weak signal of your heart rate, then passes that data to your smartphone, which can further analyze that data—whether or not it’s connected to the internet.
The overwhelming majority of AI algorithms are still trained in the cloud. They can also be retrained using more or fresher data, which lets them continually improve. Down the road, says Mr. Goodness, edge AI systems will begin to learn on their own—that is, they’ll become powerful enough to move beyond inference and actually gather data and use it to train their own algorithms.
AI that can learn all by itself, without connection to a cloud superintelligence, might eventually raise legal and ethical challenges. How can a company certify an algorithm that’s been off evolving in the real world for years after its initial release, asks Dr. Bertino. And in future wars, who will be willing to let their robots decide when to pull the trigger? Whoever does might end up with an advantage—but also all the collateral damage that happens when, inevitably, AI makes mistakes.
Copyright 2020, Dow Jones & Company, Inc. All Rights Reserved Worldwide. LEARN MORE
This stylish family home combines a classic palette and finishes with a flexible floorplan
Just 55 minutes from Sydney, make this your creative getaway located in the majestic Hawkesbury region.
Playful 1950s style spotlights details like coloured cabinets, checkerboard and mosaic tile patterns, vintage lighting, and SMEG appliances
6 min
The 1950s spawned society’s view of kitchens as the heart of the home, a hub for gathering, cooking, eating and socializing. Thus, it makes perfect sense that the same decade could inspire today’s luxury kitchens.
“The deliberate playfulness and genius of the era’s designers have enabled the mid-century style to remain a classic design and one that still sparks joy,” said James Yarosh, an interior designer and gallerist in New Jersey.
That playful style spotlights details like coloured cabinets, checkerboard and mosaic tile patterns, vintage lighting, and SMEG appliances—all of which are a conspicuous rejection of the sterile, monochrome kitchens that have defined luxury home design for years. One of the hottest brands to incorporate into retro-style kitchens, SMEG is turning up more these days. But the question is: How do you infuse a colourful refrigerator and other elements from this nostalgic era without creating a kitschy room?
“The key to a modern, fresh look in your kitchen is to reference, not imitate, signature looks of the 1950s,” said New York-based designer Andrew Suvalsky, who often laces retro style throughout the rooms he designs. He said using the period as inspiration will steer you away from imagining a garish space.
“When it comes to incorporating that retro-esque look, it’s a fine dance between looking beautiful and looking kitschy,” added Lisa Gilmore, a designer in Tampa, Florida. Gilmore suggested balancing contemporary pieces with vintage touches. That balance forges a functional yet attractive design that’s easy to live with while evoking a homey atmosphere––and ultimately, a room everyone wants to be in.
Colour Reigns Supreme
Suvalsky said one way to avoid a kitschy appearance is to mingle woods and colours, such as lacquered base cabinets and walnut wall cabinets, as he did in his Montclair, New Jersey, kitchen.
“Mixing colours into your kitchen is most effective when it’s done by colour-blocking––using a single colour across large areas of a space––in this case, zones of cabinetry,” he explained. He tends to lean toward “Easter egg colours,” such as baby chick yellow and pale tangerine. These soft pastels can suggest a starting point for the design while lending that retro vibe. But other hues can spark a vintage feel as well.
Natalia Robert
“Shades of green and blue are a timeless base foundation that work for a 1950s vintage look,” said designer Jennifer Verruto of Blythe Interiors in San Diego. But wood isn’t off the table for her, either. “To embrace the character of a mid-century home, we like a Kodiak stain to enhance the gorgeous walnut grain,” she said. “This mid-tone wood is perfect for contrasting other lighter finishes in the kitchen for a Mid-Century Modern feel.”
Since colour is subjective, a kitchen lined with white cabinetry can assume a retro aesthetic through accoutrements and other materials, emanating that ’50s vibe.
“The fun of retro designs is that you can embrace colour and create something that feels individual to the house and its homeowner, reflecting their tastes and personality,” Yaosh said. He recommended wallpaper as an option to transform a kitchen but suggested marrying the pattern with the bones of the house. “Wallpaper can create a mid-century or retro look with colours and hand-blocked craftsmanship,” he said. “Mauny wallpapers at Zuber are a particular favourite of mine.”
Suvalsky suggested Scalamandre wallpapers, for their 1950s patterns, and grass cloth, a textile that was often used during that decade. He also likes House of Hackney, a brand that “does a great job reinventing vintage prints in luscious colours,” he noted. “Many of their colourways invert the typical relationship between light and dark, with botanical prints in dark jewel tones set over light, more playful colours.”
Materials Matter
Beyond wall covering, flooring, countertops and backsplashes can all contribute to the 1950s theme. Manufactured laminate countertops, specifically Formica, were all the rage during the decade. But today’s high-end kitchens call for more luxurious materials and finishes.
“That’s a situation where going the quartz route is appropriate,” Gilmore said. “There are quartzes that are a through-body colour and simple if someone is doing colorued cabinetry. A simplified white without veining will go a long way.” She also recommended Pompei quartz Sunny Pearl, which has a speckled appearance.
Patricia Burke
But for those who welcome vibrant colour schemes, countertops can make a bold statement in a vintage kitchen. Gilmore said solid surface materials from the era were often a colour, and quartz can replicate the look.
“Some brands have coloured quartz, like red,” she said. But keeping countertops neutral allows you to get creative with the backsplash. “I‘d pull in a terrazzo backsplash or a bold colour like a subway tile in a beautiful shade of green or blush,” Gilmore said. “Make the backsplash a piece of art.”
Suvalsky also leans toward bright and daring––such as checkerboards––for the backsplash. But depending on the kitchen’s design, he’ll go quieter with a double white herringbone [tile] pattern. “Either version works, but it must complement other choices, bold or simple, in the design,” he explained.
Native House Photography
Likewise, his flooring choice almost always draws attention. “My tendency is more toward very bold, such as a heavily veined marble or a pattern with highly contrasting tones,” he noted. Yarosh suggested slate and terrazzo as flooring, as these materials can make an excellent backdrop for layering.
Forge a Statement With Vintage Appliances
As consequential as a kitchen’s foundation is, so are the appliances and accoutrements. While stainless steel complements contemporary kitchens, homeowners can push the design envelope with companies like SMEG when making appliance selections for a retro-style kitchen. Although Suvalsky has yet to specify a SMEG fridge, he is looking forward to the project when he can.
“I think they work best when the selected colour is referenced in other parts of the kitchen, which helps to integrate these otherwise ‘look at me’ pieces into the broader design,” he noted. “They are like sculptures unto themselves.”
“For our mid-century-inspired projects, we’ve opted for Big Chill and the GE Cafe Series to bring a vintage look,” Verruto added. Similar to SMEG, Big Chill and GE offer a vintage vibe in a wide selection of colours and finishes, alongside 21st-century performance.
Can’t commit to a full-size appliance? Sometimes, a splash is enough. Gilmore tends to dust her retro kitchens with a coloured kettle or toaster since her clients are likelier to add a tinge with a countertop appliance or two. “Mint green accessories make it pop, and if in five years they are over it, it’s not a commitment,” she said. “It’s a great way to infuse fun and colour without taking a major risk.”
Deck out the Breakfast Nook
Kitchen dining areas present the opportunity to introduce retro lighting, furniture, and accessories to complete the look. Flea markets and antique markets are excellent places to hunt for accompaniments.
“Dome pendants and Sputnik chandeliers are iconic styles that will infuse vintage charm into your kitchen while also easily complementing a variety of other styles,” Verruto said.
DLux Editions
Suspend a vintage light fixture over the classic Saarinen table, and you can’t go wrong.
“Saarinen Tulip Tables are almost always guaranteed to deliver a home run in nearly any interior, especially a 1950s-themed kitchen,” Suvalsky said. “The simplicity of its form, especially in white, makes it nearly impossible to clash with.”
To really channel the vibe of this era, Verruto suggested local vintage stores and brands such as Drexel Heritage and Lexington. Dressing the windows counts, too. “Cafe curtains in a chintz pattern will make for a fabulous finishing touch,” she said.
Meanwhile, Yarosh delights in selecting tabletop items, including novelty stemware and other trappings ubiquitous in the 1950s. “Mid-century kitchens also need to have pedestal cake plates and maybe a cloche to keep a cake,” he mused. “I love the opportunity to curate these details down to the correct fork and serving pieces.”
Consumers are going to gravitate toward applications powered by the buzzy new technology, analyst Michael Wolf predicts
Just 55 minutes from Sydney, make this your creative getaway located in the majestic Hawkesbury region.
