How AI Is Taking Over Our Gadgets | Kanebridge News
Kanebridge News
Share Button

How AI Is Taking Over Our Gadgets

AI is moving from data centres to devices, impacting everything from phones to tractors.

By Christopher Mims
Wed, Jun 30, 2021 10:25amGrey Clock 5 min

If you think of AI as something futuristic and abstract, start thinking different.

We’re now witnessing a turning point for artificial intelligence, as more of it comes down from the clouds and into our smartphones and automobiles. While it’s fair to say that AI that lives on the “edge”—where you and I are—is still far less powerful than its datacentre-based counterpart, it’s potentially far more meaningful to our everyday lives.

One key example: This fall, Apple’s Siri assistant will start processing voice on iPhones. Right now, even your request to set a timer is sent as an audio recording to the cloud, where it is processed, triggering a response that’s sent back to the phone. By processing voice on the phone, says Apple, Siri will respond more quickly. This will only work on the iPhone XS and newer models, which have a compatible built-for-AI processor Apple calls a “neural engine.” People might also feel more secure knowing that their voice recordings aren’t being sent to unseen computers in faraway places.

Google actually led the way with on-phone processing: In 2019, it introduced a Pixel phone that could transcribe speech to text and perform other tasks without any connection to the cloud. One reason Google decided to build its own phones was that the company saw potential in creating custom hardware tailor-made to run AI, says Brian Rakowski, product manager of the Pixel group at Google.

These so-called edge devices can be pretty much anything with a microchip and some memory, but they tend to be the newest and most sophisticated of smartphones, automobiles, drones, home appliances, and industrial sensors and actuators. Edge AI has the potential to deliver on some of the long-delayed promises of AI, like more responsive smart assistants, better automotive safety systems, new kinds of robots, even autonomous military machines.

The challenges of making AI work at the edge—that is, making it reliable enough to do its job and then justifying the additional complexity and expense of putting it in our devices—are monumental. Existing AI can be inflexible, easily fooled, unreliable and biased. In the cloud, it can be trained on the fly to get better—think about how Alexa improves over time. When it’s in a device, it must come pre-trained, and be updated periodically. Yet the improvements in chip technology in recent years have made it possible for real breakthroughs in how we experience AI, and the commercial demand for this sort of functionality is high.

From swords to plowshares

Shield AI, a contractor for the Department of Defense, has put a great deal of AI into quadcopter-style drones which have already carried out—and continue to be used in—real-world combat missions. One mission is to help soldiers scan for enemy combatants in buildings that must be cleared. The DoD has been eager to use the company’s drones, says Shield AI’s co-founder, Brandon Tseng, because even if they fail, they can be used to reduce human casualties.

“In 2016 and early 2017, we had early prototypes with something like 75% reliability, something you would never take to market, and the DoD were saying, ‘We’ll take that overseas and use that in combat right now,’” Mr. Tseng says. When he protested that the system wasn’t ready, the response from within the military was that anything was better than soldiers going through a door and being shot.

In a combat zone, you can’t count on a fast, robust, wireless cloud connection, especially now that enemies often jam wireless communication and GPS signals. When on a mission, processing and image recognition must occur on the company’s drones themselves.

Shield AI uses a small, efficient computer made by Nvidia, designed for running AI on devices, to create a quadcopter drone no bigger than a typical camera-wielding consumer model. The Nova 2 can fly long enough to enter a building, and use AI to recognize and examine dozens of hallways, stairwells and rooms, cataloging objects and people it sees along its way.

Meanwhile, in the town of Salinas, Calif., birthplace of “Grapes of Wrath” author John Steinbeck and an agricultural center to this day, a robot the size of an SUV is spending this year’s growing season raking the earth with its 12 robotic arms. Made by FarmWise Labs Inc., the robot trundles along fields of celery as if it were any other tractor. Underneath its metal shroud, it uses computer vision and an edge AI system to decide, in less than a second, whether a plant is a food crop or a weed, and directs its plow-like claws to avoid or eradicate the plant accordingly.

FarmWise’s huge, diesel robo-weeder can generate its own electricity, enabling it to carry a veritable supercomputer’s worth of processing power—four GPUs and 16 CPUs which together draw 500 watts of electricity.

In our everyday lives, things like voice transcription that work whether or not we have a connection, or how good it is, could mean shifts in how we prefer to interact with our mobile devices. Getting always-available voice transcription to work on Google’s Pixel phone “required a lot of breakthroughs to run on the phone as well as it runs on a remote server,” says Mr. Rakowski.

Google has almost unlimited resources to experiment with AI in the cloud, but getting those same algorithms, for everything from voice transcription and power management to real-time translation and image processing, to work on phones required the introduction of custom microprocessors like the Pixel Neural Core, adds Mr. Rakowski.

Turning cats into pure math

What nearly all edge AI systems have in common is that, as pre-trained AI, they are only performing “inference,” says Dennis Laudick, vice president of marketing for AI and machine learning at Arm Holdings, which licenses chip designs and instructions to companies such as Apple, Samsung, Qualcomm, Nvidia and others.

Generally speaking, machine-learning AI consists of four phases:

  • Data is captured or collected: Say, for example, in the form of millions of cat pictures.
  • Humans label the data: Yes, these are cat photos.
  • AI is trained with the labelled data: This process selects for models that identify cats.
  • Then the resulting pile of code is turned into an algorithm and implemented in software: Here’s a camera app for cat lovers!

(Note: If this doesn’t exist yet, consider it your million-dollar idea of the day.)

The last bit of the process—something like that cat-identifying software—is the inference phase. The software on many smart surveillance cameras, for example, is performing inference, says Eric Goodness, a research vice president at technology-consulting firm Gartner. These systems can already identify how many patrons are in the restaurant, if any are engaging in undesirable behaviour, or if the fries have been in the fryer too long.

It’s all just mathematical functions, ones so complicated that it would take a monumental effort by humans to write them, but which machine-learning systems can create when trained on enough data.

Robot pratfalls

While all of this technology has enormous promise, making AI work on individual devices, whether or not they can connect to the cloud, comes with a daunting set of challenges, says Elisa Bertino, a professor of computer science at Purdue University.

Modern AI, which is primarily used to recognize patterns, can have difficulty coping with inputs outside of the data it was trained on. Operating in the real world only makes it tougher—just consider the classic example of a Tesla that brakes when it sees a stop sign on a billboard.

To make edge AI systems more competent, one edge device might gather some data but then pair with another, more powerful device, which can integrate data from a variety of sensors, says Dr. Bertino. If you’re wearing a smartwatch with a heart-rate monitor, you’re already witnessing this: The watch’s edge AI pre-processes the weak signal of your heart rate, then passes that data to your smartphone, which can further analyze that data—whether or not it’s connected to the internet.

The overwhelming majority of AI algorithms are still trained in the cloud. They can also be retrained using more or fresher data, which lets them continually improve. Down the road, says Mr. Goodness, edge AI systems will begin to learn on their own—that is, they’ll become powerful enough to move beyond inference and actually gather data and use it to train their own algorithms.

AI that can learn all by itself, without connection to a cloud superintelligence, might eventually raise legal and ethical challenges. How can a company certify an algorithm that’s been off evolving in the real world for years after its initial release, asks Dr. Bertino. And in future wars, who will be willing to let their robots decide when to pull the trigger? Whoever does might end up with an advantage—but also all the collateral damage that happens when, inevitably, AI makes mistakes.

MOST POPULAR

Chris Dixon, a partner who led the charge, says he has a ‘very long-term horizon’

Americans now think they need at least $1.25 million for retirement, a 20% increase from a year ago, according to a survey by Northwestern Mutual

Related Stories
Lifestyle
For the Best Interior Design Finds, Take a Guided Shopping Tour to Paris, Istanbul and More
By ANTONIA VAN DER MEER 06/02/2023
Lifestyle
The Seawater Cure: How the French Slim Down
By ALEXANDER LOBRANO 03/02/2023
Lifestyle
‘De-Influencers’ Want You to Think Twice Before Buying That Mascara
By SARA ASHLEY O'BRIEN 03/02/2023
For the Best Interior Design Finds, Take a Guided Shopping Tour to Paris, Istanbul and More

Passionate about both decor and travel? Design industry pros are leading global tours to share their secret shopping sources—and help you score one-of-a-kind pieces.

By ANTONIA VAN DER MEER
Mon, Feb 6, 2023 6 min

WHEN MELANIE BURNS of Oklahoma City first entered the Grand Bazaar in Istanbul, she was stunned by its sheer size and the pathways winding through its tented structures like a tangle of yarn. Though well-traveled and an old hand at hunting one-of-a-kind objets, she’d never experienced such an onslaught of potential riches. “The bazaar is intimidating,” she said, “the size of about five football fields.”

She had expert allies, however: Clare Louise Frost and Elizabeth Hewitt of Tamam, a lifestyle brand and Manhattan store specialising in Turkish antiques and their own collections. The duo led Ms. Burns to a shop layered deep behind other shops. “It was no more than about 14 feet square, and stacked high with the most beautiful hand-woven vintage tapestries I’ve ever seen,” Ms. Burns recalled. “I would never have tackled the place without these women. They are walking encyclopedias, they speak the language and when you shop with them, you don’t overpay.”

Ms. Frost, who calls the bazaar “her second home,” lived in Istanbul for nine years, and her business partners, Ms. Hewitt and Hüseyin Kaplan, still live there. Together they host trips to Turkey, capped at 14 participants, all eager to buy décor to take back home. Overseas shopping sprees like this are an increasingly popular new category of travel. Interior-design pros immerse travellers in a country’s culture and guide them to fabulous finds, whether an ornate vintage camel bag from Turkey or a contemporary French sculpture.

Indagare, a travel company in Manhattan, is seeing a growing market for overseas shopping trips. The 30 Insider Journey trips it ran in 2022, including seven design-centred jaunts, drew 540 travellers, twice as many as in 2019. Sicily, Japan and Mallorca are locales Indagare is eyeing for future design trips. Penta, a magazine that, like The Wall Street Journal, is published by Dow Jones & Co., has a partnership with Indagare to organise trips.

“Covid taught us we need to go when we have the opportunity,” said Grant K. Gibson, a San Francisco interior designer who himself has led eight trips to India and two to Morocco and is adding excursions to Egypt, Mexico and Turkey.

Trips are as cultural as they are commercial. Before Mr. Gibson’s group of 10 globetrotters start looking for linens or bargaining for bowls, they tour Jaipur by electric rickshaw and visit a textile museum. “I want them to understand the history and know where design ideas come from,” he said. Cynthia Smith, a biotech exec from San Francisco who traveled with Mr. Gibson to Morocco, came home with pottery in a vibrant green glaze unique to Tamegroute, a village that edges the Sahara. “Everyone asks me about the vase, and I have a story to tell about Tamegroute pottery,” she said. “It gives character to my house.”

The packages don’t come cheap—from around $4,000 to $18,000 (not including flights) depending on location and length—but offer you insider access. Designer Chloe Mackintosh of Boxwood Avenue Interiors in Reno, Nev., is leading her first trip this year to parts of Italy and France she knows well. One focus will be the weekend antique markets in L’isle-sur-la-Sorgue, in southeast France, but she’ll also introduce guests to local artisans, including a fifth-generation ceramist. Her group will take a pottery-making class to understand the process behind the product.

Known as “the huntress” because of her many years buying and selling vintage furniture, Ariene C. Bethea says people began asking her to lead a trip so they could hunt alongside her. The owner of Dressing Rooms Interiors, a shop and design studio in Charlotte, N.C., teamed with TrovaTrip to create a journey to the Paris flea markets this May. With Ms. Bethea’s input, the Portland, Ore., group-travel managers lined up accommodations, vendors, translators and tickets to museums. “I plan to help my guests shop, give them ideas and help them learn to tell stories in a space,” said Ms. Bethea, known for her playful use of colours, bold patterns and culturally inspired designs.

Lodging on these guided forays offers design cred, too. Ms. Mackintosh has reserved an entire 16-room château in the French countryside for just 12 people. Tamam’s Istanbul guests stay in a marble-floored hotel that was a late 19th-century Ottoman bank—with a vault that doubles as a wine cellar—and for excursions to Cappadocia, a region in central Turkey, they bed down in a traditional cavelike home carved out of soft rock.

On a trip to the South of France with Los Angeles-based designer Kathryn M. Ireland, visitors stay in Ms. Ireland’s farmhouse near Toulouse. Her trademark fabrics and colourful Bohemian and English-country style are on display in every bedroom lamp shade and living room chair. “Guests shop my house, and then I point them in the right direction to buy similar things,” she said. Ms. Ireland has been leading groups (a maximum of 10 people) for over a decade, taking them to neighbours’ villas, antique markets and out-of-the-way bakeries and bee yards.

Abby Landers first visited Ms. Ireland’s home as a high-school senior, traveling with her mother. Now five years out of college and living in Boston, she recently returned. “Kathryn embraced us, and she has been a mentor for me ever since.” Inspired by that first trip, Ms. Landers earned a master’s degree in interior architecture, and her current boss is someone she met on that trip. “You’re there for a week, and it’s a whirlwind of meeting artists and artisans, all friends of Kathryn’s.”

Kirstan Barnett, a tech investor from Palm Beach Gardens, Fla., traveled to Tangier with Melissa Biggs Bradley, founder of Indagare. Ms. Barnett was particularly moved by dinner at the 300-year-old, whitewashed, riad-style residence of Jamie Creel and Marco Scarani, two of the many designers she met at private events. The home was so richly layered and eclectic, she said, it inspired her to approach her own décor more bravely and reject the notion that a room must adhere to one style.

Some pros who organise such tours offer itinerary planning to folks who don’t want to travel with strangers. Mr. Gibson recently created a program for a group of four going to Jaipur. Though he won’t be joining them, he’s chosen the lodging and booked the restaurants and the experiences.

Travelers laser-focused on in-the-know shopping minus the touring can hire Chicago-based Skin Interior Design in cities such as London, Paris and Milan. The company arranges excursions so clients are shown exactly what they want—whether French midcentury chairs or Venetian-glass chandeliers. “We have an education in art history and antiques, and we help find pieces that keep value,” said Lauren Lozano Ziol, one of the founders. A recent two-day antique-furniture and art expedition in London cost $10,000.

How to get all the booty home? Mr. Gibson advises guests to travel with at least one empty suitcase. Bulky items can be packed and airfreighted home using DHL or FedEx. (Most carriers will pick up at the hotel.) Some vendors ship direct to the States from their stores at reasonable rates. For those who travel with Tamam to Turkey, easy shipping—including having your purchases collected from the vendors—is one of the perks. Ms. Burns, who bought ceramics, four suzani bedspreads and six rugs, said Tamam handled shipping for about $400. “Some of my things arrived before I even got home,” she said.

International Harvest / Souvenirs that guests collected on their design-focused journeys abroad
DESIGN JAUNTS ON THE HORIZON

Five 2023 trips abroad devised and helmed by interiors experts imparting their insider info

Ready to shop your way around the world? Here are just some of the available packages that focus on home design. Prices are per person and generally include accommodations, meals and beverages, guided touring, activities and local transportation.

Flea Market Foraging | May 4-10, 2023

The owner of Dressing Rooms Interiors, a vintage-home-furnishings boutique and design studio in Charlotte, N.C., Ariene C. Bethea takes travellers shopping the Paris vintage markets and art galleries and on visits to lesser-known museums such as the Museum Nationale Gustave Moreau. Also on the agenda: a foray to Versailles and its gardens, a tour of Montmartre street art and a tasting at the Museum of Wine. From $3,649, Trips.TrovaTrip.com

Ciao, Italia | May 15-19, 2023 (wait list only)

Chloe Mackintosh, owner of Boxwood Avenue Interiors, a Reno, Nev., studio and shop, leads a 4-night trip in Florence, Italy. Travelers stay at the five-star Il Salviatino, a restored 15th-century villa that mixes Renaissance and contemporary décor. Along with shopping excursions, antiquing and a workshop at a local artisan’s studio, the trip includes wine tasting and cooking lessons. Florence, from $5,500, Learn.BoxwoodAvenue.com

Turkey Club | May 17-26, 2023

Designer Clare Louise Frost, Tulu Textiles owner Elizabeth Hewitt and carpet dealer Hüseyin Kaplan teamed up to create Tamam, located in Manhattan and Istanbul and specialising in antique and vintage Turkish textiles, rugs and ceramics. Travelers tour Istanbul, Konya and Cappadocia, shopping the Grand Bazaar and the Spice Bazaar and visiting textiles and antique dealers. Plus: a hot-air-balloon ride and cooking class. Tamam in Turkey, from $3,600, Shop-Tamam.com

English Town and Country | June 11-17, 2023

In London, South African interior designer Serena Crawford guides travellers through Kensington Palace’s Sunken Garden (Diana’s favourite) as well as shops such as heritage brand Fortnum & Mason. In the university town of Oxford, architectural highlights range from medieval to modern, and in the bucolic Cotswolds, guests visit private homes and gardens of renowned interior designers. London & the Cotswolds with Serena Crawford, from $15,350, Indagare.com

Joie de Vivre in France | Sept. 9-16, 2023

Los Angeles-based designer Kathryn M. Ireland takes you on private museum tours, flea market hunts and a trend-spotting tour of design fair Maison et Objet in Paris (ticket not included), followed by leisurely days in the French countryside at her farmhouse outside Toulouse. Paris & La Castellane, from $7,900, Paris hotel not included, KathrynIreland.com

India, Indeed | Dec. 11-18, 2023

San Francisco interior designer Grant K. Gibson shares his passion for India with a guided tour of Jaipur and Taj Mahal. Participants stay in a guesthouse once part of a maharajah’s gardens; enjoy traditional Indian feasts; learn the history of block printing; rendezvous with rescue elephants; and conquer the chaotic bazaar, comprising flower and spice markets and rug and textiles vendors. Travel with Grant from $9,500, GrantKGibson.com

The Wall Street Journal is not compensated by retailers listed in its articles as outlets for products. Listed retailers frequently are not the sole retail outlets.

MOST POPULAR
Pamela Anderson House

Inspired by some of California’s best known Modernist architecture.

FIVE PERTH PROPERTIES UNDER $750K

What a quarter-million dollars gets you in the western capital.

0
    Your Cart
    Your cart is emptyReturn to Shop