A few years ago, cops raided the home of a teenager in Brooklyn and arrested him for posting a sequence of emoji on Facebook. Two officers on patrol had recently been killed in the borough, and tensions were running high. The offending post contained an emoji of a police officer followed by three emoji handguns, enough to procure an arrest warrant, apparently, on charges of making a “terroristic threat.” (The teen’s bail was set at $150,000, though a grand jury later declined to indict.) According to the criminal complaint, the images had caused police officers “to fear for their safety.” At the time, Facebook’s design for the handgun emoji resembled a fairly realistic revolver. In 2018, it was remade to look like a neon-green squirt gun. Zuckerberg waves his wand; a felony becomes a joke. Or does it?
In cases of alleged criminality, courts have always been tasked with sussing out intent. The evidence ranges from the verbal (did the suspect tell his friends he was going to rob the liquor store?) to the gestural (did he slide his fingers across his throat?) to the tonal (was he screaming a threat or bellowing song lyrics?). Emoji, with their ever-evolving connotations and dramatic variation in appearance, present a unique challenge. The question of how to interpret them is not just a matter of theoretical inquiry. Eric Goldman, a legal scholar at Santa Clara University and an expert in the burgeoning field of emoji law, has documented more than 150 cases in which emoji appear. Recently, he led an emoji crash course for a group of New York state trial judges. “A lot walked in thinking, ‘What do I need to know about emoji?’” Goldman told me. “And, at the end, they were flabbergasted.”
Facebook isn’t the only technology company that moved its pistol emoji to the toy aisle. Apple paved the way in August 2016, after a string of highly public shootings by police of black men—Philando Castile, Alton Sterling, Delrawn Small. The rest of the major technology companies followed suit. By late 2018, which saw massacres at Marjory Stoneman Douglas and Santa Fe high schools, the pistol emoji had, with a few exceptions, been almost completely recast. In keeping with the committed apoliticism we’ve come to expect from Silicon Valley, none of the companies cited gun violence or pressure from activists as a reason for the change. Even if they had, the notion that gun violence can be diminished by eliminating our ability to reference it visually feels tendentious at best. We may as well ask Apple to render the word “gun” with curlicues and flowers.
Emoji lawsuits don’t often involve “terroristic threats.” The most common category is sexual predation, but the cases run the gamut from child custody to workplace harassment to murder. By Goldman’s count, the most litigated emoji is the plain ol’ smiley, followed by the winky and sad faces. In a recent sex trafficking case, the court had to decide whether a crown emoji sent by the defendant conveyed an offer to become a pimp. In 2017, an Israeli couple were ordered to pay damages to a prospective landlord after they sent emoji of a champagne bottle, a chipmunk, and a dancing woman in the midst of an apartment negotiation, communications that led the landlord to conclude they wanted the place, and to take the listing off the market. Then the couple stopped responding. The court found them liable because the emoji signaled “great optimism” and suggested the defendants were acting in “bad faith.” One man’s chipmunk is another man’s security deposit.
For years, the standard view has been that emoji enrich comprehension by helping us to read between the lines. According to a recent study by Vyvyan Evans, a U.K.-based linguistics professor and author of The Emoji Code, a lively volume of emoji ephemera and theory, 72 percent of young adults find it easier to express emotion using emoji rather than words alone. “What we mean is often not what we say, in the sense of what our words literally convey,” Evans notes. Emoji stand in for knowing nods, ironic eye rolls, a tongue in cheek—those bits of paralinguistic information that are not easily conducted via text.
And yet, though meant to clarify, or at least to provide some sort of meta-comment on accompanying text, emoji remain elusive to define. Not for lack of trying: The web site Emojipedia, which is viewed 33 million times per month, catalogs and explicates emoji on a rolling basis. It also maintains an archive that tracks the visual evolution of each character. “We’re the Oxford English Dictionary of emoji,” John Kelly, Emojipedia’s senior lexicographer, told me recently. “It’s wonderfully ridiculous!” (Kelly engages in more traditional lexicographic pursuits as a research editor for Dictionary.com.)
The web site does its best to grapple with definitional ambiguities. For instance, those associated with woozy face, a newer emoji that depicts a “yellow face with a crumpled mouth and a cockeyed expression, as if tired and emotional from inebriation or smitten with love,” Kelly explained. “It’s popularly thought of as the drunk-face emoji, but, if you look at the data, you see people drop it at the end of a sentence like, ‘I just got back from California and I got my In-N-Out fix, woozy face,’ or, ‘Did you hear the latest thing Trump said, woozy face?’ There’s an endless amount of color and variation.” The greasy satisfaction of gorging on fries and a milkshake, the disgust aroused by a presidential tweet; woozy face contains multitudes. Pity the judge who must decide among them.
Making up a new word is easy. You just start saying it. A couple of composters at the weekend farmer’s market grasp for a term to describe their love of nearby foods, and inspiration strikes: locavore. It catches on, or it doesn’t, as selective forces do their work. Someone uses it on Instagram, or tells a friend from out of town; other social groups begin to weigh its merits. The local news clues in, a grocer slaps it on a billboard, and, suddenly, there it is, zipping around the culture, a new descriptive unit.
Emoji sink or swim on less democratic tides. They aren’t quite words, of course, though they’re certainly word-adjacent. (Three out of four Americans regularly deploy emoji in text messages, and at least six billion emoji are sent across the major social media platforms each day.) Oxford Dictionaries, for its part, seems to have embraced the slippage; in 2015, it named the face with tears of joy emoji “Word of the Year.” And yet, unlike neologisms—try-hard, chillax, bling, hangry, bougie, the list is long—emoji cannot spring forth from the collective mind. A new emoji isn’t something you invent. It’s something you apply for.
In Picture Character, a documentary that premiered earlier this year at the Tribeca Film Festival, directors Martha Shane and Ian Cheney explore the vicissitudes and absurdities of the emoji approval process. The film centers on three eager applicants seeking to expand the pictographic corpus. To achieve their ambitions, they must submit their pitches to the Unicode Consortium, which maintains global standards for digital text display, and approves and accepts all new emoji.
A Muslim teenager wants a hijab, a group of Argentines pine for a maté gourd, and an international nonprofit aims to destigmatize menstruation by adding an emoji that evokes a period. (After much internal debate, the group settles on a pair of blood-stained underwear.) The push for more equitable and inclusive representation goes beyond political correctness; two university researchers found that watching television decreased self-esteem in every demographic they surveyed except white boys—the only group who mostly saw themselves. “Five months ago, I created a group chat for my friends on WhatsApp,” says Rayouf Alhumedi, the hijabi teen. “Both my friends … had an emoji to represent them. But I didn’t have one.” The writer Rhianna Jones recently proposed an Afro emoji to reflect the “diversity of Black, Afro-Latinx and other Diasporic communities with kinkier, spherical, coily hair.” This year, the Unicode Consortium approved emoji of hearing aids, mechanical limbs, and gender-neutral couples.
Unicode publishes a guideline for submitting new emoji proposals. It is more than 6,000 words long. Among the 16 factors and subfactors considered: Does the candidate emoji have notable metaphorical references or symbolism? Is there a clearly recognizable image of a physical object that could serve as a paradigm? Curiously, near the top, following a red X, it scolds: Don’t justify the addition of emoji because they further a “cause,” no matter how worthwhile. On some level, such conservatism makes sense. Unicode can’t delete emoji—doing so would defeat the purpose of maintaining a universal standard—and a trigger-happy approvals process risks cluttering up the library with paeans to passing fads. For this reason, the consortium also prohibits submissions based on the likenesses of celebrities, brands, or logos. Nevertheless, some brands manage to work a shadow game. The pickup truck emoji—which made it past the “provisional” round and is currently on the cusp of Unicode immortality—was masterminded by Ford. The proposal form lists its author as Nathan Maggio, formerly creative director at the consulting firm Blue State Digital, which counts Ford as a client. (Maggio has since left the agency and now works for Elizabeth Warren.) In retrospect, a phrase from the proposal’s introductory text—“it is time for our global visual language to get a little bit tougher”—should have been a giveaway.
That these requests are adjudicated by an organization composed of ten technology companies—Adobe, Apple, Facebook, Google, Huawei, IBM, Microsoft, Netflix, Oracle, SAP—and, curiously, the government of Oman (where journalists and activists are routinely imprisoned for criticizing the state on social media 🤷🏻) should perhaps raise an eyebrow. Do we consider these companies responsible stewards of our digital conversations? The YouTube recommendation algorithm created a breeding ground for child pornography enthusiasts. Facebook allowed military personnel in Myanmar to use the site to mount a years-long disinformation campaign that culminated in the Rohingya genocide. This track record doesn’t quite evoke thoughtful upkeep. When the blood-stained underwear languishes in subcommittee—“I don’t know exactly what that means,” one forlorn activist admits—the group tries again with a sanitized version, perhaps more palatable to the powers that be: a simple drop of blood. A colleague chimes in, “It’s difficult to get change done when you’ve got the Unicode Consortium, this shady organization based in Silicon Valley, controlling everything.”
Most of our digital conversations are mediated and monitored by corporations and governments; this is hardly a revelation. But Apple doesn’t own the letter A. It does own smiley face with hearts and all the rest. We have, in a sense, begun to cede to corporations the building blocks of speech—and thought—itself. A friend of mine recently described a mildly horrifying feedback loop: “Now, when I experience a tender moment, instead of sending an emoji, I imagine myself as the crying cat.”
Lest we forget, emoji are not universal. The Unicode Consortium builds a common set of “code points”—like a skeleton—for each emoji, but the various companies that want to provide emoji keyboards to their users get to dress up the skeletons however they please. “In the past couple of years, many of the larger vendors have converged,” Kelly told me. “But there are still some discrepancies.” An ear of corn tilted left instead of right, a rowboat moving through choppy instead of placid waters, a chocolate chip cookie that transforms, en route from an iPhone to a Samsung device, from one of grandma’s specialties into what appears to be a Saltine cracker—perhaps these little unlikenesses won’t leave anyone scratching their head for too long.
Some incongruities are significant. For a couple of years, a squirt gun sent from that same iPhone became a revolver when viewed anywhere else. Goldman, the emoji law expert, insists that when emoji appear as evidence in court, they should always come in pairs, so the judge and the jury can see what the image looked like to the sender and to the recipient. “One of the platforms interprets astonished face with Xs for eyes,” he told me. “Someone might send astonished face, meaning to convey astonishment, which is reinterpreted by the recipient as a death threat.”
In The Emoji Code, Evans cites research conducted at the University of Minnesota that makes plain the divergences, however subtle. Participants were shown 22 emoji across different platforms and asked to rank how positive or negative they understood their meaning to be, on a scale of -5 to 5. Most depictions of grinning face with smiling eyes hovered around 4, near the most positive pole. Apple’s strangely pained-seeming version got a -1. In the presentation Goldman gives to judges, he refers to a survey of Twitter users in which one-fourth of respondents were unaware that emoji are rendered differently across platforms. Twenty percent said they would not have sent their tweets had they known.
Emoji first started showing up on cell phones and pagers in Japan. The first widely used library, designed by Shigetaka Kurita, was released in 1999, and contained 176 images, each composed of no more than 144 pixels. (Kurita developed some of the early prototypes by coloring in squares of twelve-by-twelve grids on sheets of graph paper.) In 2016, the Museum of Modern Art acquired Kurita’s set, touting it as the “original.”
But origin stories are often slippery; Jeremy Burge, the founder and editor-in-chief of Emojipedia, claims to have identified, along with other emoji experts, a smaller library, designed by SoftBank, which predates Kurita’s by two years. (Burge recently traveled to Japan to get some “hand time” with old devices that are hard to find online. “I just wandered around old phone stores, checking phone models to pin down the status of these discontinued emoji sets,” he said.) In The Story of Emoji, a Prestel art book packed with colorful illustrations, the writer Gavin Lucas points to the nascent technological conditions of mid-’90s Japan as the perfect emoji crucible. Widespread digital communication had begun to emerge—via new devices, like pagers, and technologies, like email—and old customs had a hard time keeping up. “In Japanese culture,” he writes, “personal letters are traditionally long and verbose, full of seasonal greetings and honorific expressions that convey the sender’s goodwill to the recipient. The shorter, more casual nature of email was at odds with this tradition.” Emoji did the work for all the words that got left out.
Presently, there exist 3,019 emoji, a 17-fold increase over Kurita’s set. In the film Picture Character, we see him stare lovingly at his designs, displayed on a digital board in the MoMA lobby. “To be honest, the current emoji have become so complicated, in some ways they’ve strayed from my original creation,” he says.
One senses a shift in this history from vague yet capacious ideographic concepts to literal, pictographic representation. The impulse toward comprehensive mapping recalls Borges’s famous short story, “On Exactitude in Science,” in which the Cartographers Guild of an unnamed nation endeavors to make ever more exhaustive maps of its territory, increasing in size and scope until eventually it produces a map “whose size was that of the Empire, and which coincided point for point with it.” But the utility of a map depends as much on what it omits as what it includes, and subsequent generations “saw that the vast map was Useless,” and let it decay into tatters.
The rudimentary appearance and limited purview of early emoji sets forced users to fill in the gaps. Today, many of the most popular emoji—upside down smiley, smiling face with horns, pile of poo, wide eyes—operate in similarly ambiguous territory. The requisite interpretative gloss, the need to uncover a precise meaning, becomes part of the appeal. “The problem is, at this point, something like 90 percent of emoji are literal, and don’t get used that often,” Kelly told me. “The other 10 percent that aren’t so literal, those are like the well-worn pair of jeans that get a ton of everyday use.” Eventually, that favorite pair of jeans gets buried under a mountain of other clothes. “If you have an emoji for everything, you kind of have an emoji for nothing. When we start having total, one-to-one representation, the value goes down, because the creativity goes down. We’re chipping away at the flexibility.”
On Change.org, the online petition platform, there are campaigns for beard emoji (“Beards are amazing and I would like to express my love for the bearded community”), worm emoji (“Worms deserve love and recognition!”), white wine emoji (“Time to take a stand!”), and an emoji of Lemmy Kilmister, the deceased lead singer of the band Motörhead, “the badest [sic] motherfucker the world has ever known.”
Picture Character contains a montage of similarly outlandish requests. One avid user angles for an anglerfish: “We still have only twelve ocean emojis, two of which are whales, most of which are shallow water animals. And anglerfish in general are iconic deep sea species.” Another desires vegetables: “I thought the set of emoji could be improved by adding members of the allium family. Garlic, onions, ramps, leeks, scallions, shallots.” Senator Angus King of Maine, gesticulating on a dock, shills for a local delicacy: “We sent a letter to the Unicode Consortium saying, you know, this is outrageous. We’ve got crab emoji, we’ve got all these other kinds of emoji. No lobster!” These pleas are less about representation than imposition: an emoji library that reflects what people like, rather than what they are like.
Even the Unicode folks seem to want a way out. “We only hope that there would be a better answer,” says Craig Cummings, vice chair of the Unicode Technical Committee, with the sporting yet exasperated expression of someone who agreed to cook for a party that has quadrupled in size. On its web site, buried at the bottom of a long technical document, Unicode identifies its vision for the future as one in which people can use and upload their own “arbitrary emoji symbols,” rather than relying on libraries produced by tech companies. The idea sounds simple, but such a shift would require “significant infrastructure changes.” Until then, we’ll have to play with toys we’re given.
And so we do. Hearts when we love someone, kissy faces when we miss them, a slice of pizza when we want a slice of pizza. Banality begins to ossify. The tight control exerted by the consortium has largely insulated emoji from the anarchic thrum of creativity that characterizes the trade of memes and GIFs, which flame in and out of being at dizzying speeds. Some inventive repurposing manages to take hold—an eggplant here, a peach there—but emoji are made not by us, but for us, and their usage tends to reflect these circumstances.
Aberrations, though few and far between, glimmer with possibility. This summer, two enormous, spray-painted emoji—one sticking out its tongue, the other’s mouth replaced with a zipper—appeared on the side of a house in Manhattan Beach, California. The owner, Kathryn Kidd, circumvented Unicode’s hostility to customization and slapped bushy eyelashes on each. According to Easy Reader News, a local beachfront paper, neighbors had reported the property for violating a city ordinance banning short-term rentals, like Airbnb, and Kidd was fined $4,000. One of those neighbors, Susan Wieland, had recently gotten eyelash extensions. “I feel like I’ve been directly attacked,” she told Easy Reader. Kidd denies targeting her, though the emoji stare directly into Wieland’s house, and the artist Kidd commissioned posted an image of his handiwork on Instagram with the hashtag “eyelashextensions.” Snitches get stitches, I suppose. Hope the cops don’t nab her for making a terroristic threat.