High-frequency flickering patterns from LED lighting can ruin photos or videos taken by smartphone-grade cameras in a room
Rules that prohibit photos or videos can prove almost impossible to enforce when nearly everyone carries a smartphone. But a new indoor privacy system has shown how the power of smart LED lighting could prevent people from taking illegal videos of a live Broadway show or surreptitiously snapping photos during a secretive trade show presentation.
The prototype privacy measure uses a smart LED to give off a high-frequency flickering pattern that interferes with the camera sensors on mobile devices such as smartphones. Such flickering creates a vertical striped pattern effect in the photo or video frames taken by digital smartphone cameras without interfering with human eyesight or harming human eyes. It’s a first step toward enabling a cloak of privacy to be thrown over live events or exhibits, even in the presence of dozens of smartphones.
“Previously, when you are in locations like gyms or exam rooms or trade shows, in those locations the cameras are forbidden but there was no way to enforce the compliance,” says Xinyu Zhang, assistant professor in electrical and computer engineering at the University of California San Diego. “Our main achievement with LiShield is automating digital privacy.”
The “LiShield” privacy system specifically targets consumer digital cameras such as smartphone-grade cameras by exploiting their limitations in capturing images. Zhang and his colleagues presented their research on the first day of the ACM Mobicom 2017 conference held from 16-20 October at the Snowbird Ski Resort near Salt Lake City.
Digital cameras use a rolling shutter sampling approach to capture images by exposing each row of pixels in sequence. The on-off flickering patterns of the LiShield system ensure that some rows of pixels are overexposed and appear bright while other rows are underexposed and appear dark. That creates the striped patterning in the captured images or video frames to ruin the overall photo or video.
Such a privacy system is imperceptible to human eyes because it uses high-frequency flickering patterns beyond the limits of human eye sensitivity at around 80 Hertz. And while human eyes have the capacity to perceive very different levels of brightness across a dynamic range within the same scene, cameras tend to suffer from the overexposure or underexposure problems when exposed to lighting with vastly different intensities within the same scene. This is what makes the LiShield’s flickering patterns so effective.
LiShield’s capability to corrupt digital camera images and videos without interfering with human eyesight is an advantage over some past attempts to enable similar privacy protection. “Strong infrared light can saturate the camera and block it from capturing images, but that infrared light would burn human eye retinas,” says Shilin Zhu, a Ph.D. student in computer science at the University of California San Diego and co-primary author on the paper. “We have to use the visible light spectrum for safety reasons.”
LiShield can also allow specific “authorized users” to use their smartphones or digital cameras. For capturing video, authorized camera must use secure side channels to communicate its exposure time setting and synchronize its clock with the smart LED’s clock. That allows the smart LED to send recoverable waveform information about the flickering pattern in use during the authorized camera’s operation. For capturing still images, the smart LED synchronizes with the authorized camera to help reconstruct a still image taken from several complementary frames in a very short video.
The current LiShield prototype relies upon custom-built smart LED hardware and software algorithms used to control both the defensive flickering patterns and synchronization with an authorized camera. Such a system could easily be integrated into existing commercial smart LED lights already on the market, according to the researchers.
This system still only works within indoor areas that can be illuminated by smart LED lighting, and only under certain situations. For example, LiShield cannot disrupt the photos or videos taken by professional global-shutter cameras that capture the entire scene all at once. Its flickering pattern defense will also fail when the smart LED lighting is overwhelmed by the brightness of stronger ambient light such as sunlight.
Still, LiShield has one more trick to play if strong ambient light threatens to weaken the protection of its flickering pattern defense. Its patterns can still form the equivalent of faint, low-contrast stripes that act as nearly invisible barcodes within smartphone camera images or videos. Such barcodes could then be automatically detected by the online servers of social media networks such as Facebook or Snapchat, alerting the social media servers to block such images or videos taken without permission.
To make LiShield truly practical, the researchers need to go beyond their initial test phase, which involved a single smart LED and a fairly small indoor space. Eventually, the researchers hope to find commercial partners who might be willing to team up for more ambitious real-world testing during live events.
“If you want to protect a large area like a theater, you need to install multiple smart LEDs,” says Chi Zhang, a Ph.D. student in electrical and computer engineering at the University of Wisconsin-Madison and the second co-primary author on the paper. “Synchronization between multiple LEDs in this entire area is a problem to solve in the future.”
The team also wants to figure out ways to enable multiple authorized camera or smartphone users to take photos or videos during LiShield’s operation—a challenge given how the current system can only synchronize between a single digital camera and the smart LED. One possibility is that LiShield could quickly hop between authorized users and enable them to take photos or videos in a round-robin scheduling fashion.
There is also a possibility of taking LiShield’s privacy protection even further sometime in the future. Some potential military customers asked if the researchers could develop a privacy system that fully protects a target area by blacking out the video or images taken at the scene.
“That’s a kind of demanding application,” Xinyu Zhang says. “We still want the system to be as simple as replacing a light bulb, but we probably need to design smart algorithms like specialized waveforms that can realize that full protection.”
On May 18th, 2012, attorneys for Oracle and Google were battling over nine lines of code in a hearing before Judge William H. Alsup of the northern district of California. The first jury trial in Oracle v. Google, the fight over whether Google had hijacked code from Oracle for its Android system, was wrapping up.
The argument centered on a function called rangeCheck. Of all the lines of code that Oracle had tested — 15 million in total — these were the only ones that were “literally” copied. Every keystroke, a perfect duplicate. It was in Oracle’s interest to play up the significance of rangeCheck as much as possible, and David Boies, Oracle’s lawyer, began to argue that Google had copied rangeCheck so that it could take Android to market more quickly. Judge Alsup was not buying it.
“I couldn't have told you the first thing about Java before this trial,” said the judge. “But, I have done and still do a lot of programming myself in other languages. I have written blocks of code like rangeCheck a hundred times or more. I could do it. You could do it. It is so simple.”
It was an offhand comment that would snowball out of control, much to Alsup’s chagrin. It was first repeated among lawyers and legal wonks, then by tech publications. With every repetition, Alsup’s skill grew, until eventually he became “the judge who learned Java” — Alsup the programmer, the black-robed nerd hero, the 10x judge, the “master of the court and of Java.”
Judge Alsup would like everyone to know that he doesn’t know Java.
Not very well, anyway. He can, however, definitely code. He’s been coding in BASIC for decades, actually, writing programs for the fun of it: a program to play Bridge, written as a gift for his wife; an automatic solution for the board game Mastermind, which he is immensely fond of; and most ambitiously, a sprawling multifunctional program with a graphical interface that helps him with yet another of his many hobbies, ham radio.
His interests have served him well on the judicial bench, informing his outlook on the multibillion-dollar intellectual property cases that come to him. The fortunes of tech companies can rise or fall depending on his rulings. Oracle v. Google has wide repercussions for big companies and smaller developers alike, to say nothing of the $9 billion at stake. The yet-to-be-totaled billions Alphabet is seeking from Uber in the ongoing Waymo v. Uber suit could make or break Uber as a player in the nascent self-driving car market.
By sheer coincidence, these major cases have wound up in the docket of maybe the one judge in America capable of understanding their technical details: a judge who can code. Alsup’s long-cherished hobby illuminated issues at the very heart of Oracle v. Google, and his off-hours tinkering with photography, lenses, and the science of light will inform him in Waymo v. Uber, a case involving LIDAR, a laser-based technology for self-driving car navigation.
The tech industry has long despaired of the law’s inability to comprehend it, making much of the legal system’s struggle to keep up with the rapid pace of progress. The belief that the law will never “catch up” to technology is borne in part of tech exceptionalism, a libertarian elitism that derides any kind of legal or regulatory impediment as Luddism. But it’s also fueled by genuine frustration with the state of law. The patent office is perceived to be rubber-stamping obvious technologies. Supreme Court justices appear befuddled by the basic process of coding. And attorneys stack juries with non-technical jurors who return massive verdicts for patents on online shopping carts.
In this landscape, Alsup is an outlier — a mystifying exception to the accepted wisdom that the law cannot make sense of the fast-changing tech industry. Alsup’s secret is simple: he’s a lifelong geek.
Alsup is notorious among San Francisco’s attorneys for the early hours he keeps (and forces upon the lawyers who appear before him). At 9AM, most of the judges’ chambers at the federal courthouse were still dark, and doors were closed. But when I got to Alsup’s chambers, the door was flung open and the bustle inside suggested that everyone had already been at work for hours.
A white-haired man with rectangular wire-frame glasses and a soft Southern accent, Alsup is of normal stature, but his imposing presence leaves you with the impression that he towers over you.
Alsup’s chambers have many of the classic aesthetics of the legal profession: the shelves upon shelves of leather-bound books, the stained wood paneling, a small black-and-white portrait of Abraham Lincoln hanging over an intimidatingly large desk. Off to the side sits a sofa strewn with dog toys. Alsup’s Jack Russell Terrier, who he often brings to work, was not at the office that day.
The judge sat me down on the sofa and walked me through his programs on a 2011 court-issued Dell laptop. He couldn’t run the same programs on his desktop computer, he said with some irritation, so the Dell was here to stay. “It’s the last one that will support QuickBASIC, which is kind of a shame, because it’s the only language I really know.”
The judge is not a hardware fanatic. He uses computers and whatever smartphone the court has provided him with. He has a court-issued iPhone, but if the Northern District of California issued him an Android, he’d use an Android, he said.
I asked him if I could put his code on GitHub, and he asked me what GitHub was. In lieu of that, he handed me printouts of his computer programs, three stacks of paper that had been neatly stapled at the corners. The one on the top, he apologized, had several dependencies that he hadn’t had the time to print out. Long before he became the judge presiding over Silicon Valley, Alsup had been a hobbyist operating in isolation; he’s a geek, but he’s a geek from another era.
Alsup was born in 1945 in Jackson, Mississippi, the son of two rural Texans — a nurse and a civil engineer who got his start under Roosevelt’s Works Progress Administration. Even as a boy, Alsup displayed the freewheeling curiosity and technical bent that would characterize his later life. He spent much of his childhood alongside Hubert Feild, now a professor at Auburn University. Friends since the age of six, the two built forts together, buried time capsules, launched lighter-than-air balloons made of laundry bags, shot flaming matches from clothespins (an Alsup invention), and had “dirt clod wars,” which, Feild said, were “not recommended.”
“Bill was an extremely bright kid,” Feild said. Alsup taught himself piano, but, dissatisfied with the sound, modified the instrument by pressing thumb tacks into the hammers that strike the piano wire. His hack made the piano sound like it came from a long-lost cowboy saloon. “I still have recordings of Bill playing classic songs (such as those by Ray Charles) on his ‘new and improved’ version of his piano,” Feild said.
But if there was one gadget that caught the boys’ imaginations, it was ham radios. The two spent hours listening to shortwave broadcasts, combing frequencies on a Zenith Transoceanic radio with a seven-foot telescoping antenna. They tuned into faraway stations like Radio Moscow, Radio Quito, and Radio Havana, but most of their time was spent listening to the amateur radio operators — the so-called “hams.”
“As we listened to the conversations from hams in various states in the US as well as foreign countries, it was as if a ‘new world’ had been discovered beyond the confines of Mississippi,” Feild said. He would listen as Alsup, who became a licensed ham operator while still in high school, carried on conversations from his bedroom with far-off interlocutors, sometimes in Morse Code.
“Bill had, and continues to have, a very strong influence on my life,” Feild said. “For the last 20 years or so, Bill and I talk via ham radio each Saturday morning.”
After high school, Alsup began as an engineering major at Mississippi State, intending to be a civil engineer like his father. But this was the ‘60s, and the civil rights movement was in full swing: the March on Washington happened during his freshman year in college; the Civil Rights Act of 1964 was passed in his sophomore year. As Alsup took an interest in broader legal issues, something clicked. “I wanted to be another Atticus Finch,” Alsup said to me. After college he attended Harvard Law School, and went on to clerk for Supreme Court Justice William O. Douglas.
He set up shop in Mississippi doing civil rights work, but found it financially unsustainable. Eventually he and his wife relocated to San Francisco in 1973, and over the years he worked both in private practice and at the Department of Justice. In 1999, he was commissioned as a federal judge by President Bill Clinton.
In the Northern District of California, Alsup has a fearsome reputation among lawyers. His early hours are the bane of attorneys, who are forced to argue motions as early as seven, sometimes even earlier if the judge sees that everyone is present. Litigators are timed down to the minute, and juries are let out at the exact time he specifies.
In the second Oracle v. Google trial, he refused to allow lawyers to continue to question Eric Schmidt past 1PM, even though this meant that the president of Alphabet would have to return to court the next day. The business of a $570 billion mega corporation would have to make way for jurors picking up kids from daycare, putting dinner on the table, and catching the train for a long commute back home. “I know the witness is a busy man,” Alsup said. “But the jurors’ time counts a lot more right now.”
He keeps his courtroom significantly colder than the rest of the building; it’s rumored that the air conditioning is cranked up high to keep the jurors awake. If someone coughs in the gallery, Alsup pauses the trial to demand to know who did it. Once the cougher is identified, the judge produces a cough drop — he keeps them by the judge’s bench for such eventualities — and the cough drop of shame is passed down through the ranks of attorneys and into the gallery. If the cough persists, the cougher must exit the courtroom, as swiftly and as quietly as possible.
Out of earshot, when the jurors are closed off inside their deliberations room, he can be harsh to attorneys. But the judge that jurors see is a grandfatherly, solicitous Southern gentleman, one who takes the time to ask after individual jurors and to thank them for the time they’re putting in.
When one of the Oracle v. Google jurors was stung by a bee during the trial, Alsup asked her if she could still follow the proceedings. When she hemmed and hawed, he said, “Let me rephrase. Can you understand what is going on just as well as you could before you were stung by a bee?”
“Yes,” she replied without hesitation.
The American system allows attorneys a great deal of power over who gets to be a juror. That’s why a jury in a software copyright trial taking place in San Francisco, California, the tech capital of the world, ended up with zero jurors with experience in the computer industry. But attorneys can’t pick the judges they end up with. And the litigators in the tech-dominated Northern District of California have learned they can’t pull a fast one on Judge Alsup.
Oracle v. Google is a vast, sprawling piece of litigation over the Android platform, one where the billions of dollars at stake were the least-significant possible consequence of the lawsuit. There’s a reason why over 70 notable computer programmers signed onto an amicus curiae “Brief of Computer Scientists” to the Federal Circuit, and later to the Supreme Court, in an attempt to explain the technical question at the heart of the case. Each one of them feared what Oracle v. Google could do to their profession.
Oracle brought multiple patent claims and a copyright claim against Google in 2010, only to lose across the board in 2012 in a trial presided over by Judge Alsup. But Oracle appealed, and the Federal Circuit ruled in its favor. When Google tried to appeal again, the Supreme Court declined to hear the case, and sent it back down to Judge Alsup at the district court. The case was tried again in 2016. Again, a jury came back in favor of Google, and again, the appeal is pending before the Federal Circuit. After seven years of litigation, the lawsuit at this stage has boiled down to a single question: did Google’s use of 37 Java APIs infringe Oracle’s copyright?
Software copyright is difficult to grapple with. When it comes to music, movies, literature, paintings, and even Bikram yoga, it’s pretty easy to have an opinion about whether something has been copied. Software, on the other hand, was an awkward late addition to the original Copyright Act of 1976, shoehorned into section 102(a) as a “literary work.”
Copyright is only supposed to cover creative works, not what’s useful or functional. That’s why the functional elements of anything — from mannequins to accounting ledgers to computer program menus — are barred from being copyrightable.
Is code a functional tool, or is it a creative expression? To the extent that code “conveys meaning,” it does seem like an art form with a valid claim to copyright. Think of programmers who refer to “elegant code” or “badly written code.” But when that code is being executed to move a robot arm to pick up and fasten a bolt, that seems entirely functional — and therefore, not the kind of thing that can be addressed by copyright.
You can’t copyright a urinal. But you could probably copyright a sculpture of a urinal. And like Duchamp’s famous work, code is both, at the same time.
“You've seen courts wrestling with [this] for decades,” says James Grimmelmann, a law professor at the Cornell Tech Institute who once worked as a Microsoft programmer. “It turns out that actually carving up a piece of software into those functional and nonfunctional parts is really hard,” he says. “It requires a really nuanced understanding of just what is doing something and what means something in software.”
The Oracle v. Google case concerns a specific component of software: the application programming interface. APIs are a collection of well-defined interactions, a sort of shorthand to quickly access services, libraries, other functionalities. APIs have been compared to dictionaries of words with their definitions, but John Bergmayer, a senior staff attorney at Public Knowledge, says they’re more like collections of proverbs or idioms. You don’t need to know idioms to be able to speak grammatically correct English, but as many ESL students know, you’re going to have a hell of a time communicating without them. An idiom might be a lovely turn of phrase, but the more common it is in a pool of speakers, the more it simply serves as a shorthand for something that might take more time to spell out. Similarly, APIs often condense commonly used or particularly complex code.
This is the question at the heart of Oracle v. Google. Section 102(b) of the Copyright Act excludes copyright protection for “any idea, procedure, process, system, method of operation.” Is an application programming interface a process, system, or method of operation? Or is it a creative expression that warrants copyright protection?
When Google first created Android, the company made the decision that it would be compatible with Java, a popular programming language. By using Java, Android could tap into an existing community of developers, and maybe even their existing code. Anyone could write in Java, but Sun Microsystems, which had developed the language, maintained strict control over Java Standard Edition and Java Mobile Edition, which allow Java code to be deployed on desktop computers and mobile phones.
After negotiations to license Java broke down, an army of engineers at Google wrote a clean room implementation of Java SE — meaning that the code was reverse engineered by a team that was forbidden from accessing the original code. Oracle acquired Sun in 2010, and in a matter of months, filed a lawsuit against Google over Android.
Google’s reimplementation of the Java APIs was almost entirely written from whole cloth. But it shared declaring code — code that identifies the names of other constituent parts of code — with the APIs of Java Standard Edition. Not only that, the structure, sequence, and organization of the implementation looked similar. And then there was rangeCheck, the infamous nine lines of code. They made their way into Android by way of Joshua Bloch, who had, suspiciously, previously worked at Sun Microsystems and had authored many of the Java APIs. (This was only an unfortunate coincidence, Judge Alsup later determined. Bloch had continued to contribute to OpenJDK, the open-source implementation of Java, after he left Sun for Google, and his code had wound up in both Android and Java SE through innocuous circumstances.)
In order to be compatible with Java, certain calls to certain APIs should look about the same. For example, the method that finds that maximum value in a set of numbers is quite sensibly named java.lang.Math.max. Oracle argued that Google could have just called it java.lang.Arith.larger.Google argued that the Java APIs were similar to the QWERTY keyboard layout. Sure, a keyboard could be organized any other way, but keyboard manufacturers keep making QWERTY keyboards, because people are used to that setup.
Languages build on top of other languages, and part of that means that their APIs look similar. The Java regular expression API is a reimplementation of Perl 5 and the Java string formatting API is a reimplementation from C. This is one of the reasons why programmers get up in arms about Oracle v. Google; it just doesn’t make any sense to police Google for something that everyone else has been doing for forever. There is a general consensus among software developers that Oracle was wrong: that APIs are meant to be used, that to restrict their usage would subvert their purpose.
Indeed, even Oracle had a hard time being consistent about its own position. In 2015, a corporate witness for Oracle said in a deposition that the Java APIs and the Java language — which is free to use — were “inseparable,” only to backtrack after his lunch break while sweating profusely.
When Oracle v. Google went to trial the first time, in 2012, the jury found in favor of Google on every patent claim. But the copyright question had since split into two stages. Were the claimed elements of the APIs copyrightable to begin with? And if they were copyrightable, were they fair use? The first question was decided by Judge Alsup; the second was by the jury.
The jury was hung on the question of whether Google’s use of the Java APIs was fair use. But it turned out not to matter, it seemed, because Judge Alsup ruled that Oracle didn’t have copyright in the declaring code or the structure, sequence, and organization of the implementing code. There was no infringement because there was nothing to infringe. And this was, Alsup concluded, because structure, sequence, and organization for API implementing code was functional rather than creative in nature.
In a case where witnesses and lawyers had struggled to explain APIs, comparing them to everything from file cabinets to electric wall sockets, Alsup’s opinion was distinctive for its meticulousness and technical savvy. For pages on end, it describes how code works, from the difference between source code and object code, to classes, declarations, headers, subroutines, methods, interfaces, and packages. It even includes sample code.
It’s hard to imagine a judge without Alsup’s long experience as a coder coming up with such an opinion. Alsup’s background certainly came in handy when he ruled on rangeCheck, the infamous nine lines of code.
“It was the kind of thing I had done many times myself in QuickBASIC,” he says, five years after that hearing. (The judge uses Microsoft’s QuickBASIC, which is an integrated development environment and compiler for BASIC, to program in). “And if you had given me that problem in QuickBASIC, I was certain I could go back within an hour and I would have a working QuickBASIC model of that.”
When we spoke, the judge was careful when talking about Oracle v. Google, since a second appeal is still pending at the Federal Circuit. But it seemed like he was still irritated with Oracle’s attempts to cast the copied lines of rangeCheck as a “big deal.” The coder in him may have even felt a little sorry for the beleaguered author of rangeCheck. The incident bothered Alsup so much that he spent an entire section of his opinion on it. “Oracle has made much of nine lines of code that crept into both Android and Java. This circumstance is so innocuous and overblown by Oracle that the actual facts, as found herein by the judge, will be set forth below for the benefit of the court of appeals.” (The nine lines of code never came up again in the case.)
Alsup’s 2013 opinion in Oracle v. Google is, as Grimmelmann says, “the most detailed, most difficult, most nuanced engagement” that the judiciary has ever had with software copyright. He teaches Oracle v. Google to his own IP class. “It’s a framing of this is how Java works, this is what the different elements of source code are… it’s not just judging the case before him. It’s a piece of writing that’s pedagogical.”
Thanks to the meticulousness of his opinion, a great deal of Alsup’s understanding of Java and of software development remains preserved in the law, and is being passed down to young would-be lawyers. That may be the most lasting mark left by the remarkable ruling of the coder judge — because the judgment itself has been completely overturned.
Alsup’s 2013 ruling in Oracle v. Google went up on appeal almost immediately, and in 2014, the Court of Appeals for the Federal Circuit came back with a shocking reversal, one that resulted in countless law review articles penned by irate scholars of copyright law. The appeals court wrote that Oracle had “unlimited options as to the selection and arrangement of the 7,000 lines Google copied.” Oracle didn’t have to name the function java.lang.Math.max. It could have been called “any number of things,” like “Math.maximum” or “Arith.larger.”
It’s small tells like that one that suggest an overall unfamiliarity with code. (In another part of the opinion, the court says, “Google was free to develop its own API packages and to ‘lobby’ programmers to adopt them.”)
Google appealed the Federal Circuit decision up to the Supreme Court, which declined to take the case. The suit was sent right back to where it started: with Judge Alsup in San Francisco, for a second jury trial, and on May 26th, 2016, a jury found fair use in favor of Google.
Google may have won, but not only is the case still pending — Oracle has appealed again to the Federal Circuit — it’s not clear what the actual effects of the case will be over time. The jury verdict of fair use doesn’t provide any guidelines on when it is or isn’t okay to copy declaring code, or the structure, sequence, and organization of an API.
Whatever insight or clarity Alsup had written into his opinion, built upon his long experience with BASIC, had simply evaporated away, blown to bits by the appellate court.
Photo by Sarah Jeong / The Verge
Alsup is not shy about his coding chops: “I do think I am a good programmer, and I think when you read these programs you can see that — since I’ve taught myself everything — there are some pretty nifty programming devices in these QuickBASIC programs.” He added, “But they’re not Java.”
The judge has been coding ever since he got his first computer in 1985: an old IBM that he’s since consigned to a dark and dusty corner of a barn on his Yosemite ranch. It used a 5 ¼-inch floppy disk and didn’t have a hard drive. You could get the version with one floppy drive, or you could get the version with two. He and his wife sprang for the luxe two-drive version.
The computer came with two books, one about the DOS operating system, and the other about BASIC. “At some point, I looked at the BASIC book and decided I would learn that.” He taught himself straight from the book, which he recalls was “pretty straightforward.”
The first programs he wrote were the example ones, simple routines that did arithmetic. They became increasingly complex: one that would play blackjack, one that played seven-card stud. They were all stored on a 5 ¼-inch floppy disk that was destroyed in 1988 when his two-year-old son ravaged it with the stapler.
“He was so proud of himself,” Alsup said wistfully.
Photo by Sarah Jeong / The Verge
Of all the programs he’s written, it’s his shortwave radio propagation predictor that he’s most proud of — and with good reason. It’s a relatively complex piece of software with multiple dependencies and retro-looking graphic interfaces, including an Azimuthal projection map based on the starting location you pick, complete with colored lines that track the movement of the sun, and extensive databases that he compiled by hand from atlases. (He is immensely proud of the massive amount of time he has spent doing data entry in his off hours, both before and after becoming an Article III judge appointed by the president of the United States.)
The program predicts the best times to target ham radio signals to various parts of the world by calculating between two endpoints that he specifies, or even generating tables of data about key locations he’s selected from all over the globe.
Even in 1995, when he started coding his program, there were commercial versions available of similar programs. But, he says, “I just wanted the fun of being able to see if I could do it.”
He uses his program to calibrate his Yaesu Mk V Field radio to talk his friends all over the world, including his childhood friend “Junior” Feild, and operator friends he’s made in places as far away as Japan and New Zealand. He still spends a couple of hours operating his radio every month, mostly from the Sierra foothills. His call-sign is N6XMW, or as he puts it, “November Six X-Ray Mike Whiskey.”
The judge spent almost an hour explaining this particular program to me, going over each of the various inputs that can change shortwave radio propagation, as well as the science behind it. The interview turned into an impromptu physics tutorial as he patiently explained the solar flux, K-index, and the ionosphere to me.
Predicting radio propagation takes into account more than a few of these constantly fluctuating variables, all of which change depending on where you are and where you’re trying to reach, as well as the time of year and time of day.
Once he puts in the variables, he hits enter, and the computer begins to calculate. “See,” he said. “It’s thinking.”
Indeed, the computer screen is now blank, except for the words, “Thinking…”
During the lengthy demonstration, the program does run into trouble once: a dependency breaks, and for some reason, it won’t let him input New York City as a location. “That’s not good,” he mutters to himself. “Alright… so I goofed up,” he admits to me. We agree to try a different location entirely, and the program runs smoothly from there on.
Alsup has coded in relative isolation for decades, learning from books and compiling databases by hand. It’s a marked difference from the typical practices of the current generation of software developers, whose workflow and habits often tap into a larger community. He doesn’t Google for solutions, he doesn’t check StackExchange, and he doesn’t use preexisting libraries. Everything he’s written, he’s written from scratch.
In fact, Alsup’s closest encounter with the culture and community that has sprung up around programming seems to be Oracle v. Google.
In an infamous exchange in the second trial, erstwhile Sun CEO Jonathan Schwartz tried to explain free and open-source software to the jury, starting with GNU, a project integral to Linux that can be loosely described as both an operating system and associated suite of software.
“What does GNU stand for?” Alsup interrupted to ask.
“GNU is Not Unix,” said Schwartz.
“The G part stands for GNU?”
“That doesn’t make any sense,” the judge replied. There was some laughter in the courtroom, but it was nothing like the uproar on Twitter afterward, as hundreds of nerds across the world facepalmed. (I even later saw a webcomic about this exact exchange.)
The GNU acronym is recursive, meaning that it invokes itself in an infinite loop — a move that frequently comes up in computer programming. There is a host of recursive acronyms in programming, including PHP (PHP: Hypertext Processor), cURL (cURL URL Request LIbrary), and unofficially, the search engine Bing (Bing Is Not Google). These are bad inside jokes, embarrassing markers of an insular culture that never anticipated having to explain itself in a court of law.
And Alsup, despite having been a coder for so long, did not know what GNU stood for until that very moment. He was a little chagrined about it when I asked. Apparently an engineer friend of his (one of his backpacking buddies) had teased him for the GNU remark. “I did not know this recursive feature of the definition,” Alsup said. “Once it was explained to me, I was like, ‘Okay, that’s kind of cute.’”
Recent shifts in computer programming have made it difficult for Alsup to keep up with his hobby. A few years ago, he made an effort to learn Python, but that fell by the wayside when he “got too busy,” presumably with his day job as a federal judge. Microsoft has since stopped bundling QuickBASIC with Windows, making it impossible for Alsup to run his programs on newer computers.
It’s poetic in its own way: the judge who presided over a major compatibility case is now himself a victim of compatibility issues.
Photo by Sarah Jeong / The Verge
In December, Alsup will preside over Waymo v. Uber, a potentially multibillion-dollar lawsuit in which Uber is accused of stealing intellectual property around self-driving car technology from a division of Alphabet. Waymo doesn’t grapple with core principles of intellectual property law the way that Oracle v. Google does, but like that case, the outcome could change the face of the industry forever.
In our conversations, Alsup has been very careful not to discuss the ongoing Waymo case. But he’s enthusiastic about the science surrounding the lawsuit. This time, it’s his interest in photography and optics that might be most relevant to Waymo v. Uber. In March, Alsup requested that each side name a book or a treatise about LIDAR, a laser-based detection system used for self-driving cars that is at issue in the case, so that he could read up on it. But a court order sternly cautioned the parties not to patronize him.
Please keep in mind that the judge is already familiar with basic light and optics principles involving lens, such as focal lengths, the non-linear nature of focal points as a function of distance of an object from the lens, where objects get focused to on a screen behind the lens, and the use of a lens to project as well as to focus. So, most useful would be literature on adapting LiDAR to self-driving vehicles, including various strategies for positioning light-emitting diodes behind the lens for best overall effect, as well as use of a single lens to project outgoing light as well as to focus incoming reflections (other than, of course, the patents in suit).
Alsup also asked for a tutorial in LIDAR, presented by lawyers from both sides. It’s something he does for a lot of different cases. He enjoys the tutorials, he said to me, and he listens carefully. But still, he suspects that he understands the technology better than the lawyers do in a great number of cases.
He has a long memory for lawyers he suspects of trying to get one over him, only to run into his engineering background and his acid tongue. Like any geek, he bears a grudge against any dodgy obfuscation of technology.
In a case that took place about a decade ago, regarding a patent involving FastTrak (the Bay Area’s automated bridge toll pay tracking device), the two parties had come to a legally binding agreement about the technical aspects of how the patent was to be interpreted. Instead of rubber-stamping the stipulation, Judge Alsup pored over the patent himself, only to come to the conclusion that their stipulation made no sense at all.
“I knew what the technology was. And I told them. And I wrote an order that said I refuse to accept this stipulation, and here’s what it really means. And then they both agreed with me that I was right.”
The parties settled the case shortly thereafter.
“I guess they thought the judge was crazy.”
Alsup’s aggressive handling of attorneys is on full display in Waymo, perhaps fueled by deep disapproval of the millions of dollars that Alphabet and Uber are burning on some of the best litigators in the business. In the past, he’s been open about his distaste for the money that corporations fling around in court. In one post-trial hearing in Oracle v. Google, he snapped, “Do you know how many Social Security claimants I can’t rule on because you’re arguing over a costs bill?”
In Waymo in particular he has called on the press to scrutinize how the two companies behave in court, telling journalists to keep an eye on which party chooses to get rid of jurors with technical backgrounds during jury selection. Meanwhile, Alphabet and even Lyft are fighting to keep portions of the trial closed to press, claiming that valuable trade secrets might be exposed to the public.
A group of media companies (including The Verge’s parent company, Vox Media) has since intervened in the lawsuit, asking to keep the courtroom open. While it’s certain that parts of the lawsuit — which does involve a fair number of trade secrets — will remain sealed, Alsup has been adamant that the press has a right to know as much as they can.
In one hearing, lawyers for Uber asked to appear in camera, making the meeting closed to the public. But once the hearing got going, Judge Alsup concluded that this was an overstep, a use of secrecy that was motivated by embarrassment rather than legitimate reasons.
“Listen, please don't do this to me again,” he said. “There's going to be a lot of adverse headlines in this case on both sides. And I can't stop that.”
“The public has a right,” said the judge. Then, in a classic Alsup twist, “In fact, this whole transcript? I’m going to make it public.”
Since 1999, Alsup has divided his docket between two clerks: what he calls the criminal desk and the IP desk. Cases that don’t fall into either category are evenly split between the two.
These days, he often looks for some kind of STEM background for the IP desk. It’s not necessary, but it helps. Bill Toth, the IP clerk during Oracle v. Google, didn’t have a STEM background, but he told me that the judge had specifically asked him to take a computer science course in preparation for his clerkship. When I asked Alsup about it, he laughed a little — he had no recollection of “making” Toth take any classes — but he did acknowledge that sometimes he gives clerks a heads up about what kind of cases are coming their way, and what kind of classes might be useful ahead of time.
Bill Toth is now clerking at the Federal Circuit, for Chief Judge Sharon Prost. He won’t be allowed to work on the Oracle v. Google appeal, of course; it would be a conflict of interest.
The tech community maintains an entrenched belief that the law will never understand what it does, and certainly the higher-level court decisions made in Oracle v. Google do nothing to dispel that notion. Yet Alsup’s very existence is a challenge to this belief: a 72-year-old former engineering student who has been quietly and delightedly tinkering away in BASIC for decades, playing with his radios and his cameras, teaching his clerks and random journalists alike the things that he knows.
In the same patient and meticulous manner that Alsup explained the ionosphere to me, he explained software to the Federal Circuit in his Oracle opinion. And because it was so precise and so specific, he forced the Federal Circuit to engage with software development more than it otherwise might have. Regardless of the Federal Circuit’s ultimate decision, a great deal of Alsup’s understanding of software remains preserved in appellate caselaw.
That opinion is now taught in law school intellectual property classes. As the tech sector attracts more and more money, tech-specialized lawyers are emerging to meet that demand. Many of them will be reared on Alsup’s careful pedagogy — for most of them, transmitted through words on paper, and for a lucky few, transmitted in person.
A few hours after we conclude the interview and I leave the courthouse, he emails me with the subject line “Found the bug,” informing me that he had figured out the error from when he was showing me his shortwave radio propagation program. “I had remmed out earlier a key line for reasons I can't remember and just reactivated it, so now it's fixed,” he writes.
I thought back to my last moments in his office. As I was packing up my recorder and notebook, I had called him the “geek judge,” only for him to look perplexed and ask, “Is that a good thing?”
When I insist that it is, he chuckles a little in response. “In my day, a geek was not something you wanted to be.”
As part of an ongoing legal battle to get the New York City Police Department to track money police have grabbed in cash forfeitures, an attorney for the city told a Manhattan judge on October 17 that part of the reason the NYPD can't comply with such requests is that the department's evidence database has no backup. If the database servers that power NYPD's Property and Evidence Tracking System (PETS)—designed and installed by Capgemini under a $25.5 million contract between 2009 and 2012—were to fail, all data on stored evidence would simply cease to exist.
Courthouse News reported that Manhattan Supreme Court judge Arlene Bluth responded repeatedly to the city's attorney with the same phrase: “That’s insane.”
Last year, NYPD’s Assistant Deputy Commissioner Robert Messner told the City Council's public safety committee that “attempts to perform the types of searches envisioned in the bill will lead to system crashes and significant delays during the intake and release process.” The claim was key to the department’s refusal to provide the data accounting for the approximately $6 million seized in cash and property every year. As of 2013, according to the nonprofit group Bronx Defenders, the NYPD was carrying a balance sheet of more than $68 million in cash seized.
City attorney Neil Giovanatti continued that line of argument. He claimed that the NYPD doesn’t have the technical capability to pull an audit report from its forfeiture database—because the system wasn’t designed to do that.
But an expert witness for Bronx Defenders, which is suing for access to the data, undercut claims that the system could not produce a report on the cash. Robert Pesner, former chief enterprise architect for New York City’s Department of Housing Preservation and Development, told the court, “Based on the information I have reviewed about the technical specifications of PETS’s hardware and software, it is my opinion that it is technologically feasible to retrieve much of the data sought from PETS by running queries directly on the underlying [IBM] DB2 database.”
When it was activated in 2012, Capgemini vaunted PETS—which was built using SAP’s enterprise resource planning (ERP) software platform as well as IBM DB2 databases—as a flagship public sector project. The company went as far as submitting PETS as a nominee for the 2012 Computerworld Honors awards. But the system was apparently designed without any scheme for backing up the database or any sort of data warehouse to perform analytics on the data.
When told by Giovanatti that the police department’s IT department did not keep backups and only knew that the database “is in IBM," Judge Bluth responded, “Do you want the Daily News to be reporting that you have no copy of the data?... That deserves an exposé in the New York Times.”
So, I came across this astounding Yelp page for a family medicine clinic in Riverdale, NY, proving that sometimes, you just can’t win in my line of work.
I don’t work there but do know some of the staff. They are drilled to be extremely professional to the point of being colorless automatons, which should be kept in mind with some of these reviews.
“Deserves no stars… My mom called on Friday because her left eye was super red to the point where it looked like something popped… called them to make sure that they accept our insurance and have the appropriate doctor who can take a look at her eye and diagnose what is going on… After asking a series of questions over the phone, Dwell Family Doctors confirmed saying that they have an eye doctor who can take a look at my mom’s eye on Saturday…
“After filling out necessary paperwork, before we even meet the doctor, they ask us to pay the co-pay. I found that to be a bit unprofessional and odd but regardless we paid the co-pay before we met the doctor… We wait about 5-10 minutes and the DO finally comes in.”
The visit immediately goes negative over the simple issue of the co-pay, which is a standard requirement of most insurance plans, and one that the insurance carriers actually require medical providers collect. Yet, this person is already primed for a negative encounter over something she would still have to pay at literally any other clinic. And she had to wait 5-10 minutes? Heavens to Betsy, the outrage! The Department of Health should shut them down at once!
“He then asks “what brings you here today?” So my mom explains her condition in regards to her left eye, and the only advice he’s able to give is “you probably need to see an eye specialist”. At that point I’m baffled by his response because the whole reason why we stopped by was because of her EYE and we were told specifically that an eye specialist would be able to check her condition to make sure everything is okay. They gave us false information from the beginning saying that a specialist would be able to diagnose her symptom and when we get there, we’re told that there is no optometrist available. We paid the copay to hear from this place that she needs to see an eye doctor. So ridiculous.”
She was told a doctor could look at her mother’s eye, and chose to interpret this as an eye doctor would be on site. Hardly any urgent care has subspecialists on site, because how is that even feasible economically? Why would they pay an ophthalmologist to sit around all day for the, perhaps, 1 patient at most with an eye problem more severe than simple pinkeye? If her mom were having a bad migraine, would she have expected a neurologist to be on site too? A cardiologist for chest pain? Pulmonologist? Interventional radiologist, perhaps?
Same review: “After expressing my frustration , the DO goes to his office and brings me a medical textbook and tells me to read it. Does he really think we went all the way there to read a textbook instead of getting proper medical treatment? He expects a book to resolve all issues. Is he serious? What good is reading a textbook going to do for my mom’s eye?”
It’s clear by this point the visit has become entirely hostile, so the poor DO has probably pulled a reference as some sort of support against the daughter’s verbally abusive barrage. But the truth is: a primary or urgent-care doctor simply will not have the fancy and expensive eye equipment that only an ophthalmologist would have to diagnose advanced eye problems.
There was literally no way this clinic could have ducked this 1-star review. The daughter went in with completely unreasonable and irrational expectations and already started to become hostile over the routine matter of the co-pay… for an issue the daughter was clearly comfortable enough with to let sit overnight as opposed to going to the ER. And flips out to find that the clinic does not have a friggin’ ophthalmologist on site — on a weekend, no less.
This brings to mind another charmer I saw a few years ago. Came in complaining of chest pain x2 years, vague, unable to really specify beyond that in terms of pain quality, triggers, relieving factors, whatever. He says he got all kinds of workup from several different cardiologists, echocardiograms, nuclear stress tests, MRIs, MRAs, you name it. Everything came back negative. He came in to my simple clinic with no specialized hardware available demanding that I tell him what was wrong with his chest right then and there. After stating it wouldn’t be so simple, yep, he completely flipped out and served me a 1-star review. He even stated he threw out the cardiology referral I had given him because he felt I was useless. Out of spite!
Sometimes in medicine, you are Holland and Belgium circa 1940. Sometimes, there is literally nothing you can do to stave off the unprovoked aggression.
Returning to that Yelp page for another review:
“My daughter has been to this place twice and has been given the wrong medication for her condition. This office was convenient for her because she lives down the street. She was diagnosed with an ear infection but she was given EYE drops from the person who saw her (who is not even a doctor) someone should be looking over every prescription that is given to patients at this facility by anyone who is not an MD.”
This actually happens all the time. I prescribe antibiotic ear drops; get the call from the pharmacy that they’re completely out, so they’ll have to sub in the eye drops instead.
Guess what? They are the exact same medication. In drop form. The eye drop one might be more benign for your far more sensitive eyes but they do the same damn thing. Oh well — because the bottle said “ophthalmic” and not “otic,” the patient’s mother literally wants the clinic shut down. (The clueless doctor she saw the next day didn’t exactly help matters, either, tbf.)
Also, given the shortage of MDs, especially since competent but non-board-certified MDs are all but prohibited from practicing, the gap has to be filled somehow. Don’t like PAs or NPs? Too bad. If you refuse to wait until the doctor is in, get used to seeing them.
“Went to Dwell and after waiting for an hour they told me that couldn’t verify my insurance. Woman at the front desk was incredibly rude claiming she was bending over backwards to help me while swearing at me. Don’t waste your time, you get better service at McDonald’s.”
If you really think she had to wait an entire hour while the staff member tried her best to accommodate her and her likely garbage insurance plan, I got a Nigerian prince on the other line for ya. The “swearing at me” bit in a clinic with multiple people around and with likely video surveillance systems proves that online reviews are like the presidency — lying simply has zero consequences.
“I went there today because I had a bad cough and sore throat. The staff at the desk were polite and thorough. The doctor was good. He examined me and explained everything. I was pleasantly surprised. I would go back if I couldn’t get an appointment with my primary MD.”
What’s so wrong about this one? Nothing, except they then proceeded to give the clinic 3 stars out of 5. “Staff was great! Doctor was good! Explained everything! Pleasantly surprised! 3 stars, mediocre at best!” What on earth could she have wanted, an erotic massage thrown in gratis? And yes, 3 stars is almost as bad as 1. Would you buy a printer off of Amazon if it were anything less than 4 stars?
Walked in to an empty waiting room with what I suspected was a sinus infection: coughing, low fever, head pressure. Just as I’m signing in, a woman comes storming out of the back saying, “I will not see him, he’s an idiot! Don’t charge my insurance because he doesn’t know what he’s doing! He doesn’t act professional or dress professional!” So now the 4 (count em FOUR) people at the front desk all turn to her with open mouths and sit there staring and do nothing. I say to the woman directly in front of me, “can you help me and let someone else take care of this? There are 4 of you.” She comes back and says, “Excuse me, we , are taking care of this now, then we’ll talk to you.” Really rude and terrible attitude.
Ah yes, the 1-star by association. Didn’t even wait to see the provider herself — after all, if the raving lunatic says they suck, then what more does she need? And, yes, how dare the staff be distracted by the screaming crazy person in the lobby? “Really rude,” indeed!
This is all a significant problem in medicine, especially as these people are far more likely to file frivolous malpractice lawsuits. But I should make clear that these people are the exceptions. The vast majority of patients just wanted to be seen and have reasonable expectations for a busy primary- or urgent-care clinic. Being a health care provider isn’t about completely avoiding the above outcomes, as that is literally impossible. Being a health care provider is about rolling with the punches and not letting such people color your attitude towards the 99% of your patients who are not like this.
And if that takes authoring a long, online tirade to let off steam, so be it.
Dieter Bohn, reviewing the new Pixel phones for The Verge:
Without fail, every person who has picked up the Pixel 2 XL has
said virtually the same thing: “It feels like it’s made out of
plastic.” I said it myself when I first held it. Of course,
neither the Pixel 2 nor the Pixel 2 XL are made out of plastic.
They’re made out of Gorilla Glass and aluminum, just like every
other high-end phone these days.
But Google coated all that aluminum with a textured finish that
hides most of the antenna lines and also makes the phones easier
to grip. Google took what could have been a visually impressive
design and covered it up in the name of ergonomics. It literally
made a metal phone feel like a plastic one. It chose function
Interesting design decision. Apple has moved from aluminum to glass (and last year a glassy-feeling coating on the jet black iPhones). Samsung has moved to glass. Part of this is that most top-tier phones this year support inductive charging (which doesn’t work through aluminum), but even here with the Pixels, they’ve moved away from feeling like aluminum.
On the display colors:
The screen, especially on the 2 XL, has been polarizing. Google
opted to tune the display to sRGB (the Galaxy S8, by comparison,
offers four gamut options), so it looks a little more like the
iPhone’s screen. But more than that, on the 2 XL the colors look
muted in a way that many Android users I’ve shown it to found
distasteful (even with the “vivid colors” setting turned on). I
think many Android phones, especially from Samsung, are so vivid
as to be phantasmagoric, so Google’s choice was to make this more
My take ever since last year (I bought a Pixel 1) is that the Pixels are targeting people whose taste runs toward the iPhone hardware-wise, but who prefer Android over iOS. Actually, not Android over iOS, but the Google ecosystem over Apple’s. They’re iPhones for Google people. I find Samsung displays to be technically impressive but downright garish in terms of saturation.
95% this: "Pixels are targeting people whose taste runs toward the iPhone hardware-wise, but who prefer Android over iOS. Actually, not Android over iOS, but the Google ecosystem over Apple’s."
The other 5% is not only wanting more ram than Apple's stock offerings, but also wanting Google's cutting edge software for things like its camera. Seriously though, integrating perfectly into the google ecosystem is a really big plus.
At the Interior Department's headquarters in downtown Washington, Secretary Ryan Zinke has revived an arcane military ritual that no one can remember ever happening in the federal government.
A security staffer takes the elevator to the seventh floor, climbs the stairs to the roof and hoists a special secretarial flag whenever Zinke enters the building. When the secretary goes home for the day or travels, the flag -- a blue banner emblazoned with the agency's bison seal flanked by seven white stars representing the Interior bureaus -- comes down.
In Zinke's absence, the ritual is repeated to raise an equally obscure flag for Deputy Secretary David Bernhardt.
Responding this week to questions from The Washington Post, a spokeswoman for Zinke, a former Navy SEAL commander, defended the Navy flag-flying tradition as "a major sign of transparency." [...]
Zinke, a Stetson-wearing former Montana congressman who has cultivated an image as a rugged outdoorsman, has come under a harsh spotlight in recent weeks for behavior criticized as extravagant for a public official. The agency's inspector general opened an investigation after he ran up bills for travel on chartered jets and mixed business with political appearances, sometimes accompanied by his wife, Lola. It's one of five probes underway of Cabinet secretaries' travel. [...]
Zinke rode to work on horseback on his first day in office and displays animal heads on his wood-paneled office walls. For a while, he kept a glass-case display of hunting knives but was asked to remove them because of security risks, according to people familiar with the decision.
He has commissioned commemorative coins with his name on them to give to staff and visitors, but the cost to taxpayers is unclear. Zinke's predecessors and some other Cabinet secretaries have coins bearing agency seals, but not personalized ones.
The flag ritual is unique in President Trump's administration. The White House does not raise the presidential flag when Trump alights at 1600 Pennsylvania Ave. There is no Defense secretary's flag atop the Pentagon.