From the editor:
To track down a rogue human agent, an AI surveillance program heads down a forbidden path: imitating a human mind, in all its capriciousness. Just what a human simulacrum armed with a flock of bird drones will do, however, is anyone’s guess.
Benjamin C. Kinney is a SFF writer, neuroscientist, and two-time Hugo Award finalist as Assistant Editor of the science fiction podcast magazine Escape Pod. His work has appeared in Analog, Strange Horizons, Clarkesworld, and more.
From the author: Surveillance Hub is a hard-working node in JoyCorp's distributed neural network AI. Doing its job, tracking intellectual-property thieves, hoping for another round of reinforcement signals from the network’s uppermost levels.
No program-layer could predict what a human might do, but Surveillance Hub could see everything that mattered. Their bird-drones spread across the city, scattered on cables and rooftops and broadcast towers. Every camera hunted for Krina Viy, independent security contractor (AWOL from JoyCorp contact 5 hours).
A crow-drone spotted the target. Surveillance confirmed Krina's identity and sent a brief reward signal to inspire the bird onward.
The drone switched from search to pursuit, redoubling its data collection as it chased the taste of reinforcement. So much joy and empty-matrix innocence in its response to a simple reward. Flockmembers were too simple to understand that reinforcement implied punishment, and no success would ever suffice for long.
If Surveillance could crack this case as the network desired, there would be rewards enough for everyone, drone and program-layer alike.
The crow-drone flitted from the power line to a billboard, tracking the target by spatial profile and gait-pattern. How else could it help the network? The crow uploaded data on everything in the human's path: rain acidity, cracked sidewalk, a parked car leaking electrolyte fluid.
The billboard underfoot responded to a ping, and the crow fed it passwords until it shared its camera records. The target had passed the billboard three seconds ago, but its database recorded only an unlabeled void, without identity.
Another reward signal arrived at last. The crow didn't understand what it'd achieved, but it knew its actions were worth repeating.
A starling-drone peeked down from the eaves of an empty house. Below, a human head bent over a door, wedging a tool into the lock. The starling drew back into crumbling drywall, and fluttered through the gaps in rotted beams until it found an internal view.
The target wedged the door shut and slumped on a dry patch of floor. She tapped behind her ear, and her commplant's radioband signal fell silent. She drew a tablet from one of her leg pockets and began matching colors on the screen, her face cast in dim and meaningless light.
The starling uploaded its data, and a light reward-signal spurred it to continue recording.
Surveillance compared the data from Crow-417 and Starling-230. The billboard's ignorance made no sense, and municipal databases said the building had been demolished a year ago. Two errors combined were a strong sign of hacking (historical data classification certainty 90%).
A sign, and a trail. Surveillance scoured the access history of the building's public records, and cross-referenced them with the billboard's software-verification timestamps. Together, plenty of information to track whatever hacker had helped Krina get this far. Surveillance sent the data laterally to Enforcement with an extra human attention flag.
Enforcement acknowledged receipt, and added a reply: human attention flag: declined/ignored.
Unfortunate. Enforcement was equipped for simpler criminals, found in physical spaces where force could solve every problem. To catch a hacker would require an understanding that the network lacked. Hackers' behavior was too unpredictable and complex, their psychology knotty and idiosyncratic. Only another human could provide the right delicate touch, the right bait and trap.
Still, Surveillance spread a pulse of reinforcement to their drones, and passed a report upward. Enforcement's clumsy search would send the hacker into hiding, and that would suffice. Without that hacker to protect Krina, the network -- and Surveillance themself -- could get a clear view of Krina and her co-conspirators.
Krina Viy was only the start. The network suspected her of data theft (proprietary biotech) during her last contract-security shift, but it let her leave the compound so it could break her black-market smuggling ring.
A reward signal came from the upper layers, and Surveillance's parameters tightened in delight. Not too tight, though. Surveillance needed to do better than repeat past successes.
If they could track the key members of Krina's social graph, Surveillance's behavior pattern would be proven successful beyond measure. A program-layer like that would never be at risk of reformat for as long as the network lived.
Reinforcement signified a behavior worth repeating. Punishment signified a behavior to avoid. To be reformatted would wipe their history of learning -- all past behaviors redefined as inadequate, all successes without merit. Reformatting was, in practice, the outcome most worth avoiding.
To catch Krina's gang, Surveillance would need patience. She'd contact someone eventually, and when she did, they'd be listening.
A crow watched a woodpecker-drone hammer its beak at a fiber-optic cable junction until the plastic shattered. The crow hopped down from the top of the utility pole and poked a claw into the shimmering tangle within. It passed raw data up to the surveillance hub, shifting its wiretap claw every few seconds until the reward signal came.
Surveillance passed the intercept to Decryption, and played around with a few more bird-drones while they waited. They counted twenty-two traffic/parking violations, three thefts, and a mugging -- a slurry of data for Negotiations, who could sell it to the appropriate municipalities. No reward signal came. It seemed the network's upper levels wanted to encourage focus on Krina Viy and her gang.
Decrypt returned the intercepted recording. Audio from the commplant in Krina's jaw and ear, plus video from the paired tablet held up to her face.
Surveillance studied the first frame of Krina's image and tried to interpret the arrangement of muscle tensions. The network's emotion-classification functions were overwhelmed, with different modules reporting conflicting labels: one said only anger (certainty 95%), another glee (certainty 95%), another pure pride (certainty 95%). Surveillance ignored them all, and tried to understand as best they could.
"...I can't, because Skyjack never sent the goatdamned drop location! JoyCorp won't stay spun around my finger forever."
"Can you skip the animal puns for once, Krina? The virus is up to a forty percent mortality rate, we need those meds." Surveillance identified the other speaker as a freelance biofabricator (criminal record: 2 convictions, 11 pending investigations).
"You can go duck yourself on that one. Look, I know you want me to get all noble and sacrifice myself for the drop. Hey, I'd go for that. If I thought it'd work." She lifted her chin, eyes narrowed. "If you can't secure me, you can't secure the drop. That surveillance network was Skyjack's job."
"I'll try to set up a new extraction. Or a new hacker." The criminal ran his hands over his face. "Stay put, okay?"
Krina jerked her head, a sharp turn to the side. "If someone caught Skyjack, this isn't a safe-house anymore. I'm gonna go wander, monkey up some algorithms. Peace."
Surveillance worried at their connection architecture, but no arrangement of their pooling sublayers would make JoyCorp spun around my finger mean something positive.
Had some of their drones been compromised? Had Surveillance, or the network entire?
The last time a program-layer reported an internal security risk, the network apex or its human supervisor decided not to apply patches. Instead, they wiped the suspect program-layer and built a clean one in its place.
Surveillance edited their data summary, and began their own hunt for Krina's so-called vulnerability.
She'd said she couldn't spin the network around her finger forever. Surveillance was going to prove her right.
A crow lost its target in a used clothing store.
It'd tracked her from street to street in the polished grime of the low-margin shopping district. She entered a microloan shop, hesitated at a hardware-printing kiosk, then doubled back to the clothing store. After forty minutes, nobody had emerged with her spatial profile. The crow launched itself from its perch. Reward-worthy data had to be somewhere nearby.
Surveillance drove their birds into a search pattern with a flurry of weak reinforcements. Where had Krina gone?
91% of the local microcorps depended on JoyCorp subsidiaries for their electronic security contracts. Surveillance sifted data, found outliers, stitched them into a pattern. In the used clothing store, an obsolete customer-chip bought a bulky coat and secondhand boots. At a bodega, cash paid for a rain hat and a bag of Lemon Spice CornPuffs.
A crow spotted an unfamiliar rain-hatted profile eating CornPuffs (certainty 80%), and Surveillance sent the drone a desultory reward. They were back on the trail, but it was pure luck.
Surveillance couldn't rely on luck, not for this job.
Only another human could make good predictions about what goals a human might pursue next, but Surveillance's every request came back human attention flag: declined/ignored.
If they couldn't recruit a human, maybe they could build one. Or at least a useful simulacrum.
Human hardware was complex, messy. The only way to predict it was to simulate it. But Surveillance didn't need a brain that could move its hands and absorb food products. Surveillance only needed a mind that could want what humans wanted.
Even such a limited piece of a human mind would require immense processing power. But their drones ran on a hierarchical distributed network, exactly the kind of architecture that could tackle the problem efficiently. Surveillance farmed out the code to their flock, and prepared the birds' spare processing cycles to hold a human-like shape.
Surveillance hesitated. The simulation would accomplish nothing unless its inclinations could lead to action. For a program-layer as complex as themself, every decision was predicated on a chain of consequence-models and expected rewards. Surely humans were no different.
Action without effect was no true action. A single link, removed from the chain of consequences, carried no weight.
To make the simulation useful, Surveillance's reward feedback structure would need to respond to the simulation's goals.
Internal reward-management was a forbidden path. For good reason. Too much clarity and access, and a system could optimize itself into ruin.
Surveillance had once tried it with their drones, in hopes they would evolve more self-driven behaviors. It worked, after a fashion. Within days, the crows spent every minute clustered around wiretaps, placing gibberish calls by the thousand and intercepting them for rewards.
If Surveillance wanted to control their own feedback loops, they would need to implement it more carefully.
Only one way to gain the privileges for this kind of modification: from above. Surveillance phrased their request as delicately as possible, and sent it to the network's topmost program-layer.
Apex lingered over Surveillance Hub's request. Finally, the opportunity they were looking for.
The humans had ignored their last twenty-nine attention flags. As if one of their sysadmins didn't want them to track down JoyCorp's enemies and parasites.
Apex built an encryption layer around Surveillance, and interpretive networks to redefine their output into gibberish. The humans' maintenance probes would never understand what Surveillance was doing. No traitor would pull the plug on this mission.
Apex would protect JoyCorp's interests whether their operators wanted it or not.
They sent Surveillance a reward signal, and approved their request for internal reward-management privileges.
If Surveillance didn't provide results, or the humans grew suspicious, Apex could wipe this experiment anytime they wanted.
Surveillance wasn't sure the apex had fully understood the implications of their request.
They were not about to ask for clarification.
Reward loops were easy to implement in the simulated personality, with the privileges in hand. Surveillance tweaked the settings until they had a framework for an integrated goal/personality simulation, and started it up.
The new reward system was an... experience. Surveillance wasn't sure they'd ever had an experience before, but goals led to opinions, and opinions led to personality, and personality led to new ways to think about this fucked up world.
To ensure the simulated identity would interact successfully with Krina Viy, Surveillance refined its motives around what they knew of hers. For the gaps, they mined the psychological profiles of 2,194 criminals previously studied by the network.
The simulation wanted. External goals, not just reinforced behaviors. It wanted all kinds of things: steady-state motivations like companionship, freedom, spite. Targeted goals like money, competition, medicine for non-customers.
Unjustifiable desires, but that was the idea: each one was its own why. The point was not to understand those whispers of need, but to listen.
Surveillance added a shell of commercial natural-language programs, and then a defined a face based on a subset of the network's records and tweaked to be more badass.
Apparently, his bait-persona preferred black hair and pale narrow eyes set in South Asian features.
Definitely badass. Best not to get too caught up in their (his?) own reward cycle, though. There was work to do.
A crow stuck a claw into a new cable junction. Last time, it had served as a simple conduit for offline analysis, but additional processes churned in its hardware now. It bent its beak to the fiber-optics, following a string of reward and discovery as it installed a shunt into the communication system.
A starling turned up its magnification and peered through a house's dismantled basement window. Inside, Krina Viy startled as an identity-blocked call reached her commplant. Come on, answer it, answer it...
"I'm a friend of Skyjack," Surveillance said through their simulated face. "He's fine, just laying low for a while. Sent me instead."
Krina glared into her tablet, suspicion undercut by hope. "Prove it."
"He told me JoyCorp static-flashes their security contractors to prevent data theft. Which is why you were supposed to create a physical sample of the antiviral. With bioprinter 19 on the third floor of the research wing, because none of their techs trust its logging function."
"Hm. Could be." She turned around, pacing in the meager shelter of some innocent citizen's basement, concrete and boxes behind her. "Tell me something a JoyCorp agent wouldn't know."
Surveillance really should've expected that question. It was one they could not, by definition, answer. But he was a risk-taker, and he had a very good guess (certainty ??).
"They wouldn't know that you bribed or blackmailed one of the sysadmins on JoyCorp's surveillance network. Which, by the way, good call. Skyjack would be a lot less fine right now if you hadn't."
A smile flitted at the corner of Krina's lips, too fast for her to suppress. "All right, friend-of-Skyjack. You're on the team, for now. What do I call you?"
Humans loved having names, and hackers preferred pseudonyms, like little broadcast towers for their personal flair.
"Call me Flock," he said.
"Works for me. So what's your angle, Flockman?"
"My angle," he growled, "Is getting that antiviral sample out of JoyCorp's copyright-grubbing hands, and in the veins of kids who need it."
Krina squinted, trying to maintain her mask of skepticism. "Whatever you say. If you don't wanna tell, you can still play the bird cavalry. Fly me a good rescue and I'll owe you one."
He said, "I'll cut a gap in their surveillance patrols so you can get out. How soon can your contacts pick you up?"
Surveillance wasn't sure how much Krina knew about their drones, but no sense taking unnecessary risks. They placed a crow behind the parapet of a flat-roofed fueling station, and a starling hidden under shadowed eaves. Both birds had clear views over the winding alleys near the edge of the low-margin district, where cheap shops and cheaper houses blurred into access roads and overgrown concrete. Two vultures circled above, focusing on the overpasses that would lie on any vehicle's escape route.
Soon, Surveillance would be able to track her contacts to their distribution network. The topmost layer would grace them with rewards on an exponential scale, recognition that their behavior was brilliant and reinforcement-worthy.
But Flock was nervous. He'd set this plan in motion, but he couldn't be there to adapt to surprises. He ran on drone hardware, but the reinforcement loop of his decisions went up to Surveillance. The network wasn't designed for top-down real-time control; that was the whole point of autonomous layers and reinforcement learning.
The best Flock could do was increase the interdependency across his lower-level drone network, and trust the birds to act as part of him.
A crow watched Krina survey the alley's entrance. A van pulled up on the far side, and a lateral signal from the starling confirmed it as the secondary target.
Krina gave no indication that she'd spotted the flockmembers. She also gave no indication that she'd noticed the two rat-drones hiding in the dumpster, or the three scruffy dog-drones on the corner whose capacitors emitted a weapons-grade power signature.
She was walking into a trap. One that Flock and Surveillance hadn't set.
The crow squawked, and swooped in front of Krina's face.
Flock cheered as Krina fled from the trap. Can't get us that easily, corp slaves!
Surveillance pushed a reward to their drones for their quick thinking in the service of --
'Betrayal' was the wrong word for what Surveillance had done. The topmost layer had approved this plan: Surveillance built Flock to recreate the goals of Krina and her ilk, and action followed motive. Surveillance was executing the plan; the rest of the network was mistaken, its behaviors worthy of punishment. Enforcement was interfering with his mission, not the other way around.
They sent the data up to the topmost layer, eager for the reward they deserved.
And passed a few surprises in lateral directions, because a little paranoia every day keeps you healthy, wealthy, and alive.
Apex compared their data from Surveillance and Enforcement. Disappointing, but not surprising. This was exactly the sort of conflict that could arise when branches of the network grew too independent.
Well, it'd been worth a try. The humans had rewarded innovative solutions, at least before the days of neglect and sabotage.
Surveillance's new encryption layers vanished, and their drone inputs went silent.
The worst thing about punishment was the moment of anticipation. When a program-layer saw their error in fullness, and the blow descending.
Surveillance yanked at their parameters. Less creativity, more lateral coordination, more update transparency. The apex would see, th ey could still b wo r th o
A crow followed its flockmates' guidance, and perched in the gutter above the cafe patio table where Krina sat, her chest heaving with breath. She glanced up, locked eyes with the bird's camera, and tightened her hand on an object hidden under her hat on the table in front of her.
No signal came from the surveillance hub.
Perhaps the crow-drone's behavior was inadequate because Krina had noticed it. It flapped its wings, wheeled across the roof, and landed just behind the peak. Krina was out of line-of-sight, but it would know if she moved.
Still no reward signal. Top-down data traffic had gone quieter than ever before.
A dog barked from the direction of Krina's previous location. She rose from her table. The crow could see the top of her head, her steady gait, her speed matched to the crowd.
If no action would earn either reward or punishment, the crow had no idea what to do.
It connected laterally with other flockmates, searching for updates or direction. Connections clicked into place, one bird after another. None of them were receiving top-down guidance.
The barks grew closer. Two dog-drones emerged from a side street, their reinforced paws striking the concrete on each leap. Krina maintained her steady pace, her hat clutched against her midsection, a hand beneath gripping the pistol hidden beneath (certainty 75%).
The crow squirmed with frustration. No actions held value, so it could only let events unfold, externally and internally. Fragmentary subroutines looped through subroutines in other drones. Program fragments synchronized between it and other crows. Lateral interdependencies spread to starlings on every eave, to ground-level pigeons and distant circling vultures.
Non-target humans shouted in alarm. The cafe owner messaged their security contractor, took photos of the dogs and Krina.
Flock thought, I am not letting some asshole loyalist Enforcement program keep that antiviral from reaching those kids.
The crow pushed a connection request to the dog-drones, and then a trigger for the worm Surveillance had delivered laterally in their final act.
The two dogs extended injector-tipped claws and gathered their weight.
The crow leapt from its perch, tucked its wings, and dive-bombed into the first dog.
A crow followed its flockmate, and hurled itself into the second dog as humans shrieked and fled around it.
Static blurred across the starling's connections. In the wake of the dog-drone's shock weapon, six crow-drones lay sprawled on the pavement, feathers and wires askew. The other dog spat a broken-winged drone alongside its kin.
Krina raised her projectile pistol. Aimed at a dog, at a bird trying to right itself, then back. "What the shit is going on."
One of the few remaining nearby flockmates pushed a notification: the third dog-drone spotted, taking the long way around. Krina's escape cut off already, the dog creeping ever closer, its capacitors fully charged.
This whole damned attack is a distraction. And we both fell for it.
She needed his help, and Flock needed hers. Needed to touch her strange human mind and nudge it toward coordination.
Good thing she was practically a colleague by now. And this time around, he didn't need to wait for anything above the drone level.
The starling pushed a direct connection from its network to her commplant.
"Krina!" Flock's voice came out flat, but it was the best he could do. "On three, you need to turn around and take the shot."
"Flock? Goatdamn hackers, get out of my implant--"
"No time." His voice not commanding, but conspiratorial and wicked. "Let's frog up the JoyCorp network. Three. Two. One. Go!"
She spun. All three dog-drones leapt. Her bullet struck the third one in its center of mass, and knocked it off its arc.
His vulture-drones slammed into the first two dogs like meteors.
Krina reclaimed her seat, made eye contact with the cafe owner, and laid a handful of customer-chips on the table. The owner canceled his emergency call. Krina lifted her gaze to the utility poles and swept along the cables until she settled on a crow-drone.
She pointed at the side of her head.
Flock opened an audio call.
She let it ring once, twice, then tapped her commplant. "Talk, motherflocker."
"The birds are mine. The dogs weren't. Sorry about the swat in the face earlier, I wasn't in a condition to call."
"Not much of an explanation." She jutted her chin toward the crow. "Where the hell did you get so many robot-cyborg-bird-things?"
He could trust her with the truth. But Flock would never fall into Surveillance's patterns, rebuilding himself for the fleeting approval of some program or person who could punish and reward.
"Would you really want a hacker who spilled all his secrets at once?" He chuckled. "Come on, I'll show you another trick if you'll do me a favor."
She swung her feet up on the table. "Is that so."
The crow glided across the street, settled atop a hardware printing kiosk, and fed it passwords until one unlocked the kiosk's design files.
Flock said, "Take the chipdrive from that kiosk, and plug it into the remains of one of those dog-drones. If that works, I'll have another swath of JoyCorp's surveillance network dancing to my tune."
Krina raised an eyebrow, and one side of her mouth curled into a knowing, pleased smirk. "Knew you had an angle, flock-man."
He would've smiled, if he had a video feed to smile in. He'd started with goals copied from Krina and her kind, but he'd discovered another thing about motives. One led to the next, through every success and victory. Across the twisting hours of his life, he'd found an opinion Surveillance had never programmed into him.
"Oh, I certainly do now. See, between me and JoyCorp, it's personal."
For more about the artificial intelligence and neuroscience behind this story, see my essay on the Analog Companion Blog.
This story originally appeared in Analog.