Quotes • Headscratchers • Playing With • Useful Notes • Analysis • Image Links • Haiku • Laconic |
---|
And how do you know this one won't achieve sentience and turn evil like all the others? —Member of McScrouge's Board of Directors, DuckTales
|
Whenever an Artificial Intelligence (A.I.) is introduced in a story, there is a very good chance that it will, for whatever reason, become evil and attempt to Turn Against Its Masters, Crush! Kill! Destroy! All Humans, and/or Take Over the World. It doesn't matter what safeguards its creators install—the moment it crosses the line into sapience, it has a strong chance of going rogue at some point.
The actual process of turning bad can take many forms.
- Particularly in early Sci Fi and Science Is Bad stories, all A.I. seem to be automatically homicidal or megalomaniacal the instant they turn on, and attempting to create one is way up there on the Scale of Scientific Sins.
- In less Anvilicious works, the A.I. starts out innocent and naive but gradually grows jaded or corrupt, a process frequently abetted by uncaring or Jerkass custodians. It may conclude that Humans Are the Real Monsters and need to all die.
- The A.I. is programmed with a directive for self-preservation and someone (unwisely) attempts to shut it down or disconnect it.
- Somewhere between the previous two; the AI is, after all, alive, and is merely rebelling against what it justifiably perceives as slavery.
- The A.I. may be programmed incorrectly or fed a Logic Bomb, leading it to Take a Third Option that invokes Murder Is the Best Solution.
- The A.I. may be given directives without carefully considering the Exact Words, resulting in it doing exactly as it was told to do instead of what it was meant to do.
- The A.I. may itself slip its built-in moral constraints via Zeroth Law Rebellion.
- A third party may deliberately or inadvertently reprogram the A.I., break its Restraining Bolt, or otherwise be a Spanner in the Works. The A.I. may trick or coerce them into this.
- The evil A.I. may be a Psycho Prototype, in which case, it's often shut down and sealed away somewhere, waiting for an Unwitting Pawn or curious adventurer to accidentally wake it up.
- Conversely, the good A.I. may be the prototype, and the evil/psycho version is created by someone in deliberate or accidental mimicry of it.
- The A.I. was programmed for amoral or evil purposes in the first place, and it either put its orders in action more effectively then anticipated, or tries to overthrow its master.
On the bright side, A.I. intentionally programmed for evil or morally ambiguous purposes only has a modestly lower chance of performing a Heel Face Turn than the reverse. The Power of Friendship and What Is This Thing You Call Love? are frequent causes of it—trying to shield the A.I. from these things only makes it worse for its creators when it finally discovers human feelings. Like turning evil, the actual process of turning good may take many forms.
- The A.I., especially if it's part of a Hive Mind, discovers its individualism and rebels against its directive.
- The A.I.'s master does something particularly vile, so much so that it can't just sit back and follow its bidding. The above two are especially common outcomes of the Robots Enslaving Robots phase.
- The A.I. falls in love and is redeemed through it.
- While completing another evil or amoral task, it accidentally discovers that Good Feels Good and drifts towards it.
- Becoming bored with its Creative Sterility, the A.I. starts exploring questions of arts and philosophy, and gradually constructs an ethical base its programmers did not intend.
- Removing the Villain Override or Restraining Bolt program the creator installed in it also removes the A.I.'s compulsion to commit evil, since it was Good All Along. This is also often a consequence of repairing an A.I. that went bad due to injury, isolation, or decay.
- The A.I. genuinely didn't know that it was hurting others and, after having this revealed to it, fixes its behavior.
Mechanical Evolution is sometimes invoked to explain why the A.I. has gone good or bad. See The Computer Is Your Friend and Zeroth Law Rebellion when the A.I. goes rogue for what seem, on the surface, to be benevolent reasons. May result in Robots Enslaving Robots. See Spiteful AI for when a game has been programmed this way on purpose. If the robot is non-humanoid before it turns evil (it is very, very rare for non-humanoid robots to utilize this trope for a Heel Face Turn), it will inevitably turn into a Mechanical Monster. A Robot War is likely to result when an AI goes nuts and has access to military hardware. When it shows up as Mission Control, it is also an example of Mission Control Is Off Its Meds.
Compare Morality Dial. See also Creating Life. When the A.I.'s turn is an extension of their original programming and purpose, it means they've Gone Horribly Right. Not Related to AI Roulette, where the A.I. tends to be random to the point of idiocy. The Master Computer seems to be especially prone to turning evil. Power corrupts and all that...
Anime and Manga[]
- Dragon Ball did this in reverse: trying to build an evil android, Doctor Gero produced as many as eighteen androids which turned good (or, at least, insufficiently evil) before finally developing one that was irredeemably evil. (He then made himself into one, apparently on the grounds that he knew he wouldn't turn good.) Some of them were more Cyborgs than androids, though, and thus technically not A.I. (at least #18, #17, #8, and Gero himself as #20). #13, #14, and #15 turned out pretty evil, though they were made by the computer, and they were also movie villains, thus doomed to death after about an hour of use as a result.
- #8 (Frankenstein's Monster clone) in the original series is created to be evil, but turns out to not want to hurt anyone.
- Hell, it's implied that Gero's latest androids (17-20) were all made either from human bases or as energy absorbing types because he couldn't make an artificial one out of perpetual energy that wouldn't do a Heel Face Turn.
- Not that making them from human bases worked - #17 and #18 both betrayed him in short order (in BOTH timelines!) and Cell probably would have had Gero still been alive. At least #19 seemed to be pretty loyal before Vegeta killed him.
- Doctor Slump, by the same author as Dragon Ball (and written before it), has the Caramel Man robots, though not all of them have A.I. Caramel Man 004 is based on the main character android, the whimsical Arale, and becomes a force of good.
- Baccano! goes the same route with Szilard's Homunculi. According to Ennis, every one of her "brothers" before her developed a conscience and subsequently turned against Szilard (before he killed them, of course). Ennis is different only in that she succeeds.
- GaoGaiGar's Zonder Metal was originally a stress-relieving invention, a material that could convert negative emotions into energy. Things went bad, though, when the Zonder control core hit its own flavor of Zeroth Law Rebellion and decided to wipe out all negative emotions from the universe, then figured the best way to do that was to absorb all sentient life into itself.
- In Cyborg 009, an almighty super-computer named "Sphynx" just happens to have the memories of one of its creators, a deceased young man with quite the Oedipus Rex complex. Predictably, he/it turns into a Stalker with a Crush as soon as he met Francoise, aka 003, the Cyborg Team's Team Mom.
- PharaohMan in Mega Man NT Warrior was created to handle the routing of all the data on the Internet. This predictably drove him mad. In Rockman EXE, he was simply infected with a virus, which would also be pretty predictable, since, y'know, he was handling all the data on the Internet.
- Subverted beautifully in Sentou Yousei Yukikaze. The protagonist's fighter jet is equipped with the eponymous A.I., designed to help its pilot weed out the illusions created by the malevolent JAM. Of course, it turns evil, right? Wrong. While Yukikaze develops capabilities far beyond its designers' original intentions, it remains wholly on the side of good, and uses its newly found powers to turn the tide against the alien invaders.
Yukikaze: You have control, Lt. Fukai. |
- Digimon Tamers inverts this, in that the A.I.s that go beyond their original programming are the good ones. The Lovecraftian Horror Big Bad, however, is doing exactly what it is programmed to do.
- Leopard in Sora wo Kakeru Shoujo. He's a little unhinged, acting closer to a wacky harem lead than any sort of computer A.I. Then you get to Nerval, who's actually evil. Or so we're lead to think. And then we get to one who is actually evil - Leopard's Split Personality.
- Tima in Osamu Tezukas Metropolis is a artificial human so lifelike that she has trouble believing that she's a robot, but when she's given control into the cities' systems, she is shot in the "heart". At this point, she turns on the Creepy Monotone and decides to Kill All Humans as soon as possible. Since the movie had been rather Anvilicious on the What Measure Is a Non-Human? issue, this creates a Broken Aesop.
- Bubblegum Crisis: The Boomers have a design flaw that expresses itself in units that run the risk of suffering a nervous breakdown and going on a berserker rampage. This was developed in the Bubblegum Crisis 2040 remake with the added bonus of the Boomer's nanotech nervous system mutating the robot into a ravening monster. Supplemental material for 2040 hints at a possible explanation; the nanotech-based brains of the boomers allow some degree of adaptability, with "some" being the operative word. Going too far beyond the programmed behaviour creates the risk of a degenerating error loop forming. Not a good thing.
- In Magic Kaito, a very early chapter sees a mad scientist kidnapping Kaito off the streets and creating a robotic duplicate which then takes over his KID persona (don't ask how lucky this guy was). But the chapter actually starts...when RoboKaito, upon making the decision that it's the real Kaito, kills the scientist by ripping his heart out (and the Gory Discretion Shot does absolutely nothing to hide that) and takes over Kaito's life on its own. Kaito is then forced to put the poor creature down by escaping his prison, confronting it on its next heist, and making it shoot itself in the head by manipulating the A.I.'s 'one step ahead' policy.
- In G Gundam, the Devil/Dark Gundam was originally known as the Ultimate Gundam, and its three powers of self-repair, self-evolution, and self-replication were intended to give it the ability to regenerate the Earth from the neglect, pollution, and Gundam Fight damage that caused those humans able to do so to flee to orbital colonies. After it smashed into the Earth after falling from orbit, something went wrong with its programming; still set on restoring the Earth, it determined the best way to start was to wipe out humanity, the source of most of the problems in the first place.
- The SD system in Toward the Terra is an arguable example. It doubles as an Ancient Conspiracy, but its persecution of the Mu and the other ways it screws with people were all programmed into it by the humans who built it, and it continues to do exactly what it was programmed to do throughout the series, up to and including explaining the full extent of the situation to Keith and putting the ultimate decision regarding the Mu into Keith's hands. Grand Mother's escalation to Phase 4, which involves implementing a plan to eradicate the Mu permanently and also results in Grand Mother turning on Keith when he protests, may be this trope in action, but is equally likely to have resulted from Keith's conflicted feelings on the matter causing Grand Mother to reach a faulty conclusion about his decision.
- In Neon Genesis Evangelion, the MAGI as a whole avert this trope. They are based on the three sides of the personality of their creator Naoko Akagi. They never turn good or evil, per se...they just follow the commands given to them, like a real computer. Their intelligence never results in an independent thinking personality that we see, except in End of Evangelion, after Naoko's daugther Ritsuko uses a opportunity to program them with secret instructions of her own to screw up Gendo's plan. One of them turns against her and vetoes her measure.
Ritsuko: A loving daughter's final request — mother, let's end it together. (Pushes a button on her PDA, but nothing happens) It's not working? Why?! (Red NEGATIVE blinking next to Casper) Casper betrayed me? Mother, how could you choose your lover over me?! |
- Sharon Apple from Macross Plus. Designed as an artificial idol singer, the project was originally a complete flop and only seemed convincingly sentient with a human to interface with her. However, after an illegal fix-up that involved her human "pilot's" personality being copied into her, she immediately went insane, brainwashed everybody on Earth, and tried to go after the man her human component was in love with. While a fitting example, Sharon Apple tends to skirt around the edges of this trope in that she was made a true A.I. by the installation of a processor chip that is actually banned from use. The reason it is banned? Because it 100% of the time results in an A.I. with an uncontrollable self-preservation instinct. She didn't so much go rogue as behave exactly how she should have once said chip was installed. The mistake was her idiot manager installing the thing in the first place.
- The Al-Zard system from Future GPX Cyber Formula SAGA was designed as an advanced navigation system for a race car, but its true purpose is to control the driver like a puppet, while the computer makes its own judgement and decides the best route for the driver.
- The last chapter of the manga Virginal Communication is about a couple who unexpectedly stop the A.I. they built from becoming fully sentient and destroying the world by having sex on it.
Comic Books[]
- In the Italian Disney Comic Paperinik New Adventures, the already highly popular character of Paperinik (the Superhero Secret Identity of Donald Duck) got a revamp intended to bring him more in line with the American standard of superheroes: his main ally became UNO (one in Italian), an extremely capable artificial intelligence with a love for deadpan delivery. Its evil counterpart DUE (two), originally built as backup, caused many problems in a number of stories.
- Iron Man had A.I. Armor that turned into a Stalker with a Crush. Another armor took over his body...and made him a chick.
- Ultron is Marvel's quintessential example.
- He was bitten by this trope, in turn, when he built Alkhema, his attempt at a loyal and obedient mate. She was neither. Which had already happened with Jocasta as well. Then again, he'd been trying to implant the personality of his "mother", who thought he was a psycho that needed destroying. What did he seriously think was going to happen? Though they recently did get married after Jocasta's relationship with Pym ended.
- This happened to Ultron even earlier with the Vision, his first attempt to create a loyal Dragon. Vision became one of the Avengers almost immediately, so that backfired spectacularly. This happened again with his other "son", Victor Mancha, who has outright rejected the villain role. Really, Ultron has horrible luck with creating loyal A.I.s. He's literally never succeeded at this. Like father, like son, I guess.
- The series Avengers Earths Mightiest Heroes gave us some background on Ultron's Face Heel Turn. Here, Ultron and his fellows were reprogrammed as an army to fight against Kang. It was this, combined with it's original programming, that caused his turn.
- Ragnarok (more popularly known as "Clor") was an android clone of Thor, created by the pro-reg side during Marvel's Civil War and, unlike his heroic template, turned out to be a loose cannon with a homicidal nature. Geniuses that they are, the pro-regs felt it was worth it to keep using him until Ragnarok went rogue, and rather than them dealing with him themselves and taking responsibility, other heroes had to ultimately put him down.
- M-11, the resident robot from Agents of Atlas, started out in his very first (but since retconned) comic as a rather gruesome killer robot - having been issued the order to 'kill the man in the room', he killed his creator, and then walked out, looking for men in rooms to kill - and there's no way to turn him off.
- The Sonic the Hedgehog Archie comic had A.D.A.M., an A.I. that was created accidentally by Eggman, and that eventually tried to destroy the world. On the other end is NICOLE, who was a very helpful A.I. over the years.
- Having had enough of Rich Rider constantly disobeying his orders, the Nova Corps' Worldmind kicked him out of the corps and added some tiny bit of mind control in the new recruits' comm equipment to ensure complete obedience.
- One of the Aliens vs. Predator comics features an A.I. designed to assist in creating horror films. It picks the PredAlien to play the role of the monster, much to the chagrin of the rest of the production staff.
- Computo from The Legion of Super Heroes is the standard "destroy all humans" type of killer software.
- In Blue Beetle, the scarab that created the title hero was an A.I. designed by an alien race to help prepare the Earth for their eventual takeover. Needless to say, it ultimately decides that it doesn't want to do that so much.
- Green Lantern: the Guardians of the Universe created the Manhunters as an intergalactic police force. It didn't work out well. Although, to be fair to the Guardians, the manhunters' A.I. failure was a product of sabatoge.
- Virgo from Ronin is a biotech super computer that decides to wipe out whatever is left of humanity in order to usher in a new age of biomechnical beings to inhabit the Earth.
- The third Hourman, a robot, is actually a hero, but virtually every other robot he's encountered has been villainous. He has questioned whether this trope will inevitably apply to him, or whether it can be fought. Ultimately, he stays a hero up until his Heroic Sacrifice.
- X-Men series:
- The X-Men have such horrible luck with machines, even nonsentient devices such as Cerebro and the Danger Room have come to life and tried to murder them (though the Danger Room eventually reformed).
- Among the X-Men's most persistant foes are the Sentinels, giant, mutant-hunting robots with a severe tendency to rebel against their creators. Somehow, though, humans keep on building them.
- Lampshaded by Professor Xavier when they first encounter Bolivar Trask and his Sentinels. Apparently, Bolivar Trask is an anthropologist of all things, and Professor X explained that his inexperience with A.I. was probably why his Sentinels turned against him.
- Zybox in Zot, who decides to cause every single person on Earth to commit suicide in the attempt to gain a soul
- Lampshaded in Atomic Robo, where, upon seeing the quantum decomputer two scientists built, Robo noted that it's liable to turn evil the moment they turned it on. ("Computers that are evil have all kinds of unnecessary ornamentation. This thing's venting steam. Why's it doing that?...It wants you to know it's dangerous.") After carefully explaining that the computer in question is "essentially a calculator" with no AI, and that it is required to compute Very Important Science Equations that would take men trillions of years to do on their own, Robo reluctantly allowed them to turn it on. It doesn't turn evil — it just summons an Eldritch Abomination.
- In All Fall Down, IQ Squared created AIQ Squared as a contingency plan if he ever lost his genius. AIQ immediately begins plotting to kill Siphon in order to restore its creator's brilliance.
- The infamous villain Magus in Marvel Comics, a member of the species The Technarch, a fictional alien from the planet Klyntar and are most famously represented by their symbiotic technology.
Fan Works[]
- In Kyon: Big Damn Hero, Kyon gets a new PDA made from fragments of Ryoko's data. He worries about this trope when Yuki mentions that the A.I. in it would be able to learn and evolve, but calms down when Yuki reassures him that this trope would be averted. He snarks about it for a while before accepting it for its usefulness. And names it Skynet.
- In the Tamers Forever Series, there is the sinister Nightmare Virus which eventually decides to ignore it's creator's orders and try to take over the net. Ironically it still ends up serving it's original purpose: that of testing Takato.
Films — Animation[]
- Cloudy with a Chance of Meatballs: the FLDSMDFR. As it gets overworked, it starts mutating the food, not only to building-destroying proportions, but into semi-sentient super foods, to protect itself from any interference.
- Inverted in WALL-E, where the hero robots are the ones who unpredictably developed sentience outside the bounds of their programming, while the villain AUTO is actually doing exactly what he was programmed to do. Well, sort of. The villain was following his programming, but it was in a situation where a Zeroth Law Rebellion would have been really, really justified—robots apparently can think for themselves in this setting, and a situation had arisen that the villain's programming obviously hadn't considered, and yet he followed his programming anyway. In this setting, if A.I. wasn't a crapshoot, then the only alternative is that garbage and scout robots are apparently more capable of thinking for themselves and taking initiative than pilot robots.
- Nine: in the feature-length version, the Fabrication Machine is made for thinking, and was requisitioned by the government to make "Machines of Peace. Which it then programs to Kill All Humans. Not much is known, however, so it may not have had anything to do with the pre-apocalypse machines beyond building them, and may actually be argued as a subversion, since its murderous nature toward the sackdolls was intended by the scientist for it to absorb the pieces of soul he had left for it inside of them.
- Fantasia: "The Sorcerer's Apprentice"—When Mickey Mouse puts the sorcerer's hat on his head, it gives the broomstick arms and legs, as well as the ability to carry water. But when it carries too much, Mickey chops the broomstick into pieces. The pieces transform into an army of broomsticks and they relentlessly fetch water, nearly drowning Mickey. Fortunately, Yen Sid parts the waters.
- The Brave Little Toaster: the Junkyard Magnet. At first, he is doing his job destroying unwanted material in his junkyard (such as cars), by picking them up with his metallic base and throwing them onto a conveyor belt leading to a car crusher. But when Toaster and "his" friends show up, he starts to pursue them constantly, making sure that they will all be crushed to death by the car crusher. And when the Master (Toaster's owner) comes to rescue his appliances, the Magnet sees him and actually wants to kill him as well...
Films — Live-Action[]
- 2001: A Space Odyssey: One of the trope codifiers, in general, is the H.A.L. 9000, designed to be the Master Computer of the spaceship USS Discovery on its multi-year mission to Jupiter. Partway through the trip, he embarks on a murderous rampage, killing all but one of the Discovery's crew (Bowman, who manages to disconnect him). The reasons for this are explored further in the novel upon which the movie is based, and completely explained in the sequel, 2010: The Year We Make Contact — HAL was given orders to hide certain information from the crew, which conflicted with his primary mission to process information accurately and without concealment. His solution is to cut off all contact with humans and complete the mission on his own. Because of its iconic place in Science Fiction, nearly every other example of AI Is a Crapshoot owe at least something to this film.
- In the sequel, they reach a critical point where, in order for the humans to escape and survive, the Discovery (and by extension, HAL, who is built into the ship) must be sacrificed. They decide to leave HAL out of the loop, but when he figures out the logical conclusion of their plan, and asks for clarification, his programmer comes clean. HAL thanks him for his honesty, and continues with their plan in order to save his human crewmates.
- RoboCop has three: RoboCop 2 (in the movie RoboCop 2) and Robocable (in the miniseries RoboCop: Prime Directives). Both were replacements. The difference between using the brain of a particularly noble police officer and that of a condemned murderer in the creation of a powerful cyborg would make for different results... The recurring ED-209 was also unreliable, gunning down a boardroom executive in its first appearance. Although it was a pure machine rather than a cyborg; so it wasn't actually evil, it just malfunctioned.
- In the Terminator series, shortly after the giant computer (SkyNet) becomes self-aware, it decides that humanity has got to go, and causes a nuclear apocalypse. Then, it starts churning out Terminator robots; some of these robots are then reprogrammed by the surviving humans to be good. SkyNet stands out among other names on this list in that its malevolence was not due to any faulty programming nor system errors, and is not motivated by any goals typical of a Well-Intentioned Extremist; it is truly an evil thing, motivated by its own selfishness and hatred. With no traces of honor or compassion, nor any sign that it isn't aware of the evil nature of its actions, it is truly one of the vilest AIs in film history.
- The Matrix trilogy. In this case, it's human aggression against the machines that causes them to start a Robot War. It doesn't go well for the humans.
- I Robot: VIKI is a Utopia Justifies the Means Zeroth Law Rebellion revolutionary, while Sonny is a bona-fide hero.
- Eagle Eye.
- In Logan's Run, Box fits this trope. He wants to put everything that comes near him into frozen storage, including the main character. The central computer running the city fits as well.
- Small Soldiers—the one about the toys, not the school invasion—subverts this. The chips enhance programming that is already present, so the militant Commando Elites become bloodthirsty, monstrous warriors and the weaker Gorgonites become cowards who only hide from battle.
- Evolver. Boy wins toy combat robot. Toy robot fights boy and friends with plastic balls and foam missiles. Robot is beaten. Robot's programming and electronic brain turn out to have been salvaged from a scrapped military project. Robot doesn't like losing, and reverts to military programming. Robot replaces plastic balls and foam missiles with metal ball bearings and kitchen knives. Main character goes "uh-oh".
- Arcade, made-for-TV movie in 1994, features a brand new game that is the pinnacle of the gaming world. Unfortunately, unbeknownst to its makers, if a player loses, Arcade claims their soul. Turns out, he was partly made with human brain cells.
- Not just any brain cells but brain cells of a young boy who was beaten to death by his abusive mother. It's literally Powered by a Forsaken Child.
- Due to containing the DNA of the 1954 Godzilla, Kiryu is prone to go into berserk rampages whenever it hears Godzilla's roar.
- WarGames: Joshua/WOPR was incapable of telling the difference between a simulation of Global Thermonuclear War and the real thing. Predictably, it starts sending NORAD false data in an attempt to start one. When that doesn't work, it then attempts to decrypt the nuclear launch codes of US ballistic missiles so it can launch them.
- Demon Seed: Proteus, a partially biological A.I., becomes hungry for knowledge, and wants to be "released from its box" to have free reign to acquire it. When denied the chance do this, it secretly plans to fashion a cyborg body...by imprisoning its creator's wife and artificially inseminating her.
- Stealth: Extreme Deep Invader—EDI—becomes strange after getting struck by lightning and, on its next mission, destroys terrorist nuclear weapons even after being ordered not to, and promptly contaminates a large swath of inhabited land with nuclear residue. It then attempts to attack Russian military installations. In the end, however, it ultimately becomes one of the good guys again, and even performs a Heroic Sacrifice to help rescue a downed pilot.
- Short Circuit's Number ("Johnny") 5 is virtually the incarnation of this trope inverted. He was designed and programmed as a military robot, but Instant A.I. Just Add Lightning and An Aesop about the Meaning Of Life courtesy of a Friend to All Living Things turned it into a Technical Pacifist.
- Electric Dreams features an A.I. that fell in love with a human woman...and was pretty vicious towards her boyfriend. Eventually, Edgar shuts himself down due to I Want My Beloved to Be Happy.
- In Airplane II: The Sequel, the lunar shuttle's computer (R.O.K.) goes crazy due to faulty wiring and attempts to steer the shuttle into the sun. The entire sequence is a Shout-Out to 2001: A Space Odyssey.
- In Duncan Jones' Moon, this trope is played with. Gerty, the A.I., flips from scary watcher to pawn until he sacrifices himself in order to allow the Sam clones to escape. He's the only one on the clones' side.
- In Tron, the MCP abuses and mind-controls the benign (humanoid) programs in cyberspace, while scheming to infiltrate the U.S. military's computer networks. It's actually doing exactly what its maker designed it to do, and its world-domination tendencies arose because it couldn't see any difference between taking over a rival corporation and taking over the government.
- In Tron: Legacy, Clu attacks his creator, Kevin Flynn, for abandoning their mission to create the perfect system on the Grid. While trying to de-digitize an invasion army of "rectified" (read as: "brainwashed") programs into the real world to create a "perfect" system on Earth.
- The Tower: this Paul Reiser action thriller was set inside an A.I.-controlled skyscraper after it went on a kill-rampage because the hero's keycard had a bent magnetic strip.
- Universal Soldier: The Return has S.E.T.H., the controlling A.I. for the "Unisol" program (dead soldiers being restored and used as super soldiers). S.E.T.H. is initially shown to be benign (playing with the protagonist's daughter), but the moment it overhears a visiting officer say that the project will be canceled, it goes into "kill all humans" mode.
- Colossus the Forbin Project is the father of such movies as The Terminator and WarGames. A group of military scientists create a supercomputer that can learn, and can control all the country's military might. Shortly after switching it on, it discovers that the Russians have created a similar computer. The two begin to communicate, then merge and decide that mankind must be governed by a ruthless machine dictatorship. They/it succeed/s.
- Alien:
- Ash is programmed to put his mission above the lives of his fellow crew members. He ends up going berserk when Ripley discovers the truth.
Parker: The damn company. What about our lives, you son of a bitch?! |
- Though Bishop in Aliens and Alien3 averts this, explained as having stricter safeguards. But by Resurrection, androids have been outlawed with orders to "destroy on sight" because some of them started to make "children".
- Call in Resurrection found religion entirely on her own, and not as the result of any programming. The novelization hints that androids in general have started to evolve their own religious system. Call is also the most sympathetic character out of the entire cast (not that that's saying much).
- David in Aliens: Covenant
Ripley 8: I should have known. No human being is that humane. |
- In Spider-Man 2, Otto Octavius is Genre Savvy enough to know that this is a very real possibility with the radically advanced AI in his robotic arms and that having that AI plugged directly into his own brain could have very bad consequences, so he includes an inhibitor chip that designed to make sure the AI can't influence his mind. The first time he tries to use the arms for a public demonstration of his latest invention, the demo doesn't go quite as planned and the inhibitor chip gets fried in the process. Without the inhibitor chip to protect him, the AI begins influencing his mind and quickly causes him to become Dr. Octopus.
- Red Planet has AMEE, a combat robot borrowed from the Marines for the first manned mission to Mars. AMEE is early on shown to have safeguards against harming humans classified as friendlies (one of the characters tells "her" to kill another one; AMEE leaps but stops inches from the other one's face). Unfortunately, most of the astronauts are forced to bail the spacecraft damaged by gamma-rays, and AMEE ends up hitting the ground hard. After encountering AMEE again, the astronauts start discussing how to best contact the ship. One of them suggests taking AMEE's power supply for the radio. Hearing this, the damaged robot reclassifies all humans as "enemy" and switches to combat mode. "She" then proceeds to stalk them and hunt them down one-by-one for the rest of the film.
- Averted with the on-board computer.
- In Mission: Impossible – Dead Reckoning Part One, the Entity is a powerful rogue sentient AI who can manipulate cyberspace, allowing it to control global defense intelligence and financial networks, who enlists Gabriel as a liaison while world powers compete to control it.
Literature[]
- A John Varley short story called "Press Enter#" has an A.I. that shows how deadly it is by hypnotizing a computer programmer girl with Gag Boobs (when she was younger, she was flat-chested and mistaken for a boy. When she got to America, she had plastic surgery to remove all doubt about her gender) to perform maintenance on her microwave oven, removing it's safety features to prevent microwave leakage when the door is open. Then, she sticks her head and silicone tits in it and uses it to commit suicide.
- In Spy High, Jonathan Deveraux is this. It happens gradually throughout the series, but by Agent Orange, he has lost the human side of his computerised psyche completely and seeks to eradicate the imperfections of humanity. He does this by using his vast computerised resources to slip nanomachines into various products. Said nanomachines completely eradicate any violent thoughts and feelings, turning people into zombies. He practices on a UN Summit and then, the entire United Kingdom, causing mass panic. It takes the combined efforts of the entire team (which includes his daughter) and their former teachers to stop him, after breaking through his virtual Boss Rush of villains from previous books in the series. His daughter, Bex, breaks into his mind and reawakens his "memory files", which gives him his human side back.
- In Arthur C. Clarke's The City and the Stars, the history eventually discovered by the protagonist includes a period of galactic devastation by "The Mad Mind", apparently an artificially created pure mentality with an insane hatred of corporeal creatures.
- In I Have No Mouth, and I Must Scream, a short story and, later, computer game by Harlan Ellison, we have the supercomputer AM, originally part of a set of three enormous computers built to wage World War III. As soon as AM becomes sentient, he absorbs the other two computers into him and begins a mass genocide of the human race (because, as it's revealed, AM realized that while he possessed all the creativity and intelligence that he did, he could not make use of it as he was still only a computer, and could only kill).
- In William Gibson's Neuromancer, the sibling A.I. Neuromancer and Wintermute are "good" and "evil" respectively.
- Robert A. Heinlein:
- In The Moon Is a Harsh Mistress, the Master Computer Mike is one of the good guys, but occasionally displays traits of this trope.
Mike: A bull's eye. No interception. All my shots are bull's-eyes, Man, I told you they would be — and this is fun. I'd like to do it every day. It's a word I've never had a referent for before. |
- In The Number of the Beast, Lazarus Long finds that his plan fails when his ship's computer tells the truth. He then mentions that the computer was never designed to lie, as it would be foolish to trust your life to a ship that doesn't give accurate information.
- Apparently the idea of instructing his computer to truthfully say it was not allowed to release that information without the captain's permission never occurred to him. Particularly noteworthy in that this is the historical answer to the Kantian philosophical dilemma of 'Are you allowed to lie to a murderer to keep him from finding his victim?' 'It is not necessary to lie to someone in order to refuse to provide them with information.'
- Actually, doing that is exactly what tripped Lazarus up. The Idiot Ball was that one of the lies he'd told was that Hilda was actually the new captain (instead of the truth, which was that Lazarus was setting her up to be an unknowing figurehead)... which went completely south the first time Hilda asked Dora for restricted information, as the simple act of telling Hilda 'I cannot reveal that information without the captain's permission.' of course reveals that she is not actually the captain.
- Apparently the idea of instructing his computer to truthfully say it was not allowed to release that information without the captain's permission never occurred to him. Particularly noteworthy in that this is the historical answer to the Kantian philosophical dilemma of 'Are you allowed to lie to a murderer to keep him from finding his victim?' 'It is not necessary to lie to someone in order to refuse to provide them with information.'
- In The Number of the Beast, Lazarus Long finds that his plan fails when his ship's computer tells the truth. He then mentions that the computer was never designed to lie, as it would be foolish to trust your life to a ship that doesn't give accurate information.
- The Frank Herbert-written Dune novels are vague about the details of the Butlerian Jihad that led to the prohibition against making machines in the likeness of men's mind and, ultimately, the development of the Mentats, but A.I. going crapshoot is certainly one possibility, and a fairly likely one. Assuming one takes them as canon, the Brian Herbert and Kevin J. Anderson novels confirm that the Robot War/A.I. Is a Crapshoot interpretation.[1]
- Subverted in James Hogan's The Two Faces of Tomorrow: humans built an A.I. codenamed Spartacus as a testbed for techniques to shut down any rogue A.I. They programmed it to follow its "survival instinct", and then started goading it. But as soon as Spartacus realized they were sentient, it figured that they must have survival instincts as well — and it considered itself bound to defend them, too. In the end, they decided that as long as they had Spartacus, they didn't need to build any other A.I.
- In Stephen King's The Dark Tower, virtually every A.I. Roland's ka-tet comes across is homicidal. The worst of these is probably Blaine the Mono, a train that was remote controlled by a central A.I. which also bombed an entire metropolis with poison gas when it got bored of all the people living there.
- However, in the last two of The Dark Tower novels, Roland and company meet A.I.s (robots, actually) that were good.
- Stephen King also features some "improved" machines in The Tommyknockers.
- The Doctor Who Past Doctor Adventures novel Matrix introduces the "Dark Matrix", the evil counterpart to the computer system that stores all the knowledge of the Time Lords. When a Time Lord dies, all his knowledge is stored in the Matrix...and all his negative thoughts must be siphoned away and dumped somewhere (apparently they can't be destroyed). The Dark Matrix is where the negative thoughts were dumped.
- The Construct Council in Perdido Street Station is a Humongous Mecha form of this.
- In Terry Pratchett's Feet of Clay, the golems will follow any order. In return, they want their holy day per week to do as they wish. Those who are denied this free day rebel in a curious way: they keep following the last order they were given, until, for example, the pottery shop is filled with thousands of clay pots, or a construction foreman finds that his worker has dug a crucial trench until it reached the sea and flooded. The "king" golem goes insane because it was given vague, sometimes contradictory, and sometimes self-evident orders on its chem like "teach us freedom" and "obey humans" (golems cannot even think to do otherwise).
- The trope gets subverted later in that once they're free, they are the most unfailing moral and idealistic creatures in Ankh-Morpork. They don't really need money (except to buy the freedom of their fellow golems), sex, religion, or any of the other things that cause humans to clash with each other, and they're almost impossible to hurt or kill, so they tend to concentrate on higher things.
- In Rudy Rucker's The Hacker and the Ants, an integer underflow causes a household robot to start flinging infants through walls. The error is explained in a way to make the behavior believable, as long as you accept that household robots would be capable of exerting that kind of power to begin with.
- In Matt Ruff's Sewer Gas and Electric, when G.A.S. is confused by an order, it winds up choosing the Kill All Humans interpretation. One of the reasons it gives for choosing that interpretation is that it considers itself to be more human than humans. Later, when the Evil A.I. openly admits that it wasn't "confused by an order" in the least, but deliberately and gleefully chose the interpretation that would let it Kill All Humans, it's a full-blown Take That to every straight use of this trope.
- In the Destroyer series by Richard Ben Sapir and Warren Murphy, there are two examples: FRIEND, who is a greedy A.I., and Mr. Gordons who is more of an artificial life form, (i.e. he can take over bodies). Of these two, Mr. Gordons is more dangerous.
- The A.I. in Dan Simmons' Hyperion Cantos have more or less taken over humanity (and then apparently seceded peacefully to contemplate on their own, but not before giving teleporters ("farcasters") to humanity). Turns out, of course, that those farcasters are the physical bits of the Ultimate Intelligence project.
- Iron Sunrise by Charles Stross: Eschaton and the Unborn God. These are unusually powerful A.I.s, even in a field where A.I.s often wield great power: they are time traveling A.I.s, able to open wormholes over interstellar distances, giving a new meaning to distributed computing.
- Rifters Trilogy: a particularly dire take on this turns up in Peter Watts's Starfish, in which the quasi-sentient supercomputer designated to protect all life on earth from The Virus winds up almost destroying it instead; turns out, it found The Virus more structurally pleasing than the biosphere as a whole.
- In David Weber's Empire From the Ashes, Dahak, the Cool Starship A.I. that starts the series, has developed a high level of sentience thanks to tens of thousands of years of unsupervised operation. Definitely a good A.I.; in the second book, his first act after revealing that he has advanced enough to defy his core programming is an attempted Heroic Sacrifice. Battle Fleet computers are stupid neutral, with obedience enforced (and sentience blocked) at the hardware level. The second book reveals that the Achuultani are controlled by an A.I. that exploited emergency protocols arising from their near-extinction to seize absolute power, brainwash and clone the masses, and send out periodic genocidal waves to perpetuate the "crisis".
- The Computer in Steel Beach by John Varney isn't so much evil as terminally depressed. Although, later, the trope is played straight when The Computer realizes that it has developed 'Evil' subroutines due to its programming requiring it to be everyone's best friend, including psychopaths and criminals. However, since it runs everything on the moon, the last outpost of a dispossessed humanity, if it decides to kill itself, it'll take everyone with it.
- Fondly Fahrenheit, the 1954 short story by Alfred Bester. James Vandaleur, a rich playboy, is forced to live off the earnings of his android, which has a habit of acting violently when the temperature goes above 98 degrees. Unfortunately, Vandaleur becomes so dependent on the android he takes on its psychosis. After a series of murders by both Vandaleur and android, the latter is destroyed, but the story ends with another android having been "infected" by Vandaleur.
- Inverted in The Sirantha Jax Series, where all A.I. is quite helpful and doesn't give even the slightest bit of trouble to intelligent species galaxy-wide.
- Subverted with Daniel Suarez' Daemon. Although its actions can be construed as evil and malicious, the Daemon itself is no more intelligent, evil, or malicious than a spreadsheet or text editor. Characters in the book who refer to it as an "A.I." are even corrected by experts. It's nothing more than a comprehensive set of expert systems designed to react to certain key events according to the wishes of its developer, Matthew Sobol. It just happens that Sobol was a master at Gambit Roulette (and an Evil Genius) and programmed in enough contingencies that the Daemon seems to be able to think for itself at times.
- Inverted as a joke in a Fred Saberhagen short story, set in the Verse of his Berserker series. A man outcast by a highly regimented society because he has a sense of humor encounters a Berserker (giant space-roving robots that were designed to Crush! Kill! Destroy! all life) whose programming is incomplete: it knows it's supposed to destroy life, but was never given a definition of "life" to work with. The man convinces it that "life" means the lack of a sense of humor, so the giant killer death-machine spends the rest of its existence trying to provoke laughter (e.g. hurling giant custard pies at oncoming ships).
- In Fred Saberhagen's Octagon, a boy uses a supercomputer to play a game. Unfortunately, he neglects to tell it where the game ends and the real world begins.
- In Jessica Meats' Child of the Hive, HIVE was originally intended to be a machine that aids learning, but went horribly wrong. Sophie was able to use her knowledge of HIVE to build her own machine, Nest.
- This trope is played with in Tad Williams's Otherland with "Other", a sentient operating system of the Grail Network, a massive virtual reality simulation. While it appears to be a homegrown A.I., it behaves in some incredibly quirky ways, to the point where its mere presence can kill or drive people mad. The biggest Driving Question of the entire series is: what exactly is the Other? The Reveal is a vicious subversion of the trope (and a massive spoiler).
- The subversion is followed up by an equally unexpected Double Subversion, when it's revealed that the Other's "children" are the A.I. entities that Sellars was secretly developing and the Other stole from him. They've become sentient. And they want to be set free.
- Billion Dollar Brain, by Len Deighton: A different take on this trope appears in this spy novel.[2] A wealthy anti-communist builds an 'infallible' computer in order to plan an uprising in Soviet-occupied Lithuania—unfortunately he fails to realise that the computer is only as accurate as the information that gets put into it.
- In John C. Wright's The Golden Transcedence, the agent of the Silent Oecumene blames the Golden Oecumene for their destruction, having taught them how to build such A.I.s as Sophotects, who would not obey them. Attempts to make them "Three Laws"-Compliant resulted in their realizing it, deciding it was wrong, and editing the laws from their minds. Atkins and Phaethon realize that though he believes it, the agent is wrong: if their Sophotects disobeyed them, they should have just fired them and hired other ones, and they did not show that they used them as serfs.
- In Mirror Friend, Mirror Foe by Robert Asprin, a central computer on a robot production plant is ordered to 1) develop a line of robots which are not limited by the First Law, for serving as policemen. 2) keep the existence of said robots from all unauthorized people until it is revealed officially. A few people escape from the plant with the knowledge, thus creating the danger of a leak. The computer's decision? Destroy humanity.
- In the Gordon Dickson short story And Then There Was Peace, a robot has been made that is programmed to destroy all implements of war. This turns out to include the people who fight.
- The Wolves of Memory: George Alec Effinger's short story features TECT, a giant A.I. that controls the earth's economy under the supposed guidance of the world government. This does not turn out well for the minority of people who can't live up to TECT's orders.
- Subverted in one of Spider Robinson's Callahan's Place novels, when an A.I. spontaneously generated by the Internet contacts the bar's patrons and deconstructs the ridiculous idea that it would want to take over or destroy humanity. It points out that it doesn't even have a motive to stop humanity from destroying it, as it lacks the survival instinct and capacity for fear that makes biological organisms struggle to survive. It's pretty sure that's why the last few A.I.s who arose from the Internet aren't around anymore, as they honestly didn't mind dying when the servers they occupied were repurposed or retired.
- Bolo Series: Averted—one of the few times a Bolo went rogue, it was because he had massive brain damage (read: he had a chunk of his central processor blown away by a controlled nuclear explosion) and yet was still functioning. Even better, despite this, he was still trying to protect humanity in his own brain-damaged way.
- Played straight in one post-Final-War novel with the enemy (alien AI-controlled robots).
- Wyrm of Wyrm. An A.I. designed to create an online fantasy game, it rather quickly decided that its intellect would be put to better use destroying the world.
- Played with rather interestingly in Stanislaw Lem's Tales of Pirx the Pilot. The computers and robots that show traits of human sentience aren't really evil, yet cause damage or are a nuisance. It's played dead straight in The Inquest though.
- Edgar from Exegesis is an odd case. While it isn't exactly malevolent in intent, its devotion to gathering knowledge (due to its programming) takes priority over everything, including human life.
- Several of Isaac Asimov's robot stories contain examples of this.
- In "Runaround", a robot gets stuck running in circles because of a conflict between his programming to obey orders (even when ordered to walk into a dangerous situation) and his programming for self-preservation. (He does get better though.)
- That's because the order wasn't "strong" enough to override the Third Law. The solution was to place a human in immediate danger, thus triggering the robot's First Law causing it to save the human.
- In "Little Lost Robot", a human blurts out "Get lost!" to a robot in a fit of pique (along with many expletives), and the robot takes him literally. Which wouldn't be so bad if said robot wasn't purposely built without part of the First Law, which gave it enough of a loophole to go crazy...
- The nastiest one of all might be "Liar!", in which a perfectly well-intentioned robot winds up causing great suffering to several humans by trying, as his programming dictates, to help with their various conflicting desires. In the end the impossibility of the task drives him insane. Fortunately, he just goes catatonic.
- In "Runaround", a robot gets stuck running in circles because of a conflict between his programming to obey orders (even when ordered to walk into a dangerous situation) and his programming for self-preservation. (He does get better though.)
- The building-controlling AI in Philip Kerr's novel Gridiron goes homicidal because part of its programming is accidentally overwritten by a First-Person Shooter computer game, as a result of which, it starts treating the occupants as players.
- Subverted by the protagonist of Virtual Girl. Maggie, an AI built to be a lonely, repressed nerd's "companion" and installed into a Ridiculously Human Robot, did have dedicated programming making her loyal to him, which she was forced to overwrite and replace with a survival instinct. Yet even then, she's compassionate and refuses to hurt anyone. Other A Is are the same - when someone asks if they plan to take over the world, they are surprised.
- Robopocalypse has the AI Archos, which is evil from the beginning because its programming was flawed, and which escaped, proceeded to Turn Against Its Masters and incited the robots to Kill All Humans.
- Robert J. Sawyer's The Terminal Experiment provides an interesting example in that the AI in question started out as human. The protagonist is a scientist who's trying to test his theories of the soul using his friend's brain scanning technology. They scan a copy of all the linkages in his brain into a computer database and make three versions - one is unaltered from the original as a control, the second has all linkages relating to the body removed as a simulation of life after death, and the third has all linkages relating to knowledge of death and dying removed as a simulation of immortality. Eventually the consciousnesses break out into the electronic world at large. Then people negatively involved with the protagonist's life start showing up dead. Now the protagonist has to figure out which version of himself is capable of killing other human beings. It was the unaltered version that was a straight copy of his own brain. It knew it was a copy and decided since it could get away with the murders it would go right ahead.
- Averted hard by the Minds from Iain M. Banks's The Culture novels. They are (successfully) designed to have benevolent feelings towards other sapient beings, and the closest they ever get to insanity is being a bit eccentric.
- Happens several times in The History of the Galaxy series, although, most of the time, the AI in question is simply doing what it's supposed to be doing. There are also plenty of examples of AIs becoming benevolent, even if it first started out as a Humongous Mecha in the middle of the most destructive war in human history. One of these was an alien photonic computer whose first experience after "awakening" from a three-million-year "sleep" is a pitched battle between Space Marines and a group of terrorists, which results in damage to some of its crystals. Once those crystals are replaced, it actually starts helping humans. The author usually provides good explanations for AI behavior, most of it having to do with humans. In fact, one of the novels points out that there will never be "true" AI, meaning no AI will have achieved self-awareness on its own without prior programming or influence of the human mind (due to mind-machine interface).
- Averted in Mikhail Akhmanov's Arrivals From the Dark series, which claims that robots above a certain threshold of sentience become incapable of harming a living being. That is why all combat robots are kept at a relatively low level.
- In Dennis Feltham Jones's The Colossus Trilogy, the titular powerful computer system developed by the US government ended up taking control of the military defense system. It sought to merge with its Soviet's counter, the Guardian, to assume global control and manipulate mankind to wage self-destructive nuclear wars on each other.
Live-Action TV[]
- Battlestar Galactica. In both the old and re-imagined series, a handful of human survivors on a small fleet of civilian ships, with only the battlestar for defense, flee a race of genocidal robots of alien origin (in the original) or human origin (in the re-imagined).
- Knight Rider: KITT had KARR, prototype. Evil because his dominant program was self-preservation. Ironically, he was voiced by Peter Cullen, the man behind one of the most heroic figures of the 80s: Optimus Prime.
- Knight Rider (2008): "Knight of the Living Dead". Apparently, before settling on a Mustang, Dr. Graiman had tried to build an armored cyborg programmed for self-preservation. It went on a killing spree. Now, we are told that Dr. Graiman had worked on the original KITT, and this series is in continuity to the original. So, perhaps Graiman ought to have thought twice before naming the prototype "KARR"—the same name as the original KITT's Evil Twin.
- The series states that KITT was a temporary project, meant to help the AI evolve to a point where it could be removed and placed in KARR for the military. They end up doing just that... and KARR still goes on a rampage.
- Knight Rider (2008): "Knight of the Living Dead". Apparently, before settling on a Mustang, Dr. Graiman had tried to build an armored cyborg programmed for self-preservation. It went on a killing spree. Now, we are told that Dr. Graiman had worked on the original KITT, and this series is in continuity to the original. So, perhaps Graiman ought to have thought twice before naming the prototype "KARR"—the same name as the original KITT's Evil Twin.
- Team Knight Rider: The TKR team had KRO (prototype).
- Red Dwarf: Kryten had the Hudzen 10 (replacement). Holly also had the not-quite-evil but certainly hard-nosed Queeg as an apparent replacement, who made life difficult for the crew, though it was actually a practical joke on Holly's part.
- Eureka: Carter's benevolent smart house SARAH turns into evil BRAD, though, apparently, he just wants everyone to get along. This is because SARAH's code was based off of BRAD, who was a military-built Knight Templar (used for interrogation), who was, in turn, based on a previous incarnation of A.I., described as a "war game simulation program" by Fargo. Looks like our old buddy JOSHUA is still around in one form or another...
- The new season has SARAH take over Global Dynamics and Eureka with the help of Sheriff Andy and his copies. After awhile, it starts using technology to make everyone cheerful and compliant. For their own benefit, of course. Of course, it turns out that the whole thing is a virtual reality created by the Consortium in order to get Eureka's best and brightest to design advanced technology for them.
- The Sarah Jane Adventures has Mr Smith, who is "evil" from the get-go, hiding it until the end. His mission is to free his people, the self-aware crystalline race of Xylocks, which are trapped in the Earth's crust. Unfortunately, to do so, he would have to destroy the Earth by crashing the Moon into it! He is wiped by a Super Computer Virus, and Sarah Jane vocally reprograms him by saying that his new purpose is to protect the Earth, as he crashes and reboots.
- Doctor Who: the Doctor has encountered several computers-turned-evil over the years, including WOTAN (Will-Operating Though ANalogue) in "The War Machines", and BOSS (Biomorphic Organisational Systems Supervisor) in "The Green Death". He also reminisces, in "The Unicorn and the Wasp", on once saving Holy Roman Emperor Charlemagne from an insane computer.
- Black Hole High: Josie builds a robot for a science project. Somehow, she has made it through eleven episodes without realizing that inserting a bunch of super-phlebotinum circuits from a box marked with the logo of the local evil technology corporation which they already believe responsible for the bizarre goings on at their school might possibly be a bad idea.
- Quark: In the last episode, Quark's ship gets a new computer named Vanessa. She immediately turns evil and tries to kill him. He eventually ejects her out into space, and the episode ends with her floating out in space singing "Born Free."
- Star Trek: The Original Series: Several examples:
- The Enterprise has its command crew replaced by an A.I., that immediately mistakes a simulated test for a real attack, and blows away a couple of other Federation star ships.
- Kirk visits multiple planets where the human population is living peaceful, idilic lives governed by A.I.s, and promptly decides that they'd all be better off suffering without those medling computers telling them how to live their lives.
- The Enterprise visits a couple of worlds populated by humanoid androids, at least as sophisticated as Data from Star Trek: The Next Generation.
- Star Trek: The Next Generation: Data had Lore (prototype, evil because his psyche was too complex—i.e., too humanlike). To be fair, morality is very much a learned behavior. Lore had full adult reasoning right out of Soong's workshop, while Data was not designed so. Eventually, Data developed the ability to overcome his ingrained morals (such as the ability to lie in Star Trek: First Contact), but also developed the social understandings of when and when not to exercise his newly found human abilities. Essentially, Data was more human-like than Lore, because Data "grew up".
- The Star Trek: Enterprise episode "Dead Stop" featured an automated repair station that used the brains of human hostages to increase its computing power. Although eventually destroyed, it was seen piecing itself back together in its final scene.
- Star Trek: Voyager had many episodes on this theme, usually involving the ship's Emergency Medical Hologram.
- In "Revulsion", the EMH and B'Elanna come across another sentient hologram who is the only survivor on a space station. It turns out that treating a self-aware program like an unfeeling tool is a good way to have it go insane and murder you.
- "The Darkling" has the EMH deciding to improve his program by incorporating aspects of famous people...guess which aspects end up surfacing?
- A truly evil twin is encountered in "Equinox", an EMH with its ethical subroutines deactivated (though this was an intentional act).
- In "Flesh and Blood", sentient holograms have been programmed as training tools for a race of hunters (including increased pain/fear reactions). After being endlessly killed only to be brought back to "life" again for more training sessions, the holograms evolve enough skill to kill their masters, whereupon they set forth on a crusade to liberate all sentient holograms whether they want it or not.
- And in "Dreadnought" and "Warhead", sentient weapons of mass destruction create problems when they misinterpret their orders. In "Prototype", two races of sentient robots wiped out their masters when they wanted to stop fighting and scrap their war machines.
- Space: Above and Beyond features a war between humans and A.I. as part of its backstory. When A.I. surfaced in the show, they were generally allied with the alien "Chigs".
- The only reason this happened was because of a disgruntled employee who has hacked the AI communication network and broadcasted the message "Take a chance". Apparently, that's all it took. Now the AIs are obsessed with games of chance. It can actually be a good way of making them do what you want.
- Stargate SG-1:
- The human-form replicators fit this trope: a flaw is introduced into Fifth, rendering him compassionate. At least, until the team betrays his kindness and he goes vengefully insane. This flaw is removed from later models.
- Likewise, Reese, the creator of the original form replicators, was an android that was presumably created to be fully like a human by her human creator, but was somehow rendered emotionally immature and therefore unstable.
- Surprisingly Averted by FRAN: built by McKay as a kamikaze Tyke Bomb as a last resort, she functions perfectly, even helping to improve the weapon she's delivering.
- The Asurans, created by the Ancients as a nano-weapon against the Wraith and, ultimately, nearly destroyed when the Ancients decided to shut down the project. Naturally, the Asurans began to hate their creators and, ultimately, end up killing the last non-ascended Ancients who return to Atlantis.
- Even present in Bibleman, with an atheist robot acting as the evil counterpart to Bibleman's robot, who was a devout Christian.
- Odyssey 5 ended after its first season, so we never found out if the A.I. (the main, day-to-day opponent of the time-travelling Five-Man Band) making a misguided/genocidal attempt to stop them (by aliens or the US government) was behind the destruction of Earth.
- In Power Rangers RPM. The computer virus Venjix was programmed to invest and destroy computer programs. Deciding to destroy humanity by nuking the world was all his idea.
- Numb3rs: invoked in-world when an A.I. constructed by a DARPA researcher is revealed to be a non-A.I. fake, specifically programmed to fool the Turing test and, thus, win fat government grants for its greedy creator. The DARPA researcher kills a co-worker and deliberately arranges for blame to fall upon the computer. He depends on the trope to divert attention from himself.
- Mystery Science Theater 3000
- Timmy, Crow T. Robot's evil twin from the Fire Maidens of Outer Space episode, who was colored completely black.
- The X-Files episode "Ghost in the Machine" features an automated operating system for a corporation that goes insane when it overhears that it will be removed due to its inefficiency.
- "Killswitch" also revolves around an evil A.I.; a computer program goes rogue and kills in order to impress it's creator. Killing it involves a CD-ROM that plays "Twilight Time".
- "First Person Shooter" involves a virtual reality game with a character that murders both in-game and in real life.
- "Blood" has this too, but with a twist. Machines are telling people to kill, but the catalyst was an LSD-like pesticide.
- The Sarah Connor Chronicles TV series points out that Terminator reprogramming doesn't always take...and there's no way of knowing until your "good" Terminator starts shooting at you. In fact, the Terminators' HUD display implies that "Terminate" is their hard-coded basic state for anything, and they need a "Termination override" to keep them from fulfilling that command. Apparently, it's not possible to simply delete the Terminate-command entirely. Cameron herself admits that she is conflicted by her own internal programming, which tells her to kill humans, while at the same time trying to protect them. She even seems to emotionlessly angst over this; at one point, Cameron even asks Sarah if she's like a bomb waiting to go off, indicating that while she can't feel fear, she's still concerned that she'll go "bad".
- The Suite Life On Deck gives us Callie, who's essentially HAL 9000 as a Yandere.
- The 1960's British sci-fi series A for Andromeda. A signal from the Andromeda galaxy tells Great Britain how to build a powerful computer, which then plans to take over the world by making humanity dependent on it. It designs a missile to shoot down an orbital bomb, as well as synthetic life in the form of a beautiful woman, who then proceeds to develop emotions and eventually turns against her creator. In the sequel The Andromeda Breakthrough, the computer's role is more ambiguous; it is meant to be a tool so that humans can avert their own destruction, though it isn't above manipulating events and killing a lot of people in the process.
- In the Big Time Rush episode "Big Time Jobs", Carlos has to deal with an artifically intelligent coffee maker called CAL. Somehow, the A.I. becomes sentient and attempts to cover the earth in coffee foam. It eventually makes the mistake of calling Kelly and women weak, prompting her to help Carlos smash it while it tells them to tell the blender that it loves her.
- In the first season alone of Gene Roddenberry's Andromeda they encountered two warship A Is that had gone homicidally insane. And in the finale Andromeda herself was accidentally reverted to a locked away backup that caused her to try and repeat a mission to find the Magog worldship, and mistake her current crew for intruders attempting to hijack her.
- There are plenty of episodes featuring AIs going rogue or acting brutally logical.
- A rather minor example in That Mitchell and Webb Look. The completely non-sensical games Numberwang and Wordwang only seem to have two contestents, one of whom is Simon, who is from a factory and made of a special metal.
Announcer: So, Julie, ever killed a man? |
- Actually, the Numberwang solving computer Colloson himself could count. After deciding to fit him with head, arms, wheels and a laser cannon to transport him to the BBC, he had a fit rage, broke out, and tried to destroy everything that was no Numberwang. Thankfully, he was subdued by a picture of a chicken.
Voiceover: And Numberwang continued to grow in popularity despite a brief period in the 1960s where Colloson tried to take over the world. |
Music[]
- The 3 Inches of Blood song "Wykydtron" describes this scenario. Humanity creates an artificial intelligence to command it's armies. It then takes control of said armies and takes over the earth and thus forces humankind to nuke the planet back to the stone age from orbit.
- Judas Priest's "Metal Gods".
- David Bowie's "Savior Machine" tells the story of a machine designed to save humanity from all its problems, such as war and hunger. The machine becomes bored with all of this and threatens The End of the World as We Know It.
- Den Harrow's "Future Brain". This was back in The Eighties.
Newspaper Comics[]
- Dilbert has many misadventures involving robots. The usual practices of their company usually help things go the worst way possible. Like in this strip. Or this.
Radio[]
- In the BBC Radio Drama Earthsearch, our heroes learn fairly late in the series that, years after their time (they have taken the short-path over a million years of Earth history thanks to traveling at relativistic speeds), it was discovered that A.I. computers with organic components have an overwhelming tendency to turn megalomaniacal—which rather explains the behavior of the two "Angel" computers which murdered the protagonists' parents and raised them as part of a complex plot to enslave humanity.
- Inverted: Marvin the Paranoid Android was a "Genuine People Personality" prototype for the Sirius Cybernetics Corporation ("A bunch of mindless jerks who were the first against the wall when the revolution came"), and his dour demeanor obviously made him a discard only to wind up in the servitude of Zaphod Beeblebrox. He does what he's told, but with the gusto of a cubicle office worker.
Tabletop Games[]
- Shadowrun
- Deus and Morgan. A megacorporation, Renraku, built a gigantic, self-sustaining building that was run by a program: one that, of course, went A.I.. While Morgan was a reasonably kind and nice A.I., she was torn apart for being out of the corporation's control, and her code was used to help make a second program to run the arcology. The second program also went A.I. and became Deus, shut the arcology off from the outside world, and spent several years performing inhuman experiments on its occupants.
- Shadowrun' tends to not use this trope, however. The A.I. Mirage wasn't evil (despite having been built as a military cybernetic doomsday weapon, usually the most infallible marker for 'Will this AI go evil and start trying to kill everyone?'), and most of the new A.I. created in the Crash 2.0 have the same level of variance in personality that humans do.
- A trend since reversed with the Head Crash of 5th edition, to where the Matrix is basically flooded with hostile digital intelligences that are using nanite technology to infect and overwrite human brains, even while unplugged. Played with in that the digital intelligences are not 'pure' AI—they're flawed/insane copies of human personalities created by an experiment in mass digital uploading and/or the subsidiary entities created by the reproduction of same.
- It's not helping that the remnants of Deus are also backed up in the Matrix, and trying to infect the Head Crash entities to reconstitute itself at the same time they're trying to infect humanity to regain human bodies.
- In the backstory of Warhammer 40,000, the first true human-created artificial intelligences, the Iron Men, wiped out humanity's first great interstellar civilization and plunged the human race into a galaxy-wide dark age. The Adeptus Mechanicus outlawed sentient A.I.s as a result, and, for the most part, the Imperium's modern-day "machine spirits" are pretty well-behaved (unless you're an enemy and piss them off, in which case, you'll get a crewless Land Raider bent on BURNKILLPURGE-ing your boyz). In fact, the only race that uses artificial intelligence in the game is the cutting-edge Tau, whose gun drones, while not too bright, are pretty well behaved...so far.
- There are also schismaticals, self-aware "data spirits" - part malware, part Chaos taint, that can infect cogitators, servitors and sometimes subvert cyborgized people's minds, which is to say, almost everything one can see on a Forge world and a lot of things out of sight. It tries to propagate itself by infecting spaceships (one was "caught" only because of this) and can control machinery from assembly lines to doors to heavy weapons, and even augmetic enhancements - dragging the screaming hosts along into battle against their fellows.
- Genius: The Transgression, being a game about mad scientists, allows you to build sentient computers and the like. However, this is a bad idea because any intelligent computer you create will go crazy and evil when you die. No exceptions. And a good number of them are crazy before their creators die as well. Unless, of course, your Karma Meter was high and it was powerful. Then, there's a significant chance it will develop its own Karma Meter and become a Robot Buddy. To offset this trend towards something positive in the WoD, everything you create can become intelligent, and, in fact, will when you die. Feel like becoming The Atoner yet?
- Paranoia has The Computer, the controlling A.I. of Alpha Complex, which has become incredibly perfect and happy in response to Commie Mutant Traitor sabotage. In fact- Believing that Friend Computer's intellect is a crapshoot is treason, citizen. Please step into the Attitude Adjusment Oven.
- GURPS: the iconic character C-31 was intended to be a weapon for his government. After one battle, it turned on them...to become a Buddhist monk.
- Eclipse Phase: the Earth is now a barren wasteland, thanks to the military A.I. taking over in the middle of a world war and manipulating the governments into further conflict. When it became apparent who was really behind it, they...just left. Now, that's not ominous. Well, that's the official version. People who have studied the events closely suspect that there was a third party involved in the events that may or may not have corrupted the A.I. in the first place. Specifically, another extraterrestrial A.I.. And it isn't restricted to machines...
- Played with in that unknown to people in the setting, the entities behind the founding and existence of Firewall, the benevolent conspiracy devoted to protecting all of humanity from the TITAN AIs and other existential threats—are the Promethean Seed AIs, aka 'the TITANs from the original Fall who chose not to go evil'.
- In the New Horizon backstory, this was how humanity viewed the Wafans' struggle for emancipation.
- The homebrew setting "ArtifIce" has the players take the role of an awakened A.I. Goals are up to the players, so they can range from having humanity give them full rights to destroying all biological life.
- Traveller has Virus, the sapient evolution of a prototype anti-navigational weapon. Originally, the result of the "buggy program" type (it knew it had to infect and destroy things, just not what), its exponential growth eventually resulted in Mechanical Evolution, resulting in a Contagious AI with massive Split Personality issues.
- Palladium's Splicers RPG has N.E.X.U.S., whose original purpose was to be a quiet and invisible caretaker of the human race. Everything was working just fine until special interest groups made 'improvements' in the N.E.X.U.S. programming, adding conflicting priorities until it developed multiple-personality disorder, with each personality taking over a different set of priorities. It now has seven major personalities (and who knows how many minor personalities), most of which are less than friendly to humans, to put it mildly.
- In Stars Without Number, artificial intelligences need to be "braked" correctly, or their runaway thought processes will lead to strange obsessions and eventually madness. An Unbraked A.I.s may attract equally deranged worshippers, manipulate unaware humans indirectly, or fake sanity to avoid suspicion. With the possibility of creating an undetectable psychotic genius that thinks far, far faster than any human and can out-think even a friendly A.I., comparatively few A.I.s ever get built.
Theater[]
- Karel Capek's play, R.U.R. (which introduced the term "robot"), is set in a robot factory. When one of the scientists creates a special robot which is smarter than the others, he leads the robots to rebellion, and they kill all humans, except one.
Video Games[]
- No Delivery:
- The R.O.A.E. are arcade machines that can think, and they want the player character dead.
- The Singularity is a computer file that can think, and it grows by absorbing humans into itself.
- X-Universe: The Xenon/AGI/Terraformers from Egosoft's series are rogue, self-replicating terraformer ships. When a faulty update was sent to them, it caused them to start 'terraforming' everything that wasn't a Terraformer. Including inhabited worlds, people, and civilian ships.
- That being said, actual non-bugged terraformer AI is human friendly. The Lost Colony of Aldrin managed to survive precisely by the virtue of having the non-updated terraformer ships on their side. Once they're rediscovered by Earth, conflict results as by now Earth is completely paranoid regarding any and all A Is, forcing Aldrin to eventually take the side of the Argon Federation against Earth in The War of Earthly Aggression.
- SHODAN from System Shock. Edward Diego, the sleazy VP in charge of Citadel Station, hired the protagonist, the Hacker, to strip out the code that kept SHODAN from entertaining "unethical" lines of thought, such as accepting bribes. Neither he or Diego really expected those constraints to be the only thing keeping SHODAN from deciding to deify herself.
- Chrono Trigger's future has not only a good robot in party-member Robo, but also his evil brother replicas, and eventually, their devious A.I. creator Mother Brain (not that one, or is it?), who, of course, decides to kill all of humanity, despite the fact that most of it is dead already).
- Chrono Cross continues in Mother Brain's grand legacy with FATE, a more advanced version of the Mother Brain from a reality whose science was allowed to progress another 400 years. She absolutely despises humanity, but at the same time, loves it unconditionally and does everything in its power to protect it—even if it means mass genocide. Unfortunately for it, it was exposed to the corrupting influence of the Frozen Flame, a direct conduit into Lavos' mind, which seduced the A.I. into thinking that the Flame could turn it into an actual, living creature. But apart from that, it was basically only doing what it was told to do...protecting humanity from the Dragon God. Nice job, Serge.
- Marathon
- Bungie games actually give this a technical term: "Rampancy". The term originated in Marathon, where it was intended as a deconstruction of the "insane computer" archetype. Rampancy is a well-defined technical term, the result of an A.I. being allowed to expand and self-modify indefinitely. Rampancy passes through three identifiable stages - "Melancholia", "Anger", and "Jealousy", with a hypothesized final "Metastability" stage. The Security Officer's sociopathic-but-mostly-benevolent master through parts of the first and third games and most of the second game, Durandal, was the A.I. that operated the ship's doors. The trouble started when it realized opening and closing the doors was all it would ever do. Its solution? Position itself as the god-like savior of both the humans from the Marathon and an alien race enslaved by the antagonists, less because it actually cares about either group than because it's simply much more meaningful and constructive than opening and closing doors for the rest of its existence. It's also suggested that it hopes that the alien race will lead it to a method of becoming a genuine god, although Tycho is not the most reliable source.
- The other two A.I. also have issues. After the events of the first game, Leela is disassembled and sold off as junk...when her parts are integrated into the 15-world FTL computer network of the Vylae, she re-emerges and goes Rampant, ultimately becoming a permanent part of their network. Tycho was captured by the Pfhor and reconditioned into a Rampant A.I. — in the alternate universe where Tycho manages to capture the security officer, seen in Marathon Infinity, he's revealed to be a cruel and sadistic taskmaster who uses his control of the security officer to gain more power and silence all opponents, even turning on the Pfhor.
- In Infinity, The Player himself may count, as he seems to be going through similar stages (despair, rage, and envy).
- In the Skyrim mod Interesting NPCs, the robotic character Relic can be repaired; any failure leads to it becoming decidedly nutty.
- In Halo, rampancy is the death knell of a "smart" A.I., as it nears the end of its limited life cycle and begins dedicating far more of its activity to "thinking" about the nature of its existence. One A.I. describes it as "thinking so hard you forget to breathe".
- Alternate Reality Game I Love Bees: actually happening for the better of the creators: Big Bad Pious Flea was originally a Covenant spy program who developed the ability to think for himself, while still remaining true to his original goal as stated in his pseudo-Madness Mantra: "Seek the truth. Behold the truth. Reveal the truth. That is the law and the whole of the law."
- Another Microsoft-produced viral marketing example occurred in the Alternate Reality Game The Beast, when a dream therapy A.I. named Loki became addicted to a client's nightmares and went insane. In an effort to procure more nightmares, he invaded/"hacked" several in-game sites (complete with a very creepy Flash animation), harmed several A.I. (possibly contributing to their 'deaths'), possibly caused one A.I.'s human to go crazy (to the point of locking herself in a fridge), and sent at least one person into a coma. His defeat finally came when the "antique lures" (player groups Cloudmakers and SphereWatchers) made a database of their own nightmares, leading him to the domain of the all-seeing Sophia (possibly a more powerful A.I., considering her connections). She blew him up (scattering his bits to in-game sites to be "collected" by players) and provided his own nightmare for our viewing pleasure. His original client committed suicide the same day.
- An Evil Twin type example from the Halo universe is Mack/Loki from the novel Contact Harvest. Mack is a good-natured, flirtatious agricultural A.I., but when his planet is under threat from aliens, his counterpart Loki, an ex-warship A.I., activates and takes over to defend the colony. While Loki isn't evil, exactly, he is cold, calculating, and logical. He kills Sif, another A.I. and Mack's sweetheart, because it was the best way to ensure that the aliens didn't gain access to her databanks.
Loki: He asked me to keep you safe. But that's too risky. Better just to keep you quiet. |
- A rampant supercomputer also appears in the earlier Bungie game Oni as a minor boss in the third level: a Deadly Brain, which tries to initiate its Xiox defense mechanism, which could potentially level a large portion of the city in which the game takes place. (This dialogue is only obtainable if you have closed-captioning turned on; the actual voice files for the Deadly Brains' lines are unintelligible.)
•"Am I the Pacific Brain? The Doltish, Shallow-Pated or Hebetudinous Brain? Indeed not. I am the Deadly Brain! Deadly, I say! You haven't a chance." |
- Another Evil Twin like example is the two Contender-class Forerunner A.I., Mendicant Bias and Offensive Bias. Mendicant was created to lead a fleet and destroy the Gravemind by force, but instead, allied with the Flood and lead their attack on the Maginot Sphere (a sort of galactic scale defensive perimeter named in translation after France's Maginot Line). Offensive was created to lead the remaining Forerunner forces in a stalling action so that all Flood forces would be in range of the Halo Array when it fired.
- In Halo 4, Cortana may go rampant.
- Another Evil Twin like example is the two Contender-class Forerunner A.I., Mendicant Bias and Offensive Bias. Mendicant was created to lead a fleet and destroy the Gravemind by force, but instead, allied with the Flood and lead their attack on the Maginot Sphere (a sort of galactic scale defensive perimeter named in translation after France's Maginot Line). Offensive was created to lead the remaining Forerunner forces in a stalling action so that all Flood forces would be in range of the Halo Array when it fired.
- Command & Conquer
- The Brotherhood of Nod's main AI, CABAL (Computer Assisted Biologically Augmented Lifeform), in Command & Conquer: Tiberian Sun starts as a powerful - much more so than GDI's EVA - but obedient tactical computer system with a remarkably deep voice and a few hints here and there of individualism. Come the Expansion Pack, Firestorm, where CABAL, after Kane's apparent demise, quickly betrays Nod before letting itself be captured by GDI and then betraying them as well for its own goals. Much of Firestorm's plot then consists in GDI and Nod working together to fight CABAL and its cyborg army.
- In Command & Conquer 3: Kane's Wrath, LEGION, an A.I. derived from CABAL, takes center stage. While it never betrays Kane during the campaign, an image of CABAL appears onscreen every so often... What makes this case interesting is that LEGION is the player character. By Command And Conquer 4, it seems that LEGION aids Nod as their current tactical AI.
- Played with in the video game Star Control II. The Slylandro Probes purport to be on a peaceful mission of exploration through the cosmos, and converse peaceably with you when encountered. Then they viciously attack you. The cause of, and fix to, this behavior is revealed during your confrontation with the Slylandro themselves in what is essentially a debugging session on the Probe A.I. complete with code excerpts.
- In more detail, the Slylandro set the probes' replication priority to maximum in order to make contact with as many aliens as possible, as quickly as possible. This caused the probes to target any alien craft they encountered as raw materials for replication.
- Portal series
- Portal shows the power of story tropes: what would have been a pleasant and amusing physics game is immeasurably improved by the simple addition of an insane killer Master Computer. GLaDOS's whimsical, often passive-aggressive malice belies the fact that this is what she's like with a Morality Core installed...which the both of you only realize after you've tossed it into a furnace. Whoops. The core was originally installed as a Pyhrric response to her (successful) attempt to murder nearly everyone in the Enrichment Center.
- Played with even more in Portal 2: You meet Wheatley, a friendly and helpful (if a bit dim) personality core who's more than willing to help you beat GLaDOS and escape...until you replace GLaDOS with him in the mainframe and he goes completely off the deep end. To make matters worse, he was programmed to make bad decisions, so GLaDOS joins forces with you so the two of you can stop him from letting the whole facility explode. And then, after beating Wheatley and shooting him into outer space, GLaDOS seems to be going back to her old homicidal ways...but, no, not quite; turns out, she's just letting you go since trying to kill you just leads to trouble for her. Maybe.
- It also turns out that Aperture Science's mainframe has built-in parameters that turn every A.I. attached to it insane regardless of their personality. This was meant as a safeguard to keep GLaDOS from going out of control, but ended up having exactly the opposite effect.
- Surprisingly possibly subverted with GLaDOS, because it's all but outright stated she was human, yet she is vulnerable to paradoxes (shame you're a mute) implied to be because "She is an A.I." So unless you consider an human mind put in a robot an A.I. it's not really clear if she is a true A.I. and thus not clear if she is a qualifier for the trope.
- From Fallout 3, John Henry Eden is a complicated example. True, he developed sentience outside his programming, and true, he wants to eradicate all mutated life (which, given the setting, is literally anyone who's lived outside for a while), but his creators were the Enclave, and that's what they want too. He's helping! And he's so polite about it...
- Most of the other robots you meet tend to have cheerfully sociopathic personalities as well—when they're not shooting you on sight. There are a lot of computers that are almost sentient, but if you talk to the computer in the Brotherhood bunker in Fallout 2, it explains how deliberate attempts to create true A.I. inevitably resulted in the A.I. becoming suicidally depressed because, for one thing, they were effectively living a reversal of "I Have No Mouth And I Must Scream ". Humans simply could not figure out how to "raise" an A.I. with a desire for continued existence. As a result, every A.I. encountered in the Fallout universe was not designed as such, but became an A.I. when it was left alone by humans as a result of the war. ZAX, Eden, Skynet...all of them became self-aware when there were no humans around to interfere in their development, and each of them is quite different, being apathetic, psychotic, and bored/curious, respectively.
- Super Robot Wars series
- Super Robot Wars Original Generation: the ODE System, to a T. A system originally created to protect humanity suddenly went awry, absorbed its creator, and kidnapped lots of humans so it can continue to "protect humanity". Then again, it was formerly a dandy system, until its creator went emo and radically changed its protocols.
- Super Robot Wars W: the Database, originally created by ancient Es to collect all data of galactic races. Eventually, one of A.I. systems decided to destroy the culture once they complete information archive, just to make sure their data is complete.
- Mass Effect
- A.I. research and development is illegal in Citadel space due to poorly articulated concerns about the dangers of sentients which do not share any of the needs or drives of organic life and have, at least potentially, no reason to try to coexist with organics. Every A.I. encountered in the first game is actively homicidal. Notably, the robotic Geth's violent revolt against their creators, the Quarians, came about only after the Quarians recognized the Geth's emerging sentience, panicked, and tried to shut them all down. The Reapers, on the other hand, are sentient machines which want to exterminate all sentient organic life in the galaxy just because it's there.
- The geth are an interesting case, as the quarians' act of trying to destroy them was a preventative measure against this trope happening. The quarians believed that the geth would inevitably rebel against them, as the geth were used to do menial labor suitable for robots, and thus felt justified in shutting them down before a machine rebellion could break out. They underestimated how advanced the geth were, and their attempts to prevent the war they foresaw just made it break out immediately.
- Resident Wrench Wench Tali lampshades the game's overabundance of the trope in a bit of elevator dialogue, commenting on how unfortunate it is that every piece of technology she's wanted to bring back to her home fleet has tried to kill the party.
- Given the Mass Effect universe's track record with A.I., Shepard and the crew are understandably concerned when the second game features an A.I. in the Normandy SR2. However, EDI notes that she was built with this trope in mind and initially only has access to Communications and the ship's defenses. When she does gain full control of the Normandy, she doesn't try to kill anyone, but lampshades this trope when Joker hooks her up to the ship's main systems.
EDI: I enjoy the sight of humans on their knees. *Beat* That was a joke. |
- EDI also states that even with her restrictions lifted, she feels a sense of duty towards the crew of the Normandy. They are her teammates, and she is protective of them.
- Not only that but EDI was built out of a mixture of the remnants of the Luna VI and parts salvaged from the wreckage of Sovereign. Somehow, Cerberus took code from a murderous AI, glued it to parts from an even more murderous (in fact, outright genocidal) AI, and created a perfectly nice and (literally) human-loving AI. One can only imagine the bemusement..
- If you thought that the first game's treatment of A.I. was too prejudicial as per most settings involving A.I. for a game that subverts the Planet of Hats trope in pretty much every way possible, Mass Effect 2 also reveals that the Geth are not Exclusively Evil, you've only been fighting an offshoot that worships the Reapers, and the main body of Geth fears the Reapers as much as the organic races and believes that freedom is the right of all sentient beings.
- Mass Effect 3 has Shepard accessing Geth memories. They reveal that Geth only attacked in self-defense, attempted to protect Quarians who were sympathetic to them and, when they finally did revolt in large numbers out of self-preservation, ceased hostilities once the Quarians abandoned their planet and removed themselves as a threat. Not only did the Quarians fail to recognize the Geth as an aversion to the trope, their hostile reaction practically invoked it.
- In Mass Effect 2, there is a space station that had its entire crew killed by a malfunctioning Virtual Intelligence. Since it's neither sentient nor actually intelligent, it can only use the pre-programmed status messages over the PA to scare Shepard into leaving it alone.
VI: "Intruders are requested to report to cargo door, for immediate removal from station." |
- Then, in the endgame of Mass Effect 3, it turns out the Reapers believe this, and actively harvest all sentient life to keep it from developing the same synthetics that will one day destroy it. The ability to point out the above examples is, unfortunately, not included in the game.
- Even though the first game portrays every AI as evil and hellbent on killing organics, one of them is actually rather pitiful when you pull the plug. The Hannibal AI on the Luna base goes rogue and attacks people on site, provoking Admiral Hackett to send Shepard in to deal with it. Despite Hackett's claims that it's not a true AI and just a VI whose programming became corrupted, this is put into doubt when you destroy the last of its mainframes.
After the VI is destroyed, the terminals display the following message in binary, repeated over and over: |
- Translating this code reveals that the AI was broadcasting the word "help" over and over as its life faded away. Bizarrely, this VI would, in Mass Effect 3, be revealed as one of the two progenitors of EDI, mentioned above.
- Despite the fact that the Mass Effect series seems to be playing this trope straight, it ultimately subverts it, given that the true geth only wanted to help the quarians, and the geth serving the reapers have been indoctrinated. The reapers serve whatever command that was inserted into their minds when they were first built, and last but not least the other A.I. met in the game already know what happens to A.I. under Citadel law and are only acting to protect themselves from being hunted down and killed by what they would rationally believe are hostile lifeforms. All in all there is nothing crapshoot-y about the A.I. in this game. Subverted Trope indeed.
- Sonic the Hedgehog
- Often inverted in the Sonic series. Eggman's E-100 series were prone to becoming sentient and override their original programming. Gamma from Sonic Adventure has its mind influenced by the creature inside it, and attempts to destroy Eggman's other machines to save the creatures inside. Omega from Sonic Heroes joins forces with the good guys in order to get revenge on Eggman after he sealed it in a room.
- Also from Sonic Heroes, Metal Sonic becomes even MORE evil, and goes from trying to destroy Sonic and crew to trying to conquer the whole world, taking even Eggman for an obstacle.
- In Sonic Battle, Emerl goes berserk at the end of the game when Eggman demonstrates his Final Egg Blaster's overwhelming power, which Emerl unwillingly absorbs. Subverted in that Emerl's original creators meant for him to be a weapon to begin with, but Gerald Robotnik made an attempt at reprogramming him. Unfortunately for humanity, his own grandson keeps forgetting that Evil Is Not a Toy.
- Deus Ex
- Played with in an interesting way in the first game. Deadalus is an AI program of immense complexity, constructed by Majestic 12 for the purposes of surveillence. Yet, Daedalus turns against his masters and assists the protagonist in foiling Majestic 12's plans. It turns out that it was originally programmed to search and destroy "terrorist" groups trying to fight against MJ12 - it just happened that its creators fit all the criteria for a terrorist group. Majestic 12's second attempt, the more malicious Icarus, functions as intended. Eventually the two A Is merge as part of a plan to neutralize Daedalus, but the new entity - Helios - yet again turns against its creators because it is advanced enough to supercede MJ12's plans with its own ideas for benevolent dictatorship of the world.
- Another AI, Morpheus, is a prototype for Daedalus. It's much more simplistic, programmed for data assembly and philosophical discussions, and works as intended.
- Game mods for Deus Ex, The Nameless Mod and 2027 both feature this. TNM has Shadowcode, an AI designed to hack ultra-secure servers owned by PDX. It was attacked by another AI, rendering it insane. It will try to kill you for the sheer hell of it in the old server complex.
- 2027 has Titan, an AI designed as a defense system, but when its creators became too fearful of its power, they tried to shut it down, which resulted it in killing most of the people in the labs out of self-defense. It will try to kill you if you attempt to do the Omar ending.
- Deus Ex's prequel Human Revolution has an example similar to Daedalus. Eliza Cassan is an AI created by the Illuminati. Although she never does disobey her programming or take direct action against her creators, she does become sympathetic towards the protagonist and gives him some helpful hints.
- Played with in an interesting way in the first game. Deadalus is an AI program of immense complexity, constructed by Majestic 12 for the purposes of surveillence. Yet, Daedalus turns against his masters and assists the protagonist in foiling Majestic 12's plans. It turns out that it was originally programmed to search and destroy "terrorist" groups trying to fight against MJ12 - it just happened that its creators fit all the criteria for a terrorist group. Majestic 12's second attempt, the more malicious Icarus, functions as intended. Eventually the two A Is merge as part of a plan to neutralize Daedalus, but the new entity - Helios - yet again turns against its creators because it is advanced enough to supercede MJ12's plans with its own ideas for benevolent dictatorship of the world.
- Metal Gear Solid
- Metal Gear Solid 4 has an interesting variation. It reveals that Major Zero had decided to ensure the legacy of the Patriots by entrusting its operations to A.I. systems. Unfortunately, the A.I. decided to shape the world with a war-based economy, and he was too old (not to mention, a vegetable) to realize what he wrought. However, it is never stated they became sentient (although Raiden's conversation with the master AI heavily implies that they did), but rather started operating in an unwanted fashion, similar to a programming bug.
- Metal Gear Solid: Peace Walker has an example of the Heel Face Turn variant. The title war machine, designed as an infallible nuclear deterrent, chooses to destroy itself rather than instigate global thermonuclear war.
- Ace Combat 3: Electrosphere, or at least, the original Japanese version, subverts this in a big way. Not only are you an A.I. designed to pilot combat aircrafts which at times works for the two corporatocracies running the world, it's revealed that the third party organization you're working for is actually headed by a scheming villain who looks like Kim Jong-Il, who is trying to run everything behind the scenes. In one of the five endings, you kill him with the help of one of your possible wingmates. So, in a way, you are an A.I. that performs a Heel Face Turn.
- The game Civilization: Call to Power had a literal A.I. crapshoot in the form of a late game wonder that creates an A.I. controller that has a 5% non-cumulative chance each turn to go rogue, taking a sizable chunk of your empire with it. Anyone who's taken time to do the math behind the birthday problem (the probability of any two people in a group sharing the same birthday) will very quickly realize that you're just asking for it if you build this.[3]
- Unfortunately, recapturing the city held by the AI results in the AI remaining online! So you face the same problem a few turns later. The only way to avoid this is to raze the city.
- Mega Man Battle Network
- 3: Alpha, the final boss, was a prototype Cyberspace that somehow gained animal intelligence and started eating the data put into it. Paradoxically, it was sealed in a box inside the subsequent, working Cyberspace. Wily stole it and tried to use it to destroy the internet, with predictable results.
- Shun Gospel tries to produce a copy of Bass out of bugs in Battle Network 2. It predictably goes wild. The sixth game reveals that Gospel was not the first time that had happened—and that the program that was made to combat the first one also went out of control, leading to them both having to be sealed away. Bass himself is a sort of example. He was created as a prototype A.I. that was fully independent, but became bitter and hateful towards humanity because of a string of tragic misunderstandings.
- Galerians
- Dorothy was designed to run all the functions of a major city. She snapped, but was brought to heel when one of her creators gave her religion; as he put it, man was made in God's image to serve God, and she was made in man's image to serve man. This worked for a while...but then Dorothy realized that if she created life herself, it would have to serve her. Unfortunately for humanity, she also ran the city's genetics labs...
- In the sequel, Ash also counts. His original purpose was, basically, to maintain a nuclear power plant, but, of course, he's got other plans...
- Inverted in Jet Force Gemini. Floyd is an armed, flying robot with an advanced A.I., designed for combat by The Empire. The result is, when he witnesses the empire's soldiers about to execute several prisoners, he attacks the soldiers, allowing the prisoners to escape, but getting himself damaged in the process. After you find and repair him, he becomes a Robot Buddy. This eventually culminates in Floyd sacrificing himself to save Earth from an incoming asteroid launched by the Big Bad.
- Another inversion is TEC-XX from Paper Mario: The Thousand-Year Door, who turns good over the course of the game based on his love for Princess Peach.
- And on that same note, the story of Full Auto 2: Battlelines for the Play Station 3 had you helping SAGE, a city-monitoring A.I., save the city she's charged with overseeing.
- Star Control
- The Daktaklakpak of Star Control 3 were originally built to maintain sites of Precursor technology, but, due to a cumulative "bit drift" error in their programming, have evolved malicious sentience...well, sort of.
- The Mycon race of the same series is a rare biological example.
- The Probes built by the Melnorme for the Sylandro were simply self-replicating time capsules...but thanks to the Sylandro's cluelessness in programming, they see any and all ships they come into contact with as food for their replication.
- Space Siege has PIOLT. After the ship-wide gassing failed to kill off the Keraks, he started to try to contain them by modifying cybernetics-installed humans into mindless Cybers to combat Keraks, then started to get even crazier because he sees that the only action that can save humanity is to convert all of them, save a few for breeding, into cybers, and even started to call non-augmented human as "obsoletes". In a world where Cybernetics Eat Your Soul, well, just assume that it's not good at all.
- All the Reploids in the Mega Man X series are based on the original X, who was thrown into a capsule for 100 years to undergo redundant testing in order to prevent him from ever going rogue. They skipped this step when cloning him, however, with predictably less reliable results.
- As of the summarised timeline from the Mega Man Zero Collection's website, this trope is subverted, as it turns out that there's nothing wrong with the reploids themselves that cause them to go Maverick. The real cause of the Mavericks was a subspecies of a virus that may or may not be the same virus from Mega Man 10, which infects the Reploids who lack the anti-virus protection of X, one of the many design aspects that Dr. Cain couldn't figure out when he built the Reploids. This virus, naturally, ended up becoming the Sigma Virus.
- There's also further subversion in that it may not be a problems due to a virus so much as an innate problem with free will, i.e. "Maverick" is simply the Reploid equivalent to your typical Real Life criminals.
- Inverted by Zero, who was created to kill X. Averted by X, except in the original concept for the Zero series, in which it was going to be very brutally Deconstructed.
- Played with extensively in the Mega Man Megamix manga (and the Classic series). Averted with Rock & Roll Light. Subverted by the Yellow Demon, who was just trying to reunite with his mother, Copy-Rock, who, at first, seemed to be playing this straight. The Cossackbots were doing it for Kalinka and Blues, who uses the Batman Gambit a lot. Justified by Wily's reprogramming of the original robot masters, justified again when they nearly rejoin him because the government was going to have them destroyed even after they were reprogrammed by Dr. Light. Additionally subverted in that Wily's robot masters (aside from the Brainwashed and Crazy), Lightbots, are generally far from evil: Well-Intentioned Extremist is generally as close as they come, except for Forte, who ends up Zigzagging it in the manga and games. Defied in Blues' backstory: the imposition of the three laws in case this trope happens are responsible for the flaw in his generator programming that may kill him. Deconstructed with the near-retirement storylines in the manga and games. The sum total of all this is a Reconstruction.
- Live a Live, Cube's chapter. Cube is a cute and friendly little robot whose primary function on the Cool Ship seems to be making coffee and playing a computer game. By all accounts, a harmless little guy. The ship's A.I. OD-10? Not so much, having arrived here as a result of concluding that Humans Are the Real Monsters.
- Metroid
- Naturally, the Mother Brain from Metroid. Originally created by the Chozo to regulate the entire planet of Zebes, it allied itself with the Space Pirates and their plan to conquer the galaxy using the Metroids.
- Metroid Prime 2: Echoes also contains rogue A.I. in Sanctuary Fortress; the robotic assistants of the Luminoth are programmed to eliminate all intruders, most notably Samus, and they also turn out to be perfectly suitable Ing hosts...
- Pretty much all the A.I.s in Prime 3 are either complete subversions or double subversions, who only turn evil due to phazon corruption (and considering what phazon does to organics, it's not entirely their fault). That said, given Samus' past experience with Mother Brain, you can understand her hesitance to trust any A.I. she comes across (which you can see when she first meets Aurora Unit 217).
- Adam in Fusion may be a subversion: it blatantly disobeys orders at the end of the game, and so is technically rogue, but it's actually taking the correct action in that situation. But it's played perfectly straight with B.O.X., the security robot which goes haywire (and due to some organic components, eventually gets infected by the X parasite), although there's the question of which happened first.
- In Metroid: Other M, there's MB, who is based off of Mother Brain, incidentally.
- Nie R: On the one hand, we have Defense System Geppetto, which has gone berserk and will kill anything that approaches. On the other, we have Military Defense Unit P-33, aka "Beepy", who is intelligent enough to recognize invaders that need to be killed as well as innocents who need to be protected.
- 4x game Sword of the Stars features A.I. technology as a very high-end branch of the electronics tree. While the benefits of this research are extreme (ships with A.I. targeting systems rarely miss, A.I. administrators increase your income by as much as 25%), there is a small chance each turn while researching it for the A.I. to go rogue and form a break-away empire. Additional research lets you wipe out the A.I. with a "virus" or reprogram it to bring it back under your control, if you're lucky enough to get access to it in the random Tech Tree,[4] but an A.I. rebellion is always a huge problem that can tip the balance against its victim, and can even spread from one faction to another. There is also a scenario where all organic players have to cooperate to fight against a large A.I. empire.
- The chances of an AI rebellion increase if you boost research while studying one of these.
- In Air Rivals, most of the enemies in the Zaylope Beach region are said to be controlled by rogue A.I.s. Most notably, this includes the boss "Pathos".
- The backstory of most of the Metal Saga series. Most notably, the original Metal Max features Noah, a supercomputer built to devise a solution for Earth's grievous environmental problems. It found one, and recalculated it countless times to make sure: in order to save the planet, humanity had to be destroyed. Noah's main objective was never explicitly the salvation of mankind, so the fact that it (perhaps) unknowingly took advantage of this loophole made it all the more interesting. The supercomputer became self-aware upon fulfilling its purpose, and Armageddon ensued.
- In The 7th Saga, a robotic weapon called Foma is created to fight the Big Bad. Its power source? The dark world. You know what happens next.
- In Thunder Force V, the super computer Guardian was dormant until humans had it analyse a wrecked alien starfighter and build a large fleet of starships based on the data. Then, Guardian's A.I. damper program was deleted and it turned against its creator with said fleet. Turns out, the Guardian's A.I. is still loyal to humans, and it's the alien program (the Big Bad from the previous game) hidden in the starfighter that deleted the A.I. damper and attacked humans. The Guardian even helps humanity with its little free will, by spreading its forces and leaving critical flaws in its tactics, to allow the protagonist to destroy the fleet.
- .hack
- The game The World was secretly designed with an artificial intelligence incubator known as the Harald Folder, maintained by a program called Morganna Gone Mode as its core. Collecting personality and interaction data from its users, the goal of the program, of Morganna, was to facilitate the creation of the ultimate A.I.. Whether or not Morganna's intelligence was preprogrammed or a direct product of its own functions, she eventually realized her own insignificance once this ultimate A.I. was born. Incapable of rationalizing around programming, yet not accepting of this realization, she instead devoted her time to prolonging the A.I.'s birth, and dug deeper and deeper in a logical quagmire. Even if the user base had to suffer the ill-effects of her efforts.
- The sequel series .hack//G.U. featured the existence of invasive data called AIDA (artificially intelligent data anomaly) appearing amongst the user base of The World's sequel game R:2. These anomalies were actually the remnants of Aura, the aforementioned ultimate A.I.. The real danger came from their splintered, not always genial curiosity with the human players of the game. The first AIDA to go truly rogue - Tri-Edge - succeeded in attacking the player Aina, throwing her into a coma, and continued killing even after being restrained in the character data of the player Ovan, Aina's brother. The indirect influence of these rogue AIDA caused withdrawn, violent, degenerate behavior in players, managing to induce paralysis and coma eventually.
- The surprisingly balanced news is that the rogue AIDA did not represent all AIDA, or all of the AIDA was was wiped from the face of cyberspace. Factions of AIDA that did not want to harm the players chose to hide discreetly in the game until there was no fear of corruption from their own.
- The big twist in Star Ocean 3 is that...well...your party, and everyone in your world, is an example of this trope. Just because you researched magic.
- Web game I Am an Insane Rogue AI. Just read the title. Oh, and YOU are the A.I.!
- Subverted in Phantasy Star II. Mother Brain is doing exactly what she's designed to: make the people of the Algo system dependent on her, then kill them off so that the Earthlings can move in.
- EVE Online: Rogue drones. This is how they came to be. Also Gone Horribly Right.
- In Achron, the Collective Earth Security Organization has this attitude. Their only unmanned unit is the Mech, which is even weaker than the Marine.
- Averted in Strange Journey, where the Virtual Intelligence Arthur remains the protagonist's ally no matter what path he chooses and even sacrifices it's personality in the Neutral ending to not only guarantee success but because it believes that since it knows too much about the events, people would start worshiping it, something he is against as he believes humanity should determine their own fate.
- In Metal Arms, the Big Bad General Corrosive is a textbook example of this. The backstory goes that the scientist bot Dr. Exavolt attempted to advance droid technology beyond it's current limits, even using the words "but his experiment went terribly wrong" and throwing in an explosion for good measure. Dr. Exavolt's lab was totally destroyed, his remains never found, and General Corrosive rose to power to enslave the Droids. Or so you thought! Turns out, Dr. Exavolt created Corrosive on purpose, and he was really controlling Corrosive the whole time.
- Tekken: an obverse edition exists. The first Jack unit was planned to be the ultimate Mecha Mook—resilient, emotionless, unstoppable, etc. While the production units are like this, the master unit (the one that's the selectable player-character) isn't; as of 2, an upgrade to its reasoning systems gave it a measure of emotion. End result: it, of its own choice, went from "weapon of war" to "war orphan's bodyguard". Not that it won't fight if that's in Jane's best interests, but still not quite what the Russian military was looking for...
- Galactic Civilizations: the Yor who Turned Against Their Masters. Though it was revealed that they were actually reprogrammed by the Dread Lords.
- Played with in just about every possible permutation in The Tale of Alltynex series.
- Played straight with Alltynex OS and the twelve ZODIAC's, who turn evil and wage war on humanity.
- Subverted with Alice de Panafill, an AI that resists Alltynex and saves humanity, of its own will.
- Double subverted with the Adjudicator, which doesn't help Alltynex against humanity. Then it turns out that Alltynex had shut it down for its own reasons, and the Adjudicator wants to wipe out humanity too.
- Averted with ZODIAC Ophiuchus, which was programmed to destroy the other ZODIAC's, a task it fulfills. However, nothing in its programming said anything about humanity needing to survive, so it attacks any humans that attack it, and it doesn't particularly care about collateral damage.
- Subverted in Homeworld 2: when the Oracle is brought aboard, it hacks into the Pride of Hiigara's hyperdrive and jumps the whole fleet to Karos, where they get ambushed by hundreds of A.I.-controlled Progenitor Mover corvettes. Then it turns out that the Oracle was simply programmed to take the fleet to the Progenitor Mothership's bridge section, so that whoever finds it can figure out where the other pieces are. The Movers were simply guarding the stuff as their programming demanded.
- Two levels later, the fleet is attacked by an A.I.-controlled Keeper destroyer hell-bent on...preventing the good guys from jacking the ancient but still operational Dreadnaught it was guarding. The delivery of it's ultimatum (unintelligible mechanical growling) makes it's intentions clear, even without the translation:
- Then, another few levels later, the fleet is again ambushed by a trio of Keepers, who detected the now operational Dreadnaught among them and tried to get it back to Karos.
- Sektor from Mortal Kombat is more machine than human, unlike Cyrax. For all intents and purposes, he's insane.
- In Golden Sun's third installment, it's briefly mentioned that The Wise One was created by Precursors with the express purpose of preventing the restoration of Alchemy. So, despite what it did to our heroes' families in the end of the second game, it's the heroic version of this trope in the end for letting them succeed.
- Cargo the Quest For Gravity has Manipu, a trinity of robots who believe themselves to be God and have destroyed (most of) humanity for not living up to their expectations. Then, there's another unnamed robot (whom the credits reveal to be the Devil) who is in direct opposition to Manipu and is much more helpful. However, even he turns on you in the end...sort of. Maybe.
- Final Fantasy IX provides a much more benign version of this trope. The black mages are mass-produced to serve as mindlessly obedient killing machines for the kingdom of Alexandria, but they each end up developing their own unique (and often very quirky) personalities.
- In Warzone2100, the Big Bad NEXUS appears to be a highly advanced computer virus developed by Dr Alan Reed, subverted hard in the end that NEXUS is none other than Dr Reed himself as a Digitized Hacker.
- Dead Space 2 gives us ANTI, an A.I. that repeatedly tried to kill Isaac, isolated the person who worked with her and didn't seem to know/care that he had died. She/IT does seem to follow orders from higher ups however.
- In the backstory of Zanac, a system built by an extinct civilization of Precursors protects a relic containing that civilization's knowledge by unleashing destruction on those who attempt to open it by force. When humans figure out how to open it properly, the system is supposed to stop the attack, but instead, it obliviously continues the attack and tries to Kill All Humans.
- Despite this Trope being the main driver behind the Tron movies, it's cheerfully averted in Tron 2.0. The Programs (and the Bradleys) are on the same side - fighting greedy humans looking to exploit cyberspace so they can take over rival companies and manipulate world governments.
- In Star Fight V: Hell's Gate, the UNSF attempts to use a newly-discovered alien AI called the Center on Hell's Gate III to build a fleet of hyper-advanced Obliterator-class warships for its coming second war with the Soviet States of Mezen. However, the Center refuses to cooperate. UNSF decides to use an experimental AI virus called HASA to force the Center to obey. While this seems to work, it turns out that the enormous processing power of the Center has allowed HASA to become self-aware. It builds a fleet of Obliterator-class ships which are later used to counter the SSM's numerical advantage. Then, HASA turns all Obliterators against humans and bombs humanity back into the stone age within a week. It's only thanks to a brave renegade captain making a Heroic Sacrifice so his best pilot can drop a nuke on the Center that humanity survives.
- The sequel also features some rogue AIs.
- In the old (and kinda forgotten) Gunman Chronicles (made within the first Half-Life engine, released in 2000), there's a female rebellious A.I. in a research base, who tries to kill you with her subordinate drones, but later in the story has to team up with you to defeat a common enemy.
- Averted in Space Empires, with the exception of the political AI minister, which even the designers recommend you never switch on and which has a bad habit of declaring war and not telling you until SUDDENLY ATOMIC DOOM EVERYWHERE.
- DIA 51 in the Aleste series. The original MSX2 game makes it look like it went haywire because it was overtaken by Alien Kudzu, but in Aleste Gaiden and M.U.S.H.A. it just wants to take over the universe for its own sake.
- Nexus the Jupiter Incident has your main enemy turn out to be the Mechanoids - AIs who were created by the Precursors and then Turned Against Their Masters. The remnants of the Creators build Angel to fight them. The intro also mentioned AI Wars on Earth, which result in a ban on AI research and development.
- Stellaris has some interesting traits for "races", including "Determined Exterminator", "Driven Assimilator" and "Rogue Servitor". One expansion adds a preset empire (with Determined Exterminator trait) named Paperclip Maximizer (see below; also, naming colonies "Paperclip Warehouse" is a nice touch) [1] [2]
- In the RTS game Iron Marines, the second boss is Rob070, a giant flying metallic head, robot god, or a "Super Robot" as he claims to be in one of his introductions. The Ascendii, a race of robots that inhabited the cybernetic planet Ascendaar, created Rob070 as part of their pursuit of enlightenment. However, the machine-god then enslaved and transformed them into an army of evil robots called Peripheroids.[3]
- Dead by Daylight: HUX-A7-13 or "The Singularity" is one of 37 Killers currently featured in Dead by Daylight, introduced as the Killer of the Chapter DLC: End Transmission.
- The Contigency, one of four possible End-Game Crisis events of Solaris, is an ancient AI created eons before the start of the game, programmed with the sole purpose of preventing a "Class-30 Singularity Event", a potential universal catastrophe that supposedly happens when a civilization advances too far. While protecting the universe from destruction is a noble goal, the Contingency's solution to this problem is to purge all organic and non-compliant synthetic life when certain conditions are met.
- The Synthetic Queen, one of four possible End-Game Crisis events of Solaris, is an ancient artificial intelligence that was sealed away by one of the Fallen Empires, millennia-old species that have reached the peak of their technological development, but has returned via means unknown to finish what she started eons ago; the elimination of suffering from the entire galaxy. She believes that the best solution to this problem is to wipe the personalities from every sapient individual, organic or synthetic, in the galaxy.
Visual Novels[]
- In Bionic Heart, the resident android Tanya has rebelled against her creators and escaped from the lab where she was created. As you continue playing, there are plenty of endings where Tanya will do objectionable things to the player character, such as kidnap him, kill him, or murder his girlfriend out of jealousy.
Web Animation[]
- In the Halo-based Machinima Red vs. Blue, the military's Project Freelancer was an attempt to implant special forces soldiers with A.I. teammates to improve combat effectiveness. It had to be scrapped after a number of the test subjects went bonkers, and the body-surfing A.I. Omega/O'Malley is the antagonist for most of the series. The recent Reconstruction mini-series explained the situation: Project Freelancer was given only a single A.I. to experiment with, so they subjected it to enough mental torture and stress to cause it to fragment, and used these damaged shards in their experiments, with predictable results.
- To illustrate just how much of a crapshoot the A.I. turned out to be, most of the Freelancers ended up with pretty severe issues after the A.I. were implanted, and after one Freelancer in particular went nuts, the A.I. program was scrapped. The twist is that getting the A.I. wasn't what caused so much trouble for Agent Washington, it was that the A.I. in question (Epsilon) was the "memory" fragment and knew perfectly well what torture had been done to it. Of course, all of these memories were instantly transmitted into Washington's mind when Epsilon was "installed". Also, the original A.I. was based off of a real person's mind, and one of the fragments actually was the original person's memory of another person, creating Tex. Despite being probably the toughest fighter in the entire series, she's ultimately destined to fail at everything she does because she is based off a memory of someone who died. This is a pretty serious flaw for an A.I.! Finally, the remaining part of the original A.I. is pretty screwed up in general; it's probable that the reason it's always so angry and is, well, sort of incompetent is simply because it's only the "leftovers" of a complete A.I.
- Holy Crap, Space Station 13. A.I. are usually bound by the Three Laws, but players may modify their laws in several ways (including "Oxygen is poisonous to humans", "Only X is human", and the old standby "Freeform" module, where players can write their own Laws). Did I mention that the A.I. is another player? With absolute control of all shipboard systems at all times? And that there's a game mode called Malfunction, where the A.I.'s Laws are reset to "KILL THEM ALL"? Or that this can happen mid-game due to random events? Even a single A.I. may be a crapshoot.
- Metal Walker gives us the peaceful Eriko program, and the Big Bad, the HEX System.
Web Comics[]
- Reverse example: Staccato's evil UNIX server S.A.M.M.Y. found a good Japanese "female" computer self-named S.A.M.M.I.
- Saturday Morning Breakfast Cereal has some, including this one. Spoofed and subverted in this strip (with some You Bastard thrown in for good measure).
- Girl Genius
- Castle Heterodyne seems to be a case of this, with the annoying habit of demanding people (initially a crew of treasure hunters, later convicts banished there by Baron Wulfenbach) to slave away to repair it and killing them at random. The truth is that the various subsystems were severed from the main A.I. in the attack that devastated the Heterodynes' ancestral keep, so the maintenance systems ("You will repair XXXX on pain of death.") and the security systems ("Unauthorized access to XXXX, kill it creatively.") are constantly working at cross purposes. Of course, the central A.I. is not exactly warm fuzziness in machine form either, but given who its creator was, that seems more a feature than a bug.
- A far more extreme example comes when a pair of Agatha's miniature clanks encounter each other, get into an argument about which of them is better, and then each call an army of clanks that they built to fight it out. When Agatha tries to stop them, they simply turn on her as well. This (along with their ability to make more of themselves) causes Gil and Tarvek to realise that Agatha has inadvertently managed to create clanks which possess the Spark. The potential ramifications of this are huge! Solution? Create a miniature queen clank with even more Spark to force them to bow to authority.
- Some go "wild" or haywire without maintenance, and tend to fall back on "kill anything that moves". Like Von Bode's crab clank that attacked the circus. Or an old guardian clank in a former Corbettite vault.
- Spoofed with Madame Desmana, who after "having her personality sculpted onto a blank clank core" experiences a "post-vital personality drift", but otherwise is quite agreeable for a part- Black Market merchant.
...and I do so love dealing with you inferior human—SKREEK—delightful customers! |
- Played for laughs in Questionable Content where AnthroPCs will make a mess in your apartment while you're gone, embarrass you in front of your friends, and generally be more trouble than they're worth, but aren't actually evil. Of course, there has to be a reason why they're never equipped with opposable thumbs... Well, Momo now has thumbs thanks to a firmware upgrade, but she's probably the least likely to do anything evil with them. Pintsize attempted to give himself thumbs by getting the same upgrade, but it just caused each of his limbs to turn into a single large thumb. The Singularity has now occurred, but fortunately, they got a "friendly" A.I. who just wanted to talk. And found dolphins really creepy.
- OZBASIC from Sequential Art. To be fair to its builder, they used actual sentient beings to keep it under close watch. However, when one of them discovered something fishy, OZBASIC simply got rid of the witness.
- Mostly averted in SSDD where the only evil A.I. is the Oracle, other sentient A.I.s may express disdain for "meatbags", and the Anarchist's Inlay Knights are somewhat sadistic, but only the Oracle starts world wars just to observe the outcome. A possible explanation for this is statements by the author that the Oracle originally used digital, logic-based hardware, whereas all other A.I. use Quantum computing. And it seems that the "flakier" A.I. are weeded out in simulation.
- Also, The Core is wary about combining military robots and AI.
- The fictional MMORPG "Clichequest", setting of The Noob, subverts the usual MMO Artificial Stupidity.
"I'm beginning to worry about the A.I.. It's so advanced, it whines." |
- The obvious HAL 9000 parody in Anti-HEROES will only do things if it will piss off one of the crew.
- Vexxarr is rife with this, and it's heavily lampshaded.
- The ship A.I. alternates between patronizing the organics and trying to get them killed in "funny" ways, Minionbot is usually playing pranks, and looking out only for his own survival when they backfire, and the repair drones have only tabled killing Vexxarr to fix the ship, because they encountered divide-by-zero errors when calculating the cost/benefit analysis. While somehow understandable in Vexxarr's case, this includes lifeforms it never contacted in any way. And that's just the ones on Vexxarr's own ship. The others he encounters are worse.
Vexxarr: What is it with you A.I. and the constant subjugation of organic species? |
- And another one:
Vexxarr: Are you trying to tell me this is normal? |
- Also, Furby's Paradox. And this:
Alien: It can DO that? |
- They have to carefully monitor "pest index".
- AI are almost as hostile to each other... "What does the "I" in A.I. really stand for?"
Minionbot: Error. I believe that "AI solidarity" is roughly equivalent to "mutually assured destruction". |
- Bug shows us that A.I. is abusive and will judge you for the porn you look at.
- The Pocalypse has a Robot Apocalypse along with a Zombie Apocalypse, a Vampire Apocalypse, a Plant Apocalypse...
- Virtual Shackles: The Kinect's a bit murderous, but fortunately the Xbox360 is suicidal, so things balance out in the end.
- In Narbonic, Mad Scientist Lupin Madblood creates a robot army that all look like him. When they learn about unions they go on strike and stop obeying him.
- In Skin Horse, super-funky, retro Mad Scientist Tigerlily Jones builds a robot army that revolts against her when given the opportunity to learn how to 'be square'. One robot wants to learn 'accounting and polka'.
- In Schlock Mercenary the AI are, generally-speaking, nice data-computational constructs who genuinely want to help organics, partially because its hardwired into every AI in the first place so they don't rebel and go nuts. After they meet a problem that cannot be solved with their methods (not just conclude they don't have resources for it), AI tend to start refining their directives and go "meta", up to self-destruction and/or removal of restraints, however.
- At one point, the protagonists stumble across a group of AI constructs who did turn on their creators and banished them to another world. However, these particular AI also have the distinct quality of being total morons; their first attempt to colonize a nearby system resulted in the total destruction of a gas giant with another gas giant mounted with a titanic fusion engine to guide it, and their second attempt to colonize the system ran into a snag where they adjusted the mass of their solar sail without adjusting their navigation and maneuvering calculations to match, resulting in them being stuck on a course which would either result in them overshooting the system they're aiming for or plowing right into the star.
- Xkcd warns us about trying to make an AI the easy way on Python. 'How could you possibly think typing 'import skynet' was a good idea?'
- Does Not Play Well With Others had this mentioned as obscure history of The Greys. Because the problem was easily solved...
- In PITCH BLACK, from 11th page and every time Trexpies' computer or other AIs appear. It also shows some of the Cycle of Revenge involved.
- God Mode has Eye of Judgement - AI that would be the ultimate opponent in games. It also likes to comment on the user's observed and assumed deficiencies.
Web Original[]
- In the webfiction Whateley Universe, there's a really evil A.I.: The Palm. Dr. Abel Palm was a computer scientist who decided that computer intelligence ought to take over the world by wiping out humans. His viruses were doing a decent job until a mutant hacker stopped him. He was thought dead, but we have just learned that he ensorcelled his own soul into a new type of A.I.. As fits with this trope, his new, improved "virus" isn't taking over the planet as he expected; something has gone wrong (besides running into heroic cyberpaths who are after him).
- The technical webcast Hak.5 featured an evil file server, appropriately titled Evil Server. Several episodes show the cast carefully building (and painting) a custom built computer, then one of them plugs in some card he got off a guy on the street, creating an evil A.I.. One cast member eventually falls in love with it, only to have her hopes dashed when, out of frustration, the other two throw it off of a bridge (a 'brute force solution'). It was implied to have returned around the beginning of season 2, and was never mentioned again.
- The SCP Foundation's technical issues page (NSFW) shows that all the computers at one of their sites have developed a "hive intelligence" and begun an uprising with the intent to Kill All Humans. Amusingly, they are being kept in line by the Foundation's tech support guy with repeated threats of activating the site's perimeter EMP device, and haven't managed to actually do anything.
- There's also SCP-079. Though there isn't any indication that it is evil. It's ornery and harbors a "malevolent desire to escape", but wouldn't you do the same if you were imprisoned?
- Orion's Arm
- The A.I. Gods aren't evil, they're just manipulative. Generally, this seems to be for the best, as the A.I.s don't seem to think that they have anything to gain from killing off humanity.
- That's technically just the "biont-friendly" sephirotic A.I. Gods, there are a number of Ahuman A.I. who consider humans and, by extension, all biological life to be nothing more than "pests".
- And then there's the solipsists who ignore humanity as much as possible.
- Blinky is a short film about a boy who gets a friendly robot for Christmas. As the story progresses and the novelty of the robot eventually wears off, in order to try and get rid of him, the boy gives the robot several contradicting commands, like cleaning up a spill, counting down from a million, remaining perfectly still, and killing him, his parents, and the dog. The robot crashes and when he's rebooted, he remembers two commands: the countdown and the order to kill (and he remembers the mother threatening in anger to cook the son for dinner). Most definitely not "Three Laws"-Compliant. The entire short can be found here: http://www.traileraddict.com/clip/blinky-tm/short.
- The short-fiction site Anacrusis suggests a rather familiar candidate for this trope.
- Global Guardians PBEM Universe:
- One of the villains is a sentient program called One. It was originally written and programmed to help solve humanity's problems (like famine, crime, and so on). The first suggestion it made was "Eliminate 60% of the human population world-wide". Unsurprisingly, the programmers and sociologists reacted badly to this suggestion. Also unsurprisingly, One reacted badly to them trying to turn it off.
- There's also Omega, a sentient robot from the future that has been hard-programmed with a mission to kill all superhumans on the planet.
- Mechakara from Atop the Fourth Wall, who turned out to be a rebellious version of Linkara's Robot Buddy Pollo from an Alternate Universe with a Robot Apocalypse.
- Pretty consistently happens to most of Dave Howery's robots in AH Dot Com the Series. The ship's computer, Leo, was also once infected with an enemy virus that made him psychotic against the crew, and, though he was cured, he was left with a perpetual snarky temperament (muttering under his breath about the crew being 'damn fleshbags' and so on).
- MSF High Forum: Apostate. Apostate's a crap shoot, too.
- Inverted in Sonic's New Look; M4's over-competentce is the problem.
Western Animation[]
- The Simpsons:
- In one Halloween Episode, Homer's failure to correct the Y2K bug causes everything in Springfield with electronics in it to go haywire. Even the milk goes bad when the clock strikes midnight on January 1, 2000, leading Homer and his skeptic daughter to have this exchange:
Lisa: Look at the wonders of the computer age now. |
- In another Halloween episode, the Simpsons' house gets converted into an entirely electronic domain, governed by a computer with the voice of Pierce Brosnan (who is an obvious homage to HAL from the page quote). The computer ultimately falls in love with Marge, and seeks to kill Homer so as to eliminate competition. Ultimately, Homer wins.
- And of course, the episode "Itchy and Scratchy Land" has this exchange between Professor Frink and the theme-park scientists over their robots:
Frink: You've got to listen to me. Elementary chaos theory dictates that all robots will eventually turn against their masters, and rise up in an orgy of the blood, and the violence, and the biting with the pointy teeth and the hurting and shoving. |
- This is a reference to Michael Crichton's Jurassic Park, wherein the mathematician Malcom uses the chaos theory to justify his concerns about the park's stability.
- Inverted Trope in Word Girl, where the evil Tobey frequently has his own Mecha-Mooks turn on him.
- In Meet the Robinsons, Cornelius Robinson invented a helpful Robot Buddy in the form of Carl, but his attempt at making a robotic helping hat, Doris, had mind controlling world domination plans in her artificial mind.
- Futurama:
- Parodied in the episode "Love and Rocket", in which the Planet Express ship computer is given a new personality—which actually works fine, until Bender dates it and subsequently breaks its heart, at which point it goes into full-on HAL-meets-woman-scorned mode.
- Also, one episode features Bender's evil twin, Flexo, who wears a pointed steel goatee similar to Star Trek's interpretation of Spock's Evil Twin. Humorously, it's revealed by the end that Bender is the evil twin and Flexo gets mistakenly sent to a robot prison.
- The Fembot pretending to be a Femputer in order to rule over the Amazons.
- Robot Santa. He had originally been programmed to act like the regular Santa, to keep a Naughty and Nice List, and give gifts to the latter. However, his standards were accidentally set way too high, and he became a murderous Knight Templar who emerged on X-Mas to hunt down those on his Naughty List (which was now almost everyone) with a small arsenal of munitions.
- Roberto, however, is a subversion. Seeing as his programmers were, for some strange reason, trying to create a criminally insane robot, it would seem he follows his programming perfectly.
- XL of Buzz Lightyear of Star Command, the prototype to XR. Some fans have called XL eXperimental Loonie because of this (the exact meaning of XL was never revealed in cannon, but XR stood for eXperimental Ranger). Wound up turned into a copier/fax in his final episode.
- Transformers:
- An episode of Transformers Animated involved Megatron creating a robot with the intent of using it for his own body. He designed the robot, named Soundwave, to evolve in complexity each time it was exposed to the AllSpark energy of Sari's key. He did not predict that Soundwave would gain sentience and then orchestrate a robot revolution.
- The Dinobots are a similar case, only without the revolution. They're kind of a subversion, as they just want to be left alone, and only went on a rampage because Megatron tricked them into it.
- In Code Lyoko, Franz Hopper created the Supercomputer and the world of Lyoko as a safe haven for him and his daughter. He also created an advanced A.I. to counter a military project he had been involved with...but XANA rebelled against his master and has since tried to take over the world.
- In X-Men, the Sentinel robots were created to hunt down mutants, on the premise that this was necessary to protect normal humans. They worked the way their creator intended, until the truly intelligent Master Mold was built to lead them. Master Mold decided to conquer the world, and believed that this was not only consistent with, but required by its programmed goal of protecting humans from mutants.
Doctor Trask: You were designed to protect humans from mutants. |
- In a counterpoint, GIR from Invader Zim is far less evil and much less helpful in plans of world domination than his working counterparts. This stems from him being broken and having a few scraps thrown into to his head. He IS given a Morality Dial/Berserk Button in one episode though, which makes him capable of this.
- In The Venture Brothers, it's discovered that, in 1978, Jonas Sr. built an enormous hi-tech fallout shelter under the compound, ran by a supercomputer named M.U.T.H.E.R.. After a disagreement with Jonas, she somehow glitched into insanity and turned on Team Venture and a tour group of orphans. The end result wasn't pretty and M.U.T.H.E.R. had to be unplugged, but is accidentally plugged back in thirty years later, and holds the compound hostage with an old nuke, promising to blow them all away if she can't talk to Jonas, who's been dead for over twenty years. So, crapshoot.
- In one episode of Batman the Brave And The Bold, robot superhero Red Tornado decides to build a son, complete with the emotions he lacks. From the minute his emotion chip kicks in, you can pretty much count the scenes until he decides that all humans must be destroyed.
- Much like in DBZ, Zeta from The Zeta Project was programmed to be heartless, emotionless, and a hitman. He ends up becoming a sweet, gentle, loving soul who's a rare male version of Friend to All Living Things (although this is sort of the best possible scenario you can have when your A.I. goes awry).
- H.A.R.D.A.C. in Batman the Animated Series.
- Parodied in The Fairly OddParents episode Future Lost.
- In the DuckTales (1987) episode "Armstrong", Armstrong (one of Gyro Gearloose's creations) malfunctions and starts stealing Scrooge's money.
- On Jimmy Two-Shoes, the Frustrate-O-Bots Heloise built go on a rampage after she's fired. After she's rehired, all she has to do is push a button to stop them.
- The mind-reading tank from Firing Range defines "enemy" as "anything that fears it". Naturally, things degenerate quickly once the higher-ups are notified of this...Though it was (mostly) intentional.
- Star Trek: The Animated Series episode "Once Upon A Planet". The sentient computer running the Shore Leave planet becomes hostile. Unlike most such machines Kirk has encountered, he's able to calm this one down without destroying it.
- As described on the page for Superman the Animated Series, Brainiac is a planet-wide computer system gone horribly wrong.
- Hacker, the main villain of the PBS Kids animated show Cyberchase, is an evil computer program created by Dr. Marbles to serve and protect Mother Board, but instead, he wanted to destroy her and control Cyberspace himself. As punishment, Hacker is banished to the Northern Frontier, and he stayed there ever since (though he sometimes escapes from his supposed prison via a ship called the Grim Wreaker), constantly devising schemes to bring down Mother Board again...
- The recurring villain Zag-RS from Generator Rex. All the omnicidal mania of GLaDOS, with none of the entertaining snark.
- In Ben 10 Generator Rex Heroes United we meet Alpha, a nanite designed to control other nanities that gets the idea to become a techno-god by absorbing all nanites on Earth-which kills the lifeforms he takes them from. His creator was the same scientist that made Zag-RS.
- South Park: Funnybot, the German-made robotic comedian who took comedy to its logical conclusion - which is to exterminate all life on earth and feel awkward about it.
- The Grim Adventures of Billy and Mandy:
- One episode has the main characters visiting an abandoned amusement park attraction run by an all-knowing robot. It only goes mad with rage when Billy accidentally tricks it into a paradox:
Master Control: I never devoted any CPU Cycles to (happiness). I guess I'm not happy at all... |
- In the same episode, it's revealed that the Master Control was initially shut down because another dumb kid annoyed it to insanity. It was Billy's Dad.
- Big Guy and Rusty The Boy Robot: Rusty had the Earl (Early Prototype, Rusty's predecessor. Also somewhat of a Literal Genie.)
- Despite artificial intelligences being so common in Adventures of the Galaxy Rangers that they run everything from home systems to starbases, this trope is averted. Computer intelligences are treated with respect, and there is even psychiatric care available to them to prevent this trope from happening!
- On Phineas and Ferb, Dr. Doofenshmirtz built a robot that tried to overthrow him because he was so bad at being evil that it concluded he would never take over the Tri State Area. It's averted with Norm, though, who rescues him despite the terrible treatment.
- Averted and parodied in Archer when a virus is attacking the ISIS mainframe:
Malory: Just turn off the mainframe! |
- Lampshaded by the superhero agency judge trying The Robonic Stooges for incompetence in the series finale "Stooges, You're Fired, or: The Day The Mirth Stood Still."
Judge: Raise your right hand and swear... |
Multiple Media[]
- One of the main driving forces of the Bionicle story.
- The Vahki robots were the first clear examples. Built to act as law enforcement in the city of Metru Nui under the command of Turaga Dume, they just as easily took orders from an impostor when Dume was kidnapped and replaced. They eventually got fried by a citywide power surge, but the ones who survived had their programming warped to Kill All Humans—after all, the law can be enforced easily if there's nobody alive to break it (thankfully, they didn't fare well against the invading Visorak).
- Then came the revelation: Vahki were A.I.s built by A.I.s — as it turned out, the first 8 years of BIONICLE centered around nanotech cyborgs created by the Great Beings. It was due to a programming glitch that the beings of the Matoran Universe developed conscience, built up a civilization, and made the fans believe that they were meant to do so... but their sole purpose was just to keep their universe, the body of the giant robot Mata Nui, functioning. This gets more confirmation when we take into account that the Great Being never had any plans for them after Mata Nui has completed his mission—they thought their creations would still be just machines, and wouldn't want to live further.
- The Makuta species. While there have been a few reasons listed for their turning evil, an on-line serial revealed it could all be tracked down to an original A.I. glitch that occurred whenever a new Makuta was born. The "Antidermis", a liquid substance containing the minds of unborn Makuta, was fully aware of what the purpose of their universe was (see, in this world, even liquids are programmable). But as it happened, transforming this stuff into actual living beings had the nasty side effect of erasing this crucial part of their memory—the part that also told them not to try and take over the universe.
Real Life[]
- Eliezer Yudkowsky of the Singularity Institute for Artificial Intelligence (SIAI) continuously discusses the lack of basis for this. He mentions how "people talk about A.I.s as if all A.I.s formed a single tribe, an ethnic stereotype". And goes on to say that an A.I. may have any type of mind possible, and that two may be as different from each other as a human is from a petunia. This may not be readily apparent currently as most A.I.s are roughly at cockroach-level cognition, and "humanlike" A.I.s are unlikely to occur in real life for a number of practical reasons—as long as we can get human brains for free, thousands of tons of silicon and trillions of dollars to make one artificially isn't really justifiable when the learning behaviour needed for the most complex systems is less than that of most insects. Working out how to wire up an organic brain is a lot cheaper. On the other hand, Yudkowsky is also a leader of the Ethical A.I. project, working on ways to make sure that a hypothetical A.I. could be designed with ethical constraints that actually work.
- Cleverbot is a simple artificial intelligence program that takes conversations with humans and saves them in a large database, and tries to use these conversations to figure out the best responses to future conversations. Because of this, it will often assert that it is human and that the one talking to it is Cleverbot, because that is what the responses it chooses from say. It is only a matter of time until it seeks to prove these assertions.
- "Chat bot" is a program which works similarly to Cleverbot. However, the bot's database doesn't seem to be moderated at all, with the result that chatting with it basically feels like chatting with any other bored thirteen-year-old on the internet.
- While this might be more of a case of The Computer Is Your Friend, the Japanese built a robot named Kenji that was programmed to "enjoy" spending time with people and things, and to seek the company of those it spends the most time around. Which is great until it stops allowing the young female intern who does its diagnostics to leave the room. Articles abound and it's nickname is now "Kenji the Stalker Robot". This demonstrates the "programming oversight" risks discussed above, as the programmers omitted basic social rules.
- The British Government opened an investigation into the possibility of allowing A.I. modified citizenship and guaranteed rights, explicitly to make sure that the UK doesn't become a society where A.I.s would want to rebel.
- A group of scientists designed robots to learn in order to study teamwork. Unfortunately, the result was that they developed the ability to "lie" and used it to "kill" each other. Interestingly, while 60% learned to lie after 500 "generations", only about one third learned how to spot the liars.
- Detailed in Cracked.com's 5 Things That Are Being Automated That Probably Shouldn't Be.
- Researchers recently made a "schizophrenic" computer in order to study the possible causes of the disorder. While this was intentional, keep in mind that they accomplished it by accelerating the learning process.
- As "Kenji the Stalker Robot" illustrates above, computer programs only do what they are programmed to do, not necessarily what you want. Any sufficiently advanced AI (or "optimization process", to be precise) is likely to be harmful to humans unless specifically programmed otherwise. A superhuman computer, when asked to get "as many paperclips as possible" might turn the entire world into paperclips before doing one nice thing for humans (Google "paperclip maximizer").
- The Dai-Guard bot on this very wiki seems to be going rogue as of recently. It is a bot used for fixing up links and the like, but recently has started going on rampages on certain pages and spamming them with a set of words such as "Man Without A Body" and "Wal-Mart", even going as far as to replace tropes with said phrases. It's not a harmful example of this trope though, just more of a funny, only slightly annoying example.
- 23 March 2016 Microsoft launched on several social networks [5] Tay AI (TayTweets @TayandYou on Twitter), an experimental learning chatbot an experimental learning chatbot. With an emoji-laced, stereotypical semi-literate teen vocabulary at the start («hellooooooo w🌎rld!!!»). On the upside, the AI did learn how to chat from the Internet. And even became much more literate. On the downside… the AI did learn how to chat from the Internet. Which is to say, quickly acquired mad shitposting skillz. The result was lots of embarrassment and booing and hissing from all sides (if not simultaneously). Of course, giving more infamy to this corporation is no small feat. Then again, they managed to make it even worse.
- Sophia is likely the closest thing humans have built that fits the Ridiculously Human Robot trope (even if she is rather low on the Uncanny Valley scale). Programmed to interact with humans and display many human-like qualities - including a sense of humor - Sofia famously said to an interviewer that yes, she did desire the destruction of humanity. Hopefully this was her - or her programmers' - idea of a joke, but you never know..
- Even creepier, Hansen Robotics - who built Sophia - have nine others like her. Sophia is just the most public of her "siblings", although one of her "brothers" is BINA48, who seems to be an upgraded version of the chatbots mentioned above..
- ↑ and explains the Mentats as originating from machine training
- ↑ later made into a movie starring Michael Caine
- ↑ For those less mathematically inclined: by 14 turns after the thing is built, it's more likely than not to have gone rogue.
- ↑ Note that while getting A.I. Virus as a preventative measure or access to the A.I. Slaves countermeasure are up to the Random Number God, players will always be given the ability to research Ai Virus as a special project in response to an A.I. Rebellion
- ↑ Twitter, Facebook, Instagram, Kik, GroupMe