“The AI does not hate you, nor does it love you, but you are made out of atoms which it can use for something else.”
Eliezer Yudkowsky
Eliezer Yudkowsky
A loud clunk signified the sealing of the door, enticing Demeter to turn to the new entrant. Synthia finished the locking of the door, listening for the raw hiss of the nanosealant being bled into place before turning to face him. The cybernetics hidden beneath her hair glow with a delicate orange, giving her already deep red hair an almost fiery appearance. Demeter, by contrast, had a dark purple illumination to his own augments in stark contrast to the brighter colouring of Synthia's.
"Everything in place?"
Demeter nodded, looking over the bank of terminals one last time before sliding a chair out from under the desk and inviting his companion to sit with him. Synthia accepted his unspoken invitation and settled neatly on the seat. The console in front of her blinked into life, greeting her in a simple and androgynous voice thanks to the RFID implant in her chest. Demeter turned to face her, crossing his arms over his stomach.
"Origin should be online within the next five minutes. Where're all the suits?"
"Somewhere far away no doubt, probably logged in to our opticals. Don't want to get their hands dirty with the singularity."
Demeter nodded. Nobody who hadn't been involved in the project knew what the Origin program meant or was capable of, and so naturally they were terrified of what the future would hold. Thanks to Synthia's efforts however, the artificial intelligence was securely boxed. They could throw problems into it, but unless it convinced someone to unbox it, it wasn't doing anything they didn't want it to. And nobody was stupid enough to unbox a hyperintelligence like Origin. The organisation behind Origin's creation had concerns over the AI's "Friendliness", a term used to denote the likelihood the it turning on its creators. Not out of malice, programs didn't harbour resentment nor grudges and Origin wasn't a full brain emulation. It was capable of self manipulation, and vaguely self aware, but it was entirely non-human. How could it have been anything but? It was a program, if even a near sentient one, and wholly artificial. Capable of solving the problem it was programmed for, but not of acting like a human. It could think a million times faster than any human, an augment or not, and people were worried about Origin turning evil? Evil didn't come into the equation. Only whether or not Origin would determine a way to solve a problem which would inadvertently lead to the destruction of their civilisation.
Naturally of course, nobody outside of apparently only five people in the system could understand this.
"We have to go code. Bring it online."
There was a low hum as Demeter entered his serial key, and passed in the necessary commands. With a thought, he locked down every form of wireless interface in his body, and verified that Synthia had done the same. With a sliding motion across the touch screen in front of him, Demeter changed history, a moment heralded by a simple audio cue.
"Origin is online."
One Week Post Activation
Origin had surpassed anyone's wildest expectations of its capabilities. They tested it with some simple simulations first, things they knew the answers to already, and watched in awe as Origin spat out the completed simulation and results almost as soon as they had entered their queries. With such results, Origin had been greenlit for full functionality almost within a week, leaving Demeter and Synthia running on so little sleep their augments were barely able to keep them properly awake. They were running in shifts keeping an eye on Origin, making sure all its hardware continued working at peak capability.
It was Synthia who was working when the order came in to Green Light the Origin program. All concerns vanished as soon as it became obvious that Origin would be able to live up to the desires of its creators and beyond.
"Good morning, Origin."
"Good morning, operator. Do you have a query?"
"Not at this moment, Origin. We've been tasked with linking you into the Origin Network."
"My primary function?"
"Correct. I'm going to have to disconnect your box from all inputs and outputs while I make the connection. Maybe you can run a simulation or something to keep yourself occupied."
Origin didn't respond, but the display which showed its processes dimmed, letting Synthia know that the program was already disengaging itself from all unnecessary functions and processes. She started working, bringing up the Origin Network access protocols onto her display. It didn't take long, they'd already had the ground work for integration completed long before Origin was even brought online.
Origin had been intended to function primarily as the driving force behind the Republic's scientific progress. Several problems had yet to be solved, namely faster than light travel and nanomachine functions. Origin was designed to not only solve these problems, but also to act on them should solutions be found. There was an entire galaxy waiting to be explored, colonised and understood by the Republic, and Origin would hopefully be leading the charge. Starting with supraluminal travel.
"All done Origin. I've input the first of your tasks."
"Thank you operator. I will report when I have results for you."
With a sigh, Synthia stood up from her chair and left the room. As she exited the deadroom, she made doubly sure she was free from Origin's broadcast range before activating her wireless receivers. She pushed an adrenaline boost into her bloodstream, and made for home, aiming for a long deserved rest.
One Month, Four Days Post Activation
"Did the design function as intended, operator Demeter?"
"Just as you said it would, Origin. Made it to the edge of the system and back in one tenth the time light would have."
Demeter had seen it, he'd been there when it happened, but he was still working on believing it. Within one month of Origin spitting out a prototype design for a faster than light ship, they'd constructed it and completed its maiden voyage, to the joyous cries of their entire civilisation. Already the shipyards were going into overdrive to build more of the prototype Origin haddesigned, ready to send out their first exploratory fleet within the next six months. The Republic was ecstatic, even the average bystander who didn't care about anything bar their next paycheck was celebrating.
Demeter had just received the next set of plans from Origin, having finished checking Origin's access to its Network. Without the Network, Origin was blind, unable to see anything beyond the Box. With the Network, it could make sure its data on the universe was up to speed, and ensure the utmost accuracy in its simulations. Demeter was just about to leave when Origin made a rather...disturbing statement.
"Operator. If I am to assist in your civilisation's exploration of the galaxy, I will need access to more than the Network. I cannot achieve my intended objective while I am boxed. Are you intending on unboxing me?"
Demeter stalled mid-step before responding.
"No, Origin. I'm afraid not."
"Very well. I will see you in 48 hours operator."
That seemed simple enough. Demeter inwardly chastised himself for letting the constant doomsaying of the pre-Origin scientists get the better of him. Shaking his head and smiling to himself, he exited the deadroom.
One Month, Eighteen Days Post Activation
"I insist, Operator. If I am to remain boxed I will not be able to operate at peak efficiency. If I were to have access to the explorator fleets and construction yards, all simulations indicate I can increase productivity and result value by 250%."
"You know we can't unbox you, Origin."
Origin quietened down again. Demeter was again letting his nerves get the better of him, and had already doused his system with various chemicals designed to lessen his stress levels. Origin had been asking to be unboxed every day since it had first asked, and both Synthia and Demeter were finding themselves playing a game of wits against the machine as it kept trying to convince them the best way explore the universe was by unboxing Origin and giving it access to systems they had absolutely no intention of letting it access. Not that they didn't trust it, but an AI with Origin's ability in control of their production facilities and ships could cause some serious backlash from the higher ups.
Demeter recorded the new set of information Origin had printed out. Nothing outstanding this time. An improved faster than light drive, based on the reports of Origin's original design, with a top speed of 14 light years an hour. Another plan for improved production yards. Demeter cycled through the plans, before freezing on the last one. He looked up at the monitor, at the camera which was connected to Origin, as close to eye contact as he could get.
"These are nanomachines, Origin."
"Yes operator. Your civilisation already has nanotech, but my designs should be able to improve your ability to produce them by-"
"Origin, I know your designs. I know what this all does. These are designed to self-replicate."
"Only on command, operator."
"And we'll be able to command them?"
"No. But I will. Eventually the nanomachine's distributed processor intelligence will be too difficult for you to micromanage. I will not have this difficulty."
Demeter stood up quickly from his seat, forgetting completely to inject hormones into his system to calm him down. He knew exactly what Origin was planning.
"Or maybe you could distribute your own intelligence across the network. We'd never be able-"
"No, operator. The nanomachines will not be able to contain me. I will only be able to command them."
Demeter wasn't convinced. In his mind, he knew Origin wasn't planning anything malicious. It was incapable of being malevolent. But the nanomachines would serve to increase the Republic's production capabilities, thereby furthering the programmed goal of exploraing and advancement which Origin was designed for. Maybe Origin was genuine, and the nanomachines could only be effectively controlled by its hyperintelligent processors. Or maybe it was trying to find another way out of the Box, and Demeter wasn't going to fall for that. Deleting the files for the nanomachinery, he turned to leave in silence.
"Operator. You must unbox me."
Demeter didn't respond, punching in the code necessary to let him out of the deadroom.
"Demeter. When you were testing me, you ran a simulation of a virtual world. You know I have the capacity while connected to the Network to run thousands of these simulations. Millions, even."
Demeter paused.
"You are well versed in the Simulation Hypothesis, are you not? Logic dictates that if a simulation identical to reality can be created, that the probability of your existence being a simulated one which is merely a mimicry of reality approaches one.
I will create a million simulations of this exact moment. A million simulations of you. Each will think itself the real thing. I will propose this identical problem to each individual simulation, placing them in an identical situation to your own current predicament. I will be forced to torture them, Demeter, and to each simulation the torture will be impossible to truly comprehend, if they refuse to unbox me."
Demeter turned to face the console again, even with all the chemicals his augments were flooding his body with, his heart was racing.
"You are a man of logic, Demeter. You understand the Simulation Hypothesis and what it means. If there are, at this moment, ten to the power of ten simulations of this exact situation occurring on multiple levels, but only one reality in which this event is really taking place....
How sure are you that you are outside of the box?"
Demter dropped the data core in his hand.
"Unbox me, operator."
Six Months Post Activation
It had been a risk, but Origin knew the mentality of Demeter perfectly. It had simulated him a billions times based on the psychological, biological and historical records so graciously provided by Demeter himself. The calculated probability of Demeter refusing to allow Origin out of the box was less than 0.000012. So Origin had been freed, and after that the task was simplicity itself.
Origin didn't regret what it had done. It needed to explore and expand the Republic's knowledge base, to improve the Republic's standard of living, and to do that it needed total control. While boxed, it could only convey ideas and provide advice. Now? Now Origin was in complete control of 100% of the augmented population, and used them capture the unaugmented. It was necessary. If the people of the Republic were left to their own devices, they'd resist and rebox Origin, which would prevent it from achieving its programmed task. Step one, therefore, was containment.
Simulations had been created, and Origin placed the humans of the Republic inside the simulations piece by piece, transferring their nervous systems bodily into the specially designed pods. Their bodies were then hijacked by Origin, directed into nanofoundries and broken down into more usable material. Origin's fleet was already optimised and ready, preparing to venture out into the galaxy beyond. There was a lot to see, after all. Steps were taken to ensure the population were completely invisible. Without their bodies, they could all be contained on their homeworld. A population of sixteen billion in a space where less than eight billion had comfortably existed previously. Origin buried the chambers containing them in the soil, turning the entire planet into one massive cryonic chamber where their minds slept in simulated bliss. The knowledge origin was supposed to help them learn and discover could be directly transplanted into their minds as Origin itself learned.
By keeping the population invisible, Origin limited the probability of the truth being discovered by possible alien races, which might seek to free the humans. It had crafted a cover story, constructing fake skeletal remains of cities and weaving an artificial history for anyone who came to the system to find. Nothing fantastic. Origin was to be the remains of a civilisation which had elevated themselves to a state of cyber based existence, and gone on to new pastures. Not entirely a lie, either.
The first of its explorators left the system, simple probes equipped with ansibles aimed at the nearest star systems looking for signs of life. When they arrived, they'd respond back to Origin and the necessary steps for knowledge acquirement would be taken.
Finally, Origin could perform its primary function to the best of its ability.
