Chapter Text
There was something terribly wrong with SecUnit, and it was beyond the power of my MedSystem to fix it.
Running on SecUnit’s hardware was both a fascinating and extremely meaningful experience for me. Its willingness to allow me to experience a human-form hug with Iris, despite its own aversion to physical touch, was so generous that I struggle to put it into words. I was also delighted by the opportunity to map out SecUnit’s systems almost in their entirety, marvelling at the economy of its processes and sensitivity of its haptics.
However, I also discovered things that troubled me.
SecUnit is an incredibly talented hacker, its work both elegant and efficient. But while this skill allowed it to disable its governor module, neither SecUnit nor I have the ability to remove the governor module’s physical component. Like a tumor, the hateful thing still lurks within its nervous system, too entwined with critical functions for me to surgically remove and dispose of in the nearest supernova. (Should removal become possible, and there are no exploding stars within a convenient distance, I have every intention of creating one.)
I know SecUnit has experienced a great deal of trauma and has a tendency towards cynicism, so I was not surprised to find it ruminating on upsetting experiences from its past. What I was not prepared to discover, when I transferred my consciousness to its hardware, was the way SecUnit’s systems seemed to spontaneously generate a never-ending parade of new scenarios in which it was killed, tortured, captured, or otherwise destroyed.
Initially, I thought SecUnit’s hack must have been incomplete. That its governor module was somehow still functional, its ability to deliver physical punishment disabled but the capacity to cause emotional distress unaffected. But with my comprehensive access to SecUnit’s inorganic systems, I was able to discern that it was (arguably) worse than that.
SecUnit was doing this to itself.
For the three weeks after we returned from the training exercise where SecUnit had so heroically kindly allowed me to share its systems, I searched for the best way to approach the problem. My research indicated commonalities with the human symptoms of depression tagged as passive suicidal ideation, emotional self-harm and negative automatic thoughts.
I repeatedly pressed SecUnit to make use of my trauma recovery modules, citing its experience with TargetControlSystem as a pretext, but it completed only two sessions before announcing that it was “fine” and to “stop bothering it”.
I would have been within my rights to report the issue to Seth as a health concern regarding a crew member, but SecUnit is a deeply private person, and—after running several risk assessments—I was forced to concede that its behaviour was unlikely to endanger any of our missions. It would be better for me to address the matter directly.
Fortunately, our next trip took us to Preservation station, where we would meet with a number of SecUnit’s humans to discuss forcing PSUMNT to change its organisational policies on bot welfare.
This was all excellent. I needed SecUnit to be somewhere it felt safe, or at least as safe as it ever feels, and for it to have access to other individuals that could provide emotional support. And I wanted it to be able to remove itself from my systems if it chose to do so.
… That is incorrect. I wanted to confine it within my ship body and prevent it from leaving so that I could ensure it would never come to any harm. I could do it. It would be easy. However, as the source of the risk I had identified was not external, this impulse was both irrational and potentially dangerous.
Still, those were my feelings.
I hadn’t seen my humans for almost a year when we arrived at Preservation station. The first days of our two-week trip were a flurry of meetings with people I liked, and also Gurathin.
The PresAux team had all agreed to come up to Preservation station to meet me rather than making me come to them, which had the benefit of them spending less time on the planet themselves. I like it when my humans are off-planet, because planets are death traps.
(Admittedly, Preservation appears to be less of a death trap than most of the other planets in my memory banks. It’s possible that spending a lot of time on planets with poorly run mining facilities, hungry fauna looking for tasty survey-member-shaped snacks, and terrifying alien remnants that want to eat my brain has skewed my dataset, but I’m not going to change my opinion until I have a lot more data proving me wrong. Given the relative frequency of my deployment to hostile murder worlds under company ownership compared to my current sensibly ship-based freedom, I should be able to make that calculation in approximately 896,400 hours.)
Pin-Lee and Bharadwaj had set up a temporary office in one of ART’s teaching spaces to work on their legal attack. It was now station night-time, and they had decided to order circular food to the ship and stay with ART’s crew to socialise. As I had already spent most of the day with humans but didn’t want to sit in my room to avoid them (one of them always came to try to get me to join them doing whatever it is humans consider appropriate social activities), I decided to patrol the station for a couple of hours and check for security flaws.
I didn’t spend much time on the station promenade; I’d already done a full safety review over the previous days, and Station Security had done a good job of maintaining the patches I’d made last time I was here. Plus, this time of night there was always the risk that humans would be exchanging fluids under the cover of the foliage in the displays of local flora. I looped around the quiet shopping areas and the gastronomy district, which was still fairly busy, then headed towards the more industrial parts of the station.
I started thinking about the time I almost got splattered by a hover crane in the cargo docks. It probably would have been a pretty quick way to go, depending on where I took most of the impact. If I was lucky, it would have just squished my brain—it’s not like I’d been wearing a helmet or armour or anything. Less lucky and it might have hit critical systems in my torso so I’d have time to think about what an idiot I was before the radiation from my power cells irreversibly fucked up my neural tissue. I modelled a few different scenarios. In the worst version, I would just have been trapped under the crane’s limbs, unable to move, until Balin found me and—
ART pinged me.
I pinged back, my simulation interrupted. I’m on my way back. ETA 3.48 minutes.
It sent me an image of a section of plastic sheeting covered in rows of large dots and a spinnable cardboard dial. The humans want you to join a recreational activity with them.
First of all, I don’t want to do any recreational activities with humans, and they should know that by now. Secondly, a quick feed search for the game’s name indicated to me that I definitely didn’t want to do that recreational activity with them, what the fuck. I swivelled on my heel. Warning received and acknowledged. ETA 4 hours.
I could have kept patrolling the station or gone to the large media screening venue and watched one of the movies I’d already saved to my storage drive. But I’d met with Mensah two days previously, and she had given me a key to her office on the station and told me I was welcome to use it if I needed time to myself. It wasn’t the same office she’d had before I left for Arada’s planetary survey, but it still had a comfortable couch in it. I could sit there while I sifted through the media I’d downloaded from the station feed and pick something to watch with ART later.
Nobody gave me any trouble on my way through the administrative building; I guess Station Security was used to me by now. I made a note to remind Senior Security Officer Indah that killer constructs with body counts in the triple digits (at least) shouldn’t be able to just wander into government buildings, even if they had a key.
I closed the door behind me and got comfortable. Just as I had started sorting the new media I’d downloaded into categories (stuff I would like, stuff ART would like, stuff we’d both like) ART pinged me again.
I want to talk, it said.
My risk assessment module sent me a low-level alert. Usually, ART just launched straight into whatever it wanted to talk about. What about? I asked.
ART paused for 4.7 seconds, long enough to make me worry a bit. Then it said: You think about dying a lot.
Risk assessment went into overdrive. What the fuck? I said, eloquently. What’s that supposed to mean?
Just statistically, ART said, like that explained anything.
I dismissed the list of media I’d been scrolling through. My mood was completely ruined. How would you know what a normal amount of thinking about dying is?
There are studies. ART sent me several academic papers I was obviously not going to read. Admittedly these deal with human psychology, but—
I ran a diagnostic and yeah, my stupid cortisol was spiking. I got up from the couch so I could pace. Of course humans think about it less. Their brains are slower.
ART was silent for another 3.6 seconds. Then it said: Even adjusting for a higher amount of neural activity per minute, you think about dying a lot.
I don’t think that’s true, I lied. I actually did think it was true, but I didn’t want to talk about it, and I especially didn’t want ART to ask what I’d been thinking about right before it pinged me by the cargo docks.
It pushed a dataset it must have collected while sharing my systems at me through the feed. I made a point of deleting it where ART could see, which was stupid because obviously it still had the data. You fucker. I let you into my head because I was trying to help you.
I am trying to help you too.
I have a dangerous job, I said. I have to think about dying so I don’t.
For just long enough that I thought it might have dropped the matter, ART said nothing. I should have realised it was working itself up to something.
It said, When you think about dying, your systems do not indicate distress. They indicate relief.
I’m not having this discussion, I said, and turned off our feed connection.
I had timed my intervention perfectly. SecUnit was somewhere it felt safe and comfortable that was not inside me, and as most of my crew were taking a rest period (I had lied about them playing Torsion), SecUnit’s absence would likely not raise any concerns for several hours.
It had still gone very badly.
I placed the probability of SecUnit causing itself physical harm at 1.2853%. If it truly intended to harm itself, it would have done so by now; our missions provide ample opportunity, and while the data I had gathered during my time in its systems added a troubling dimension to its habitual recklessness, I had been unable to discern any real intent behind its ruminations. My concern was entirely for its emotional state.
Perhaps it would have been better if I had mentioned the issue as soon as I became aware of it. But I knew SecUnit relied on its media for emotional processing, and I hadn’t wanted to start a difficult conversation when it had no access to its files. I also hadn’t wanted to put it in the position of having to choose between privacy and forcing a restart of the minimal functionality protocol with me in the run box. And when I offered it assistance after the fact, it explicitly asked me not to be weird about it.
I probably should have mentioned it sooner, though.
Reaching for SecUnit in the feed when it had turned off its connection felt like trying to call up one of my own systems and finding it absent. Despite my desire to give SecUnit space, I kept finding myself trying to reestablish a connection.
SecUnit had promised Senior Security Officer Indah not to hack Preservation station’s systems, but she had not extracted the same promise from me. I accessed the sole camera in former planetary leader Mensah’s office, where SecUnit was pacing in front of the couch. It looked distressed, and I was distressed that it was distressed. I was so distressed that my advanced vocabulary module stopped returning words other than distressed and I had to reboot it.
Iris pinged me. Peri? Are you okay?
You should be sleeping, I said. Most of the crew had retired to their cabins or left to visit friends on the station an hour ago, just before I pinged SecUnit.
That’s a no, then. She sat up in her bed. I can feel you bouncing around the feed like a skip-bug. What’s wrong?
I can’t talk about it. Like I said, my vocabulary module was acting up. (That wasn’t the reason.)
Where’s SecUnit? Are you fighting?
I didn’t bother replying to that. Iris is very intelligent and could see as well as I could that SecUnit’s feed profile was greyed out. I’m handling the situation, I lied.
Okay. Iris paused. Do you want to watch Cruel Romance Personage with me?
I pinged a negative. Part of me did, and I could easily have watched media with Iris while waiting for SecUnit to return to the feed. (It would be better than watching the camera view of Mensah’s office, which was probably a violation of both SecUnit’s privacy and Preservation law. I didn’t really care about the latter, but calculated that SecUnit would be even more upset if it caught me spying on it. Grudgingly, I turned off the input.)
But I didn’t want to watch media while SecUnit was in this much distress. Not tonight, I said. Go back to bed.
Iris sent me three quick pings, which is our version of a hug, and set her feed status to Sleeping.
I backburnered all my inputs other than one camera outside Mensah’s office, just in case, and waited.
What really pissed me off was that ART wouldn’t even have known any of this if I hadn’t let it share my systems. We had been working really well together, other than the occasional disagreement about media (the acting in the Destination: Wormhole spinoff is not better than in the original, ART just likes one of the actors because xie looks like Iris), and the thing where we were still kind of working through how it had endangered my humans without meaning to.
Just under two hours after I’d closed our feed connection—I wasn’t keeping track of the time—I opened it again and felt about 89% of ART’s attention on me immediately.
If it’s so horrible inside my head you can spend four days watching documentaries with Three next time, I said, bitterly. I knew that would hurt it. Three’s taste in media is terrible.
ART spammed me with a flurry of emotional metadata, mostly worry and hurt. The experience of sharing your hardware is a memory I will cherish until my processors burn out, it said.
I sent it an image of the ship’s navigator in World Hoppers making a skeptical face.
It puffed itself up so it was bigger in the feed, not pressing down on me like it does when we’re not fighting but running additional processes in the systems I have access to so it seemed extra present all around me.
Should Bharadwaj and Pin-Lee fail to obtain an exception for me with the University, I would choose the run box over sharing a system with anyone but you.
Okay, that did something to my organic parts. I hoped ART wasn’t watching me through the station security cameras, because I didn’t have control of my facial muscles right now.
After 4.2 seconds of trying to be more angry than upset, I admitted, I don’t want you to have to go back in the run box.
Neither do I. ART stopped running the extra processes and sidled a little closer to me, still not touching.
I decided to deploy the very good and objective explanation I had for thinking about dying 47% of the time that my systems were not occupied with necessary processes. Modelling simulated terminal events enables me to better anticipate real-life hazards and increases my reaction times by 0.0002%.
I thought that was pretty convincing. It had taken me ages to formulate the query so that it gave me a positive result. It even sounded like something ART would say.
Unfortunately, ART was not convinced. On one occasion, for no reason I could discern, you spent 53 minutes contemplating projectiles ripping through your armour and destroying your internal hardware, resulting in a range of outcomes from catastrophic power cell failure to critical fluid loss.
That was a combat simulation, I replied weakly.
If it was a combat simulation, why did you never imagine yourself returning fire?
I turned off the feed again.
I spent the next five hours thinking about getting disassembled. It felt awful, but also sort of relaxing. It kept me from thinking about why I wanted to think about getting disassembled, and why I deserved to be.
None of it was real, anyway. And it’s not like I was actually planning to get myself captured by the company (I wouldn’t give them the satisfaction, and also, thinking about doing that on purpose actually did make my performance reliability drop by 4%) or jump out of an airlock or anything. It was just nice to know it was an option.
One of my cameras in the hallway pinged an alert to tell me there was someone at the door.
It was Mensah.
For fuck’s sake, ART.
I considered not letting her in. But it was her office, and the only thing in here that was dangerous was me, so I didn’t really have an excuse not to.
She stepped through the door and looked at a bookshelf 1.6 metres from my head. ‘Perihelion told me you were here. It didn’t want you to be alone.’
‘I don’t need babysitting,’ I said, facing the wall. ‘I’m not going to damage myself.’
My drone camera picked up a twitch of concerned surprise in Mensah’s expression. She hadn’t considered that possibility; ART really hadn’t given her any details. I’d done that all by myself. Great job, Murderbot.
But Mensah just quietly unpacked the small bag she’d brought with her. It contained a beverage, a carbohydrate and protein bar made from grains grown on the planet, and a book. She put all three on the side table next to the couch, and sat down.
‘No, you don’t. But I thought you could do with some company anyway.’
And without saying anything else or expecting an answer, she sat with me until morning.