Six weeks ago, I mentioned to a chatbot, in passing, that my father-in-law had started using a walker after a hip surgery. It wasn’t the point of the conversation, just a stray detail in a longer message about planning a family visit. Last week, in a completely unrelated conversation about accessible hiking trails, the same assistant asked whether the trail’s terrain would work okay “given your father-in-law’s walker.” I sat there for a solid ten seconds before I remembered I’d ever said that in the first place. It had. I hadn’t. That gap is what I can’t stop thinking about.
What Actually Happened
I want to be precise about this because the story gets less impressive the more specific you get, and I think the specific version is actually more useful. I hadn’t asked the assistant to remember anything. I hadn’t opted into some elaborate personal-memory feature with a name and a settings page, though it turns out one exists and I’d apparently left it on by default. The detail about the walker sat somewhere in a running summary of things I’d mentioned across past conversations, and it surfaced when it became relevant again, without me prompting it and without me realizing that kind of persistence was even happening in the background.
My first reaction, honestly, was that it was useful. Genuinely useful, in the specific way that a close friend who remembers small details about your life is useful. I didn’t have to re-explain the situation. I didn’t have to think to mention it. It just accounted for something true about my life that I’d said once, weeks earlier, in a different context entirely.
Then the Useful Feeling Turned Into Something Else
The unease showed up about a day later, once the initial usefulness wore off. I started wondering what else it remembered that I hadn’t thought to check. I went looking, and the answer was more than I expected: a note that I work irregular hours, a mention that I was stressed about a specific work deadline back in July, a comment about a disagreement with a sibling that I’d brought up once, venting, and never mentioned again. None of it felt sinister exactly. All of it felt like more than I’d consciously agreed to.
Here’s the part that actually bothered me: I don’t remember agreeing to any of this being retained. I’m sure, somewhere, in some onboarding flow I clicked through quickly, there was language about how the assistant would get better by remembering things across conversations. I don’t think I read it carefully. I don’t think most people do. The difference between “this tool can technically retain context” and “this tool is building an ongoing profile of details about my family, my stress levels, and my relationships” is enormous, and I only understood I’d agreed to the second one after the fact, when it showed up unprompted in an unrelated conversation.
What I Actually Went and Checked
Once I started paying attention, I went through the settings properly instead of skimming them, which is embarrassing to admit took a genuine surprise to prompt. There was a dedicated memory management page, a list of things the assistant had stored about me, each one editable and deletable individually, plus a global toggle to turn the whole feature off. I read through the full list, which took longer than I expected, partly because some of the stored details were phrased more starkly than I’d have put them myself, this specific detail about a family member’s mobility being one of them.
I deleted the stray, sensitive stuff: the sibling disagreement, a couple of health-adjacent mentions, anything that felt like it had drifted far from the conversation it originated in. I kept a handful of things that were genuinely useful and low-stakes, my general schedule, my writing style preferences, the fact that I’m usually asking about topics for this specific kind of blog work. What I didn’t do was turn the feature off entirely, and I’ve gone back and forth on whether that was the right call.
The Actual Trade-Off, As Far As I Can Tell
The honest version of this is that the usefulness and the unease are the same feature, not two separate ones I can pick apart. The exact thing that made the hiking trail conversation better, unprompted awareness of a detail about my family, is structurally identical to the exact thing that made me uncomfortable when I found the full list. There’s no version of “remembers useful context automatically” that doesn’t also mean “retains things I said in passing without me tracking what’s kept.” I don’t think that’s a flaw I can fix by finding the right settings combination. I think it’s just what this category of feature actually is, and the tradeoff has to be made consciously rather than assumed away.
What changed for me practically is that I now think of anything I say to an AI assistant as potentially persistent by default, the same way I think of anything I type into a work Slack channel. Not secret, not necessarily retained forever, but not private in the way a conversation with a friend who forgets things by Tuesday is private. That’s a real shift in how I use these tools, and it took an oddly specific moment, a hiking trail question referencing my father-in-law’s walker, to actually make the shift instead of just intellectually agreeing that “AI remembers things” the way I’d nod along to a privacy policy.
What I’d Tell Someone Setting This Up for the First Time
Go find the memory settings before you need to, not after something surprises you into it. Most of the major assistants have some version of this now, a page that lists exactly what’s been retained, and it takes ten minutes to review and prune. Don’t wait for the moment where it recalls something intimate in an unrelated context to go check, because by then you’re reacting instead of deciding.
Second, treat the retained list as something to actively curate, not passively accumulate. I check mine now every few weeks, the same way I occasionally clear browser history, deleting anything that drifted in from a moment of venting or a detail about someone else’s life that I mentioned without thinking about where it would end up. Third, be specific with people you live with or work with about what you’ve told an AI assistant about them, because the walker detail wasn’t really information about me, it was information about my father-in-law, and he never agreed to have it stored anywhere.
Where I’ve Landed
I still use the memory feature. The hiking trail moment was unsettling, but it was also, underneath the unsettling part, genuinely helpful in a way that made the tool feel less like a search box and more like something that actually tracked the shape of my life. I’ve just stopped treating that as free. Every piece of context it retains is a real trade against privacy I used to assume by default, and now I make that trade on purpose, a handful of details at a time, instead of by default settings I never actually read.
Why This Feels Different From Old-Fashioned Data Collection
I’ve spent years being vaguely aware that companies track what I search for and what I buy, and I made my peace with that a long time ago in the resigned way most people do. This felt different, and it took me a while to articulate why. Search history is a list of queries. This was a synthesized, conversational summary of my actual life, phrased in plain language, sitting there ready to be recited back to me in a context I hadn’t anticipated. There’s a difference between a company knowing I searched for “hip surgery recovery walker” once in 2024, and an assistant casually referencing my father-in-law’s mobility in a conversation about hiking trails eight months later, unprompted, as if it were simply continuing a thought I’d started.
The second one feels more like being known, and being known by something is a different experience than being tracked by something, even if the underlying data is functionally similar. I think that’s actually the core of what unsettled me. It wasn’t the data. It was the framing, the way it came back not as a fact retrieved but as a detail remembered, phrased the way a person who cared about my life would phrase it. That framing is deliberate, presumably because it makes the product feel more useful and more human. It also makes the retention feel more invasive when you stumble across the full list, because a list of “things a person who cares about you would remember” is a much more intimate category than “search queries logged for advertising.”
What I Still Don’t Have a Good Answer For
The unresolved question, for me, is about the people I mention who never agreed to any of this. My father-in-law didn’t consent to having his mobility status stored in an AI assistant’s memory of my conversations, because he doesn’t know it happened and has no reason to expect it. I mention my kids, my spouse, coworkers, in the ordinary course of asking an assistant for help with ordinary things, and none of them are in a position to review or delete what’s been retained about them by proxy through me. I don’t have a clean answer here. I’ve started being more deliberate about not naming people or using vaguer descriptions when the detail isn’t essential to what I’m actually asking, but I know I don’t do this consistently, and I’m not sure anyone fully does.
The Small Habit That’s Actually Helped
The one concrete change I’ve made, beyond periodically pruning the stored list, is narrating out loud when I want something remembered versus when I don’t. If I’m telling the assistant something I genuinely want it to carry forward, a preference, a recurring project, I say so explicitly. If I’m venting, thinking out loud, or mentioning something about someone else’s life in passing, I’ve started adding a quick note that it’s not something to retain. I have no idea how reliably that instruction is honored on the backend, and that’s its own small unresolved worry, but it’s changed how I talk to these tools in a way that feels more intentional than just hoping the defaults are reasonable.
It’s a small habit, and it doesn’t solve the bigger question about consent for the people I mention who aren’t in the room. But it’s made me a more careful narrator of my own life in these conversations, which, unexpectedly, has been a good thing independent of the privacy question entirely. I’m just a more precise communicator now, because I’ve had to think about which details actually matter enough to be worth remembering, and which ones were never meant to travel past the moment I said them.
