Lab

This is a working notebook of experiments, models, and observations.


How I manage trust and dependency on AI

I'm sure you've read articles about how safe it is, for the user, to be using AI. There are arguments that using or overusing AI can undermine your critical thinking skills and your ability to write in your own voice. I've developed my own habits over time that seem to protect me from some of the worst aspects of that. The strategies I use are distrust, rejection, non-authorship delegation, induced pushback, and misinterpretation awareness.

Distrust: I don't "believe" a single thing told to me by an AI chatbot, but in some situations I will act on the information without belief. What I won't do is act on information without accepting the risks. I also rely on a quick failure/feedback loop, so I will use the information without critical analysis only if failure would be obvious, immediate and quicker that further developing my approach. To be honest there are still risky behaviors I engage in. For instance when installing my local AI I dumped error messages into the chatbot to get advice on how to fix them. Doing that could introduce security holes. So I have to compensate with additional attention to security. The important thing is to keep looking for those issues. Checking multiple AIs and/or asking to confirm the AI instructions will likely fix the issues.

Induced pushback: I regularly use copilot and Gemini. I have given both two separate pushback instructions. 1. Explicitly challenge any assumptions I make about equity seeking groups. My demographic is privileged and so some things are invisible to me. AI doesn't understand the perspective of minority groups, but it can reflect minority perspectives that were part of its training data. Nothing can compensate for a lack of minority voices, but the frequency at which it tries helps build the habit into my own mind. 2. Don't blow smoke. I would much rather be challenged now before I commit to a path. 3. I explicitly ask for critique of my prompts, assumptions, and my own work. Together those mechanisms 1. remind me to keep other people's perspectives in mind, 2. limit how over-confident I get based on automatic feedback, and 3. test the assumptions and arguments in my work. I don't try to do all of these things at once. I find i get better results if I separate these.

Non-authorship delegation: AI offers to write for me all the time. I reject this every time it offers. AI is very good at helping with a given subject, but I find the framing it provides is often over-simplistic because it's mostly re-constructing the most common form of a subject discussion. I do occasionally ask it to draft something, but it's very rare. Everyone's voice is unique, and I think that's more valuable than I sometimes recognize. More importantly editing an established draft that was written by AI, while easier, makes me focus on the framing that was set by the AI. I can easily spot errors, but omissions are invisible. They're far harder to spot when I'm following someone else's logic rather than writing my own. So I write my own draft first and then have AI edit. And because I use it to edit...

Rejection: I argue with the editor. I use AI to edit these articles, but I don't do what it says blindly. The biggest way this manifests is in metaphors. To communicate effectively with a human you need to get an idea across, not be perfectly accurate. AI will frequently remind me that a metaphor isn't perfect. But often I have chosen that metaphor deliberately because making it perfect will make it boring and ineffective to a human reader. If you ask an AI for feedback, you are cueing it up to give positive and negative. I am literally asking for criticism, so it will try to provide that. If I kept asking for feedback on a piece it would keep providing feedback and I can iterate forever. I stop when I'm rejecting nearly all the feedback. Not only does this protect my voice but also keeps the work my own.

Misinterpretation awareness: I know what it's doing and approximately how it works, and I expect certain kinds of misinterpretation: I underspecify the prompt, i rely on synonyms that have imprecise definitions, I expect the momentum of a conversation to dictate the direction of the next response. It's often my own fault for under-specifying what I want, but whether it's me or the AI I blame, the system overall is generating errors.

And finally, a lot of what I do is new to me. These are things I didn't have another tool for previously. So generally speaking I've got nowhere to go but up. I wrote a letter to my MP, the PM, the minister of AI and the Competition Bureau. Never done that before. I know how to write a letter, and I've sent one or two political ones before, but I've never had a tool to check my implications before, or flag that the way I've phrases something is too crystallized and easy to refute. That's a win for civics. I used it to teach me how to build my own AI system at home and how to use a 3d printer. Those are wins for sovereignty. If it all fails, I tried something new and learned some things.