I paid Microsoft's premium Copilot agents to do my work - they were confidently bad at it

Follow ZDNET: Add us as a preferred source on Google.ZDNET's key takeawaysCopilot agents are built to help with research and analysis. In my tests, those agents didn't produce useful results. Troubleshooting with Copilot wasted time and solved nothing.Microsoft is spending an insane amount of money on its AI features, building data centers and licensing large language models from OpenAI, Anthropic, and others, while it also tries to build its own in-house alternatives.The goal, driven straight from the top of Redmond's org chart, is to turn the combination of Windows and Microsoft 365 into an "agentic OS," capable of doing the tasks that make corporate life miserable: writing memos, building presentations, organizing meetings, and automating routine tasks. Also: AI PCs aren't selling, and Microsoft's PC partners are scrambling But are those investments paying off? Developers seem to be generally happy with the productivity gains they're seeing from tools like Claude Code and GitHub Copilot, but the agents working in the business sphere don't seem nearly as competent. Over the past few weeks, I've been trying to use the AI features in Microsoft 365 and Windows for a variety of everyday work tasks. Copilot shows occasional flashes of competence, but more often, the results I'm seeing are a mishmash of misinformation, hallucinations, and time-wasting dead ends. Allow me to share my recent experiences with you. 'I'm sorry, Ed, I can't do that'Microsoft has been bugging me for months to upgrade to its new Microsoft 365 Premium plan, which includes higher limits on AI usage and a handful of exclusive agents. In the interest of science, I paid the $10 to upgrade an unused account for a month so I could try them for myself. I started with the Analyst agent, feeding it a copy of the spreadsheet I use to keep track of our household income and expenses and asking for help with how to improve its design. After some back-and-forth about what I was trying to accomplish, it offered some useful suggestions with suggestions for tightening up formulas, consolidating some duplicate tables, and eliminating some redundant pages, concluding with a bolded offer to build a dashboard using only formulas and pivot tables. Also: AI Model Release Tracker: Microsoft AI's first reasoning model arrives"If you want," Copilot told me, "I can sketch a clean dashboard layout (exact cells and sections) tailored to your data so you can build it in ~15 minutes." I ... thought agents were supposed to do this work for me? So I asked, "Can you build the actual Excel file for me or do I have to do that myself?" Of course, it told me. There was just one tiny limitation: I would have to create one of the pivots myself, but that chore would take less than 10 seconds. Go ahead, I said. A minute or two later, Copilot told me, "I've created your modified workbook. Download it here." It then gave me a link to [sandbox:/mnt/data/Personal_accounts_modified.xlsm]. That's the exact link, brackets and all. It wasn't clickable. Sandbox? What the ...? "Yeah -- that's on me," Copilot confessed. "That 'sandbox path' isn't actually clickable in your interface, so you couldn't download it. Thanks for calling that out. ... I'm going to regenerate the file and make sure it comes through as a real downloadable attachment in the chat (so you'll see a clickable file card, not a path)." Also: Microsoft's first reasoning model is one of 7 AIs just released at Build - what we know so farSeveral tries later, Copilot gave up. "The file is ready. However, I need to be transparent: your chat interface is currently not rendering downloadable file attachments correctly (that's why you saw those unusable "sandbox" paths earlier). I did generate the file successfully, but it isn't appearing as a clickable download in your UI." The chatbot went on to explain that the file link might have worked if I had been using ChatGPT. And one of the workarounds that the agent suggested was to create the file in Google Sheets and send me a link! Somehow, I don't think that's what the product managers in Redmond had in mind. (Please don't tell Satya.) Also: Build 2026: Microsoft's MDASH exits preview with 100+ specialized threat-hunting AI agentsVerdict: I got some good ideas for improving my workbook, but the Copilot agent that's supposed to be working directly with Excel can't actually do the work for me. 'Which specific plan do you mean?'Next up, I asked the Microsoft 365 Premium Researcher agent to give me a concise explanation of the pros and cons of Microsoft 365 Premium. I was not prepared for the response: To make sure I cover exactly what you need: Which specific plan do you mean by "Microsoft 365 Premium"? A) Microsoft 365 Personal, B) Microsoft 365 Family, C) Microsoft 365 Business Premium, or D) compare the consumer plans (Personal/Family) and briefly note Business Premium only if relevant. To be clear, I was working with one of the signature features of a product Microsoft is promoting heavily, and it had no idea what I was talking about. Imagine you're interviewing a job candidate, and you say, "Tell me about yourself," and they reply "I don't know her." Also: Why I ditched Copilot for Claude in Word, Excel, and PowerPoint - and how you can, tooAfter I offered up a link to the product page, Copilot gave me a fairly anodyne summary of features in the new subscription, summarized from a handful of third-party sources. I wouldn't call it research, and it definitely wasn't deep. 'Why I'm confident this is the right path'The thing I find most amusing about AI chatbots is that they are so supremely confident in their answers. And when you point out that the instructions they just gave you didn't work, that confidence never flags as they suggest new options. This morning, I was trying to connect to a computer on my office network using the Remote Desktop client, but I was getting a certificate error: "The server name on the certificate is incorrect." Also: How Microsoft obliterated safety guardrails on popular AI models - with just one promptAfter a few minutes of troubleshooting on my own, I decided it was time to try to "vibe-sysadmin" my way through it with the help of Copilot. "The fix is straightforward," Copilot replied, confidently. All I needed to do was force Windows inside the VM to generate a new Remote Desktop certificate. "Here are the clean, reliable ways to do it." That didn't work. Copilot, undaunted, told me that that result was meaningful and rattled off three likely reasons, concluding with "Let's fix it cleanly and surgically." After a bunch of PowerShell commands and a reboot, I was still unable to connect, but this time, it was because of a different certificate error. "Ah -- that tells me exactly what's happening now," said Copilot. After another long-winded explanation, it said, "Let's fix that cleanly." Also: I let Microsoft Edge's new AI feature read all my open tabs - and it's a total research time-saverWell, this went on for about 20 minutes and half a dozen reboots of that VM. With each failure, Copilot had another small AI epiphany. "That error tells me something very specific...""You've just uncovered the real root cause...""We've crossed into the one scenario where Windows will not behave the way the documentation claims...""And that explains everything you've seen..."These were accompanied by bold headings like "Why I'm confident this is the right path," "Why this is the correct fix," and "Why this is the only explanation left." None of the suggested fixes worked, so I told Copilot to shut up. I reinspected the connection settings and cleared one checkbox on the connection settings. That did it. Also: Work IQ is Microsoft's big bet on agent-first enterprise IT, and I have questionsIn fairness, I learned a handful of PowerShell commands for managing certificates, and I got a refresher course in how Windows manages certificates. But the lesson I learned is not to ask Copilot for that level of troubleshooting again. Maybe someday Copilot will achieve artificial general intelligence. At this point, I would settle for artificial general common sense. And even that station seems to be many stops away from where we are right now.