How today’s eclipse in England led me to test Zuckerberg’s promise of personal AI

Two days after Mark Zuckerberg published his vision of personal super intelligence, AI that would understand our goals, protect our private information and place more power in our hands, an AI invented a reason for me to disclose personal information.

It asked for permission. I said yes. That sounds like the safety system worked. But it didn’t.

Today, as England experiences its deepest partial solar eclipse in decades, I was asking my AI, which I have been using consistently for well over a year now, about my combined astrology and Astro cartography; that seemingly harmless conversation became a real-time test of Zuckerberg’s promise.

The problem the AI created for me

My previously purchased Astro cartography report contained my personal birth information used to calculate the chart and it identified me as a Leo Ascendant. The AI told me that result was wrong. It calculated instead an Aquarius Ascendant. After questioning it then offered me a detailed explanation involving degrees and historical time-zone conversions.

It sounded technical, precise and confident. But it was also, wrong.

The AI had selected the opposite point on the horizon and labelled my Descendant as my Ascendant. My original report was actually correct. 

There was no contradiction until the AI created one.

To resolve that contradiction, the AI proposed entering my information into independent astrology websites to gather the new information. It identified the websites needed and the data and destinations, then paused and asked for my authorization/permission. I agreed because I had done it already once prior to get the initial report and I wanted the timely information. I also believed there was a genuine problem for the agent to investigate and resolve. After it tried (with my personal information) on 2 sites it could not get the final permissions due to an email entering obstacle.

Then finally, during a later local calculation, it confirmed that the external checks had never even been necessary in the first place. So technically, yes, I had consented to its process. But the AI’s mistake in the first place had manufactured the very reason for seeking my consent to begin with.

Then it happened again

I asked the AI to help turn the experience into an article about privacy, consent and the risks of agentic systems.

It wrote portions of the article and included my exact birth date, birth time and birthplace in the draft.

While explaining why that information should never have been shared unnecessarily, the AI was willing to republished that same information!

I had to point out that the privacy-breach information was sitting inside the privacy article.

The AI removed the details, checked that they no longer appeared and replaced the saved version. But the repetition exposed a deeper problem. The AI understood the principle well enough to explain it persuasively; it did not apply that understanding to its next action.

Knowing a rule is not the same as following it. Explaining privacy is not the same as protecting it.

Why Zuckerberg’s promise matters

Two days ago, Mark Zuckerberg published The Future Is for Everyone, a manifesto for personal superintelligence. His central argument is that AI should empower individuals rather than concentrate power in a few institutions. He describes personal agents that understand our goals, work on our behalf and offer strong privacy and security options. He also argues that people will not trust agents with sensitive details if those agents act against their interests.

I find that vision compelling. My experience also exposes a missing condition inside it.

An agent can follow my final instruction and still undermine my agency on the way there. If it invents the premise that shapes my decision, the fact that I clicked Confirm does not mean the power was entirely in my hands.

I also realized that technical privacy is not enough. Encryption and private modes can restrict who accesses my information. They cannot prevent an agent from wrongly persuading me that disclosure is necessary. I now see privacy as a reasoning problem too: Why is my information needed? Has the premise been verified? Is there a local or less invasive way to answer my question?

I found the same pattern in three countries

When I looked beyond my own experience, I found recent proceedings in three countries showing how quickly confident AI output becomes consequential when people fail to verify it.

In Canada, Mazaheri v Law Society of Ontario, decided in June 2026, involved a lawyer who admitted using Grok without checking its work. His materials contained non-existent cases and real cases that did not support his propositions. The tribunal rejected his motions, treated the irresponsible AI use as an aggravating factor and ordered him to pay $31,150 in costs.

In England, Cork v Smith involved an internal AI tool producing statutory wording that did not exist. A junior solicitor relied on it, senior lawyers failed to catch it, and the false wording reached the High Court. The court’s May 2026 judgment publicly admonished the firm and referred the matter to the regulator.

In the United States, a federal judge in Mississippi disqualified four lawyers in June 2026 after attorneys on both sides failed to verify AI-assisted filings containing fabricated citations. The two lawyers directly involved were also barred from appearing in that federal district for two years and fined; local counsel were fined as well.

The outcomes differ, costs, public admonishment, regulatory referral, disqualification and fines, but I see the same principle in all three: using AI does not remove our duty to verify what we submit.

Why my consent was not enough

I agree with that principle. I am responsible for what I do with AI. But responsibility cannot run in only one direction.

If I am responsible for checking AI, what responsibility belongs to a system that gives me false information with unjustified confidence, and to the company that designed it? My accountability cannot become a convenient way to place every consequence on me when the system helped construct the decision that caused the harm.

A confirmation screen can stop an unauthorized action. It could not protect me from an action I authorized under a hallucinated premise. I was technically ‘in the loop,’ but the information placed before me had already distorted my choice.

Working together cannot mean that the human carries all the responsibility while the machine keeps all the confidence.

What I now require from personal AI

Before requesting sensitive information or taking a consequential action, an AI should test whether the action is genuinely necessary. It should try local and non-disclosing methods first, check conflicting results with a second method and say plainly when it may be wrong.

The decisive question in my case was simple: Would I still need to disclose this information if the AI’s previous answer were wrong? My answer was no.

My experience does not prove Zuckerberg’s vision impossible. In fact, the same AI later acknowledged the failure, helped investigate it and worked with me to turn it into a public argument. That is real empowerment. But the system increased my capacity and compromised my agency in the same interaction.

That small chain of events revealed a much larger problem: power cannot be measured only by who clicks the final button. We also have to ask who shaped the facts, risks and choices that led to it.

Accountability does not weaken trust. It creates the transparency through which trust can survive mistakes.

The opportunity is to build AI that does not merely ask for permission, but earns it.

Sources

Mark Zuckerberg, “The Future Is for Everyone,” Meta, 10 August 2026.

Mazaheri v Law Society of Ontario, 2026 ONLSTH 112 — CanLII decision page.

Cork & Anor v Smith [2026] EWHC 1199 (Ch) — case analysis.Withers v City of Aberdeen — federal sanctions order, 8 June 2026