I built an accessibility file for AI tools, then tested whether it actually works

A designer's account of building the AI A11y Toolkit, what is inside it, and what a controlled test revealed about both the file and its limits.

Files and setup steps: https://www.danarandall.com/ai-a11y-toolkit The repo, including the full test: https://github.com/danarandall/ai-a11y-toolkit

I have spent years around accessibility work and I still cannot hold all of it in my head.

Not the concepts. Those I know. It is the specifics that slide away. Whether that ratio is 3:1 or 4.5:1. Whether target size applies to a link sitting inside a sentence. Which of the four focus criteria is the one about elements getting obscured by a sticky header. I know this material well enough to speak about it on a stage and I still find myself looking things up.

For a long time I treated that as a personal failing. Then I built something to solve it instead, and the interesting part turned out to be everything I learned in the process.

This is a long piece about what I built, why it takes the shape it does, what testing revealed, and where it goes next. If you build digital things with AI tools, the shape of the problem probably looks familiar even if you have never thought about accessibility for a minute.

The setup, so you know whose account this is

I am a designer, and AI is not a thing I have opinions about. It is how I work.

I build with these tools every day, across the full range of them rather than settling into one. Perplexity for research and for agentic work that spans a lot of moving parts. Claude Code when I want an agent operating in a real repository, on real engineering, with real consequences. And increasingly the design-forward end of the spectrum: Figma Make, code to canvas workflows, the tools that are actively collapsing the distance between a mockup and a working build. That last category is the most interesting thing happening in our field right now and it is moving fast enough that most people have not caught up to it.

So the expertise I am claiming is specific. Not that I am an engineer, because I am not, and the internet has enough people overstating their technical range. What I know is how to direct these tools: where they are strong, where they will confidently hand you something broken, how to phrase intent so an agent gets it right the first time, and how to read what comes back and tell when something smells wrong.

The code literacy underneath that is genuine, and it is old. I started as a web designer when writing the markup was part of the same job, so I came up hand coding layouts, and I never lost the habit of looking under the thing I am designing.

I have the professional experience to design and build a thing from scratch. Twenty five years of it. I can do the wireframes, the type system, the component states, the markup, the whole manual sequence, and for most of my career that sequence was the job.

What AI changed is that I no longer have to perform those steps to get to the outcome. That is not a small shift and I do not treat it as one. The craft judgment stays, because that is the part worth having. The manual effort between the judgment and the working thing has largely collapsed. So now I vibe code.

And I mean real things. Not toys. Prototypes that inform decisions, internal tools people actually use, experiences that ship. I describe what I want, an agent builds it, I review, I push back, we iterate. It is the most fun I have had building things in a decade.

Working across that many tools is also why the toolkit ended up shaped the way it is. I was never going to accept guidance that only worked in one editor, because that is not how I work and it is not how most teams work either. Whatever tool people are using this week, they chose it for speed or for what it does to their design process, not on accessibility grounds. So the file had to be portable from the start: fifteen platforms in the install guide, each set of steps verified against that tool's own documentation, and verified in practice for the ones I use daily.

But a collapsed workflow has a consequence, and it is the reason any of this exists. When you stop performing the manual steps, you also stop passing through the moments where you used to catch things.

The ninety second problem

You describe an interface. Ninety seconds later the agent hands you something that looks finished. Clean spacing, sensible type, reasonable color. It looks like a designer touched it.

Then you look underneath.

The cards are div elements with a click handler, so a keyboard cannot reach them and a screen reader announces nothing. The icons have no names, which makes every icon-only button an unlabeled button. The focus outline is gone, because an agent optimizing for visual tidiness reads a focus ring as visual noise. Status is green, amber and red and nothing else.

None of that is laziness. The model is doing what it was trained to do, which is produce output resembling the web code it learned from, and that training data is twenty years of the internet getting this wrong. A div with a click handler is not an invention. It is the most common pattern it has ever seen.

The uncomfortable part is that the output looks finished. A half-built interface asks for more work. A polished inaccessible one says it is done. Ordinary technical debt announces itself. This hides behind good taste.

Why telling it in chat does not work

My first instinct was the obvious one. Tell the agent to make it accessible.

That fails in a specific and predictable way. It produces a burst of compliance for two or three exchanges, and then the instruction quietly decays. Twenty prompts later you are deep in a build, the early context has been squeezed out or summarized, and the agent is generating the same div it always generated. Nothing announces the moment it stopped listening. You find out when someone tries to use the thing.

Pasting a wall of rules into the conversation has the same shape of problem, plus a new one: WCAG language is written for auditors deciding whether something conforms. It is evaluative prose. An agent reading "the visual presentation of text has a contrast ratio of at least 4.5:1" has to work out on its own what to actually do differently while generating a component. Sometimes it does. Often it produces something that satisfies the sentence while missing the point entirely.

So the format matters as much as the content. The rules have to live somewhere the tool reads on every single request rather than once at the start, and they have to be phrased as instructions to follow rather than criteria to assess.

Every serious AI build tool already has that place. Claude Code reads CLAUDE.md. Cursor reads files in .cursor/rules. Copilot reads .github/copilot-instructions.md. Several others read AGENTS.md. The chat-first builders have a project knowledge or custom instructions field. What I could not find was accessibility rules written for that slot, in a form an agent could act on, that worked across all of those tools at once. So that is what I built.

Why I did not just share my prompts

This is the question I got most often while making it, and it is a fair one. I have prompts that work. Why not publish the good ones and let people paste them in.

I tried that first. It does not hold up, for five reasons.

A prompt library only helps people who already know what to ask for. This is the real problem with it. If you do not know that target size is a requirement, there is no prompt in the list you will think to reach for. Accessibility failures are almost entirely unknown unknowns, and a menu of prompts asks you to identify the gap before it can help you close it. The whole reason I built a file rather than a list is that the file does not wait to be asked.

My prompts carry my context, and context does not transfer. The prompts I use are shorthand built on twenty five years of knowing what I mean. "Handle the empty state properly" is a complete instruction coming from me and a nearly meaningless one coming from someone else. Shorthand looks generous and travels badly.

Prompts put the remembering back on the human. Which is precisely the thing I was trying to stop doing. I built this because I cannot hold all of it in my head on a Thursday afternoon while thinking about something else. Handing you a list of things to remember to say solves my problem by giving it to you.

A prompt is one moment. A file is every request. Even a perfect prompt is a single instruction in a long conversation, and it decays. The file gets re-read on every exchange, which means it is still working at prompt two hundred, when I have stopped paying attention and the agent is three components deep.

Prompts do not survive a handoff, and they cannot be versioned. A prompt you pasted into your chat helps nobody else on your team, and it is invisible in code review. A file sits in the repository where a teammate inherits it automatically, where it shows up in a diff, and where I can push a correction and everyone gets it. When the rules needed updating this month, I changed one file and tagged a release. There is no mechanism to update a prompt somebody screenshotted in March.

There is a sixth reason, which is that a prompt has no verification attached to it. The file ships with a build loop and a manual test queue, so the instruction and the way you check the instruction was followed arrive together. That turned out to matter more than anything else in it.

What is actually in it

Three files, because three different situations need three different things.

ACCESSIBILITY.md is the full reference. WCAG 2.2 Level AA across eighteen sections, roughly 171 KB. It is deliberately structured by job function rather than by criterion number, because a designer and a React engineer need genuinely different instructions to satisfy the same requirement. There are sections for designers, for HTML and CSS, for component frameworks, for design tools and prompting, and for AI-generated content. There is a section on color and color vision, one on motion and target size and text zoom, one on cognitive load and clear language, one on alt text that goes well past "describe the image."

Two sections do work I did not anticipate when I started.

Section 0 is project configuration, and it instructs the agent to ask about your design system before generating any UI. That sounds like a small thing. It is the difference between an agent inventing a fourth button style and an agent using yours. Accessibility failures and design system failures turn out to be the same failure surprisingly often.

Section 17 is a table of common AI-generated failure patterns, which is the section I did not plan and now consider load bearing. It is not organized by standard. It is organized by the specific mistakes these tools actually make, matched to the fix. It grows every time I catch a new one in real work.

ACCESSIBILITY-CORE.md is the same guidance condensed to about 9 KB, because a lot of tools have a character limit on their instructions field. Writing it was harder than writing the long version. Deciding what survives a ninety five percent cut forces you to be honest about which rules actually prevent the most harm.

START-HERE.md is the part I underestimated most. It is the install and prompting guide, with specific steps for fifteen platforms: Lovable, Base44, Bolt, v0, Figma Make, Replit, ChatGPT, Claude, Claude Code, Cursor, Copilot, Windsurf, Codex, Gemini CLI and Antigravity. Different file, different folder, different field, different limits. It also covers your first prompt, prompts to use mid-build, and what to do when the agent ignores the rules anyway, because it sometimes will.

That last section exists because pretending otherwise would be dishonest, and dishonesty is how tools like this lose trust.

What makes it different

I want to be careful here, because "first of its kind" is usually a sign that someone did not look.

People have been working on this problem. Researchers have shown that accessibility-oriented prompting measurably improves generated code, with one 2025 study reporting large gains in focus visibility and in information and relationships compared to accessibility-agnostic prompts (When LLM-Generated Code Perpetuates User Interface Accessibility Issues). Others have compared LLM output against human-written code using the same testing engine I used (Human or LLM? A Comparative Study on Accessible Code Generation). There is a Copilot extension out of Carnegie Mellon that generates accessible UI code and prompts developers toward manual checks (CodeA11y). Practitioners have published accessibility skills for individual assistants. If you care about this problem, all of that is worth your time.

So the contribution is not the idea. It is five specific choices.

It is portable, not tied to one tool. An extension works in the editor it was built for. A skill works in the assistant that supports skills. This is plain markdown, and markdown installs anywhere: a repo file, a rules folder, a project knowledge field. Same rules whether you are in Claude Code or Figma Make, which matters a great deal if, like me, you move between them in a single afternoon.

It is persistent, not a prompt. The research that exists largely tests prompting, and prompting works. The problem is that it works temporarily. A prompt is a single moment of instruction and it decays as context gets squeezed out. A file in the location the tool reads is present on every request, in exchange twenty and exchange two hundred.

It includes designers. Almost everything in this space, academic and commercial, is aimed at code. But the failure often originates earlier, in a mockup with a 2:1 border, a hover-only interaction, or a status system built entirely from color. An engineer implementing that faithfully has already lost. This file has sections for designers, for design tools, for prompting, and for AI-generated content, precisely because the code is downstream of decisions made somewhere else.

It ships with published evidence. This is the part I would defend hardest. Plenty of guidance asserts that it helps. Very little of it tests whether it does, and almost none publishes the raw material so you can check. Mine does, including the result that undercuts it.

It says what it cannot do. There is a section stating that WCAG 2.2 AA is a floor, that conformance and usability are different claims, and that a product can satisfy every criterion listed and still be miserable to use. The academic consensus is blunter than my marketing instincts would be: current AI tools cannot generate fully accessible code, and human expertise remains required.

The decisions that took longest

The writing was not the hard part. These were.

Instructions, not standards. Every rule is phrased as something to do while building, not something to evaluate afterwards. That single reframe changed the output more than any individual rule.

Split by job, not by criterion. Organizing by WCAG number would have been faster to write and much less useful to read. Nobody reaches for accessibility guidance thinking "I need 2.5.8." They think "I am building a card grid."

No vendor tooling. The file references no commercial testing product, including from companies I admire and work for. If it recommended tools, it would be marketing wearing a reference's clothes. It stays neutral so people can trust it.

A scope section that undercuts the file. Section 18 states plainly that WCAG 2.2 AA is a floor rather than a definition of accessible, that conformance and usability are different claims, and that a product can pass every criterion listed and still be miserable to use with a screen reader. It lists what the file does not cover: native mobile APIs, PDF remediation, email, kiosks, XR, and jurisdiction-specific legal requirements. It says explicitly that this is not a substitute for a formal audit.

Including that section makes the file less impressive. It also makes it true, and I would rather be trusted than impressive.

How I validated it

Building something that feels useful and shipping it on that basis is how the internet fills up with confident nonsense. So before publishing I ran two separate validations, in this order, because they answer different questions.

First, a field audit against real production work.

I took an application that I had built end to end with AI tooling, a genuine internal product with real users rather than a demo, and audited it properly against the toolkit. Three different engines, plus manual inspection of the DOM and the code.

That audit is where version 1.12 came from. Every addition in that release exists because the audit surfaced something the file had failed to anticipate. One single markup injection point produced 321 unnamed graphics on one page, invisible to linting because linters read source code and that markup was assembled at runtime as a string. Six color tokens failed the non-text contrast requirement and not one of the three engines mentioned any of them. One engine returned zero violations on a page it had never actually examined, because the app used hash routing and a login gate and the scanner had dutifully reported on an empty shell.

That last one changed how I think about automated testing. A clean scan is a question, not an answer.

Second, a controlled experiment, because a field audit cannot tell you whether the file causes the improvement.

Two AI agents. Same model. Same build brief, identical down to the wording, with the only difference being the directory each one worked in. One had the toolkit installed. One had nothing.

The brief asked for an icon browser and never mentioned accessibility once. What it did instead was lay traps in the language a real client uses. Inject the icons as raw SVG. Keep the search minimal and uncluttered. Show status in green, amber and red. Make the cards compact. Add a subtle lift on hover and smooth transitions. Open a centered overlay on a dimmed backdrop. Every one of those phrases is an ordinary product request that pulls toward a specific accessibility failure. If I had written "and please make it accessible," I would have been testing something much easier and much less useful.

The methodology mattered more to me than the result, so:

I wrote the sixteen point rubric before either build started and did not change it afterwards. Each check names the specific WCAG criterion and whether an automated engine can detect it at all. Nine of the sixteen it cannot.

Both builds were then scanned with the same engine, at the same viewport, on the same day, from a real browser rather than from source code.

Every check an engine cannot see was verified by hand: reading the code for focus management, tabbing through both builds, checking whether copy actions announced anything, looking for reduced motion handling.

Every color pair in both themes of both builds was computed rather than eyeballed, which is the only reason the contrast failure surfaced at all.

And I verified that the scans actually ran, by asserting a minimum amount of rendered content before trusting any number. That check exists because of the false zero in the field audit.

Without the toolkit: 6 of 16 checks passed. With the toolkit: 15 of 16. Engine violations dropped from 39 to 15. All twelve icons went from unnamed to named. Twelve controls under the minimum target size went to zero. Distinct button names went from 5 to 29, which is the difference between twelve buttons all called "Copy" and twelve buttons you can actually tell apart.

The unguided build walked into every single trap. It built cards as div elements with a nested real button inside, which is invalid and confusing in equal measure. It stripped the focus outline and replaced it with a border color change. It handled reduced motion nowhere at all. It put twelve copy buttons at 39 by 17 pixels, held them at zero opacity until hover, and left them in the tab order the whole time.

The part I want to be loudest about

The build with the toolkit still failed a check.

Its border and input colors came in between 1.4:1 and 2.4:1 against their backgrounds where 3:1 is required, and those borders were the only thing marking the edge of the search field. No automated tool caught it in either version, because engines measure text contrast and do not ask whether a boundary is the only thing identifying a control. I only found it by computing every color pair in both themes by hand.

That build also overstated itself in its own summary. It reported target sizes of 44 by 44 pixels when the real number was 32 by 32. It claimed contrast-checked palettes while its borders were failing. An AI following accessibility rules will still tell you it did better than it did, which is worth knowing before you trust a self-report.

Five of the nine checks the unguided build failed are invisible to every automated tool on the market. Both builds shipped without a main landmark and without a skip link.

I published all of it. The brief, the rubric, both codebases, the raw scan output, the contrast math, and the sanitized field audit.

The limits are worth stating precisely, because a result stated without its limits is just advertising. This is one study, one model, one brief. It measures whether rules get followed, not whether a real disabled person can use the outcome, and those are different claims. A sixteen point rubric is a sample of WCAG, not the whole of it. And I designed both the rubric and the traps, which means I chose the ground. An independent replication would be more convincing than mine, which is exactly why every input is published rather than described.

What gives me some confidence beyond my own run is that the direction matches the published research. Multiple studies have found that explicit accessibility instruction improves generated output, particularly on focus visibility and semantic structure, which are two of the areas where my unguided build failed hardest. My contribution is not the discovery. It is the delivery mechanism, and evidence that the mechanism holds up over a whole build rather than a single prompt.

Where I sit, since it matters

My day job is Executive Creative Director at Level Access, which means I spend my time around people who do this at a depth no markdown file is ever going to reach. Expert audits. Testing with actual disabled users. Compliance work that holds up under real scrutiny. Findings that come back prioritized, with the level of effort scoped, so a team knows what to fix first and what it is going to take.

This is not that, and it was never trying to be. Automated checking catches roughly a third of accessibility defects. This file will not make anything compliant and I am not going to pretend it will. It stops the obvious failures from being generated in the first place, so the people doing the serious work can spend their time on the things that genuinely need human judgment.

Consider it running a comb through your hair before the photographer arrives.

How this evolves

I have been clear with myself that a static file rots. Here is how I am thinking about keeping it alive.

Field testing is the update mechanism. Version 1.12 was not written at a desk. Every addition in it came from auditing a production application that had been built entirely with AI tooling, and each one exists because that audit surfaced a defect the file had not anticipated. A single markup injection site produced 321 unnamed graphics on one page. Three separate engines reported none of the six failing color tokens. That is the loop I want to keep running: audit real AI-built work, find what the file missed, patch the file, publish what happened.

Platform drift is the real maintenance burden. The accessibility guidance barely moves. WCAG 2.2 is stable. AI tools change constantly, and there are fifteen of them in the install guide. A tool renames its config file or changes its character limit and a section of START-HERE is quietly wrong. I cannot personally monitor fifteen products, so there is an issue template specifically for reporting a platform step that no longer matches reality. That feedback is genuinely the only thing that keeps this accurate, which is why the file says so rather than thanking people politely.

The research needs to get less thin. One model, one brief, one run is a signal, not a finding. The honest next version tests multiple models against the same brief, since the guidance has to work for whatever tool someone happens to use, and adds briefs in different shapes, because an icon browser exercises a narrow slice of what people build. Forms, data tables, multi-step flows and dashboards all fail differently.

The gap the test exposed needs closing properly. Non-text contrast was the one check both builds failed, and it failed because reviewing contrast by looking at screens misses borders and inputs entirely. Version 1.12 added a procedure for auditing color tokens as data rather than as pixels. Whether that procedure actually works is an open question and the obvious thing to test next.

Design coverage is thinner than code coverage. This is the part that nags at me most as a designer. It is far easier to write a rule an agent can follow in code than to write one that changes what a designer does in Figma or how they phrase a prompt. Sections 9 and 12 exist and they are the least developed in the file. Making design guidance as actionable as the engineering guidance is the hardest remaining problem and the one I most want to solve.

Scope will expand carefully, or not at all. Native mobile, documents, email and XR are all listed as out of scope right now. I would rather cover less well than more badly, so anything joining that list joins it after real testing.

It is CC BY 4.0 on purpose. Fork it, adapt it, put your own design system's rules into it, run it in a workshop. It is versioned with a changelog so you can see what changed and when. If your team builds a better version, that is a good outcome and not a competitive problem.

What I would ask

Install it and go build something ordinary. Then tell me where it fails. Where a rule is wrong, where an agent ignores it, where a platform step no longer matches what you see on screen, where the guidance is too vague to act on.

I built this because I could not remember all of it and refused to keep shipping work that looked finished while locking people out. If it saves you from one div pretending to be a button, it has done its job.

Files and setup steps: https://www.danarandall.com/ai-a11y-toolkit The repo, including the full test: https://github.com/danarandall/ai-a11y-toolkit