PACE: Prompt Accessibility Controlled Evaluation

What makes AI-generated forms accessible?

PACE tests how model choice and prompting affect the accessibility of AI-generated HTML form components. The published study evaluates 26,100 outputs using automated accessibility checks and component-specific scoring.

Read the paper (PDF) Explore the code

The study

Forms are where people sign in, make choices, and submit information. PACE focuses on the HTML components that make up those interactions.

The testbed varies the model and prompting conditions while keeping component specifications explicit. Repeated generations help reveal how consistent the output is.

Study design 2026 study
Models
5
System instructions
3
Form components
58
Request variants
3
Trials per configuration
10

26,100 generated HTML outputs

Each combination of model, system instruction, component, and request variant was tested ten times.

The published study compares Claude Opus 4.6, DeepSeek V3.2, Gemini 3.1, GPT 5.4, and Grok 4. The results describe these model versions under the study's test conditions.

Inspect the study dataset

What changes between conditions?

Five models, three levels of system instruction, 58 form components, and three request variants. Each configuration is repeated ten times.

Which components are included?

The study includes text and email fields, password inputs, selection controls, consent checkboxes, textareas, file uploads, date and time inputs, range and switch controls, and submission and state components.

Appendix C of the paper lists all 58 specifications.

What can I reproduce?

The archived dataset includes generated HTML, axe evaluation files, combined results, and component-level check scores. The PACE repository provides the configurable testbed.

Follow the testbed setup instructions.

Measurement

Node-level axe pass rate

Do the evaluated HTML nodes pass the selected automated checks? PACE uses 26 form-relevant axe rules and includes incomplete results in the denominator.

Examines the markup that was generated.

Normalized schema score

Does the output include the structure and relationships expected for the requested component? Checks cover features such as labels, descriptions, grouping, and error associations.

Examines how fully the requested pattern was implemented.

A strong automated audit result can coexist with missing component semantics. See the measurement definitions in the paper (PDF).

Findings

Findings from the published 2026 study.

  1. Prompting did not consistently improve accessibility.

    A change in instructions can help under one measure and be neutral or unfavorable under another. More detail does not consistently produce better accessibility.

  2. Model rankings depended on the measure.

    The models did not rank identically on axe pass rate and schema score. A single aggregate score can conceal differences in what the output gets right.

  3. Some components are harder than others.

    Range and switch controls had lower schema scores than every other component cluster in the study.

What these results do not establish

PACE evaluates isolated HTML form components. These measures do not establish full-page accessibility, dynamic interaction quality, or usability with assistive technology. The results describe the models and conditions tested in 2026. The study does not compare the outputs with matched human-authored HTML.

Read the limitations and discussion (PDF)

Research resources

  • The paper

    Publication

    Methods, analyses, worked examples, and appendices in the ACM Digital Library.

  • The testbed

    GitHub

    Configure models, prompts, components, and repetitions. Inspect the scoring code.

  • The dataset

    Zenodo · 2026 snapshot

    Generated HTML, audit outputs, and analysis-ready results from the study.

The researchers

Affiliations as listed in the paper.

Contact the research team

Cite this research

Yumeng Ma, Jacob O. Wobbrock, Joe Devon, Aaron Gustafson, Michael Fairchild, Jennifer Mankoff, and Venkatesh Potluri. 2026. Prompt and Circumstance: Evaluating Accessibility in LLM-Generated HTML Form Components. In the 28th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS ’26). ACM, 32 pages.

More accessibility benchmarking

AIMAC compares the accessibility of AI-generated web pages. PACE focuses on controlled experiments with individual form components.

Visit AIMAC