Apple researchers have published details about an experimental AI-powered tool called SQUIRE that helps developers explore and develop interface ideas with greater control. Here are the details.

An Interesting Approach to AI-Powered Interface Prototyping

In a new paper titled "SQUIRE: Slot QUery Intermediate REpresentations for Interactive UI Writing," a group of Apple developers proposes an innovative way to approach AI-generated interfaces.

The researchers explain that natural language brings greater flexibility to many aspects of the development process, but it also comes with two main challenges:

First, natural language can be ambiguous on its own, making it difficult for developers to convey their intentions accurately. Second, the model may respond unpredictably, requiring the developer to reissue requests through trial and error to fix unwanted changes.

This is where SQUIRE comes in. It is a visual interface that allows developers to build and refine UI prototypes step by step, providing clearer control over the results.

From the study:

In SQUIRE, users start a project by providing a prompt that defines their goals for the UI and add example data containing information that SQUIRE can use as a reference. Users guide SQUIRE to fill in gaps representing missing but expected functionality, building the UI as a tree of components from the top down. In response to such a request, SQUIRE generates a list of suitable alternatives specifically scoped to the target gap in the incomplete UI. Clicking on each alternative facilitates visualizing differences by instantly updating a live preview and the underlying code. Users can also make targeted requests to change the appearance of specific UI areas, with the guarantee that no code outside the desired scope will change. In response to such requests, SQUIRE generates temporary controls that allow the user to apply semantically related changes quickly without requiring reissuing requests. In all cases, the LLM acts like a collaborator, presenting reasonable choices for the user's evaluation while leaving the authority to accept or reject suggestions with the user.

In other words, natural language prompts were still the way developers interacted with SQUIRE, but each prompt was connected to a specific part of the UI and did not affect the entire interface at once.

Based on observations with 11 front-end developers using SQUIRE to develop interface prototypes, the researchers found that participants were able to explore and iterate different UI designs with a strong sense of control, and they rated the system highly in terms of usability and overall satisfaction.

They also note that this added sense of control made developers feel more comfortable exploring ways in which changes could be made, predicted, and reverted.

From the study again:

Through data collected from 11 front-end developers, we note that (1) interactions with SQUIRE frequently encouraged participants to explore, rather than just using SQUIRE as a code accelerator, (2) participants were encouraged to take risks when making changes, knowing that the consequences of making unusual decisions could always be reverted smoothly, (3) participants felt confident that SQUIRE met their intentions when making changes, and (4) participants were generally satisfied with the quality of the code and visuals produced by the system.

The Internal Structure of SQUIRE

Instead of directly generating interface code from user prompts, SQUIRE first creates its own intermediate representation of the interface, called SquireIR, which can be filled over time by modeling the UI as a tree of components with named gaps, as shown in the following example:

This structure can also include placeholders for yet-to-be-defined parts (e.g., a button label, an image, or a content section) and multiple possible UI alternatives. For example, it can represent the same content as a list or a grid.

After that, SQUIRE translates this representation into code using HTML, CSS, and JavaScript, managing the final UI structure through Web Components.

Another important aspect of SQUIRE is how it manages changes.

When a developer wants to adjust a button or change a layout, only that part is updated, while everything else remains unchanged.

According to the researchers, this helps prevent the trial-and-error cycles seen in many AI coding tools; due to the general unpredictability of LLMs, the model can make more changes than the developer intended.

This structure also allows SQUIRE to suggest multiple options at each step, enabling developers to quickly compare different versions without losing their previous work.

Unlike many technical papers, this study does not provide detailed information about model training, architecture, or data. Although the researchers mention that SQUIRE works with OpenAI's GPT-4o, the focus of the paper is on system design and interaction model.

SQUIRE is not generally available and its use is limited to the 11 developers participating in the study. However, it is not hard to imagine how this could be implemented in future versions of Xcode or other Apple-made development tools.

To learn more about SQUIRE, you can follow this link.

Products Worth Checking Out on Amazon

  • David Pogue – 'Apple: The First 50 Years'
  • MacBook Neo
  • Logitech MX Master 4
  • AirPods Pro 3
  • AirTag (2nd Generation) – 4 Pack
  • Apple Watch Series 11
  • Wireless CarPlay Adapter