Automating Design Advice for Designers When Adapting UIs

Overview

This project creates a creativity support tool for UI/UX researchers to design for demographic groups they are unfamiliar with. Creating a corpus through collecting real-world website data, extracting features with a custom processor using computer vision techniques, then multi-step clustering of features and extracting designer-aligned insights for advice.

The research aims to showcase the difference in localized designs of modern websites, as an updated data is lacking. The research also aims to support designers throughout their design process when localizing website designs. We propose a tool that uses multi-layered abstracted advice and structured dimensional design exploration in order to do so.

Problem Statement

Research has shown that users of different demographic backgrounds engage in different behavior when using digital technologies, such as website. Yet, localization beyond typical translation or market localization rarely occurs. In particular, design localization remains underimplemented. This is for a variety of reasons:

• Lack of resources to support this research
• Unknown benefits or tradeoffs
• Not knowing how to design for users

This project focuses on enabling and supporting designers in designing websites for users of different cultures.

Research Approach

Our methodology combined multiple research methods to gain comprehensive insights:

• Conducted focus groups with designers and used thematic analysis to derive design guidelines for our tool
• Creating a custom webscraper using Javascript to create an updated dataset of locally popular websites
• Developed a custom processing module that processes screenshots of websites to extra multiple low-level features
• Clustered image features and leveraged the use of LLMs to provide human-understandable advice
• [IN PROGRESS] Creating a creativity support tool for designers when localizing design

Key Findings

The key findings from our formative study with focus groups revealed 3 design guidelines:

1. Provide inspiration through exploratory search
2. Provide understandable metrics and advice
3. Provide a balance between freedom and structure

To adhere to these guidelines, we extract low-level features (i.e. colorfulness, top colors found on a website, percentage of whitespace, contour density, etc) used in prior work. To provide understandable metrics and advice, we use the framework of multi-layered abstracted advice and dimensional reasoning. Alone, these metrics do not indicate much. We create these explainability layers by clustering websites based on their features, then providing these features to an LLM to synthesize text-based advice. This clustering is repeated twice for a total of three layers, resulting in "high-level" advice aligned with typical design guidelines seen on the web. Below are high-fidelity mockups of the system. The frontend of the system is being built with Sveltekit while the backend is built with Flask and a NoSQL database. The custom processor to analyze designer images was developed using Python, numpy, pandas, and PyTorch.