When should we use gen AI in design?

To use gen AI effectively, it’s crucial to understand its inherent strengths and weaknesses.

In this issue, I’ll break down what gen AI is truly useful for now, where it falls short, and how that’s changing. These recommendations are based on my experience, stories from clients, and recent studies of gen AI’s in use. 

But first, let’s start with the key characteristics of gen AI and how they influence its useful applications. 

An overview of when to use and when not to use gen AI in design

Generative AI doesn’t know anything

At their core, generative AI models like GPT-4 are sophisticated guessing machines.

Large Language Models learn patterns from mountains of data through techniques such as next-token prediction. Diffusion models like Midjourney employ a relatively similar process for image pixel generation.

As a result, they form an imperfect but impressive compressed model of the world. They store probabilities and patterns, not 100% accurate facts.

With its faults, this probabilistic model of the world is becoming increasingly accurate. In a recent study, GPT-4 outperformed clinicians in patient pre-test diagnostics. After all, we humans are also just guessing machines.

When to use gen AI in design

So when should we use this compressed imperfect model of the world?

In my experience, there are currently six areas where gen AI models shine: research, learning, ideation, first drafts, summaries, and feedback.

1. Research

As a model of the world, gen AI can help us effectively explore any domain with ease, versatility, and depth.

For example, you can kick-start a project by exploring the topics you need to understand. One effective technique for this is called the “Tree of Thought”. Instead of prompting for a single answer, you’ll use a tool like GPT-4 to explore the different “branches” of the topic: 

“Help me explore the topic of onboarding for a fitness app. What are the different aspects I should consider?”

After the AI’s answer, you can follow up on any of the branches to dig deeper into the topic.

An example of a simple Tree of Thought technique starting prompt. Start with an overview of a topic and then dive into specific sub-themes in follow up prompts.

2. Learning

Similarily to research, gen AI can be an effective tool for learning. In particular, it can tailor lessons specifically for our goals and skill level. I’ve found learning prompts like these to be useful:

- “Help me understand the basic principles of Large Language Models. Explain them simply like you would to a ten-year-old”

- “Evaluate my current understanding of AI with a series of open-ended questions. Provide feedback on my level of knowledge”

I’ve also GPT-4 as a personalized learning tool to master any new technical tools. 

Providing personalized learning is a clear strength of current gen AI tools

3. Ideation 

Gen AI excels at combining patterns to produce novel ideas. According to a study, its use can give people up to a 40% boost in creative ideation tasks.

In ideation, gen AI's probabilistic nature and “hallucinations” are features, not bugs. 

Teams and individuals not using gen AI for ideation are likely leaving some solid ideas on the table.

To ideate well with gen AI, it’s important to:

- be extremely specific with your challenge at hand
- iterate by taking the gen AI ideas and combining them with your own
- giving examples to gen AI of what great looks like (known as “few-shot prompting”)
- use image generation to explore early directions, not create final outputs 

I often use GPT-4 as an ideation partner for these articles, but I always write them myself to hone my thinking and writing.

4. First drafts 

While not 100% accurate, gen AI is often immensely helpful in creating a first draft.

Examples of how I and my clients have used gen AI for drafting include:

- writing the first version of placeholder copy for applications and websites
- drafting the initial outline of a customer interview
- creating a first draft of a customer journey

5. Summaries

Large Language Models are particularly adept at analyzing language and creating a synthesis out of it.

For us working in design and innovation, they can be useful to:

- create a synthesis of interview transcripts 
- extract key insights from hundreds of survey responses, reviews or customer messages
- summarize several hundred pages of PDF documents

Naturally, it’s paramount to remember data privacy and confidentiality in these use cases.
 

6. Feedback

Have an early version of a document or design? Generative AI can detect patterns in written and visual information to give constructive feedback.

In GPT-4, you can upload an image of your design. While not always the most original, the feedback from gen AI can point to areas in your draft that you overlooked.

To elicit useful feedback, it’s important to give enough of the context of your goals and target audience to the AI.

When not to use gen AI in design

How about the flip side?

With it’s less than 100% accuracy, I would steer away from using generative AI for:

- final outputs in any project without verification and iteration
- polished user interface design    
- as a replacement for human thinking and creativity
- writing where developing your own thinking and voice are critical
- critically important matters where 100% accuracy is pivotal
- tasks requiring exact facts of names and numbers

For the last two shortcomings, gen AI models can be supplemented with RAG-models, where the model can reference an external source (like company documents) to augment its learning data. Some models like GPT-4 also use live coding to circumvent their inherent lack of mathematical prowess.

The ever-moving frontier

Currently, generative AI is like a team member with enormous strengths in creativity, research, and ideation but rather glaring weaknesses in areas of 100% accuracy and polished end results.

Putting together complex user flow with polished designs is beyond the current capabilities of gen AI models.

How about in the future?

If the near past is any indication, many of these weaknesses will turn into strengths in a few years.

Hallucination has decreased significantly from GPT 3.5 to 4, and laughable mistakes in image generation (like 3 hands) have vanished almost completely. We’re also bound to see companies like Figma release more generative design tools tackling the creation of polished UI’s.

Leaders in the field, such as Sam Altman of Open AI and Demis Hassabi of Google’s DeepMind, have alluded that general AI models will become increasingly agentive. In other words, they can handle more complex multi-step processes independently.   

Right now, we’d be wise to use gen AI models selectively, leveraging their strengths to complement our own. At the same time, we should keep an eye on a future of ever-expanding gen AI capabilities.