The State of AI Font Generation

The State of AI Font Generation

Two disclaimers: First, this article was entirely human generated; no AI was involved in the writing. Second, I have spent a lot of time this year attempting to train a glyph generation model. I want to be up front about this because I know that for certain people, this will discredit me entirely, and I don’t want you to find out afterwards and feel betrayed. But then, would you rather read an article about AI font generation written by someone who didn’t have any experience of it?

There’s been a lot of discussion recently about the ability of machine learning models to generate new fonts, or “fill in” existing fonts (by expanding the glyphset or design space), much of it prompted by this thread on Typedrawers. As part of that thread, Dave Crossland wrote:

Impallari appears to have been the first. Eric’s become second. I hope there’ll be a third.

I’m not going to talk here about the ethical or legal side of font generation, about whether you should or shouldn’t do this. These are important questions, but not for here; but my reaction when I read that was “Eric wasn’t the second; he wasn’t even the twenty-second - it just that all the others aren’t in your face right now.”

So the point of this article is to try to survey some of the development in font generation research, to present a clear-eyed view of what’s out there, how it works, and what I expect coming down the line. Because more will be coming down the line. People are going to work on this problem, whether we want them to or not. So whether your instinct is to welcome it or critique it (or of course both), let’s make sure we’re aware of it and we understand it.

But let’s also use it as an opportunity to understand ourselves as well. During my investigations, I noticed a couple of factors, a couple of gaps, and I think perhaps talking about these gaps is going to be the real takeaway from this post, rather than the technical stuff. I’m talking about the gaps between type designers and ML engineers, and also the gaps between Western type designers and Chinese/Japanese/Korean type designers.

The obligatory Two Cultures reference

One feature I’ve noticed is a very obvious lack of collaboration across disciplines, between ML researchers and type designers. I’ve certainly noticed a lack of collaboration between ML researchers and type engineers - as I look into the code for some of the models, I shudder to read how glyph outlines and rasterizations are being extracted from fonts; it’s clear that nobody from type-world has had any input into this at all. But more generally, I can see that there is also a lack at the quality control level; what ML engineers consider a typographic success would be something that a type designer would never dare share in public. I’m not just thinking of this thread, but also stuff like Neural Axis Variations - a genuinely interesting idea which probably could be well implemented, but clearly needed a type designer to review it before publication.

This is not “a fully functional variable font”.

I don’t say this to mock people’s work. The whole point is that you don’t know what you don’t know. I have a theory about why this is: when it comes to collaboration with type designers, I think there is an obvious question of perceived need.

ML researchers are really used to working with computer graphics. It’s something that they do all the time and do really well. And, as I argued in the big TypeDrawers thread, “type design is very deceptive because it’s easy to draw a series of recognisable letter forms” and that makes it hard for you to realise that you need feedback. If your ML model outputs a long string of amino acids, you know that you’re going to need a biologist to tell you whether it makes sense or not. Collaboration across disciplines is so obviously necessary in such a situation that you wouldn’t think not to.

But if you’re a researcher used to working with computer graphics and your task is to draw the letter ‘a’… well, everyone knows what an ‘a’ looks like. You don’t need professional input for that. (Spoiler alert: turns out you do!)

And at the same time… how can I put this delicately? Even if ML researchers did reach out to the typographic community in order to collaborate on AI-assisted font generation, they may not necessarily receive a warm and hearty welcome. It’s kind of a touchy subject. Type people have a habit of asking awkward questions about ethics and legality. And on top of that, nobody wants to be the one responsible for putting themselves and their peers out of work. You know, I get that.

So there is a paradox here: that AI generated font design can only come out of deep engagement with the typographic community if it’s going to be effective and high quality, and right now the typographic community doesn’t actually want effective and high quality AI generated font design.

Which I guess is good news for the typographic community, right? Researchers produce bad models because they either don’t seek out typographers or typographers won’t work with them, and type designers see the output of the bad models and feel reassured that their jobs are safe. A neat closed loop.

Except, that’s such a Western perspective.

The other two cultures

One of the main pushbacks from designers about AI tooling for font design is that it has to make sure that it’s not just “doing the fun bit”. Designers draw letterforms because they like drawing, and so AI tools which draw letterforms - even decent letterforms - but don’t help with the “boring bit” are counterproductive. The boring bits of type design are different for each designer: spacing, kerning, glyphset expansion and language support, designspace expansion. We can disagree about that. But one thing that does seem to be a clear consensus is that drawing the actual shapes is the fun bit, and there’s no desire for an AI model which takes that away.

Except, not all of us have the same objectives. When you have a hundred or even a thousand characters to draw, working out how your design inhabits each of those characters is a fun and interesting puzzle to solve.

When you have a hundred thousand characters to draw, it stops being quite so fun and interesting and turns into a royal pain in the arse. Chinese font design is a different kettle of fish altogether, requiring teams of designers many years of work to produce a single font. Of course, we have some automations such as variable components which can help bring that down, but even so… drawing is not the “fun bit” of Chinese type design.

While I was looking at the deepvecfont-v2 codebase, I noticed that all of the model training data was made available. You can download the whole data dump, all the fonts. This is kind of unusual. You don’t often see font-based ML projects making their training data available, largely because it allows you to see quite how many commercial fonts were, uh, “accidentally” caught up in the dragnet of fonts that the researchers scraped off the Internet. But this one was different. A huge cache of commercial Chinese fonts were made publicly available as part of the training data release for the deepvecfont-v2 project. How is this possible? Is this even legal?

Well, a little note at the bottom of the repository says:

Please note that all the Chinese fonts are collected from Founder, and the fonts CANNOT be used for any commercial uses without permission from Founder.

To spell that out more explicitly: While Western foundries are explicitly saying “you can’t use our fonts to train AI models”, Founder Type, one of the main Chinese type foundries, have partnered with a team researching AI-based font generation and said “We are giving anyone on the Internet a free license to our entire Chinese font catalogue but only if you use it to train an AI model.”

That… probably wouldn’t happen in the West. Different situations have different objectives. Chinese type foundries welcome AI font research because it can help them with their “boring bit”. They have a serious commercial interest in the problem of “few-shot font generation”: the idea that I draw fifty glyphs in a particular style, and the computer just fills in the rest. Western foundries do not, on the whole, share this enthusiasm. (This was a dynamic which was very much in evidence at ATypI Copenhagen last year.)

What this means is that a lot of the interesting work going on in terms of AI font generation is happening in China and Japan, and is happening with foundry support. And while the current state of Latin font generation is, well, pretty poor, AI font generation in Chinese is already production-ready.

They’re already using it.

This is bad, right? If they can generate production-ready Chinese fonts, then they’re coming for Latin type design soon. Well, maybe. But maybe not. Look around you and you will discover that there aren’t production-ready Latin font generation models right now, and those which do exist are pretty woeful. Why is this? What is going on? Some ideas later.

The revolution will not be a TypeDrawers thread

For now, let’s finally take a look at the current state of font generation. We’ve established that ML engineers are not working in collaboration with type designers, and what that means is that you aren’t going to find out about font generation advances in the typography media. You’re going to need to start looking where people in the ML community communicates with one another: conference papers and pre-prints, and more specifically Arxiv. Arxiv is where scientists share in-progress research papers, and in particular, the “Computer Vision and Pattern Recognition” (cs.CV) subsection is where all the ML-y graphic-y stuff ends up. If you want to find out what’s actually going on in font generation research, that’s where you will need to go. A search for “font” on arxiv today turns up 328 papers. Not all of them are about font generation, but a lot of them are. Most of those, as predicted, are from China, Japan and Korea. Many of those are, predictably, about the “few-shot font generation” problem: draw me a 50,000 character font from 50 references.

Filip Paldia put together quite an exhaustive list of state-of-the-art projects up to early 2024. But the field has developed considerably since then. Let’s take a quick tour of them, and bring things up to date. The following sections will be representative, not exhaustive. If you’re really into this, you’re best placed doing your own searches on Arxiv…

Generative Adversarial Networks

Early font generation work (2018-2023) tended to be built on Generative Adversarial Networks, the same technology underlying image generators like DALL-E (first edition). GANs are comprised of two models, the “forger” and the “detective”; the forger tries to create an image and the detective tries to detect whether or not it’s machine-generated. They keep iterating on this task until the forger manages to create an image which the detective thinks is real, at which point it’s good to go.

For font generation, conditional GANs were used - we don’t just try to produce any image, but we give the GAN a particular glyph (content) and a particular style, and try to “forge” a glyph image matching that style. Font GAN model naming was incredibly unimaginative. They went:

DG-Font (2021): Not quite the OG, but one of the most influential.

LF-Font (2021): Saw “style” as a hierarchical set of component level features.

MX-Font (2021): Multiple component-level “experts” learn to produce different parts of a glyph. (There’s now an MX-Font++ but I haven’t looked at it yet.)

CG-GAN (2022): A few points at least for trying a different naming scheme.

FS-Font (2022): Tries to further identify the local details (terminals etc.) which define font style.

CF-Font (2023): Fuses style and content into a single prior for training.

DA-Font (2025): Tried various ways to improve corners and an “

[truncated for AI cost control]