What the G$*g!€ Font?! — How it Works

Earlier this year I built What the G$*g!€ Font?! to solve a years-long issue I’ve had with identifying free fonts to use in projects.

I set it up so that members of my physical media covers website could easily find a good font for movies they’re designing.

I recently had a researcher reach out to me asking how it was made, so I figured I’d share it here as well.

How does this Google Font finder work?

The google font finder identifies fonts in images by combining AI vision analysis with pixel-level rendering comparison against every Google Font (and only Google fonts). This dual approach allows for some flexibility and variety to give you a lot of options when you’re searching for a Google font to use in a project.

When you upload an image, three things happen:

1. Claude (AI) looks at the image and describes the font — serif or sans-serif, heavy or light, any distinctive features — and suggests Google Fonts it thinks match.
2. A font rendering server takes the detected text, renders it in every one of ~1,900 Google Fonts, and measures how closely each rendering matches the original image pixel by pixel.
3. The results are merged — if the rendering server has a clear winner, its rankings lead. If it’s uncertain, Claude’s suggestions get promoted to the top.

You get a ranked list of up to 25 Google Fonts, each with a similarity score, live preview, and one-click CSS to use it.

So if you get an exact match, its usually because of my font rendering service, and if you get something more approximate, its because of Claude AI’s vision comparison.

Why Not Just Use AI?

AI vision models are good at identifying font categories and characteristics — they can tell you it’s a geometric sans-serif with medium weight and high x-height. They can even suggest specific fonts that would be close. But they just don’t have database to give out exact matches, at least not in its current state.

Rendering comparison is the opposite: it’s precise but dumb. It can tell you exactly which font produces the closest pixel match, but it needs clean input and known text to work with.

By combining both — (using AI to understand the image and set context, then rendering comparison to measure actual similarity) — the system gets the best of both approaches.

Step by Step

1. Image Input

Users can drag and drop, paste from clipboard, upload a file, or provide a URL. The image is converted to base64 and sent to the API. Max size is 5 MB; supported formats include JPEG, PNG, WebP, GIF, and AVIF.

2. AI Vision Analysis (Claude)

The image is sent to Claude, which performs a structured analysis:

• Category: serif, sans-serif, display, handwriting, or monospace
• Characteristics: weight, width, style, contrast level, x-height, and distinctive features (e.g. “rounded terminals”, “high x-height”, “stencil cuts”)
• Detected text: all readable text in the image
• Primary text: the single most prominent text element (headline, logo, etc.) — this is what gets used for rendering comparison
• Suggested fonts: 10–15 Google Fonts ordered by visual similarity

Claude’s response is structured via tool use, so the output is always machine-readable with consistent fields.

3. Font Rendering Server

The primary text and a cropped version of the image are sent to a dedicated Python server that does the heavy lifting.

The server uses Pillow for font rendering (which delegates to FreeType, the same font rasterizer used by browsers and most operating systems) and OpenCV for image processing and comparison. The approach is straightforward: since we know the text content (from Claude) and have the exact font files for every Google Font, we can render a ground-truth image for each candidate and compare it directly against the user’s image at the pixel level.

Text Line Extraction

Before comparing, the server isolates the relevant text region from the image:

• Finds character-shaped contours in the image
• Groups them into horizontal lines
• Picks the line that best matches the expected text (based on character count and prominence)
• Crops to just that line, removing background noise

Preprocessing

The cropped text image gets cleaned up:

• Converted to grayscale and thresholded to black-and-white
• Deskewed: if the text is italicized or tilted, the server detects the slant angle using edge analysis and straightens it for fair comparison
• Weight detected: measures the fill ratio (how much of the bounding box is ink) to determine if the text is bold or regular
• Stencil detected: counts contours per character — stencil fonts have roughly 2.5× more contours than normal because the cuts break each letter into pieces

Rendering and Comparison

The server then loops through every Google Font in its database (~1,900 families):

• Renders the primary text in each font at a matching size
• If the detected weight is bold, it prefers bold/black font file variants
• Tries multiple case variants (original, lowercase, uppercase) and keeps the best match

For each font, three similarity metrics are computed:

The combined score is: 0.45 × ink_IoU + 0.35 × edge_IoU + 0.20 × correlation

If the image was flagged as stencil and the font name contains “stencil”, a bonus of 0.35 is added.

The top 25 fonts are returned, sorted by similarity score.

4. Result Merging

The API combines Claude’s suggestions with the rendering server’s ranked results using a confidence check:

• High confidence: the server’s top match scores above 50% and there’s meaningful spread between the top and bottom results → server rankings are used as-is
• Low confidence: top score is below 50% or the top 10 results are clustered within 10 points of each other (no clear winner) → Claude’s top 4 suggestions are injected at the top of the results, scored slightly above the server’s best match

This hybrid approach means you get pixel-accurate matches when the rendering comparison has a clear answer, but fall back to Claude’s visual intuition when the comparison is ambiguous.

5. Results

The final response includes:

• Font characteristics from Claude’s analysis
• Up to 25 matched fonts, each with:
  • Similarity percentage (0–100)
  • Font family name and category
  • Available weights/styles
  • Google Fonts link and CSS import snippet
• Cropped image showing the isolated text region that was matched against
• Analysis ID for sharing results via URL

Hash-Based Fallback

When no text is detected in the image (e.g. a logo or abstract letterform), the server falls back to perceptual hash matching instead of rendering comparison:

• Computes pHash (perceptual hash — captures overall structure) and dHash (difference hash — captures local contrast patterns)
• Computes shape descriptors: Hu moments (rotation/scale-invariant shape features), aspect ratio, solidity (roundness), and contour count
• Scores each font by weighted Hamming distance: 0.40 × pHash + 0.35 × shape + 0.25 × dHash

This mode is faster but less accurate than rendering comparison since it can’t leverage known text.

Infrastructure