Skip to content
AI News HubLIVE
More
Original source1 min read

datasette-ip-rate-limit 0.1a0

Summary

Due to aggressive crawlers, Simon Willison used Codex (GPT-5.5 xhigh) to build a configurable rate limiting plugin to block IPs that were hammering specific areas of the site too quickly.

datasette-ip-rate-limit 0.1a0
Report an error

The correction channel is not available yet. You can copy the article reference below for later.

Correction instructions
Read article

Release: datasette-ip-rate-limit 0.1a0

Simon Willison’s Weblog

Subscribe

14th May 2026

Release

datasette-ip-rate-limit 0.1a0 — Rate limit Datasette requests by client IP address

The datasette.io site was being hammered by poorly-behaved crawlers, so I had Codex (GPT-5.5 xhigh) build a configurable rate limiting plugin to block IPs that were hammering specific areas of the site too quickly.

Here's the production configuration I'm using on that site for the new plugin:

datasette-ip-rate-limit: header: Fly-Client-IP max_keys: 10000 exempt_paths:

  • "/static/*"
  • "/-/turnstile*"

rules:

  • name: public-demo-multi-parameter-table-pages

paths:

  • "/global-power-plants/*"
  • "/legislators/*"

window_seconds: 60 max_requests: 60 block_seconds: 20

Recent articles

Notes on the xAI/Anthropic data center deal - 7th May 2026

Live blog: Code w/ Claude 2026 - 6th May 2026

Vibe coding and agentic engineering are getting closer than I'd like - 6th May 2026

This is a beat by Simon Willison, posted on 14th May 2026.

rate-limiting 11

datasette 1483

codex 48

Monthly briefing

Sponsor me for $10/month and get a curated email digest of the month's most important LLM developments.

Pay me to send you less!

Sponsor & subscribe

Disclosures

Colophon

©

2002

2003

2004

2005

2006

2007

2008

2009

2010

2011

2012

2013

2014

2015

2016

2017

2018

2019

2020

2021

2022

2023

2024

2025

2026

Key points and analysis

Article intelligence

EngineersAdvanced

Key points

  • datasette.io site was being hammered by poorly-behaved crawlers, prompting the development of a rate limiting plugin.
  • The plugin was built using Codex (GPT-5.5 xhigh) and supports configurable rules.
  • Production configuration includes IP header-based limits, max keys, exempt paths, and specific rules for certain paths.

Highlights and analysis are generated automatically and may contain errors. Check the original source.