datasette-ip-rate-limit 0.1a0
Due to aggressive crawlers, Simon Willison used Codex (GPT-5.5 xhigh) to build a configurable rate limiting plugin to block IPs that were hammering specific areas of the site too quickly.
Release: datasette-ip-rate-limit 0.1a0
Simon Willison’s Weblog
Subscribe
14th May 2026
Release
datasette-ip-rate-limit 0.1a0 — Rate limit Datasette requests by client IP address
The datasette.io site was being hammered by poorly-behaved crawlers, so I had Codex (GPT-5.5 xhigh) build a configurable rate limiting plugin to block IPs that were hammering specific areas of the site too quickly.
Here's the production configuration I'm using on that site for the new plugin:
datasette-ip-rate-limit: header: Fly-Client-IP max_keys: 10000 exempt_paths:
- "/static/*"
- "/-/turnstile*"
rules:
- name: public-demo-multi-parameter-table-pages
paths:
- "/global-power-plants/*"
- "/legislators/*"
window_seconds: 60 max_requests: 60 block_seconds: 20
Recent articles
Notes on the xAI/Anthropic data center deal - 7th May 2026
Live blog: Code w/ Claude 2026 - 6th May 2026
Vibe coding and agentic engineering are getting closer than I'd like - 6th May 2026
This is a beat by Simon Willison, posted on 14th May 2026.
rate-limiting 11
datasette 1483
codex 48
Monthly briefing
Sponsor me for $10/month and get a curated email digest of the month's most important LLM developments.
Pay me to send you less!
Sponsor & subscribe
Disclosures
Colophon
©
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
2025
2026