The story behind it

I was pointed to a vulnerable Django setup not too long ago, and had a closer look. Their setup would have allowed potential arbitrary remote code execution through this combination of configurations, with emphasis on combination:

1. They were using a Redis database that listens to the public internet for both a Django Celery Queue and a Django cache. With the right credentials, an attacker could talk to the database directly.
2. They had Redis credentials — user and password — in the CELERY_BROKER_URL Django setting (which is not uncommon with Celery).
3. Django's default SafeExceptionReporterFilter does not cleanse setting key CELERY_BROKER_URL, and so Django's debug mode reveals CELERY_BROKER_URL content including credentials on server errors.
4. They had Debug mode enabled by mistake and without knowing, more on how that was possible below.
5. They had a Django view that could be forced to crash to then reveal the debug error page.
6. Django would unpickle anything that an attacker puts into the Redis cache after takeover and thus run attacker code through pickle as known for years.

In isolation, none of these cause as big of a problem, but combined things get scary; looking at that same picture from the opposite side, fixing any single of these parts would have closed the attack vector as whole. The affected setup has part (4) fixed now, the related pull request is public. It's a good reminder that in Python both bool('False') and bool('0') evaluate to True even when it feels wrong in some way.

What about closing some of these open doors for everyone and by default, wouldn't that be nice?

Trying to close these open doors by default in Django

Regarding (2) and (3), the CELERY_BROKER_URL issue with SafeExceptionReporterFilter is something that I already reported over five years ago but it's easy to argue that never activating debug mode is the only true fix (and that's not wrong but also doesn't help the situation) and so it was closed as "wontfix" then, and again this year when I wanted to give it another shot with a pull request when it became clear again to be a killer in practice.

Regarding (4), the debug mode that was enabled by accident could have been prevented by Django limiting settings.DEBUG to instances of bool (or by disallowing string values like "off", "no", "0", "disabled", "false", "False" except the latter approach does not scale well beyond English if that matters). It can always be argued that we cannot protect all users from themselves and that this is beyond the line of user responsibility, but it would have saved that particular setup. The issue was closed as "wontfix".

Regarding (6), making use of pickle in caching secure by wrapping it with a layer of did-we-pickle-this-ourself-earlier protection will cost some performance (which is yet to be proven critical), but it would make a good secure default and something that only those users should turn off in order to re-gain the lost performance who understand their threat model well and whether it's really okay to own all of Django when the cache database gets owned. It was closed as "wontfix".

Because I had one more hardening issue to report that keeps coming up in the wild, I filed one more issue for the security-by-obscurity issue with /static/staticfiles.json where attackers can learn about your Django dependencies, their precise versions and get new ideas for targetted attacks from that, in particular with setups missing security updates (which is the true issue to be fixed in the setup, indeed). Almost everyone starts hiding that file once made aware of the implications but the issue was closed as "wontfix".

Four hardening issues closed as "wontfix" felt like an unfortunate pattern to me — I knew "wontfix" as an exception only, even outside of security — so I reached out to the Django security team via e-mail to be sure they were in support of these "wontfix"es and that this was not just one big misunderstanding. They are in support of it and me investing more time in discussions on the forums is their wished way forward, too. So not a misunderstanding.

I have decided to direct my time and energy elsewhere, to rather blog about it here in order to raise awareness about these issues before these doors are closed by default. I also have some hope (just a tiny bit) that maybe one of my readers — could be you — wants to be the force to advance these topics in the Django forums.

Bonus track

Regarding (5), the particular crash I found was interesting. EmailField comes with max_length=254 by default and so if you have API endpoints that ask for well-formed e-mail-addresses and store them into the database without length validation, passing a too-long-but-well-formed e-mail address may allow crashing view code (at database entrance) that looks perfectly healthy at first but doesn't do enough for validating objects.

Stay secure, and have a nice day!

Sebastian Pipping

Seth Godin: Stop Waiting for Permission!

Simon Sinek & Trevor Noah on Friendship, Loneliness, Vulnerability, and More | Full Conversation

For readers new to Expat:

libexpat is a fast streaming XML parser. Alongside libxml2, Expat is one of the most widely used software libre XML parsers written in C, specifically C99. It is cross-platform and licensed under the MIT license.

Expat 2.6.3 has been released earlier today. The key motivation for cutting a release and cutting it now are the three security findings by TaiYou that were assigned identifiers CVE-2024-45490, CVE-2024-45491 and CVE-2024-45492.

Out of the remaining bunch of fixes in and around the build system, the BSD-motivated portability contributions by Dag-Erling Smørgrav stand out with this release. Thanks to everyone who contributed to this release of Expat!

For more details about this release, please check out the change log.

If you maintain Expat packaging or a bundled copy of Expat or a pinned version of Expat somewhere, please update to 2.6.3. Thank you!

Sebastian Pipping

Anna Rudolf Lie Detector Chess Interview on YouTube

git clone allows cloning single commits without history for existing branches and tags through syntax…

git clone --depth 1 --branch <BRANCH_OR_TAG> <REMOTE_URL>

…but not for arbitrary commits. It's not impossible though, and if you want to clone arbitrary single commits — say in CI — it can be done using a trick.

The idea is simple: instead of git clone we combine git init, git remote add, git fetch and git checkout — but how?

Let me demo that for cloning commit 9c6d51b71caeb1e773cabf4ad9ded9bd6e142229 from repository hartwork/git-delete-merged-branches in practice in a Linux Bash terminal:

# 0. Jump to an empty temporary directory
cd "$(mktemp -d)"

# 1. Create an empty Git repository (rather than using "git clone"), without warnings
git -c init.defaultbranch=main init

# 2. Add the target repository as a new remote "origin"
git remote add origin https://github.com/hartwork/git-delete-merged-branches

# 3. Fetch a single commit (and all trees and blobs needed for it)
git fetch --depth 1 origin 9c6d51b71caeb1e773cabf4ad9ded9bd6e142229

# 4. Check out the commit we just fetched, without warnings
git -c advice.detachedHead=false checkout FETCH_HEAD

# 5. Done.

I myself got the idea from GitHub Action actions/checkout — thanks to them!

Did you just learn something of value? Are you struggling with any things Git? Let me know!

Sebastian Pipping

Berlin, 2024

Steve Jobs About Leadership

Selling Without Being Salesy: The Kind Way to Sell. (Chris Do)

For readers new to Expat: libexpat is a fast streaming XML parser. Alongside libxml2, Expat is one of the most widely used software libre XML parsers written in C, specifically C99. It is cross-platform and licensed under the MIT license.

Expat 2.6.2 has been released earlier today. This release is the first with a detailed call-for-help banner at the top of the change log — something will have to change. It has literally been said to me that "XKCD 2347 is libexpat". If your employer or business depends on the security of Expat — if, for example, you use Expat to parse input from uploaded files or the network, directly or through another library or application — please make sure this gets the needed attention — thanks!

Regarding actual release content, most importantly, this release fixes the security issue CVE-2024-28757 that can be used to cause denial of service for code like…

XML_Parser parser = XML_ParserCreate(NULL);
XML_Parser ext_parser
  = XML_ExternalEntityParserCreate(parser, NULL, NULL);
enum XML_Status status
  = XML_Parse(ext_parser, doc, (int)strlen(doc), XML_TRUE);

…where all input is sent to the external parser and none to the parent regular parser.
The commit message of commit 1d50b80cf31de87750103656f6eb693746854aa8 explains the problem and solution in more detail.

There is also a bugfix to reject direct parameter entity recursion and to avoid the related undefined behavior. The issue was uncovered by ClusterFuzz/OSS-Fuzz after 20+ years of being unreported; that speaks volumes for the value of fuzzing.

For more details about this release, please check out the change log.

If you maintain Expat packaging or a bundled copy of Expat or a pinned version of Expat somewhere, please update to 2.6.2. Thank you!

Sebastian Pipping

Hotel Matze: Markus Lanz — über Niederlagen, Misstrauen und deutsche Politik

Miss-Verstehen Sie Mich Richtig: Gregor Gysi & Markus Lanz