Editing Blog/2025-01-16/Superheap

[[File:{{#setmainimage:Superheap Bloomberg RSS Feed Example.png}}|thumb|Overall, not too bad]]When friends got on Substack, one of the things that annoyed me is that it doesn't have an RSS feed. This is completely untrue. It does have an RSS feed, it's just that my friend [https://porterwestling.substack.com/ Porter] had chosen the Twitter-like version of Substack instead. But I didn't realize this and decided that I'd prefer if I received this stuff as an RSS feed anyway. Since Substack refuses to accept [https://kill-the-newsletter.com/ Kill The Newsletter] emails, I wondered to myself how easy it would be to write my own, even though [https://github.com/leafac/kill-the-newsletter KTN is open source]

[https://github.com/roshan/superheap This ended up being really easy to do because of LLMs today]. I ended up using a mix of Cursor, RustRover, and Claude 3.5 Sonnet. Cursor is pretty amazing as a tool to get going, though it seems to be more of an AI text-editor than an IDE for Rustlang: it has pretty slow comprehension of code and jumping to definition and so on don't work for me out of the box.

In the end, I built the initial scaffolding with Cursor and then used RustRover to write some of the code and used a copy-paste workflow vs. Claude.

The whole thing seems fairly simple to me:

# An application mail server that writes to SQLite
# An application that generates feeds from the emails collected
# <code>stunnel</code> to do TLS in front of the app mail server
# <code>nginx</code> to serve the resulting feeds with TLS

Overall, it didn't take very much time to get the whole thing working - and honestly that's simply because LLMs have accelerated development so much! In fact, the email protocol is mostly text-based and so a plain-text version of this could be 200 lines of code in Rust without invoking any libraries. To start with, Cursor just wrote up the protocol and mail parsing as we were going and I got to 50% quite fast. At this point I was getting the impression that some class of software libraries are not that useful: with known protocols, instant re-implementation is not that costly.

But TLS is a big exception. I'm sure it's possible to easily do it at the application server, but the experience of using <code>stunnel</code> to handle TLS for the mail server and <code>nginx</code> to do so likewise for web-serving was actually quite smooth. Claude and ChatGPT made a few missteps here recommending that I use <code>nginx</code> with <code>mail_proxy</code> enabled, but ultimately this wasn't so bad. I anticipate that if I tried to support TLS direct with <code>rusttls</code> it would not be as simple as a 20 line config.

When I first went to test this it worked fine since I sent plaintext emails from Gmail. But when I tried Protonmail, it sends email base64-encoded with a <code>Content-Type-Encoding</code> header that contains that specification! And of course most senders use <code>text/html</code> rather than <code>text/plain</code>

I'm confident that either Claude directly or Cursor w/ Claude could have written me a minimal parser for all of this in half an hour, but it seemed to be less bothersome for me to just use the <code>mail-parser</code> crate.

Overall, this would have taken me a much longer time to discover and handle without an LLM to assist me. They're really phenomenal tools.

In the end, I used this tool for Matt Levine's Money Stuff feed since that one actually ships out a minimal RSS feed with content on the website but a full-featured email with content in the email. So I use this to get the full content in the RSS feed instead.

[[Category:Blog]]