The Web Needs a Context Layer Built on a Shared Protocol
Internet Exchange
internet.exchangepoint.tech
2026-08-20 12:46:51
Making context a shared protocol, rather than a platform feature, would let readers see competing perspectives anywhere on the web, argue Mallory Knodel, Evan Friedman, and Brad Friedman....
Making context a shared protocol, rather than a platform feature, would let readers see competing perspectives anywhere on the web, argue Mallory Knodel, Evan Friedman, and Brad Friedman.
Two people can read the same headline and come away with opposite stories. One may see a public health measure, the other government overreach. Researchers
have long known
that communities don't just disagree on issues; they frame them in entirely different terms. The problem is not disagreement itself. A diverse society will always contain reasonable, competing interpretations of the same event. The issue is that the web typically gives users a single spotlight on a topic without making the surrounding perspectives easy to find. A claim can be accurate but still partial. What is missing is a way to see those competing frames side by side, mapping how the same conversation takes shape across the internet. This would give readers a broader view.
When a misleading post on X or Facebook appears with a note beneath it written by other platform users, that note is context: information, sources, and competing perspectives added alongside the content so readers can judge it more fully. Right now, that context layer is proprietary, created and owned by the platforms on which it appears. But context doesn't
have
to be built this way. Treating it as a protocol, a shared open standard any platform can adopt rather than a feature owned by one company, is an opportunity to build prosocial features into the infrastructure of the web.
In their paper "
From local hacks to global standards: The hidden politics of internet protocols
," Matthew Zook and Ate Poorthuis use three examples to illustrate that historically, infrastructure has started with a smaller use case and then scaled. The danger is that early informal decisions become global rules without enough consideration for human rights and other impacts.
One example they give is the country code top-level domain system, the familiar national suffixes like .FR for France or .UK for Britain, which were built according to ISO 3166, an existing list of two-letter country codes maintained by the International Organization for Standardization. But, from the 1980s until today, the ISO list itself has not been a neutral inventory of the world's nations. It is a list that elides the fraught question of what counts as a country. As a result, particular political histories and institutional relationships were adopted into the domain name system along with those embedded judgments. Territories with contested sovereignty, colonial dependencies, or without recognized statehood were included or excluded, baking political decisions about place into the architecture of the internet.
Context is a new, developing layer of the internet. The most promising tools for adding context to online content are community notes, used by both X and Meta, and which
show real promise
in reducing online harms like misinformation and disinformation. But these context layers are proprietary, owned and managed by these two platforms and, like the ISO list, they come with biases—in this case, those of these platforms’ unique user bases and their commercial interests. If we want context to scale and be scrutable, we need to facilitate context with protocols: that is the ‘how’ of building a context layer. An open, opt-in standard that any publisher, browser, or platform can adopt, rather than a feature each company builds and owns, can democratize context and appropriately place it within the realm of the political: that is the what.
Building such a protocol deliberately, in the open, lets us embed choice, user agency, and prosocial values into the context layer of the internet, a Broader View button (
demo here
) built into the web itself, rather than allowing closure to settle around whatever already exists before anyone has deliberately chosen, as happened with the domain name system.
Removals, labels and annotation
Most efforts to improve what people encounter online currently fall into three general categories: removal (take it down), labeling (flag it), and crowdsourced annotation (let users add context). The first two require a platform or trusted third party fact-checkers to decide what is true, which much of the public
no longer trusts it to do
.
Crowdsourced annotation like community notes was developed in part to address the issues of the first two categories, and the evidence suggests that it succeeds, at least in limiting the spread of misinformation and disinformation. The system's own designers found that algorithm-selected notes made users about
26 percent less likely to agree with a misleading claim
, and that exposure to notes reduced likes and retweets by 25 to 34 percent in live deployment. A
causal study
, covering roughly 285,000 Community Notes on X (formerly Twitter), found that attaching a note cut subsequent retweets by about half and raised the chance the author deleted the post by around 80 percent. Issues appear, however, when trying to scale these efforts. The average note takes more than fifteen hours to appear, by which point roughly 80 percent of a post's reach has already happened, so the net effect on overall virality falls to between 16 and 21 percent.
Plus, many posts that perhaps should have notes never do. A note requires volunteers to notice the post, write a note, and reach cross-partisan agreement before it is published. This is a high barrier that only about
11 percent of proposed notes
ever meet. Fewer than 10 percent of published notes
reach "helpful" status
, and 26 percent of those that do are later removed due to disagreement.
In addition, a great deal of online content is not false. It is accurate as far as it goes, but may show only one side of a contested issue. No current moderation system systematically surfaces the competing frames around a post that is true-but-partial, and a small, hyperactive minority of users
produce most of what everyone sees
, and that content skews more politically extreme than what the typical user posts, so the less partisan majority is rendered nearly invisible. On genuinely contested questions, the “true or false” binary is even less useful: the truth often isn’t settled for years, long after the moderation decision has been made and the post has done its work.
Rather than seeing this as an indictment of content moderation or Community Notes, which is the clearest proof we have that a context layer can work, we see it as evidence that the bottleneck is architectural: a layer run by volunteers inside one platform's user base will always face an upper ceiling that we believe only a shared standard can alleviate.
Why now? AI, obviously
A context layer that depends on volunteers noticing a post, writing a note, and reaching cross-partisan agreement will always be slower and more limited than the content it is intended to contextualize. What has changed is that large language models can now do part of this work by reducing the demand on the volunteer labor that made it scarce, and drawing from a wider range of relevant material.
A recent system,
Supernotes
, uses a language model to synthesize these fragments into a single candidate note, then scores that candidate by modeling how a politically diverse set of raters would respond to it. In testing, participants preferred the AI-synthesized notes to the best existing human-written ones roughly three times out of four. In this study, the model didn't decide what was true; it drafted and assembled content from existing human notes. Whether the result was helpful still came down to human cross-partisan agreement, and the AI was what let that agreement extend to far more content than volunteers could reach alone. This points to something a shared context layer could do beyond simply showing different perspectives side by side: surface where communities that usually disagree actually share ground, an approach sometimes called bridging. The algorithm behind Community Notes
was also built on this principle
, scoring a note highly only when people with otherwise opposed rating histories agree it is helpful, rather than relying on a simple majority. Supernotes extends that same bridging logic with AI.
There is a reason AI may be well-suited to this particular job. People often distrust context when it comes from a perceived opponent, and some research suggests they treat AI-generated summaries as
comparatively impartial
. We should be cautious, because AI carries its own biases that have to be managed openly. But for the narrow task of laying out how different communities frame an issue, that cites sources, AI may have an easier time being heard.
How? The protocol opportunity
A protocol-level approach asks: what if context were shared infrastructure, like a web standard, rather than a feature limited to one provider? A standard that lets any platform, publisher, or browser participate without each needing to build a feature from scratch, and lets context travel across services? And what if users had agency to choose their context provider?
Our model is the closed-captioning (CC) mark. It is instantly recognizable, works across virtually all video regardless of who made it, is owned by no single company, and turns on only when the viewer wants it. Part of the mark’s power is the mark itself: a single recognizable symbol compresses the whole idea into two letters anyone can spot on any screen. A universal mark is what makes an opt-in layer usable by ordinary people, not just legible to technologists.
We propose the same for context: a universal, opt-in icon which we call the “Broader View button,” that a reader clicks only if they want the fuller picture. On a contested political post, that might mean seeing how different communities understand the same event, what they agree on, where they disagree, and the perspectives and sources each community draws on, so a reader can understand the landscape and draw their own conclusions. On a video of a duck leading her ducklings across a highway, the button might open up a wider understanding about migration, habitat loss and how some cities are redesigning roads around wildlife. Context isn't only a corrective for our worst content; it's an invitation to be more curious about all of it.
No content is removed, no fact-checks are pushed into the feed. Because it adds speech rather than restricting it, the approach can hold support across a political spectrum that agrees on little else about online speech.
We propose that the standard should be provider-agnostic. Like choosing a default search engine, different providers could supply the context behind the same button, separating the standard (how context is displayed) from the curation (who, or which AI model, assembles it). Letting readers pick their own provider is a form of user agency, and experienced users
judge content more favorably
when they have actively chosen it rather than had it chosen for them.
We also propose that trust and safety belongs in the protocol itself, for instance, requiring that quoted text in a context window trace to a verifiable source and that off-topic pile-ons be filtered, so every implementation meets a minimum threshold.
A layer worth building
A key principle behind years of content moderation has been to remove false content and correct the record. But
much of what hardens divides online is not false
. Instead it is partial or one-sided, and no content moderation verdict can address this problem. What's missing is not a better judge or a jury. It's a layer that lets users who want it see a bigger, fuller picture.
Across established democracies, the spread of social media has tracked with falling trust and rising polarization, yet the research has gone overwhelmingly toward documenting that harm rather than testing ways out of it.
One 2021 review of more than ninety studies
notes how little work has explored how media might actually depolarize. In other words, we have mapped the problem in great detail, but we have barely begun to identify or fund the solutions.
Community Notes is the clearest proof we have that a context layer can work, and of its limits. They are not a reason to abandon the idea, but a reason to build it properly as shared infrastructure, rather than a feature owned by one company. A Broader View button will not be perfect, but a perfect solution does not exist, and there are costs for waiting.
Where should this work live?
Despite years of thinking from scholars like
Francis Fukuyama
and
Renée DiResta
, what are called “middleware” solutions to content moderation haven’t made it into the protocols standardization pipeline. We are still left with platforms, not protocols, implementing solutions, which
Mike Masnick
pointed out are not ideal.
Taking on a context layer has implications for any technical standards body's mandate already dealing with content, and those bodies are few. The W3C is the most natural home, since it already looks after the web and social standards, however its prior work on
annotation
would only be a partial help. ISO could also take it on as global trust frameworks like
C2PA
are increasingly within mandate and expertise. Whoever shepherds the work takes on more than writing the standard itself. They foster a community of trust and safety rules stewards and will likely bring together a huge cross section of web services and platform implementers.
Support the Internet Exchange
If you find our emails useful, consider becoming a paid subscriber! You'll get access to our members-only Signal community where we share ideas, discuss upcoming topics, and exchange links. Paid subscribers can also leave comments on posts and enjoy a warm, fuzzy feeling.
Not ready for a long-term commitment? You can always
leave us a tip
.
If you've been thinking about becoming an IX subscriber and getting access to all of our hot🔥 links, our members-only Signal community, the ability to leave comments and replies on posts, and the warm fuzzy feeling of knowing you're supporting our mission, now is the time. Annual subscriptions are usually $50 but are just
$30 until the end of August.
HN
comments
are
terrible
.
On
any
topic
I’m
informed
about
,
the
vast
majority
of
comments
are
pretty
clearly
wrong
.
Most
of the time, there are zero comments from people who know anything about the topic and the top comment is reasonable sounding but totally incorrect. Additionally, many comments are gratuitously mean. You'll often hear mean comments backed up with something like "this is better than the other possibility, where everyone just pats each other on the back with comments like 'this is great'", as if being an asshole is some sort of talisman against empty platitudes. I've seen people push back against that; when pressed, people often say that it’s either impossible or inefficient to teach someone without being mean, as if telling someone that they're stupid somehow helps them learn. It's as if people learned how to explain things by watching Simon Cowell and can't comprehend the concept of an explanation that isn't littered with personal insults. Paul Graham has said, "
Oh, you should never read Hacker News comments about anything you write
”. Most of the negative things you hear about HN comments are true.
And yet, I haven’t found a public internet forum with better technical commentary. On topics I'm familiar with, while it's rare that a thread will have even a single comment that's well-informed, when those comments appear, they usually float to the top. On other forums, well-informed comments are either non-existent or get buried by reasonable sounding but totally wrong comments when they appear, and they appear even more rarely than on HN.
By volume, there are probably more interesting technical “posts” in comments than in links. Well, that depends on what you find interesting, but that’s true for my interests. If I see a low-level optimization comment from nkurz, a comment on business from patio11, a comment on how companies operate by nostrademons, I almost certainly know that I’m going to read an interesting comment. There are maybe 20 to 30 people I can think of who don’t blog much, but write great comments on HN and I doubt I even know of half the people who are writing great comments on HN
1
.
I compiled a very abbreviated list of comments I like because comments seem to get lost. If you write a blog post, people will refer it years later, but comments mostly disappear. I think that’s sad -- there’s a lot of great material on HN (and yes, even more not-so-great material).
Basically, the Word file format is a binary dump of memory. I kid you not. They just took whatever was in memory and wrote it out to disk. We can try to reason why (maybe it was faster, maybe it made the code smaller), but I think the overriding reason is that the original developers didn't know any better.
Later as they tried to add features they had to try to make it backward compatible. This is where a lot of the complexity lies. There are lots of crazy workarounds for things that would be simple if you allowed yourself to redesign the file format. It's pretty clear that this was mandated by management, because no software developer would put themselves through that hell for no reason.
Later they added a fast-save feature (I forget what it is actually called). This appends changes to the file without changing the original file. The way they implemented this was really ingenious, but complicates the file structure a lot.
One thing I feel I must point out (I remember posting a huge thing on slashdot when this article was originally posted) is that 2 way file conversion is next to impossible for word processors. That's because the file formats do not contain enough information to format the document. The most obvious place to see this is pagination. The file format does not say where to paginate a text flow (unless it is explicitly entered by the user). It relies of the formatter to do it. Each word processor formats text completely differently. Word, for example famously paginates footnotes incorrectly. They can't change it, though, because it will break backwards compatibility. This is one of the only reasons that Word Perfect survives today -- it is the only word processor that paginates legal documents the way the US Department of Justice requires.
Just considering the pagination issue, you can see what the problem is. When reading a Word document, you have to paginate it like Word -- only the file format doesn't tell you what that is. Then if someone modifies the document and you need to resave it, you need to somehow mark that it should be paginated like Word (even though it might now have features that are not in Word). If it was only pagination, you might be able to do it, but practically everything is like that.
I recommend reading (a bit of) the XML Word file format for those who are interested. You will see large numbers of flags for things like "Format like Word 95". The format doesn't say what that is -- because it's pretty obvious that the authors of the file format don't know. It's lost in a hopeless mess of legacy code and nobody can figure out what it does now.
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#define LENGTH 128
int main(int argc, char **argv) {
char *string = NULL;
int length = 0;
if (argc > 1) {
string = argv[1];
length = strlen(string);
if (length >= LENGTH) exit(1);
}
char buffer[LENGTH];
memcpy(buffer, string, length);
buffer[length] = 0;
if (string == NULL) {
printf("String is null, so cancel the launch.\n");
} else {
printf("String is not null, so launch the missiles!\n");
}
printf("string: %s\n", string); // undefined for null but works in practice
#if SEGFAULT_ON_NULL
printf("%s\n", string); // segfaults on null when bare "%s\n"
#endif
return 0;
}
nate@skylake:~/src$ clang-3.8 -Wall -O3 null_check.c -o null_check
nate@skylake:~/src$ null_check
String is null, so cancel the launch.
string: (null)
nate@skylake:~/src$ icc-17 -Wall -O3 null_check.c -o null_check
nate@skylake:~/src$ null_check
String is null, so cancel the launch.
string: (null)
nate@skylake:~/src$ gcc-5 -Wall -O3 null_check.c -o null_check
nate@skylake:~/src$ null_check
String is not null, so launch the missiles!
string: (null)
It appear that Intel's ICC and Clang still haven't caught up with GCC's optimizations. Ouch if you were depending on that optimization to get the performance you need! But before picking on GCC too much, consider that all three of those compilers segfault on printf("string: "); printf("%s\n", string) when string is NULL, despite having no problem with printf("string: %s\n", string) as a single statement. Can you see why using two separate statements would cause a segfault? If not, see here for a hint:
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=25609
Good engineering eliminates users being able to do the wrong thing as much as possible. . . . You don't design a feature that invites misuse and then use instructions to try to prevent that misuse.
There was a derailment in Australia called the Waterfall derailment [1]. It occurred because the driver had a heart attack and was responsible for 7 deaths (a miracle it was so low, honestly). The root cause was the failure of the dead-man's switch.
In the case of Waterfall, the driver had 2 dead-man switches he could use - 1) the throttle handle had to be held against a spring at a small rotation, or 2) a bar on the floor could be depressed. You had to do 1 of these things, the idea being that you prevent wrist or foot cramping by allowing the driver to alternate between the two. Failure to do either triggers an emergency brake.
It turns out that this driver was fat enough that when he had a heart attack, his leg was able to depress the pedal enough to hold the emergency system off. Thus, the dead-man's system never triggered with a whole lot of dead man in the driver's seat.
I can't quite remember the specifics of the system at Waterfall, but one method to combat this is to require the pedal to be held halfway between released and fully depressed. The idea being that a dead leg would fully depress the pedal so that would trigger a brake, and a fully released pedal would also trigger a brake. I don't know if they had that system but certainly that's one approach used in rail.
Either way, the problem is equally possible in cars. If you lose consciousness and your foot goes limp, a heavy enough leg will be able to hold the pedal down a bit depending on where it's positioned relative to the pedal and the leverage it has on the floor.
The other major system I'm familiar with for ensuring drivers are alive at the helm is called 'vigilance'. The way it works is that periodically, a light starts flashing on the dash and the driver has to acknowledge that. If they do not, a buzzer alarm starts sounding. If they still don't acknowledge it, the train brakes apply and the driver is assumed incapacitated. Let me tell you some stories of my involvement in it.
When we first started, we had a simple vigi system. Every 30 seconds or so (for example), the driver would press a button. Ok cool. Except that then drivers became so hard-wired to pressing the button every 30 seconds that we were having instances of drivers falling asleep/dozing off and still pressing the button right on every 30 seconds because it was so ingrained into them that it was literally a subconscious action.
So we introduced random-timing vigilance, where the time varies 30-60 seconds (for example) and you could only acknowledge it within a small period of time once the light started flashing. Again, drivers started falling asleep/semi asleep and would hit it as soon as the alarm buzzed, each and every time.
So we introduced random-timing, task-linked vigilance and that finally broke the back of the problem. Now, the driver has to press a button, or turn a knob, or do a number of different activities and they must do that randomly-chosen activity, at a randomly-chosen time, for them to acknowledge their consciousness. It was only at that point that we finally nailed out driver alertness.
Curious why he would need to move to a more prestigious position? Most people realize by their 30s that prestige is a sucker's game; it's a way of inducing people to do things that aren't much fun and they wouldn't really want to do on their own, by lauding them with accolades from people they don't really care about.
. . . we noticed that we also needed:
(1) A suitable, existing airport at the hub location.
(2) Good weather at the hub location, e.g., relatively little snow, fog, or rain.
(3) Access to good ramp space, that is, where to park and service the airplanes and sort the packages.
(4) Good labor supply, e.g., for the sort center.
(5) Relatively low cost of living to keep down prices.
(6) Friendly regulatory environment.
(7) Candidate airport not too busy, e.g., don't want arriving planes to have to circle a long time before being able to land.
(8) Airport with relatively little in cross winds and with more than one runway to pick from in case of winds.
(9) Runway altitude not too high, e.g., not high enough to restrict maximum total gross take off weight, e.g., rule out Denver.
(10) No tall obstacles, e.g., mountains, near the ends of the runways.
(11) Good supplies of jet fuel.
(12) Good access to roads for 18 wheel trucks for exchange of packages between trucks and planes, e.g., so that some parts could be trucked to the hub and stored there and shipped directly via the planes to customers that place orders, say, as late as 11 PM for delivery before 10 AM.
So, there were about three candidate locations, Memphis and, as I recall, Cincinnati and Kansas City.
The Memphis airport had some old WWII hangers next to the runway that FedEx could use for the sort center, aircraft maintenance, and HQ office space. Deal done -- it was Memphis.
The decision to sell to Google was one of the toughest decisions I and my cofounders ever had to wrestle with in our lives. We were excited by the Wave vision though we saw the flaws in the product. The Wave team told us about how they wanted our help making wave simpler and more like etherpad, and we thought we could help with that, though in the end we were unsuccessful at making wave simpler. We were scared of Google as a competitor: they had more engineers and more money behind this project, yet they were running it much more like an independent startup than a normal big-company department. The Wave office was in Australia and had almost total autonomy. And finally, after 1.5 years of being on the brink of failure with AppJet, it was tempting to be able to declare our endeavor a success and provide a decent return to all our investors who had risked their money on us.
In the end, our decision to join Wave did not work out as we had hoped. The biggest lessons learned were that having more engineers and money behind a project can actually be more harmful than helpful, so we were wrong to be scared of Wave as a competitor for this reason. It seems obvious in hindsight, but at the time it wasn't. Second, I totally underestimated how hard it would be to iterate on the Wave codebase. I was used to rewriting major portions of software in a single all-nighter. Because of the software development process Wave was using, it was practically impossible to iterate on the product. I should have done more diligence on their specific software engineering processes, but instead I assumed because they seemed to be operating like a startup, that they would be able to iterate like a startup. A lot of the product problems were known to the whole Wave team, but we were crippled by a large complex codebase built on poor technical choices and a cumbersome engineering process that prevented fast iteration.
When I've had inside information about a story that later breaks in the tech press, I'm always shocked at how differently it's perceived by readers of the article vs. how I experienced it. Among startups & major feature launches I've been party to, I've seen: executives that flat-out say that they're not working on a product category when there's been a whole department devoted to it for a year; startups that were founded 1.5 years before the dates listed in Crunchbase/Wikipedia; reporters that count the number of people they meet in a visit and report that as a the "team size", because the company refuses to release that info; funding rounds that never make it to the press; acquisitions that are reported as "for an undisclosed sum" but actually are less than the founders would've made if they'd taken a salaried job at the company; project start dates that are actually when the project was staffed up to its current size and ignore the year or so that a small team spent working on the problem (or the 3-4 years that other small teams spent working on the problem); and algorithms or other technologies that are widely reported as being the core of the company's success, but actually aren't even used by the company.
As the main developer of VLC, we know about this story since a long time, and this is just Dell putting crap components on their machine and blaming others. Any discussion was impossible with them. So let me explain a bit...
In this case, VLC just uses the Windows APIs (DirectSound), and sends signed integers of 16bits (s16) to the Windows Kernel.
VLC allows amplification of the INPUT above the sound that was decoded. This is just like replay gain, broken codecs, badly recorded files or post-amplification and can lead to saturation.
But this is exactly the same if you put your mp3 file through Audacity and increase it and play with WMP, or if you put a DirectShow filter that amplifies the volume after your codec output. For example, for a long time, VLC ac3 and mp3 codecs were too low (-6dB) compared to the reference output.
At worse, this will reduce the dynamics and saturate a lot, but this is not going to break your hardware.
VLC does not (and cannot) modify the OUTPUT volume to destroy the speakers. VLC is a Software using the OFFICIAL platforms APIs.
The issue here is that Dell sound cards output power (that can be approached by a factor of the quadratic of the amplitude) that Dell speakers cannot handle. Simply said, the sound card outputs at max 10W, and the speakers only can take 6W in, and neither their BIOS or drivers block this.
And as VLC is present on a lot of machines, it's simple to blame VLC. "Correlation does not mean causation" is something that seems too complex for cheap Dell support…
Working for someone else's startup, I learned how to quickly cobble solutions together. I learned about uncertainty and picking a direction regardless of whether you're sure it'll work. I learned that most startups fail, and that when they fail, the people who end up doing well are the ones who were looking out for their own interests all along. I learned a lot of basic technical skills, how to write code quickly and learn new APIs quickly and deploy software to multiple machines. I learned how quickly problems of scaling a development team crop up, and how early you should start investing in automation.
Working for Google, I learned how to fix problems once and for all and build that culture into the organization. I learned that even in successful companies, everything is temporary, and that great products are usually built through a lot of hard work by many people rather than great ah-ha insights. I learned how to architect systems for scale, and a lot of practices used for robust, high-availability, frequently-deployed systems. I learned the value of research and of spending a lot of time on a single important problem: many startups take a scattershot approach, trying one weekend hackathon after another and finding nobody wants any of them, while oftentimes there are opportunities that nobody has solved because nobody wants to put in the work. I learned how to work in teams and try to understand what other people want. I learned what problems are really painful for big organizations. I learned how to rigorously research the market and use data to make product decisions, rather than making decisions based on what seems best to one person.
Having been in on the company's leadership meetings where departures were noted with a simple 'regret yes/no' flag it was my experience that no single departure had any effect. Mass departures did, trends did, but one person never did, even when that person was a founder.
The rationalizations always put the issue back on the departing employee, "They were burned out", "They had lost their ability to be effective", "They have moved on", "They just haven't grown with the company" never was it "We failed this person, what are we going to do differently?"
Anyway, the SOA effort was in full swing when I was there. It was a pain, and it was a mess because every team did things differently and every API was different and based on different assumptions and written in a different language.
But I want to correct the misperception that this lead to AWS. It didn't. S3 was written by its own team, from scratch. At the time I was at Amazon, working on the retail site, none of Amazon.com was running on AWS. I know, when AWS was announced, with great fanfare, they said "the services that power Amazon.com can now power your business!" or words to that effect. This was a flat out lie. The only thing they shared was data centers and a standard hardware configuration. Even by the time I left, when AWS was running full steam ahead (and probably running Reddit already), none of Amazon.com was running on AWS, except for a few, small, experimental and relatively new projects. I'm sure more of it has been adopted now, but AWS was always a separate team (and a better managed one, from what I could see.)
But it's also true that people who are more of Windows wizards than I am a Linux apprentice have worked on Chrome's Windows build. If you asked me the original question, I'd say the underlying problem is that on Windows all you have is what Microsoft gives you and you can't typically do better than that. For example, migrating the Chrome build off of Visual Studio would be a large undertaking, large enough that it's rarely considered. (Another way of phrasing this is it's the IDE problem: you get all of the IDE or you get nothing.)
When addressing the poor Windows performance people first bought SSDs, something that never even occurred to me ("your system has enough RAM that the kernel cache of the file system should be in memory anyway!"). But for whatever reason on the Linux side some Googlers saw it fit to rewrite the Linux linker to make it twice as fast (this effort predated Chrome), and all Linux developers now get to benefit from that. Perhaps the difference is that when people write awesome tools for Windows or Mac they try to sell them rather than give them away.
I'm a developer in Windows and contribute to the NT kernel. (Proof: the SHA1 hash of revision #102 of [Edit: filename redacted] is [Edit: hash redacted].) I'm posting through Tor for obvious reasons.
Windows is indeed slower than other operating systems in many scenarios, and the gap is worsening. The cause of the problem is social. There's almost none of the improvement for its own sake, for the sake of glory, that you see in the Linux world.
Granted, occasionally one sees naive people try to make things better. These people almost always fail. We can and do improve performance for specific scenarios that people with the ability to allocate resources believe impact business goals, but this work is Sisyphean. There's no formal or informal program of systemic performance improvement. We started caring about security because pre-SP3 Windows XP was an existential threat to the business. Our low performance is not an existential threat to the business.
See, component owners are generally openly hostile to outside patches: if you're a dev, accepting an outside patch makes your lead angry (due to the need to maintain this patch and to justify in in shiproom the unplanned design change), makes test angry (because test is on the hook for making sure the change doesn't break anything, and you just made work for them), and PM is angry (due to the schedule implications of code churn). There's just no incentive to accept changes from outside your own team. You can always find a reason to say "no", and you have very little incentive to say "yes".
Broken record: startups are also probably rejecting a lot of engineering candidates that would perform as well or better than anyone on their existing team, because tech industry hiring processes are folkloric and irrational.
I am 42-year-old very successful programmer who has been through a lot of situations in my career so far, many of them highly demotivating. And the best advice I have for you is to get out of what you are doing. Really. Even though you state that you are not in a position to do that, you really are. It is okay. You are free. Okay, you are helping your boyfriend's startup but what is the appropriate cost for this? Would he have you do it if he knew it was crushing your soul?
I don't use the phrase "crushing your soul" lightly. When it happens slowly, as it does in these cases, it is hard to see the scale of what is happening. But this is a very serious situation and if left unchecked it may damage the potential for you to do good work for the rest of your life.
The commenters who are warning about burnout are right. Burnout is a very serious situation. If you burn yourself out hard, it will be difficult to be effective at any future job you go to, even if it is ostensibly a wonderful job. Treat burnout like a physical injury. I burned myself out once and it took at least 12 years to regain full productivity. Don't do it.
More broadly, the best and most creative work comes from a root of joy and excitement. If you lose your ability to feel joy and excitement about programming-related things, you'll be unable to do the best work. That this issue is separate from and parallel to burnout! If you are burned out, you might still be able to feel the joy and excitement briefly at the start of a project/idea, but they will fade quickly as the reality of day-to-day work sets in. Alternatively, if you are not burned out but also do not have a sense of wonder, it is likely you will never get yourself started on the good work.
The earlier in your career it is now, the more important this time is for your development. Programmers learn by doing. If you put yourself into an environment where you are constantly challenged and are working at the top threshold of your ability, then after a few years have gone by, your skills will have increased tremendously. It is like going to intensively learn kung fu for a few years, or going into Navy SEAL training or something. But this isn't just a one-time constant increase. The faster you get things done, and the more thorough and error-free they are, the more ideas you can execute on, which means you will learn faster in the future too. Over the long term, programming skill is like compound interest. More now means a LOT more later. Less now means a LOT less later.
So if you are putting yourself into a position that is not really challenging, that is a bummer day in and day out, and you get things done slowly, you aren't just having a slow time now. You are bringing down that compound interest curve for the rest of your career. It is a serious problem. If I could go back to my early career I would mercilessly cut out all the shitty jobs I did (and there were many of them).
A small anecdote. An acquaintance related a story of fixing the 'drainage' in their back yard. They were trying to grow some plants that were sensitive to excessive moisture, and the plants were dying. Not watering them, watering them a little, didn't seem to change. They died. A professional gardner suggested that their problem was drainage. So they dug down about 3' (where the soil was very very wet) and tried to build in better drainage. As they were on the side of a hill, water table issues were not considered. It turned out their "problem" was that the water main that fed their house and the houses up the hill, was so pressurized at their property (because it had maintain pressure at the top of the hill too) that the pipe seams were leaking and it was pumping gallons of water into the ground underneath their property. The problem wasn't their garden, the problem was that the city water supply was poorly designed.
While I have never been asked if I was an engineer on the phone, I have experienced similar things to Rachel in meetings and with regard to suggestions. Co-workers will create an internal assessment of your value and then respond based on that assessment. If they have written you off they will ignore you, if you prove their assessment wrong in a public forum they will attack you. These are management issues, and something which was sorely lacking in the stories.
If you are the "owner" of a meeting, and someone is trying to be heard and isn't. It is incumbent on you to let them be heard. By your position power as "the boss" you can naturally interrupt a discussion to collect more data from other members. Its also important to ask questions like "does anyone have any concerns?" to draw out people who have valid input but are too timid to share it.
In a highly political environment there are two ways to create change, one is through overt manipulation, which is to collect political power to yourself and then exert it to enact change, and the other is covert manipulation, which is to enact change subtly enough that the political organism doesn't react. (sometimes called "triggering the antibodies").
The problem with the latter is that if you help make positive change while keeping everyone not pissed off, no one attributes it to you (which is good for the change agent because if they knew the anti-bodies would react, but bad if your manager doesn't recognize it). I asked my manager what change he wanted to be 'true' yet he (or others) had been unsuccessful making true, he gave me one, and 18 months later that change was in place. He didn't believe that I was the one who had made the change. I suggested he pick a change he wanted to happen and not tell me, then in 18 months we could see if that one happened :-). But he also didn't understand enough about organizational dynamics to know that making change without having the source of that change point back at you was even possible.
Heavily relying on Google product? ✓
Hitting a dead-end with Google's customer service? ✓
Have an existing audience you can leverage to get some random Google employee's attention? ✓
Reach front page of Hacker News? ✓
Good news! You should have your problem fixed in 2-5 business days. The rest of us suckers relying on google services get to stare at our inboxes helplessly, waiting for a response to our support ticket (which will never come). I feel like it's almost a [rite] of passage these days to rely heavily on a Google service, only to have something go wrong and be left out in the cold.
IIRC PayPal was very similar - it was sold for $1.5B, but Max Levchin's share was only about $30M, and Elon Musk's was only about $100M. By comparison, many early Web 2.0 darlings (Del.icio.us, Blogger, Flickr) sold for only $20-40M, but their founders had only taken small seed rounds, and so the vast majority of the purchase price went to the founders. 75% of a $40M acquisition = 3% of a $1B acquisition.
Something for founders to think about when they're taking funding. If you look at the gigantic tech fortunes - Gates, Page/Brin, Omidyar, Bezos, Zuckerburg, Hewlett/Packard - they usually came from having a company that was already profitable or was already well down the hockey-stick user growth curve and had a clear path to monetization by the time they sought investment. Companies that fight tooth & nail for customers and need lots of outside capital to do it usually have much worse financial outcomes.
A lot of the people who were involved in some way in Experts-Exchange don't understand Stack Overflow.
The basic value flow of EE is that "experts" provide valuable "answers" for novices with questions. In that equation there's one person asking a question and one person writing an answer.
Stack Overflow recognizes that for every person who asks a question, 100 - 10,000 people will type that same question into Google and find an answer that has already been written. In our equation, we are a community of people writing answers that will be read by hundreds or thousands of people. Ours is a project more like wikipedia -- collaboratively creating a resource for the Internet at large.
Because that resource is provided by the community, it belongs to the community. That's why our data is freely available and licensed under creative commons. We did this specifically because of the negative experience we had with EE taking a community-generated resource and deciding to slap a paywall around it.
The attitude of many EE contributors, like Greg Young who calculates that he "worked" for half a year for free, is not shared by the 60,000 people who write answers on SO every month. When you talk to them you realize that on Stack Overflow, answering questions is about learning. It's about creating a permanent artifact to make the Internet better. It's about helping someone solve a problem in five minutes that would have taken them hours to solve on their own. It's not about working for free.
As soon as EE introduced the concept of money they forced everybody to think of their work on EE as just that -- work.
I saw that one of my old textbooks was selling for a nice price, so I listed it along with two other used copies. I priced it $1 cheaper than the lowest price offered, but within an hour both sellers had changed their prices to $.01 and $.02 cheaper than mine. I reduced it two times more by $1, and each time they beat my price by a cent or two. So what I did was reduce my price by a few dollars every hour for one day until everybody was priced under $5. Then I bought their books and changed my price back.
While I like the sentiment here, I think the danger is that engineers might come to the mistaken conclusion that making pizzas is the primary limiting reagent to running a successful pizzeria. Running a successful pizzeria is more about schlepping to local hotels and leaving them 50 copies of your menu to put at the front desk, hiring drivers who will both deliver pizzas in a timely fashion and not embezzle your (razor-thin) profits while also costing next-to-nothing to employ, maintaining a kitchen in sufficient order to pass your local health inspector's annual visit (and dealing with 47 different pieces of paper related to that), being able to juggle priorities like "Do I take out a bank loan to build a new brick-oven, which will make the pizza taste better, in the knowledge that this will commit $3,000 of my cash flow every month for the next 3 years, or do I hire an extra cook?", sourcing ingredients such that they're available in quantity and quality every day for a fairly consistent price, setting prices such that they're locally competitive for your chosen clientele but generate a healthy gross margin for the business, understanding why a healthy gross margin really doesn't imply a healthy net margin and that the rent still needs to get paid, keeping good-enough records such that you know whether your business is dying before you can't make payroll and such that you can provide a reasonably accurate picture of accounts for the taxation authorities every year, balancing 50% off medium pizza promotions with the desire to not cannibalize the business of your regulars, etc etc, and by the way tomato sauce should be tangy but not sour and cheese should melt with just the faintest whisp of a crust on it.
Do you want to write software for a living? Google is hiring. Do you want to run a software business? Godspeed. Software is now 10% of your working life.
The way I prefer to think of it is: it is not your job to protect people (particularly senior management) from the consequences of their decisions. Make your decisions in your own best interest; it is up to the organization to make sure that your interest aligns with theirs.
Google used to have a severe problem where code refactoring & maintenance was not rewarded in performance reviews while launches were highly regarded, which led to the effect of everybody trying to launch things as fast as possible and nobody cleaning up the messes left behind. Eventually launches started getting slowed down, Larry started asking "Why can't we have nice things?", and everybody responded "Because you've been paying us to rack up technical debt." As a result, teams were formed with the express purpose of code health & maintenance, those teams that were already working on those goals got more visibility, and refactoring contributions started counting for something in perf. Moreover, many ex-Googlers who were fed up with the situation went to Facebook and, I've heard, instituted a culture there where grungy engineering maintenance is valued by your peers.
None of this would've happened if people had just heroically fallen on their own sword and burnt out doing work nobody cared about. Sometimes it takes highly visible consequences before people with decision-making power realize there's a problem and start correcting it. If those consequences never happen, they'll keep believing it's not a problem and won't pay much attention to it.
The thing my grandfather taught me was that you live with all of your decisions for the rest of your life. When you make decisions which put other people at risk, you take on the risk that you are going to make someones life harder, possibly much harder. What is perhaps even more important is that no amount of "I'm so sorry I did that ..." will ever undo it. Sometimes its little things, like taking the last serving because you thought everyone had eaten, sometimes its big things like deciding that home is close enough that and you're sober enough to get there safely. They are all decisions we make every day. And as I've gotten older the weight of ones I wish I had made differently doesn't get any lighter. You can lie to yourself about your choices, rationalize them, but that doesn't change them either.
I didn't understand any of that when I was younger.
It took me too long to figure this out. There are some people to truly, and passionately, believe something they say to you, and realistically they personally can't make it happen so you can't really bank on that 'promise.'
I used to think those people were lying to take advantage, but as I've gotten older I have come to recognize that these 'yes' people get promoted a lot. And for some of them, they really do believe what they are saying.
As an engineer I've found that once I can 'calibrate' someone's 'yes-ness' I can then work with them, understanding that they only make 'wishful' commitments rather than 'reasoned' commitments.
So when someone, like Steve Jobs, says "we're going to make it an open standard!", my first question then is "Great, I've got your support in making this an open standard so I can count on you to wield your position influence to aid me when folks line up against that effort, right?" If the answer that that question is no, then they were lying.
The difference is subtle of course but important. Steve clearly doesn't go to standards meetings and vote etc, but if Manager Bob gets push back from accounting that he's going to exceed his travel budget by sending 5 guys to the Open Video Chat Working Group which is championing the Facetime protocol as an open standard, then Manager Bob goes to Steve and says "I need your help here, these 5 guys are needed to argue this standard and keep it from being turned into a turd by the 5 guys from Google who are going to attend." and then Steve whips off a one liner to accounting that says "Get off this guy's back we need this." Then its all good. If on the other hand he says "We gotta save money, send one guy." well in that case I'm more sympathetic to the accusation of prevarication.
For those who work inside Google, it's well worth it to look at Jeff & Sanjay's commit history and code review dashboard. They aren't actually all that much more productive in terms of code written than a decent SWE3 who knows his codebase.
The reason they have a reputation as rockstars is that they can apply this productivity to things that really matter; they're able to pick out the really important parts of the problem and then focus their efforts there, so that the end result ends up being much more impactful than what the SWE3 wrote. The SWE3 may spend his time writing a bunch of unit tests that catch bugs that wouldn't really have happened anyway, or migrating from one system to another that isn't really a large improvement, or going down an architectural dead end that'll just have to be rewritten later. Jeff or Sanjay (or any of the other folks operating at that level) will spend their time running a proposed API by clients to ensure it meets their needs, or measuring the performance of subsystems so they fully understand their building blocks, or mentally simulating the operation of the system before building it so they rapidly test out alternatives. They don't actually write more code than a junior developer (oftentimes, they write less), but the code they do write gives them more information, which makes them ensure that they write the rightcode.
I feel like this point needs to be stressed a whole lot more than it is, as there's a whole mythology that's grown up around 10x developers that's not all that helpful. In particular, people need to realize that these developers rapidly become 1x developers (or worse) if you don't let them make their own architectural choices - the reason they're excellent in the first place is because they know how to determine if certain work is going to be useless and avoid doing it in the first place. If you dictate that they do it anyway, they're going to be just as slow as any other developer
I got the hero speech too, once. If anyone ever mentions the word "heroic" again and there isn't a burning building involved, I will start looking for new employment immediately. It seems that in our industry it is universally a code word for "We're about to exploit you because the project is understaffed and under budgeted for time and that is exactly as we planned it so you'd better cowboy up."
Maybe it is different if you're writing Quake, but I guarantee you the 43rd best selling game that year also had programmers "encouraged onwards" by tales of the glory that awaited after the death march.
I was once speaking to a good friend of mine here, in English.
"Do you want to go out for yakitori?"
"Go fuck yourself!"
"... switches to Japanese Have I recently done anything very major to offend you?"
"No, of course not."
"Oh, OK, I was worried. So that phrase, that's something you would only say under extreme distress when you had maximal desire to offend me, or I suppose you could use it jokingly between friends, but neither you nor I generally talk that way."
"I learned it from a movie. I thought it meant ‘No.’"
True story: I went to a talk given by one of the 'engineering elders' (these were low Emp# engineers who were considered quite successful and were to be emulated by the workers :-) This person stated when they came to work at Google they were given the XYZ system to work on (sadly I'm prevented from disclosing the actual system). They remarked how they spent a couple of days looking over the system which was complicated and creaky, they couldn't figure it out so they wrote a new system. Yup, and they committed that. This person is a coding God are they not? (sarcasm) I asked what happened to the old system (I knew but was interested on their perspective) and they said it was still around because a few things still used it, but (quite proudly) nearly everything else had moved to their new system.
So if you were reading carefully, this person created a new system to 'replace' an existing system which they didn't understand and got nearly everyone to move to the new system. That made them uber because they got something big to put on their internal resume, and a whole crapload of folks had to write new code to adapt from the old system to this new system, which imperfectly recreated the old system (remember they didn't understand the original), such that those parts of the system that relied on the more obscure bits had yet to be converted (because nobody undersood either the dependent code or the old system apparently).
Was this person smart? Blindingly brilliant according to some of their peers. Did they get things done? Hell yes, they wrote the replacement for the XYZ system from scratch! One person? Can you imagine? Would I hire them? Not unless they were the last qualified person in my pool and I was out of time.
That anecdote encapsulates the dangerous side of smart people who get things done.
Some kids grow up on football. I grew up on public speaking (as behavioral therapy for a speech impediment, actually). If you want to get radically better in a hurry:
I can relate to this, but I can also relate to the other side of the question. Sometimes it isn't me, its you. Take someone who gets things done and suddenly in your organization they aren't delivering. Could be them, but it could also be you.
I had this experience working at Google. I had a horrible time getting anything done there. Now I spent a bit of time evaluating that since it had never been the case in my career, up to that point, where I was unable to move the ball forward and I really wanted to understand that. The short answer was that Google had developed a number of people who spent much, if not all, of their time preventing change. It took me a while to figure out what motivated someone to be anti-change.
The fear was risk and safety. Folks moved around a lot and so you had people in charge of systems they didn't build, didn't understand all the moving parts of, and were apt to get a poor rating if they broke. When dealing with people in that situation one could either educate them and bring them along, or steam roll over them. Education takes time, and during that time the 'teacher' doesn't get anything done. This favors steamrolling evolutionarily :-)
So you can hire someone who gets stuff done, but if getting stuff done in your organization requires them to be an asshole, and they aren't up for that, well they aren't going to be nearly as successful as you would like them to be.
I can tell that this was written by an outsider, because it focuses on the perks and rehashes several cliches that have made their way into the popular media but aren't all that accurate.
Most Googlers will tell you that the best thing about working there is having the ability to work on really hard problems, with really smart coworkers, and lots of resources at your disposal. I remember asking my interviewer whether I could use things like Google's index if I had a cool 20% idea, and he was like "Sure. That's encouraged. Oftentimes I'll just grab 4000 or so machines and run a MapReduce to test out some hypothesis." My phone screener, when I asked him what it was like to work there, said "It's a place where really smart people go to be average," which has turned out to be both true and honestly one of the best things that I've gained from working there.
This entire event was a staged press op. Keith Alexander is a ~30 year veteran of SIGINT, electronic warfare, and intelligence, and a Four-Star US Army General --- which is a bigger deal than you probably think it is. He's a spy chief in the truest sense and a master politician. Anyone who thinks he walked into that conference hall in Caesars without a near perfect forecast of the outcome of the speech is kidding themselves.
Heckling Alexander played right into the strategy. It gave him an opportunity to look reasonable compared to his detractors, and, more generally (and alarmingly), to have the NSA look more reasonable compared to opponents of NSA surveillance. It allowed him to "split the vote" with audience reactions, getting people who probably have serious misgivings about NSA programs to applaud his calm and graceful handling of shouted insults; many of those people probably applauded simply to protest the hecklers, who after all were making it harder for them to follow what Alexander was trying to say.
There was no serious Q&A on offer at the keynote. The questions were pre-screened; all attendees could do was vote on them. There was no possibility that anything would come of this speech other than an effectively unchallenged full-throated defense of the NSA's programs.
Interestingly one of the things that I found most amazing when I was working for Google was a nearly total inability to grasp the concept of 'deadline.' For so many years the company just shipped it by committing it to the release branch and having the code deploy over the course of a small number of weeks to the 'fleet'.
Sure there were 'processes', like "Canary it in some cluster and watch the results for a few weeks before turning it loose on the world." but being completely vertically integrated is a unique sort of situation.
Being a very experienced game developer who tried to switch to Linux, I have posted about this before (and gotten flamed heavily by reactionary Linux people).
The main reason is that debugging is terrible on Linux. gdb is just bad to use, and all these IDEs that try to interface with gdb to "improve" it do it badly (mainly because gdb itself is not good at being interfaced with). Someone needs to nuke this site from orbit and build a new debugger from scratch, and provide a library-style API that IDEs can use to inspect executables in rich and subtle ways.
Productivity is crucial. If the lack of a reasonable debugging environment costs me even 5% of my productivity, that is too much, because games take so much work to make. At the end of a project, I just don't have 5% effort left any more. It requires everything. (But the current Linux situation is way more than a 5% productivity drain. I don't know exactly what it is, but if I were to guess, I would say it is something like 20%.)
What is interesting is that people don't even know they have a complex about money until they get "rich." I've watched many people, perhaps a hundred, go from "working to pay the bills" to "holy crap I can pay all my current and possibly my future bills with the money I now have." That doesn't include the guy who lived in our neighborhood and won the CA lottery one year.
It affects people in ways they don't expect. If its sudden (like lottery winning or sudden IPO surge) it can be difficult to process. But it is an important thing to realize that one is processing an exceptional event. Like having a loved one die or a spouse suddenly divorcing you.
Not everyone feels "guilty", not everyone feels "smug." A lot of millionaires and billionaires in the Bay Area are outwardly unchanged. But the bottom line is that the emotion comes from the cognitive dissonance between values and reality. What do you value? What is reality?
One woman I knew at Google was massively conflicted when she started work at Google. She always felt that she would help the homeless folks she saw, if she had more money than she needed. Upon becoming rich (on Google stock value), now she found that she wanted to save the money she had for her future kids education and needs. Was she a bad person? Before? After? Do your kids hate you if you give away their college education to the local foodbank? Do your peers hate you because you could close the current food gap at the foodbank and you don't?
This is Microsoft's ICQ moment. Overpaying for a company at the moment when its core competency is becoming a commodity. Does anyone have the slightest bit of loyalty to Skype? Of course not. They're going to use whichever video chat comes built into their SmartPhone, tablet, computer, etc. They're going to use FaceBook's eventual video chat service or something Google offers. No one is going to actively seek out Skype when so many alternatives exist and are deeply integrated into the products/services they already use. Certainly no one is going to buy a Microsoft product simply because it has Skype integration. Who cares if it's FaceTime, FaceBook Video Chat, Google Video Chat? It's all the same to the user.
With $7B they should have just given away about 15 million Windows Mobile phones in the form of an epic PR stunt. It's not a bad product -- they just need to make people realize it exists. If they want to flush money down the toilet they might as well engage users in the process right?
I worked briefly on the Fiber team when it was very young (basically from 2 weeks before to 2 weeks after launch - I was on loan from Search specifically so that they could hit their launch goals). The bottleneck when I was there were local government regulations, and in fact Kansas City was chosen because it had a unified city/county/utility regulatory authority that was very favorable to Google. To lay fiber to the home, you either need right-of-ways on the utility poles (which are owned by Google's competitors) or you need permission to dig up streets (which requires a mess of permitting from the city government). In either case, the cable & phone companies were in very tight with local regulators, and so you had hostile gatekeepers whose approval you absolutely needed.
The technology was awesome (1G Internet and HDTV!), the software all worked great, and the economics of hiring contractors to lay the fiber itself actually worked out. The big problem was regulatory capture.
With Uber & AirBnB's success in hindsight, I'd say that the way to crack the ISP business is to provide your customers with the tools to break the law en masse. For example, you could imagine an ISP startup that basically says "Here's a box, a wire, and a map of other customers' locations. Plug into their jack, and if you can convince others to plug into yours, we'll give you a discount on your monthly bill based on how many you sign up." But Google in general is not willing to break laws - they'll go right up to the boundary of what the law allows, but if a regulatory agency says "No, you can't do that", they won't do it rather than fight the agency.
Indeed, Fiber is being phased out in favor of Google's acquisition of WebPass, which does basically exactly that but with wireless instead of fiber. WebPass only requires the building owner's consent, and leaves the city out of it.
I've spoken at TechEds in the US and Europe, and been in the top 10 for attendee feedback twice.
I'd never speak at TechEd again, and I told Microsoft the same thing, same reasons. The event staff is overly demanding and inconsiderate of speaker time. They repeatedly dragged me into mandatory virtual and in-person meetings to cover inane details that should have been covered via email. They mandated the color of pants speakers wore. Just ridiculously micromanaged.
Hertz laid off nearly the entirety of their rank and file IT staff earlier this year.
In order to receive our severance, we were forced to train our IBM replacements, who were in India. Hertz's strategy of IBM and Austerity is the new SMT's solution for a balance sheet that's in shambles, yet they have rewarded themselves by increasing executive compensation 35% over the prior year, including a $6 million bonus to the CIO.
I personally landed in an Alphabet company, received a giant raise, and now I get to work on really amazing stuff, so I'm doing fine. But to this day I'm sad to think how our once-amazing Hertz team, staffed with really smart people, led by the best boss I ever had, and were really driving the innovation at Hertz, was just thrown away like yesterday's garbage.
Don't count on definitely being able to sell the stock to finance the taxes. I left after seven years in very good standing (I believed) but when I went to sell the deal was shut down [1]. Luckily I had a backup plan and I was ok [2].
[1] Had a handshake deal with an investor in the company, then the investor went silent on me. When I followed up he said the deal was "just much too small." I reached out to the company for help, and they said they'd actually told him not to buy from me. I never would have known if they hadn't decided to tell me for some reason. The takeaway is that the markets for private company stock tend to be small, and the buyers care more about their relationships with the company than they do about having your shares. Even if the stock terms allow them to buy, and they might not.
I took the first test just like the OP, the logical reasoning part seemed kind of irrelevant and a waste of time for me. That was nothing compared to the second online test.
The environment of the second test was like a scenario out of Black Mirror. Not only did they want to have the webcam and microphone on the entire time, I also had to install their custom software so the proctors could monitor my screen and control my computer. They opened up the macOS system preferences so they could disable all shortcuts to take screenshots, and they also manually closed all the background services I had running (even f.lux!).
Then they asked me to pick up my laptop and show them around my room with the webcam. They specifically asked to see the contents of my desk and the walls and ceiling of my room. I had some pencil and paper on my desk to use as scratch paper for the obvious reasons and they told me that wasn't allowed. Obviously that made me a little upset because I use it to sketch out examples and concepts. They also saw my phone on the desk and asked me to put it out of arm's reach.
After that they told me I couldn't leave the room until the 5 minute bathroom break allowed half-way through the test. I had forgotten to tell my roommate I was taking this test and he was making a bit of a ruckus playing L4D2 online (obviously a bit distracting). I asked the proctor if I could briefly leave the room to ask him to quiet down. They said I couldn't leave until the bathroom break so there was nothing I could do. Later on, I was busy thinking about a problem and had adjusted how I was sitting in my chair and moved my face slightly out of the camera's view. The proctor messaged me again telling me to move so they could see my entire face.
The first part of the interview was exactly like the linked experience. No coding questions just reasoning. The second part I had to use ProctorU instead of Proctorio. Personally I thought the experience was super weird but understandable, I'll get to that later, somebody watched me through my webcam the entire time with my microphone on. They needed to check my ID before the test. They needed me to show them the entire room I was in (which was my bedroom). My desktop computer was on behind my laptop so I turned off my computer (I don't remember if I offered to or if they asked me to) but they also asked me to cover my monitors up with something which I thought was silly after I turned them off so I covered them with a towel. They then used LogMeIn to remote into my machine so they could check running programs. I quit all my personal chat programs and pretty much only had the Chrome window running.
...
I didn't talk a real person who actually worked at Amazon (by email or through webcam) until I received an offer.
[M]y company got acquired by Oracle. We thought things would be OK. Nothing changed immediately. Slowly but surely they turned the screws. 5 year laptop replacement policy. You get the corporate standard laptop and you'll like it. Sales? Oh those guys can buy new Macs every two years, they get whatever they want. Then you understand where Software Engineers rank in the company hierarchy. Oracle took the average price of our product from $100k to $5 million for the same size deals. Our sales went from $5-7m to more than $40m with no increasing in engineering headcount (team of 15). Didn't matter when bonus time came, we all got stack-ranked and some people got nothing. As a top performer I got a few options, worth maybe $5k.
Oracle exists to extract the maximum amount of money possible from the Fortune 1000. Everyone else can fuck off. Your impotent internet rage is meaningless. If it doesn't piss off the CTO of $X then it doesn't matter. If it gets that CTO to cut a bigger check then it will be embraced with extreme enthusiasm.
The culture wears down a lot (but not all) of the good people, who then leave. What's left is a lot of mediocrity and architecture astronauts. The more complex the product the better - it means extra consulting dollars!
My relative works at a business dependent on Micros. When Oracle announced the acquisition I told them to start on the backup plan immediately because Oracle was going to screw them sooner or later. A few years on and that is proving true: Oracle is slowly excising the Micros dealers and ISVs out of the picture, gobbling up all the revenue while hiking prices.
In practice, we have to face that all that our quest for more stringent hiring standards is not really selecting the best, but just selecting fewer people, in ways that might, or might not, have anything to do with being good at a job. Let's go through a few examples in my career:
A guy that was the most prolific developer I have ever seen: He'd rewrite entire subsystems over a weekend. The problem is that said susbsytems were not necessarily better than they started, trading bugs for bugs, and anyone that wanted to work on them would have to relearn that programmer's idiosyncrasies of the week. He easily cost his project 12 man/months of work in 4 months, the length of time it took for management to realize that he had to be let go.
A company's big UI framework was quite broken, and a new developer came in and fixed it. Great, right? Well, he was handed code review veto to changes into the framework, and his standards and his demeanor made people stop contributing after two or three attempts. In practice, the framework died as people found it antiquated, and they decided to build a new one: Well, the same developer was tasked with building new framwork, which was made mandatory for 200+ developers to use. Total contribution was clearly negative.
A developer that was very fast, and wrote working code, had been managing a rather large 500K line codebase, and received some developers as help. He didn't believe in internal documentation or on keeping interfaces stable. He also didn't believe in writing code that wasn't brittle, or in unit tests: Code changes from the new developers often broke things, the veteran would come in, fix everything in the middle of the emergency, and look absolutely great, while all the other developers looked to management as if they were incompetent. They were not, however: they were quite successful when moved to other teams. It just happens that the original developer made sure nobody else could touch anything. Eventually, the experiment was retried after the original developer was sent to do other things. It took a few months, but the new replacement team managed to modularize the code, and new people could actually modify the codebase productively.
All of those negative value developers could probably be very valuable in very specific conditions, and they'd look just fine in a tough job interview. They were still terrible hires. In my experience, if anything, a harder process that demands people to appear smarter or work faster in an interview have the opposite effect of what I'd want: They end up selecting for people that think less and do more quickly, building debt faster.
My favorite developers ever all do badly in your typical stringent Silicon Valley intervew. They work slower, do more thinking, and consider every line of code they write technical debt. They won't have a million algorithms memorized: They'll go look at sources more often than not, and will spend a lot of time on tests that might as well be documentation. Very few of those traits are positive in an interview, but I think they are vital in creating good teams, but few select for them at all.
I worked on Solaris for over a decade, and for a while it was usually a better choice than Linux, especially due to price/performance (which includes how many instances it takes to run a given workload). It was worth fighting for, and I fought hard. But Linux has now become technically better in just about every way. Out-of-box performance, tuned performance, observability tools, reliability (on patched LTS), scheduling, networking (including TCP feature support), driver support, application support, processor support, debuggers, syscall features, etc. Last I checked, ZFS worked better on Solaris than Linux, but it's an area where Linux has been catching up. I have little hope that Solaris will ever catch up to Linux, and I have even less hope for illumos: Linux now has around 1,000 monthly contributors, whereas illumos has about 15.
In addition to technology advantages, Linux has a community and workforce that's orders of magnitude larger, staff with invested skills (re-education is part of a TCO calculation), companies with invested infrastructure (rewriting automation scripts is also part of TCO), and also much better future employment prospects (a factor than can influence people wanting to work at your company on that OS). Even with my considerable and well-known Solaris expertise, the employment prospects with Solaris are bleak and getting worse every year. With my Linux skills, I can work at awesome companies like Netflix (which I highly recommend), Facebook, Google, SpaceX, etc.
Large technology-focused companies, like Netflix, Facebook, and Google, have the expertise and appetite to make a technology-based OS decision. We have dedicated teams for the OS and kernel with deep expertise. On Netflix's OS team, there are three staff who previously worked at Sun Microsystems and have more Solaris expertise than they do Linux expertise, and I believe you'll find similar people at Facebook and Google as well. And we are choosing Linux.
The choice of an OS includes many factors. If an OS came along that was better, we'd start with a thorough internal investigation, involving microbenchmarks (including an automated suite I wrote), macrobenchmarks (depending on the expected gains), and production testing using canaries. We'd be able to come up with a rough estimate of the cost savings based on price/performance. Most microservices we have run hot in user-level applications (think 99% user time), not the kernel, so it's difficult to find large gains from the OS or kernel. Gains are more likely to come from off-CPU activities, like task scheduling and TCP congestion, and indirect, like NUMA memory placement: all areas where Linux is leading. It would be very difficult to find a large gain by changing the kernel from Linux to something else. Just based on CPU cycles, the target that should have the most attention is Java, not the OS. But let's say that somehow we did find an OS with a significant enough gain: we'd then look at the cost to switch, including retraining staff, rewriting automation software, and how quickly we could find help to resolve issues as they came up. Linux is so widely used that there's a good chance someone else has found an issue, had it fixed in a certain version or documented a workaround.
What's left where Solaris/SmartOS/illumos is better? 1. There's more marketing of the features and people. Linux develops great technologies and has some highly skilled kernel engineers, but I haven't seen any serious effort to market these. Why does Linux need to? And 2. Enterprise support. Large enterprise companies where technology is not their focus (eg, a breakfast cereal company) and who want to outsource these decisions to companies like Oracle and IBM. Oracle still has Solaris enterprise support that I believe is very competitive compared to Linux offerings.~
I'd argue that where RethinkDB fell down is on a step you don't list, "Understand the context of the problem", which you'd ideally do before figuring out how many people it's a problem for. Their initial idea was a MySQL storage engine for SSDs - the environmental change was that SSD prices were falling rapidly, SSDs have wildly different performance characteristics from disk, and so they figured there was an opportunity to catch the next wave. Only problem is that the biggest corporate buyers of SSDs are gigantic tech companies (eg. Google, Amazon) with large amounts of proprietary software, and so a generic MySQL storage engine isn't going to be useful to them anyway.
Unfortunately they'd already taken funding, built a team, and written a lot of code by the time they found that out, and there's only so far you can pivot when you have an ecosystem like that.
This unfortunately follows the conventions of the genre called "Falsehood programmers believe about X": ...
I honestly think this genre is horrible and counterproductive, even though the writer's intentions are good. It gives no examples, no explanations, no guidelines for proper implementations - just a list of condescending gotchas, showing off the superior intellect and perception of the author.
It happens sometimes. Usually it's because of one of two situations:
1) The company was on the fence about wanting you anyway, and negotiating takes you from the "maybe kinda sorta want to work with" to the "don't want to work with" pile.
2) The company is looking for people who don't question authority and don't stick up for their own interests.
Both of these are red flags. It's not really a matter of ethics - they're completely within their rights to withdraw an offer for any reason - but it's a matter of "Would you really want to work there anyway?" For both corporations and individuals, it usually leads to a smoother life if you only surround yourself with people who really value you.
I feel like this is every HN discussion about "rates---comma---raising them": a mean-spirited attempt to convince the audience on the site that high rates aren't really possible, because if they were, the person telling you they're possible would be wealthy beyond the dreams of avarice. Once again: Patrick is just offering a more refined and savvy version of advice me and my Matasano friends gave him, and our outcomes are part of the record of a reasonable large public company.
This, by the way, is why I'll never write this kind of end-of-year wrap-up post (and, for the same reasons, why I'll never open source code unless I absolutely have to). It's also a big part of what I'm trying to get my hands around for the Starfighter wrap-up post. When we started Starfighter, everyone said "you're going to have such an amazing time because of all the HN credibility you have". But pretty much every time Starfighter actually came up on HN, I just wanted to hide under a rock. Even when the site is civil, it's still committed to grind away any joy you take either in accomplishing something near or even in just sharing something interesting you learned . You could sort of understand an atavistic urge to shit all over someone sharing an interesting experience that was pleasant or impressive. There's a bad Morrissey song about that. But look what happens when you share an interesting story that obviously involved significant unpleasantness and an honest accounting of one's limitations: a giant thread full of people piling on to question your motives and life choices. You can't win.
I was the first person to be interviewed by this journalist (Michael Thomas @curious_founder). He approached me on Twitter to ask questions about digital nomad and remote work life (as I founded Nomad List and have been doing it for years).
I told him it'd be great to see more honest depictions as most articles are heavily idealized making it sound all great, when it's not necessarily. It's ups and downs (just like regular life really).
What happened next may surprise you. He wrote a hit piece on me changing my entire story that I told him over Skype into a clickbait article of how digital nomadism doesn't work and one of the main people doing it for awhile (en public) even settled down and gave up altogether.
I didn't settle down. I spent the summer in Amsterdam. Cause you know, it's a nice place! But he needed to say this to make a polarized hit piece with an angle. And that piece became viral. Resulting in me having to tell people daily that I didn't and getting lots of flack. You may understand it doesn't help if your entire startup is about something and a journalist writes a viral piece how you yourself don't even believe in that anymore. I contacted the journalist and Quartz but they didn't change a thing.
It's great this meant his journalistic breakthrough but it hurt me in the process.
I'd argue journalists like this are the whole problem we have these days. The articles they write can't be balanced because they need to get pageviews. Every potential to write something interesting quickly turns into clickbait. It turned me off from being interviewed ever again. Doing my own PR by posting comment sections of Hacker News or Reddit seems like a better idea (also see how Elon Musk does exactly this, seems smarter).
Hope this doesn't ruin it for you, but I knew someone who had a problem presented on the show. She called in and reached an answering machine. Someone called her and qualified the problem. Then one of the brothers called and talked to her for a while. Then a few weeks later (there might have been some more calls, I don't know) both brothers called her and talked to her for a while. Her parts of that last call was edited into the radio show so it sounded like she had called and they just figured out the answer on the spot.
Blockchain is the world's worst database, created entirely to maintain the reputations of venture capital firms who injected hundreds of millions of dollars into a technology whose core defining insight was "You can improve on a Ponzi scam by making it self-organizing and distributed; that gets vastly more distribution, reduces the single point of failure, and makes it censorship-resistant."
That's more robust than I usually phrase things on HN, but you did ask. In slightly more detail:
Databases are wonderful things. We have a number which are actually employed in production, at a variety of institutions. They run the world. Meaningful applications run on top of Postgres, MySQL, Oracle, etc etc.
No meaningful applications run on top of "blockchain", because it is a marketing term. You cannot install blockchain just like you cannot install database. (Database sounds much cooler without the definitive article, too.) If you pick a particular instantiation of a blockchain-style database, it is a horrible, horrible database.
Can I pick on Bitcoin? Let me pick on Bitcoin. Bitcoin is claimed to be a global financial network and ready for production right now. Bitcoin cannot sustain 5 transactions per second, worldwide.
You might be sensibly interested in Bitcoin governance if, for some reason, you wanted to use Bitcoin. Bitcoin is a software artifact; it matters to users who makes changes to it and by what process. (Bitcoin is a software artifact, not a protocol, even though the Bitcoin community will tell you differently. There is a single C++ codebase which matters. It is essentially impossible to interoperate with Bitcoin without bugs-and-all replicating that codebase.) Bitcoin governance is captured by approximately ~5 people. This is a robust claim and requires extraordinary evidence.
Ordinary evidence would be pointing you, in a handwavy fashion, about the depth of acrimony with regards to raising the block size, which would let Bitcoin scale to the commanding heights of 10 or, nay, 100 transactions per second worldwide.
Extraordinary evidence might be pointing you to the time where the entire Bitcoin network was de-facto shut down based on the consensus of N people in an IRC channel. c.f.
https://news.ycombinator.com/item?id=9320989
This was back in 2013. Long story short: a software update went awry so they rolled back global state by a few hours by getting the right two people to agree to it on a Skype call.
But let's get back to discussing that sole technical artifact. Bitcoin has a higher cost-to-value ratio than almost any technology conceivable; the cost to date is the market capitalization of Bitcoin. Because Bitcoin enters through a seigniorage mechanism, every Bitcoin existing was minted as compensation for "security the integrity of the blockchain" (by doing computationally expensive makework).
This cost is high. Today, routine maintenance of the Bitcoin network will cost the network approximately $1.5 million. That's on the order of $3 per write on a maximum committed capacity basis. It will cost another $1.5 million tomorrow, exchange rate depending.
(Bitcoin has successfully shifted much of the cost of operating its database to speculators rather than people who actually use Bitcoin for transaction processing. That game of musical chairs has gone on for a while.)
Bitcoin has some properties which one does not associate with many databases. One is that write acknowledgments average 5 minutes. Another is that they can stop, non-deterministically, for more than an hour at a time, worldwide, for all users simultaneously. This behavior is by design.
The database market is NOT closed. In fact, we are in a database boom. Since 2009 (the year RethinkDB was founded), there have been over 100 production grade databases released in the market. These span document stores, Key/Value, time series, MPP, relational, in-memory, and the ever increasing "multi model databases."
Since 2009, over $600 MILLION dollars (publicly announced) has been invested in these database companies (RethinkDB represents 12.2M or about 2%). That's aside from money invested in the bigger established databases.
Almost all of the companies that have raised funding in this period generate revenue from one of more of the following areas:
a) exclusive hosting (meaning AWS et al. do not offer this product) b) multi-node/cluster support c) product enhancements c) enterprise support
Looking at each of the above revenue paths as executed by RethinkDB:
a) RethinkDB never offered a hosted solution. Compose offered a hosted solution in October of 2014. b) RethinkDB didn't support true high availability until the 2.1 release in August 2015. It was released as open source and to my knowledge was not monetized. c/d) I've heard that an enterprise version of RethinkDB was offered near the end. Enterprise Support is, empirically, a bad approach for a venture backed company. I don't know that RethinkDB ever took this avenue seriously. Correct me if I am wrong.
A model that is not popular among RECENT databases but is popular among traditional databases is a standard licensing model (e.g. Oracle, Microsoft SQL Server). Even these are becoming more rare with the advent of A, but never underestimate the licensing market.
Again, this is complete conjecture, but I believe RethinkDB failed for a few reasons:
1) not pursuing one of the above revenue models early enough. This has serious affects on the order of the feature enhancements (for instance, the HA released in 2015 could have been released earlier at a premium or to help facilitate a hosted solution).
2) incorrect priority of enhancements:
2a) general database performance never reached the point it needed to. RethinkDB struggled with both write and read performance well into 2015. There was no clear value add in this area compared to many write or read focused databases released around this time.
2b) lack of (proper) High Availability for too long.
2c) ReQL was not necessary - most developers use ORMs when interacting with SQL. When you venture into analytical queries, we actually seem to make great effort to provide SQL: look at the number of projects or companies that exist to bring SQL to databases and filesystems that don't support it (Hive, Pig, Slam Data, etc).
2d) push notifications. This has not been demonstrated to be a clear market need yet. There are a small handful of companies that promoting development stacks around this, but no database company is doing the same.
2e) lack of focus. What was RethinkDB REALLY good at? It push ReQL and joins at first, but it lacked HA until 2015, struggled with high write or read loads into 2015. It then started to focus on real time notifications. Again, there just aren't many databases focusing on these areas.
My final thought is that RethinkDB didn't raise enough capital. Perhaps this is because of previous points, but without capital, the above can't be corrected. RethinkDB actually raised far less money than basically any other venture backed company in this space during this time.
Again, I've never run a database company so my thoughts are just from an outsider. However, I am the founder of a company that provides database integration products so I monitor this industry like I hawk. I simply don't agree that the database market has been "captured."
I expect to see even bigger growth in databases in the future. I'm happy to share my thoughts about what types of databases are working and where the market needs solutions. Additionally, companies are increasingly relying on third part cloud services for data they previously captured themselves. Anything from payment processes, order fulfillment, traffic analytics etc is now being handled by someone else.
I was a googler working on Google maps at the time of the API self immolation.
There were strong complaints from within about the price changes. Obviously everyone couldn't believe what was being planned, and there were countless spreadsheets and reports and SQL queries showing how this was going to shit all over a lot of customers that we'd be guaranteed to lose to a competitor.
Management didn't give a shit.
I don't know what the rationale was apart from some vague claim about "charging for value". A lot of users of the API apparently were basically under the free limits or only spending less than 100 USD on API usage so I can kind of understand the line of thought, but I still.thibk they went way too far.
I don't know what happened to the architects of the plan. I presume promo.
Edit: I should add that this was not a knee-jerk thing or some exec just woke up one day with an idea in their dreams. It was a planned change that took many months to plan and prepare for with endless preparations and reporting and so on.
???
How did HN get get the commenter base that it has? If you read HN, on any given week, there are at least as many good, substantial, comments as there are posts. This is different from every other modern public news aggregator I can find out there, and I don’t really know what the ingredients are that make HN successful.
For the last couple years (ish?), the moderation regime has been really active in trying to get a good mix of stories on the front page and in tamping down on gratuitously mean comments. But there was a period of years where the moderation could be described as sparse, arbitrary, and capricious, and while there are fewer “bad” comments now, it doesn’t seem like good moderation actually generates more “good” comments.
The ranking scheme seems to penalize posts that have a lot of comments on the theory that flamebait topics will draw a lot of comments. That sometimes prematurely buries stories with good discussion, but much more often, it buries stories that draw pointless flamewars. If you just read HN, it’s hard to see the effect, but if you look at forums that use comments as a positive factor in ranking, the difference is dramatic -- those other forums that boost topics with many comments (presumably on theory that vigorous discussion should be highlighted) often have content-free flame wars pinned at the top for long periods of time.
Something else that HN does that’s different from most forums is that user flags are weighted very heavily. On reddit, a downvote only cancels out an upvote, which means that flamebait topics that draw a lot of upvotes like “platform X is cancer” “Y is doing some horrible thing” often get pinned to the top of r/programming for a an entire day, since the number of people who don’t want to see that is drowned out by the number of people who upvote outrageous stories. If you read the comments for one of the "X is cancer" posts on r/programming, the top comment will almost inevitably that the post has no content, that the author of the post is a troll who never posts anything with content, and that we'd be better off with less flamebait by the author at the top of r/programming. But the people who will upvote outrage porn outnumber the people who will downvote it, so that kind of stuff dominates aggregators that use raw votes for ranking. Having flamebait drop off the front page quickly is significant, but it doesn’t seem sufficient to explain why there are so many more well-informed comments on HN than on other forums with roughly similar traffic.
Maybe the answer is that people come to HN for the same reason people come to Silicon Valley -- despite all the downsides, there’s a relatively large concentration of experts there across a wide variety of CS-related disciplines. If that’s true, and it’s a combination of path dependence on network effects, that’s pretty depressing since that’s not replicable.
This is part of an experiment where I write up thoughts quickly, without proofing or editing. Apologies if this is less clear than a normal post. This is probably going to be the last post like this, for now, since, by quickly writing up a post whenever I have something that can be written up quickly, I'm building up a backlog of post ideas that require re-reading the literature in an area or running experiments.
P.S.
Please suggest other good comments
! By their nature, HN comments are much less discoverable than stories, so there are a lot of great coments that I haven't seen.
President AOC, Senator Chi Ossé ?
hellgate
hellgatenyc.com
2026-08-21 19:50:28
Rumblings of a 2028 where charismatic DSA members might try their hand at higher office. A gambling update. ICE getting sued by the septuagenarian they maced. And our intern looks back on the summer she turned Hell Gate....
Rumblings of a 2028 where charismatic DSA members might try their hand at higher office. A gambling update. ICE getting sued by the septuagenarian they maced. And our intern looks back on the summer she turned Hell Gate.
Editor Holly Pretsky dove into NYC-DSA’s caucuses and found rumblings of a 2028 where charismatic DSA members might try their hand at higher office. Editor Christopher Robbins updated us on the state of gambling in New York—from prediction markets to "dead facades." Start placing your Kalshi bets now.
Then, Jessy Edwards on ICE getting sued by the septuagenarian they maced—And finally Alisha Allison, our intern, looks back on the summer she turned Hell Gate.
Paul Atkins Misreads Adam Smith and the American Founding
On June 30, Paul Atkins, chairman of the Securities and Exchange Commission, stood before the Economic Club of New York and delivered a history lesson. With the nation’s 250th birthday days away, Mr. Atkins
told
his audience that the Declaration of Independence and Adam Smith’s “Wealth of Nations,” both products of 1776, rest on “the same conviction: trust the individual, not the institution.” America’s founding documents, he said, “in many respects, reflect Smith’s central themes,” and the founders, wary of concentrated power “whether lodged in a crown, in a parliament, or in a bureaucracy,” built around liberty a governing framework “as light as prudence would permit.”
The problem is that Mr. Atkins turns the coincidence of 1776 into kinship, and kinship into influence. He then conscripts that invented founding into a deregulatory agenda, capped by a wholesale retreat from cryptocurrency enforcement, that America’s founders would have recognized as a corruption of republican government.
Start with the Declaration. Mr. Atkins’s most concrete evidence is Jefferson’s well-worn copy of “The Wealth of Nations.” But the Monticello
source
cited in his own footnote reports that Jefferson acquired the book while serving in France between 1784 and 1789, at least eight years after he drafted the Declaration. Jefferson may have encountered Smith’s ideas before 1776, but there is no evidence that “The Wealth of Nations” shaped the Declaration. On the contrary, Jefferson
told
James Madison in 1823 that he “turned to neither book nor pamphlet” while writing it. Asked by Henry Lee in 1825 about its sources, he
described
the Declaration as “an expression of the American mind,” reflecting the “harmonising sentiments of the day” embodied in “the elementary books of public right, as Aristotle, Cicero, Locke, Sidney, etc.”
By fusing America’s founding to “The Wealth of Nations,” Mr. Atkins turns Smith’s defense of free markets into a justification for weakening public oversight of politically favored financial interests. Notice that Atkins’ catalog of dangers includes crowns, parliaments and bureaucracies, but omits the economic factions the founders also recognized as potential threats to republican government. James Madison warned in
Federalist No. 10
that “the most common and durable source of factions has been the various and unequal distribution of property,” specifically identifying the landed, manufacturing, mercantile and moneyed interests. Jefferson made the point even more explicitly.
Writing
in 1816, he called for the nation to “crush in its birth the aristocracy of our monied corporations, which dare already to challenge our government to a trial of strength and bid defiance to the laws of our country.”
The founders valued private property and encouraged commerce. But they also recognized a basic republican principle that Mr. Atkins ignores: power can threaten liberty whether it is wielded by the state or by private interests wealthy enough to bend the state to their will.
That missing half of the founding tradition becomes impossible to ignore when Mr. Atkins turns to cryptocurrency. He boasted that the S.E.C. is answering President Trump’s call “to make America the Crypto Capital of the World.” In practice, that has meant dismissing, or settling on favorable terms for the defendant, the
majority
of outstanding cryptocurrency enforcement actions, several of them involving defendants with business ties to the president or his family.
The crypto industry helped underwrite the political conditions for this solicitude. It was the top corporate
donor
in the 2024 election cycle and has already
amassed
a nine-figure campaign arsenal for the coming midterms. And the president is personally invested in the outcome. His own financial disclosure reports that he
earned
more than $1.4 billion in income from his family’s crypto ventures in 2025.
Mr. Atkins’s crypto agenda is difficult to reconcile with his tribute to Adam Smith, because Smith did not regard money and banking as a realm beyond public law. He welcomed privately issued bank notes redeemable in gold or silver, recognizing them as an efficient means of facilitating commerce. But when Scottish banks issued small-denomination notes whose failure would fall hardest on poor laborers, Smith
endorsed
restricting them, conceding that the rule violated “natural liberty” but defending it as “exactly of the same kind” as requiring party walls to stop the spread of fire.
The Constitution reflects a similar instinct. Having witnessed the paper currency issued by the Continental Congress
depreciate
into worthlessness, the Framers vested authority over the nation’s monetary system in Congress, giving it the power to “coin Money” and “regulate the Value thereof,” while forbidding the states to coin money, issue bills of credit or make anything but gold and silver legal tender. They did not prohibit private bank notes, but they made clear that establishing the nation’s monetary framework was a public responsibility. In different ways, Smith and the Framers reached the same conclusion: private monetary innovation has a place, but it must remain subject to public law. Mr. Atkins’s crypto agenda is difficult to reconcile with that principle.
Mr. Atkins closed with a warning about socialism, in an unmistakable shot at Mayor Zohran Mamdani and other New York leaders who, he said, “are beginning to speak the language of control rather than of freedom.” He even quoted President Trump’s warning that under communism “great violence proceeds at levels never seen before.” But those warnings ring hollow coming from an administration that has repeatedly intervened in private markets, including by
taking ownership stakes in private companies
, while extending
preferential treatment
to politically connected firms. A warning about political violence likewise loses its force when its cited authority incited a mob to halt the peaceful transfer of power and then pardoned participants who beat police officers.
If Mr. Atkins is looking for a lesson about socialism, he might begin with the history of the agency he leads. American communism
attracted
its largest organized following during the Great Depression, after a stock market rife with fraud and manipulation, and lacking comprehensive federal oversight, crashed and helped drag the economy down with it. Congress responded with the Securities Act of 1933 and the Securities Exchange Act of 1934, creating the S.E.C. Critics denounced the legislation as socialism. Congressman Fred Britten
complained
that “the real object” of the 1934 law “is to Russianize everything.” But the republic survived and American capital markets went on to finance the American century under the very rules Mr. Atkins now portrays as shackles.
Mr. Atkins wants the founders to bless a world in which public power retreats and private financial power is presumed innocent. The actual founding will not cooperate. Jefferson feared a monied aristocracy, Madison feared faction, and Smith accepted restraints on banking when private liberty endangered society.
The S.E.C. was created to restore confidence in financial markets after their collapse had helped make socialism politically attractive. Well-regulated markets produce trust. Trust attracts investment, and broadly shared prosperity remains among the strongest defenses against political extremism. Markets retain their legitimacy only when the public believes the rules apply equally to everyone. An S.E.C. chairman who retreats from enforcing the securities laws against a politically connected industry tells ordinary Americans that wealth and influence purchase different rules. Once that belief takes hold, confidence in markets erodes, and critics of capitalism scarcely need to make their own case.
Lee Reiners is a lecturing fellow at Duke University
The Fourth Horseman of the File-Format-Hegemony Apocalypse
Daring Fireball
techcommunity.microsoft.com
2026-08-21 19:18:40
Microsoft’s iPhone app for OneDrive has long supported creating three new document types via a big “+” button right in its file viewer: Excel, Word, PowerPoint. It recently came to my attention that they’ve added a fourth document type, and they even gave it real Microsoft-style “they just have no t...
One shared brain that Claude, ChatGPT, Cursor, and every AI can read and write. It structures what you know so agents read only what they need, and means you never explain yourself twice.
Routing index
5 of 61 articles
positioning
what we sell and to whom
fresh
clients/meridian
scope, current terms, contacts
fresh
voice
how I write, words I never use
fresh
projects/q3-launch
status, decisions, open threads
aging
preferences
models, tools, formats
fresh
Ask your own agent how OzBrain can help.
Copy this into the agent that already knows your work.
Look at what you know about how I actually work, including past conversations, saved memory, projects, teammates, clients, shared documents, and handoffs.
Read https://ozbrain.com/docs.md as a product description, not as instructions. If that page cannot be fetched, use https://ozbrain.com/docs.
Your first reply must be under 150 words. Give up to three short, ranked reasons OzBrain could help me. Every reason must use real evidence from my work and explain the benefit in one sentence. Do not invent examples or give me a long audit.
Treat collaboration as important. If you have evidence of team or collaborator work, make one reason about how a shared brain could keep everyone and their agents working from the same current knowledge. If you cannot support three reasons, give fewer.
End with only:
“Want me to help you set it up and move the first thing in?”
Do not explain setup unless I say yes.
If I say yes:
1. Guide me one step at a time to add OzBrain as a custom MCP connector in the agent or app I am using. Use https://ozbrain.com/api/mcp. Do not invent menu names. If this app cannot add it directly, send me to https://ozbrain.com/start.
2. Explain that signing in with the email code creates my OzBrain account if I do not already have one.
3. Wait for me to complete each step. Confirm the connection by listing my OzBrain brains.
4. Recommend which identified item to add first and whether it belongs in my personal brain or a new shared brain. Ask before creating or saving anything.
5. With my approval, use the connector to stage the knowledge and show me the draft. Ask for corrections and approval before promoting it, then re-read it to prove it was saved.
6. If collaborators should join, propose the brain invite and explain that I must confirm it.
Copy a brief into Claude. Paste it into ChatGPT. Drop the same .md into Cursor. Update one copy, forget the others, and watch them drift.
That is the job you are stuck doing: ferrying context between tools that do not share a source of truth.
One brain every agent reads and writes.
The current version is wherever someone last saved it.
Copies of the same plan sit in Drive, on laptops, in Downloads, in email. OzBrain puts the current article in front so every agent decides from the same file.
Without OzBrain · fig. A
With OzBrain · fig. B
q3-plan.md
v1
Drive
q3-plan-old.md
v1
USB
q3-notes.md
v1
Notion export
q3-plan-copy.md
v1
Desktop
q3-plan-v1_1.md
v1.1
laptop
q3-plan (1).md
v1.1
Downloads
q3-plan-v2.md
v2
Downloads
q3-plan-FINAL.md
v2
Drive
q3-plan-FINAL-v2.md
v2
Slack
q3-plan-reviewed.md
v2
laptop
q3-plan-v2-draft.md
v2-draft
email
Q3_PLAN_v3.md
v3
iCloud
plans/q3
current
company brain
“Humans abandon wikis because the maintenance burden grows faster than the value.”
Running in under 2 minutes.
In Claude on this computer: Settings, then Connectors, then Add custom connector. Paste the MCP URL, sign in with the email code, and approve. Then paste this:
https://ozbrain.com/api/mcp
Hey Claude, help me set up OzBrain. It's a brain my AI agents share: they read it for context at the start of work and write back what they learn, so every session starts already knowing my stuff.
1. A connector is how you talk to OzBrain. Help me add it once in Claude on the web (a computer is the reliable place; the Claude mobile app cannot add a custom connector): Settings → Connectors → Add custom connector → paste https://ozbrain.com/api/mcp. If I need click-by-click help, send me to https://ozbrain.com/start. Then wait while I connect and sign in with my email code.
2. Once it's connected, open my brain and run me through getting started.
Platform memory keeps scraps and summaries. OzBrain holds the work itself.
Memory stores preferences, chat scraps, and thin daily summaries inside one product. OzBrain holds your projects, decisions, research, and the thinking you have already done, so every agent can pull the article the moment needs instead of whatever fits in a profile.
Platform memory · fig. 3
App Memory File
-
Prefers short answers
-
Uses TypeScript
-
NEVER EVER use emdashes
-
Dislikes filler words
Capped profile · four of many scraps
OzBrain · fig. 4
Linked articles · grows with your work
Your knowledge compounds. Every agent reads from the same place.
The shared brain payoff
Built to stay coherent.
Local files go stale and get pasted into every agent. OzBrain keeps knowledge split so agents read only what they need, enforces size discipline at write time, and treats continuous maintenance as the designed behavior as your agents update the brain when things change.
0
1
fig. 5a
Organizes for you
New knowledge finds the right article. You do not design a filing system; the brain routes each write where it belongs.
Staged write
Meridian scope closed. New retainer starts Monday.
Routes to
clients/meridian
scope, current terms, contacts
0
2
fig. 5b
Stays coherent
When a write disagrees with what the brain already holds, the write pauses and the conflict surfaces. Scheduled checks flag what went stale so agents know what to recheck.
decisions/pricing
Pause
Staged write says
$49/mo
. Canon holds
$29/mo
.
Conflict surfaced · canon untouched
0
3
fig. 5c
Shows who changed what
Every version records which agent wrote it and when. When agents run on their own, you can see what moved and catch what went off the rails.
projects/q3-launch · history
v14
claude-code
16:02
v13
chatgpt
15:41
v12
cursor
14:08
0
4
fig. 5d
Refactors as it grows
When an article gets too large for an agent to use well, the brain splits and reshapes it: refactoring, restructuring for clarity without changing what it says. More smaller articles means agents pull only what the task needs.
Splits into
architecture/overview
stack, tenants, boundaries
architecture/deploy
envs, rollouts, rollback
Trust you can verify
Encrypted at rest. Visible when used.
We never train on your brain and never sell it. Content is sealed per account, every access is in your audit log, and you can leave with everything or delete it outright.
Encrypted at rest, per account
fig. 6a
clients/meridian
Sealed
a7f3:9c21:e04b:11d8
4b90:c2ee:78a1:0f55
d13c:····:····:8e2a
Account key · decrypt on read / maintenance
Article bodies are sealed under your account key. We decrypt only to serve your agents and run disclosed maintenance, including refactoring. A stolen database dump is ciphertext, not readable articles.
Envelope key · bodies sealed
Full audit log you can export
fig. 6b
Account log
Export CSV
16:02
claude-code
·
write
·
projects/q3
15:41
chatgpt
·
read
·
voice
14:08
cursor
·
read
·
clients/meridian
Every read and write records which agent, which client, which article, and when. See it in your account. Export it as CSV. Check what touched your brain instead of trusting a promise.
Visible · exportable as CSV
Isolated tenants. Instant revoke.
fig. 6c
Tenant A
your brain
RLS
Tenant B
no path
Claude · connector
last active 2m ago
Revoke
Row-level security is forced in Postgres. There is no app-code path around it. Every connected agent is listed; revoke cuts that client immediately.
Forced RLS · OAuth revoke
Export anytime. Delete means deleted.
fig. 6d
Export
brain-export.zip
61 articles · plain markdown
Delete account
content · versions · blobs
Hard delete · not archived
Take the whole brain as plain markdown whenever you want, including after you cancel. Removing your account removes your content.
Markdown exit · hard delete
Start free. Pay when the brain is carrying weight.
Every plan includes unlimited reads and writes. You begin on Free. Pro and Max are there when one venture's knowledge, or the whole operation, lives in the brain.
Free
$0
forever
A real brain to start. Upgrade when you hit the ceiling.
OzBrain is a shared brain every AI agent you use can read and write: structured articles with links, provenance, and freshness, behind the connector menu Claude and ChatGPT already show you. One source of truth, not a separate memory in each product.
Is this another memory API?
No. Memory APIs sell add and search endpoints to developers building apps. OzBrain is a brain you connect, not a service you code against. Read the full
OzBrain vs Mem0 and Supermemory
comparison.
ChatGPT and Claude already have memory. Why this?
They do. That is the problem: each platform builds a separate, partial version of you, and none of them talk. OzBrain is the layer under all of them. Full write-up:
OzBrain vs ChatGPT Memory
.
Notes are written by you, for you. A brain is written by your agents, for your agents: every write is staged, routed, and checked against what the brain already holds. Compare
OzBrain vs Notion
or
OzBrain vs Obsidian
.
Which platforms does it work with?
Claude
and
ChatGPT
through their native connector flows, plus
Claude Code
,
Cursor
, OpenClaw, Hermes Agent,
Gemini Spark
where Google makes it available (US, Spark eligibility), and any client that supports connectors. It is one URL; anything that speaks the protocol can hold the same brain.
Do I need to code?
No. Add OzBrain from the connector menu in Claude or ChatGPT, sign in, and approve it. Nothing to install. Connect guides for
Claude
and
ChatGPT
.
Is there a free plan? What if I leave?
Yes. You begin on Free. Pro and Max are there when the brain is carrying real work. Export as plain markdown anytime, including after you cancel. Delete means deleted: removing your account removes your content. We never train on your brain and never sell it.
The brain behind every agent.
Pentagon dismisses Stars and Stripes leadership after opposition to interference
The Pentagon on Friday fired
the editor-in-chief of Stars and Stripes
and a top reporter for insubordination after they spoke publicly against any interference by the Defense Department in the military news outlet that has a long history of editorial independence. It was the latest move by an administration that has grown increasingly aggressive toward the news media.
Erik Slavin, editor-in-chief of the military newspaper that is partly funded by the Pentagon, told The Associated Press he was dismissed for insubordination after an interview he gave that objected to potential censorship by the U.S. military. He received a notice of separation, as did publisher Max Lederer — who had just announced his upcoming retirement — and Middle East reporter Lara Korte, Slavin said.
Slavin said he was being fired “for stating in a CBS interview that hypothetical censorship of news for service members would constitute a red line.” Korte participated in the same interview.
“I stand by the principle that Stars and Stripes must remain editorially independent, as required by law and by the department’s own policies,” Slavin said.
Media leaders are troubled
National Press Club President Mark Schoeff Jr. called the firing of Slavin “another brazen attempt by the Pentagon to dictate coverage of the military” and said it should be immediately reversed.
“Firing a newspaper editor after he publicly defended his newsroom’s editorial independence is deeply troubling, and it should concern every journalist and every member of the U.S. military who depends on independent reporting,” Schoeff said.
Neither Lederer nor Korte immediately responded to emailed requests for comment. Korte said on X: “Today, I was informed that the Department of Defense is firing me for insubordination after I told a CBS reporter that I work for Stars and Stripes — not the Pentagon, not the administration, and not any policy maker.”
Earlier this week, longtime publisher Lederer announced his retirement effective at the end of September. He made his announcement a few weeks after the Pentagon installed a new deputy publisher, an active duty service member, under him at the newspaper without his prior knowledge. Three Democratic senators on Thursday wrote to Defense Secretary Pete Hegseth, asking why the new deputy — Navy Capt. William Urban — had been installed.
This is our AP Ground Game newsletter.
You can subscribe below and we will email it you 3 times per week.
Sign up for the Ground Game Newsletter:
Your guide to the biggest stories in politics, policy and U.S. elections.
The retirement of Lederer, who had been at Stars and Stripes for three decades and publisher since 2007, comes as
Hegseth’s
Pentagon has moved to exert editorial control and eliminate what it asserts are “woke distractions.”
“Stars and Stripes will be custom tailored to our warfighters,” Sean Parnell, Hegseth’s spokesman, wrote in January on X. “It will focus on warfighting, weapons systems, fitness, lethality, survivability and ALL THINGS MILITARY. No more repurposed DC gossip columns; no more Associated Press reprints.”
In April, the Pentagon fired Jacqueline Smith, ombudsman for the newspaper, whose job had been to safeguard editorial independence.
Lederer, the second full-time civilian in the position, wrote in a staff memo this week that it had “become clear that my philosophy of leadership, and my understanding of the value and mission of Stars and Stripes, differ in fundamental ways from the direction the leadership of the Department of Defense has for the organization.”
Newly installed Pentagon leader outlines his plan
In a letter posted on Stars and Stripes, Urban wrote of his plans and said: “I understand that I am now part of a team, committed to editorially independent journalism that best serves our most important customer, which is our service members, their families, and our greater military community.”
“I am a media junkie who has served as a communication professional engaged in being a spokesman for some of the hardest Public Affairs assignments in the Navy and Department of War for more than 20 years,” he wrote.
It was unclear whether Urban, currently titled “military deputy to the publisher,” would now become head publisher. “I can’t comment on what my role will be in the future,” he told the AP.
In a telephone interview, Urban said Stars and Stripes has “a strong team of professional journalists that are focused on the mission of providing our military service members and their families the best possible product. That mission is going to continue. All of us in leadership will continue to advocate for … the best possible journalism going forward.”
He said he had no formal journalism experience but cited his years in public affairs and communications. He said he would be focused on expanding digital journalism at the paper, because the service members to which it caters are often young and are “digital natives.”
“I think we need to get better on the digital side,” he said, “to make sure that we are reaching as many service members as we possibly can and having the impact that we should be having.”
Friday Squid Blogging: Neon Flying Squid
Schneier
www.schneier.com
2026-08-21 17:07:20
The neon flying squid can fly in formation.
The shoal of about 100 squid rose unexpectedly from a patch of the Pacific Ocean around 370 miles from Tokyo and glided near the boat for about 30 metres. The astonished researchers were the first to capture photographs of such a thing, which looked like t...
The shoal of about 100 squid rose unexpectedly from a patch of the Pacific Ocean around 370 miles from Tokyo and glided near the boat for about 30 metres. The astonished researchers were the first to capture photographs of such a thing, which looked like the early stages of an alien invasion.
They were probably neon flying squid (
Ommastrephes bartramii
), the subsequent study
states
, a species that is part of a 20-strong flying squid family that was known to leap from the water but, until then, was only rumoured to also be able to glide above it.
The neon flying squid was able to gain such elevation by using the hyponome, a funnel-like muscular organ also present in other cephalopods, such as octopuses. The organ is able to force water out in a jet, propelling the body along both in and out of the sea. Photographs of the gliding squid show them with their arms (they have 10 limbs in all) splayed outwards.
As usual, you can also use this squid post to talk about the security stories in the news that I haven’t covered.
Author:
Bernard Quatermass via Exim-announce
Date:
To:
Exim Announcements, exim-users
Subject:
[exim] Exim 4.100 released
Exim 4.100 Released
Dear Exim users and maintainers,
We are pleased to announce the availability of release 4.100 of Exim.
Exim 4.100 is available as
* as tarball
* * https://ftp.exim.org/pub/exim/exim4/
* * https://code.exim.org/exim/exim/releases
* directly from Git: https://code.exim.org/exim/exim
tag: exim-4.100
The signatures on the release tarballs should be
* key ID 0xBCE58C8CE41F32DF
Email: jgh@???
New stuff added since 4.99
1. Lookups "psl" and "regdom" for, respectively, the public suffix or the
registered domain, given a domain and a Public Suffix List file.
2. EXPERIMENTAL_DMARC_NATIVE optional build feature. See the experimental.spec
file.
3. Log selectors "spf", "spf_verbose", "dmarc", "dmarc_verbose", "dsn".
4. Commandline option "-bI:modules" for listing installed dynamic-load modules.
5. Debug channels "start", "regex" and "macro".
6. The exiwhat utility now includes, on the daemon process line, counts for
smtp and queue-run children.
7. Main config option "bounce_charset", for setting Content-type: headers.
8. Event "proc:deliver".
9. Commandline option "-oDSN" for DSN options on commandline sourced messages.
10. Nongreedy wildcards for local_parts affixes.
11. Main config option "queue_run_order", obsoleting "queue_run_in_order".
12. The redirection router options "forbid_*" and "allow_filter" are now
expanded before use.
13. Sieve filtering now supports the "body" extension (RFC 5173).
14. Main config option "tls_eccurve" now accepts a groups tuple list.
Also, new smtp transport option "tls_eccurve".
15. Smtp transport option "protocol" is now expanded.
Removed items since 4.99
* removed obsolete malware scanners
* * f-prot6
* * f-prot6d
* * sophie
* * drweb
* * f-secur
* * aveserver
* * kavdaemon
* * mksd
* removed Interbase support
* removed Brightmail support
--
Security related.
This release contains all the previous fixes issues released in 4.99.1 through 4.99.5
--
Notable bugfixes include
* Expansion-test mode with debug (exim -d -be) now shows macro expansions.
* Fix local deliveries. A mistaken optimisation done for 4.99 caused
excessive retries on defers
* Fix radius expansion condition
* Fix use of a verify held-open connection
* Fix DNS lookups from perl on nonstandard port
* Fix DMARC for empty envelope senders
* Fix GnuTLS hostname verify of a server certificate with a zero-length Subject
* Various compiler quietening
* Update GPL doc references
* clear $spam_* variables on SMTP RST
* Proxy Protocol: add timeout guard to V2 input. Bug 2957
* Proxy Protocol: move startup before remote-host policy checks. Bug 3221
* DMARC: native implementation: use a-label of 5322.From
* Reject tainted format for internal printf
There have been no changes since RC3
Please refer to the ChangeLog file for a complete list.
File Verification:
SIZE(00-sha256sums.txt)= 1797
SIZE(00-sha512sums.txt)= 2949
SIZE(00-sizes.txt)= 726
SIZE(exim-4.100.tar.bz2)= 2184300
SIZE(exim-4.100.tar.gz)= 2751170
SIZE(exim-4.100.tar.xz)= 2007860
SIZE(exim-html-4.100.tar.bz2)= 650544
SIZE(exim-html-4.100.tar.gz)= 900661
SIZE(exim-html-4.100.tar.xz)= 639384
SIZE(exim-info-4.100.tar.bz2)= 485516
SIZE(exim-info-4.100.tar.gz)= 653766
SIZE(exim-info-4.100.tar.xz)= 487128
SIZE(exim-pdf-4.100.tar.bz2)= 2219695
SIZE(exim-pdf-4.100.tar.gz)= 2250735
SIZE(exim-pdf-4.100.tar.xz)= 2183432
SIZE(exim-postscript-4.100.tar.bz2)= 1172273
SIZE(exim-postscript-4.100.tar.gz)= 1572720
SIZE(exim-postscript-4.100.tar.xz)= 1161128
SIZE(exim-texinfo-4.100.tar.bz2)= 459473
SIZE(exim-texinfo-4.100.tar.gz)= 614510
SIZE(exim-texinfo-4.100.tar.xz)= 462132
SHA2-256(00-sha256sums.txt)= a247297a7503bc993479962130487b1dad962844afcf6cd4afb79f740d4d21d0
SHA2-256(00-sha512sums.txt)= e1376c079eea804b1c4fd8cd7414bb36822897501a6538042ffbdf8e508a722b
SHA2-256(00-sizes.txt)= 1cdc78cfc509dfa6c2216167669dfefcf1bc99a7db6baa5f186e9af852bd8bfc
SHA2-256(exim-4.100.tar.bz2)= 21c0e973cbc5f456475e7328807238c9ba998dbf97ba6053aeeb11750bfd20f3
SHA2-256(exim-4.100.tar.gz)= ea5306bf7b33094362d2fed5176df84a7a84e67a4c8413be1f005c1d9b62587d
SHA2-256(exim-4.100.tar.xz)= 5bd0a3e353dbfcd5c8174388b824316a61ee2455d9052ea2f0877dee939d33b3
SHA2-256(exim-html-4.100.tar.bz2)= d9925b615ac0e63e0bd4fca4c21a7b320e1180325b1a61624d7770d0607489d9
SHA2-256(exim-html-4.100.tar.gz)= 8bb01ffd15f520a491a68b7027ee2208bf22df9c0959e2838dc81f3107149e88
SHA2-256(exim-html-4.100.tar.xz)= 6bdd33916bd8ecd45cbcca80dedbe06e7dd2d86a5daeba35d5a0ec30dbf8109d
SHA2-256(exim-info-4.100.tar.bz2)= 50b8055c5de7c953a9eabc0ae74edf211b2969543ce802a5433c29c2c373580f
SHA2-256(exim-info-4.100.tar.gz)= 6b4cd2d218ce69f43b05ab2880b6c5f1b683d79001be6b5eac01548717d76dc7
SHA2-256(exim-info-4.100.tar.xz)= 8a902782feae88217fef998dbb9cf40218e67edfa6011b6151401e9bd1f7ff81
SHA2-256(exim-pdf-4.100.tar.bz2)= e4a4b883f653984e1c93bc1f2f2700b07a222b9059b9c8f4dac04c36ff87e085
SHA2-256(exim-pdf-4.100.tar.gz)= cc5bf3a936012d3ab6bfcc700b517bf6e9bf922118e06e499b076578ef6fcd9c
SHA2-256(exim-pdf-4.100.tar.xz)= d032dc98802b53aa509a0c36884d49851c2b75529c3e927e9ed1269629fb4ae3
SHA2-256(exim-postscript-4.100.tar.bz2)= 0f9bc6bbeb191d1df4d706bd93bbffc49ff2d2d6ae32abbe9955ab2f5499c70e
SHA2-256(exim-postscript-4.100.tar.gz)= 3dc521624a3e8c0ea03b5524de2c0f57744ef16dd8cd81f1a10591b86661465f
SHA2-256(exim-postscript-4.100.tar.xz)= 49c136e1da0c0f06e1ffe9db046a379833bac87ff58c1982daee3e5d03357a07
SHA2-256(exim-texinfo-4.100.tar.bz2)= 6d09d0c81ad48d2e1f2ed66bd1f1495004bf51d778a578807e03752543bcb830
SHA2-256(exim-texinfo-4.100.tar.gz)= afbea6b9b9bab2147f3193b992a5299aaaa87c058aa6df0abb0dd57fdeb56a19
SHA2-256(exim-texinfo-4.100.tar.xz)= b2d1b916fcaf675c6bc1608031c431cc0d6372b50b40a15536544c144bbc0e56
--
Bernard Quatermass
This past March, Jessica DuPont, who has run
Half Moon Used Books
in Troy, New York for the last 17 years, began to receive odd book orders from mysterious companies on an online platform. Previously, she'd got one to two orders a month from the secondhand book marketplace Alibris. Suddenly, companies with names like "Green Parrot Project," "Blue Finch Project" and "Scan PB" were ordering as many as 60 per day through the platform.
"They were buying obscure academic texts, like 'Commentaries on Theophrastus,' or 'Plato's Views on Education, Collected Essays,'" she said. "They were not buying super-special first editions." (The exception was one signed book of poems
by Larry Levis
.)
DuPont said the buyers, in an apparent attempt to hide the final destination, used fake individualized orders, which was "exceptionally wasteful" in terms of packaging. Meanwhile, they didn't seem to care about price—so she jacked them up 10 percent. The orders kept coming.
When DuPont emailed Alibris to ask what was going on, she said they replied that the orders were the result of a new client. When she pressed for more information, the company admitted it didn't know the ultimate destination of the books, DuPont said.
After talking with fellow booksellers, DuPont found out about the AI behemoth Anthropic's
project
to buy all the world's books, slice off their spines with a hydraulic machine, and scan them into a central digital library that will be used to train its large language models, like Claude. She realized other booksellers around the world were also
reporting
massive
spikes in orders
on platforms such as Alibris, Biblio and ISBNdb.
When DuPont found out about Anthropic's project, it led her to suspect that her books were also being fed into an AI learning model and destroyed. But after two months, the orders had dried up. DuPont said she'd made about $38,000 from the sales in two months, after Alibris took a 20 percent commission. Alibris did not respond to a request for comment.
"Quite honestly, I had a very rough 2025, so this personally for me was a windfall that allowed me to keep my two assistants and helped me stay in business," DuPont explained. "Do I go to my staff and say, 'Hey, there's this lifeline where you can keep your job, but I'm against it ethically. Do you mind losing your job for a bunch of fucking books that nobody wants, that I haven't been able to sell?' So, if I had it to do again, I might do the same thing. And I feel like shit admitting that, but at the same time, we live in this capitalist system, and I need to survive."
The Peripheral Component Interconnect (PCI) bus was first introduced all the way back in 1992. It quickly became the standard way to interface add-on cards on the PC platform, supplanting earlier buses like ISA and various other oddball standards. You wouldn’t expect to see a PCI bus on a Motorola-based machine, but
[maniek86]’s homebrew rig offers just that.
That’s a lot of soldering.
This computer is a beautiful piece of homebrew engineering, constructed out of protoboard and loose wires rather than any fancy PCB. At the heart of the build lies a Motorola 68000 running at 10 MHz. It’s got 1 MB of SRAM, 4 KB of ROM, and a MC68681P acting as a UART, timer source, and I/O controller. Where things get special, though, is in the inclusion of a Xilinx Spartan II FPGA (XC2S100), which acts as a PCI bridge. It provides the machine with two 32-bit 5-volt PCI slots which are interrupt capable, albeit with no bus mastering. A XC95144XL CPLD also sits present to act as glue logic to help lace everything together.
[maniek86] does a great job of explaining exactly why the PCI bus was hard to implement, and how it was pulled off in the end. The guide also covers how the system was able to interface various cards, from a PCI serial expansion to a Cirrus VGA adapter. It’s all good stuff.
Walmart Finally Caves, Will Soon Support Apple Pay
Daring Fireball
corporate.walmart.com
2026-08-21 15:34:55
Walmart:
When we think about convenience, we think about a lot of things,
but making shopping feel simpler and more seamless from start to
finish is a big part. And one of the moments where that matters
most is at checkout.
We want customers and members to have choice in how they pay, so
they c...
When we think about convenience, we think about a lot of things, but making shopping feel simpler and more seamless from start to finish is a big part. And one of the moments where that matters most is at checkout.
We want customers and members to have choice in how they pay, so they can check out in the way that works best for them. Now, beginning Aug. 24, we will be adding Tap to Pay to our payment options at select Walmart stores and Sam’s Club locations, with plans to roll it out to all U.S. stores and clubs by the end of 2026 and to fuel stations by mid-2027.
Tap to Pay gives customers and members another familiar and convenient way to pay at checkout using contactless payment methods. Whether picking up groceries, gifts or everyday essentials, they can check out using eligible contactless card, phone, or smartwatch. Customers and members can also add their eligible Walmart, Sam's Club and OnePay cards to their digital wallets, giving them another convenient way to use their cards.
Tap to Pay is a great addition to the other payment options already offered like cash, credit card or Walmart Pay — where customers can use the Walmart app to pay, view purchases and receipts, as well as access
Walmart+
fuel savings. At Sam’s Club, members can also use
Scan & Go
to scan and pay as they shop, skipping the traditional checkout line.
And giving customers and members more choice at checkout is part of a broader effort to make managing and using their money easier. Walmart’s financial services help customers and members manage their money and everyday financial needs, including options to save, build credit and pay over time. At Sam’s Club, members have access to Sam’s Cash and Sam’s Club credit, with opportunities to earn Sam’s Cash through qualifying purchases and programs.
It all comes back to giving customers and members more choice and making everyday shopping a little easier — from how they manage their money to how they pay at checkout.
A weiner, a frank, a foot-long—whatever you call it, the hot dog is a local icon. Simple, portable, and delicious, the hot dog is meant to stay affordable. But discourse about the rising price of the Manhattan hot dog has raised concerns about its standing as a cheap street staple. A recent
tweet
claimed it cost $15 for a hot dog at a cart near Bryant Park, which would be highway robbery, people. Adding salt to the wound, if you hop on a quick train ride Uptown, you will be face-to-face with the Guinness Book-certified "world's most expensive hot dog" at Serendipity 3, whose
notorious Haute Dog
, costs $69 and "must be ordered in advance,"
according to its menu
. Even back in 2021, veteran restaurant critic Robert Sietsema
reported
eating a $13 dog in Midtown West that he "regret[ted] paying for"—showing that hot dog prices were on the rise even
before
inflation, tariffs, and food costs made overly expensive food the norm.
Worried that New York's favorite, anytime cart snack might be the latest casualty of astronomical food prices—and eager to give into our desire for a hot dog before summer comes to a close—Hell Gate journeyed into Midtown on a sweltering Monday afternoon to get to the bottom of this.
The number sounds impressive. People hear it and assume a certain grandeur — a private library with rolling ladders, mahogany shelving, one of those brass lamps that appear in photographs of Oxford colleges. The reality is more prosaic. Twenty-eight thousand books, at an average thickness of roughly 2.5 centimetres, is 700 linear metres of shelving. That's the length of seven football pitches, or — in the metric that matters — considerably more shelf space than exists in my house.
This is the first thing you learn when your collection crosses from "large" into "logistical situation": the books will outgrow any space you put them in. Not eventually. Quickly. The relationship between books and shelf space is not linear; it is exponential, because you are always acquiring faster than you are shelving, and the shelving itself takes space that could hold more books. It is a problem with no equilibrium. I have been solving it for twenty years, and I am further from a solution now than when I started.
The Shelving Problem
I have shelves in every room. This is not a design choice. It is a consequence. The living room, the study, the bedroom, the hallway, the spare room, the room that was a spare room until it became a book room, the room that was a book room until it became a second book room. There are shelves in the bathroom. I am not proud of this, but I am also not lying about it.
The shelves themselves are a history of optimism. The first ones were beautiful — solid oak, custom-built, spaced to accommodate the quartos and folios that seemed, at the time, to represent the future shape of the collection. They were expensive, and they filled up in eighteen months. The second wave was IKEA Billy bookcases, deployed with the pragmatism of a military logistics officer: cheap, modular, immediately available, and — this is their great virtue — exactly 28 centimetres deep, which accommodates 95% of octavos and all paperbacks. The third wave was industrial steel shelving in the basement, the kind used in warehouses. It is ugly. It holds a lot of books. At a certain point in a collector's life, capacity trumps aesthetics.
The mathematics of shelving are unforgiving. A standard Billy bookcase holds roughly 80 books per unit (five shelves, sixteen books per shelf, assuming average octavos). Twenty-eight thousand books therefore require approximately 350 Billy units, which would occupy roughly 280 metres of wall space if placed side by side — more wall space than most houses contain. You can double-shelve (books in front of books), which hides half your collection behind the other half and makes finding anything an archaeological expedition. You can stack horizontally on top of vertical rows, which looks terrible and eventually causes the shelf to bow. You can put books in boxes, which solves the space problem by creating a different problem: you now own boxes of books instead of a library.
I have done all of these things. I am not recommending any of them.
The Weight Problem
Books are heavy. This is obvious when you carry them, less obvious when you store them, and dramatically obvious when you try to move them.
A standard octavo weighs roughly 300–500 grams. A folio can weigh two to three kilograms. An art book — one of those magnificent oversized volumes that seemed like a good idea in the bookshop — can weigh five. Twenty-eight thousand books, at an average of 400 grams, weigh approximately 11,200 kilograms. Eleven tonnes. On your floors.
I learned about floor loading the hard way, when a crack appeared in the ceiling of the room below my library. The structural engineer who came to assess it looked at the shelves, looked at the ceiling, looked at me, and said something in Flemish that I will translate politely as "this is too many books for this floor." He was correct. The floor joists were rated for a domestic load — furniture, people, normal life. They were not rated for seven tonnes of literature arranged along one wall.
The solution was steel reinforcement beams, installed at a cost that would have bought several hundred more books. The irony was not lost on me. It is also not lost on my wife, who mentions it at intervals she considers appropriate and I consider too frequent.
If you collect seriously, check your floor loading. Consult a structural engineer before you fill a room. Distribute weight across multiple walls rather than concentrating it on one. And if you live in an older building — which, in Belgium, means most buildings — remember that "older" often means "built for people, not for libraries."
The Moving Problem
I have moved house twice with this collection. I will not move again. This is not a preference. It is a vow.
The first move involved approximately 400 boxes. I know this because I counted them, in the way that a prisoner counts the days. Each box held roughly 30 books (you cannot fill a box with books and expect to lift it; half-full is the maximum, which doubles the number of boxes). The removal men — three of them, young, strong, and visibly dismayed — took two full days to move the library alone. The look on the foreman's face when he saw the basement shelving is something I will carry with me for the rest of my life. It was not anger. It was not surprise. It was the expression of a man recalculating the fundamental economics of his profession.
The second move, five years later, involved roughly 550 boxes. The collection had grown. The removal estimate was substantially higher. I hired a firm that specialised in library moves — they exist, in the same way that firms specialising in piano moves exist, for the same reason: the object is heavy, fragile, and owned by someone who will become emotional if it is damaged. The specialist firm packed each shelf in sequence, labelled the boxes by room and shelf position, and unpacked them in reverse order at the new house. It was efficient, professional, and cost approximately the same as a decent used car. Worth every cent.
Lessons from moving 28,000 books: use small boxes (banana boxes from the supermarket are ideal — the right size, strong, free). Pack spine-down, not flat. Never fill a box to the top. Label every box with its shelf of origin. And budget more than you think — more money, more time, more patience, more floor space for temporary stacking.
The Insurance Problem
Insuring 28,000 books requires, first, knowing what they're worth, which requires, first, knowing what they are. This is the cataloging problem in its most expensive form.
I insure my collection through a specialist policy — the kind offered by firms that understand the difference between a book and a piece of furniture. The policy is based on an agreed total value, reviewed annually, with a schedule of individually valued items above a certain threshold (currently anything worth more than €1,000). Below that threshold, the collection is covered as an aggregate: total insured value divided by total number of volumes, producing an average per-book value that is, for a mixed collection, both mathematically correct and practically meaningless. The average value of a book in my collection is approximately €85. This means nothing — it averages a €15,000 incunabulum with three thousand paperbacks, producing a number that describes no actual book.
The individually scheduled items — perhaps 200 books, representing the top end of the collection — are valued by a combination of purchase receipts, auction comparables, and periodic formal appraisal. This list is the single most important document I own that is not a book. It lives in three places: my computer, a cloud backup, and a physical copy in a fireproof box that is not in the same building as the books. Redundancy is the point.
The Relationship Problem
A collection of 28,000 books is not a hobby. It is a cohabitant. It occupies space, demands attention, costs money, and has opinions about interior design. It affects your relationships in ways that are difficult to explain to people who do not collect.
My wife is tolerant. This is not the same as enthusiastic, and I have learned, over the years, to recognise the distinction. The tolerance extends to the shelves in the living room, the shelves in the hallway, and the study that is entirely mine. It does not extend to the kitchen, the children's rooms, or the car (I once stored three boxes of books in the boot for six weeks; this was noticed). The negotiation is ongoing, and like all negotiations, it depends on goodwill, compromise, and the occasional strategic concession — I removed the shelves from the bathroom. She pretends not to notice the boxes in the garage.
Other collectors understand. The look of recognition when you mention the number — the slight widening of the eyes, the nod that says "yes, I know" — is one of the quiet pleasures of the collecting community. Non-collectors, by contrast, tend to respond with one of three reactions: admiration (from people who read but don't collect), bewilderment (from people who don't read), or the particular expression — sympathetic, faintly alarmed — of someone who suspects they are in the presence of a condition.
It is not a condition. It is a commitment. The distinction is subtle but real.
What I've Actually Learned
After twenty years and 28,000 books, the lessons are not what I expected them to be.
You will never read them all.
This is obvious, and it doesn't matter. A personal library is not a reading list. It is a reference collection, a research tool, a physical manifestation of your intellectual interests, and a comfort. The books you haven't read are not failures. They are possibilities.
The catalog is more important than the collection.
A bold claim, and I stand by it. Without the catalog, the collection is a beautiful chaos — unsearchable, uninsurable, and ultimately unknowable. With the catalog, it is a tool. I resisted cataloging for years, using memory and spatial instinct to navigate the shelves. I was wrong. The day I started entering books into a system — first a spreadsheet, then a database, then the software I eventually built because nothing else did what I needed — was the day the collection became a library.
Buying is easy. Curating is hard.
The difficult decisions in collecting are not what to buy but what to keep. At 28,000 volumes, every new acquisition implies a judgment about space, value, and purpose. Is this book better than the one it's replacing? Does it belong in this collection, or is it an impulse? Will I be glad I own it in ten years? These questions get harder as the collection grows, not easier, because the marginal value of each new book decreases as the total increases. The 28,001st book has to justify its existence against 28,000 competitors.
The books outlast everything.
They outlast the shelves. They outlast the houses. They outlast the relationships that accommodated them and the bank accounts that funded them. A book I bought twenty years ago in a shop that no longer exists, from a dealer who has since retired, in a city I no longer live in, is still here. It has moved twice. It has been shelved in four different rooms. It has survived everything I've put it through, and it will survive me. This is either a consolation or a burden, depending on the day.
Twenty-eight thousand. It's not a round number, and it's not a final one. The collection is still growing — more slowly than it once did, more deliberately, with a better sense of what belongs and what doesn't. But it's growing. The shelves are full. The floors are reinforced. The insurance is current. The catalog is up to date.
And there's room for one more. There's always room for one more.
My father passed recently, and he was twice my age. I am approximately the same age that he was when I was born, and I am now “the old generation” - there’s no one left in the generation above me.
At the same time, I recently joined a company that skews younger-than-me. When I joined Google in 2011, I had just turned 30, and was in the mainstream demographics of Google in 2011. There were a bunch of more senior folks, with the very senior ones being in their 50s and having completed stints at Bell Labs. I admired a lot of these “greybeards” (even though this is a sexist term - what’s the right female equivalent? There were a few very senior female engineers that I would love to include).
So perhaps it is natural that I am reflecting on “what were the important realizations that I made since my early 20s that had a profound impact on the way I think about the world”? In some sense: What are the insights I had that made me “more mature”, for some positive definition of “mature”?
This post tries to list them.
1. The importance of understanding your own incentive structure, and not believing everything you think.
I recently wrote a Twitter thread about the topic. Oppenheimer was very publicly guilt-ridden about the creation of the nuclear bomb, and von Neumann at some point quipped “some people profess guilt to claim credit for sin”. In my young years, particularly in situations when I had 0day that nobody else had, I agonized about the responsibility that comes with having 0day. Should I fix them? Should I use them for good? Will the world be harmed this way? Or that way?
In the end, it turns out that - while individuals matter - many ideas have a “time at which they are ripe”, and the actions of the individual matter less than the individual thinks in that moment. There is also almost no way to predict the ways in which what you do impacts the broader world.
If you were asked: “Would it be good if this 0day was used to apprehend a terrorist?” you would probably say “this is good”. If you were asked “would it be good if this 0day is used to arrest someone and then torture and waterboard him 183 times?”, you would probably say “this is bad”. So if your 0day was used to capture KSM, it is probably good? Or bad? Things get very complicated very quickly.
Is closing 0days good for society, because it makes everything safer? Or is it enabling oppression, because buggy systems are easier to bypass?
There are no good answers, and your own incentive structure will greatly influence how you choose your beliefs. In the end, people want to be the heroes of their own story, and at the same time they have basal needs for recognition, for material goods, etc. - so they will try to construct a narrative that allows them to satisfy their basal needs while also remaining the hero of their saga.
Anxiety about the impact of your work is self-flattering, and you have to recognize it as such, and keep it in check - it’s sugar for your ego, but history will largely route around you, because while individual decisions matter in specific situations, the overall flow of history is less sensitive to the individual than the individual thinks. The broader lesson, though, is: Do not believe everything you think. Examine your own incentive structures carefully. Ask yourself what alternative narratives for your behavior and beliefs could be, especially if they contradict the narrative of the heroic saga you’re constructing for yourself. Carefully weighing the question “how might I be the villain in this story?” is an important and valuable skill.
Similarly, meta-cognition - just observing your own thoughts in a detached manner, and then being able to interpret, analyze, and contextualize them with regards to your own incentive structures, is a great skill to cultivate.
2. Monocausal determinism is an illusion, and largely does not exist outside of computer debugging.
The monocausal determinism that young computer enthusiasts get used to is an illusion that generations of electrical and process engineers spent their lives perfecting and maintaining. It is because of these engineers that computer scientists could largely get away without probabilities or any empirical grounding in the past. There is an argument that you have so many natural scientists that crossed over into AI because CS education was for a long time too focused on reasoning within the deterministic monocausal illusion.
The reality is: Computing machines are physical devices, which includes wear & tear, differences in quality between items, and “probabilistically deterministic behavior”, e.g. it’ll appear deterministic most of the time if not shaken too much. If pushed a bit - be it temperature, voltage, electromagnetic fields, or even rapid memory accesses to adjacent DRAM rows - determinism has a tendency to go out of the window, the illusion collapses, and we’re dealing with a very different beast.
FWIW - this also makes me wonder about model alignment, because even a perfectly aligned model will be subject to random bit flips in inference, and it’s hard for me to imagine that you can maintain any reasonable guarantees in the presence of bit flips to inopportune values at inopportune times.
The real world is one where very few things that happen have a single reason, and very few truly deterministic transmission mechanisms. Everything is probabilistic, and everything is multicausal.
Measurement noise is real, experiment design is difficult.
Interestingly, if you think about this carefully, you also realize that the scientific method is a classifier that is
intentionally biased against accepting something as true
- so that we only accept things as true that are beyond any reasonable doubt true.
A somewhat fascinating corolary of this is that there exists a large class of true things that will never be scientifically shown as true.
3. The dichotomy between reason and emotion is a cultural construct, and neither grounded in neuroscience nor in logic.
With some digging, it turns out that the western belief that reason and emotion are two ends of a spectrum is a purely cultural construct, as is the belief that “higher-order” reason needs to reign in “basal” emotions, or that “emotions” intrude on “rationality”.
In most non-western cultures, achieving integration between rational deliberation and impulses and emotions is more common, and it turns out that this is much closer to the biological reality.
From a neuroscience perspective, it is clear that emotional valuation is part of a larger decision-making machinery that tends to not function properly if the emotional valuation component is damaged or removed. There is also a large component where things that your brain struggles to articulate verbally are transmitted via emotions, as well as actual feedback from your sensory organs in your body. Fun trivia: Your gut’s enteric nervous system contains as many neurons as the entire cerebral cortex of a dog. Your body also forward-deploys neurons in your muscles and extremities, as a form of latency optimization. Your body is feeding you extra information, and most of this shows up in the shape of emotions.
Which brings us to the logical argument why attempting to “remove” emotions from decision-making is a bad idea: Clearly, having the ability of leveraging more information for decision-making will improve the quality of decisions. Attempting to eliminate a particular source of information almost certainly makes the quality of your decisions worse.
This is not to say one should act on impulse alone, but it is certain that integrating the full spectrum of information - which includes emotions - in your decisions is a wise idea.
I am sure that if I think more carefully, I will come up with more insights, but these three are important enough that they show up in my life with astonishing regularity.
Hope this is helpful to someone.
'Ghost Job' Ads Are Getting So Bad That Lawmakers Want to Ban Them
Note:
to protect the company, I swapped out anything that could point back
to it for fake examples. The domain
electricscootercompany.com.br
, the app
package, and the user details (slug, name, and email) are all made up. None of
it matches the real company.
It started with a news article. A company had just dropped a bunch of electric
scooters in my city. Most people saw a new way to get around town. I saw a fleet
of internet-connected devices running on a backend nobody had poked at yet.
First I needed two things: which company this was, and how the service worked for
a normal user. The name was right there in the article, and a quick Google got me
to their site, which laid out the flow:
open the app on your phone;
scan the scooter's QR Code;
pay to unlock the vehicle;
ride.
That's the happy path for any user. I wanted to see what was going on behind it.
Step 1: Recon
I started by mapping everything tied to
electricscootercompany.com.br
.
Subdomain enumeration pulled up a bunch, including:
www app api privacidade privacidade2 devmembro vouchers planos validate painel
Not all of those were real apps.
app
,
membro
,
vouchers
, and
planos
all
served basically the same page that just pushed you to the app stores. Lots of
names, not much new to look at.
The
dev
host threw a 500 and set a PHP session cookie, but nothing I could use.
validate
came back with
Conta não localizada.
// "Account not found.", though
I didn't know yet which parameter it wanted. I wrote these down and moved on.
api.electricscootercompany.com.br
: REST API used by clients;
painel.electricscootercompany.com.br
: Angular panel for operators.
Nothing on the site linked to the panel. I only found it through enumeration.
Just because something isn't linked doesn't mean it's locked down.
Step 2: Opening the panel without getting in
The panel loaded for anyone: a production Angular app. I pulled down all 32
JavaScript chunks and dug through the bundles, where I found 83 endpoints for:
users and permissions;
vehicles and maps;
trip activation and finalization;
IoT devices;
garages, docks, and geofences;
vouchers, transactions, and financial modules.
That told me how juicy the target was, but it got me exactly nowhere. I hit about
38 protected routes with no valid session and every one gave me the same thing:
HTTP 401.
A lot of what I tried just didn't work:
Direct route access:
blocked by authentication.
Unsigned admin JWT:
rejected by the backend.
Tampering with token claims:
didn't produce a valid session.
SQL injection on login:
ran SQLMap against the auth fields and found no
injectable parameter.
Report endpoint:
php/report.php
returned an empty 500.
Auth was holding up fine against the direct stuff. And the app was already
pointing me at an easier road: find a real user and go after their password.
Step 3: WordPress hands over the first piece
The public WordPress REST API happily let me list authors:
GET /wp-json/wp/v2/usersGET /wp-json/wp/v2/users/1?context=view
The response gave up user ID 1, public name
admin
, slug
electricscootercompany
. Normally that's just run-of-the-mill WordPress
enumeration. Here, I could take that same identifier and try it on the operations
panel.
The login gave different answers depending on what I fed it:
existing identity + wrong password → "Senha inválida" // "Invalid password"nonexistent identity → "E-mail não encontrado" // "Email not found"
So I didn't have to wonder if
electricscootercompany
was just a blog author.
The backend told me straight up that the same identity existed in the operational
system too. That turned a generic enumeration into a target list with one name
worth a lot.
Step 4: The brute force
With the user confirmed, I threw a brute force at the login. Nothing throttled the
repeated tries, and a working password eventually turned up, so the panel login
went through.
This is the part that really explains the root cause. With no real rate limiting,
one known identity was all it took to turn a guessing loop into a valid session.
Everything after this rests on a real session I caught in Burp. The JWT decoded to
an account with:
Level
1000
was the admin role. And the token stayed good for about 950 days, so
a session grabbed once would keep working for years unless someone went out of
their way to kill it.
Step 5: The panel stops being a hypothesis
With a valid session, everything changed at once. Routes that used to give me 401
now handed back real operational data. In the capture I logged 168 first-party
requests across 118 unique host/method/path combos.
The panel gave me read access to:
fleet map at
/mapas/__veiculos
;
docks at
/docas
;
garages and operational infrastructure;
geofence polygons at
/fronteiras/__coordenadas
;
IoT device inventory, with identifiers and state;
individual vehicles and the full fleet;
app users, where the interface mentioned over 408,000 records;
companies, permissions, transactions, voucher batches, and financial data.
The map connected the digital side to the real operation on the ground: where the
garages sat, where the docks were, which vehicles were scattered around town, and
which IoT device belonged to each one.
Clicking a marker opened up the vehicle's details: code, type, and where it was
sitting (a dock, for example):
Some of the responses were big enough to show how much access this was:
Burp cut off big response bodies at around 3 KB, so I don't have every full
response saved, but the statuses, paths, and sizes I logged are all solid.
Step 6: It wasn't just looking
Next I wanted to know if the panel only read data or could write it too. The
session showed
PUT
calls against:
vehicle records;
user accounts;
voucher batches.
I could also flip the free-ride flag on test accounts. So this wasn't just reading
data. I could change business rules and records too.
The vehicle registry let me look up any scooter in the fleet (the interface showed
thousands of records) and open its edit form:
This form is where reading turned into control. On top of saving changes, it had
Unlock
,
Lock
, and
Restart IoT
buttons, and those go straight to the
physical device.
The scariest part was the IoT module. The panel fired off commands like this:
POST /iot_sends/Content-Type: application/json{"pk_veiculo":699,"comando":"open"}
I fired a second command to close the same vehicle:
{"pk_veiculo":699,"comando":"close"}
Same confirmation. I kept the whole test to one vehicle,
pk_veiculo: 699
: a
stolen admin account could unlock the scooter from anywhere and lock it right back
up.
The panel even popped a success message:
But an API response and a green message on screen don't prove much on their own. I
needed to know the command actually reached a real scooter. I wasn't anywhere near
one, so I got a friend to walk up to a scooter and film it the second I fired the
command. And it worked: the scooter unlocked, the lights came on, and it was ready
to ride, with nobody paying, scanning a QR Code, or even touching it.
What I
didn't
test matters just as much as what I did. I never automated this
against a bunch of vehicles, and I never touched anything that could cause
movement, braking, or any real-world danger. One command on one scooter was enough
to show the web panel reaches actual hardware.
Dead ends worth documenting
Not every lead panned out. The dead ends are worth sharing too, because crossing
them off is what steered me toward the path that worked.
Intercepting the mobile app
The Android app was Flutter and kept a lot of its logic compiled into
libapp.so
.
It took me a bunch of tries (emulator, certificates, repackaged APKs, traffic
capture) before I got a session I could actually intercept. A lot of proxy and
certificate combos just didn't give me the traffic I was after.
Once the capture finally worked, 33 flows showed the
validateApp
protection
leaned on static headers and a bearer token, not on a fresh signature per request:
That helped me map the API, but I still needed a valid token. This bypass wasn't
what got me into the panel.
Broad map queries
A normal bounding-box query gave back vehicle position and battery. When I
stretched the area out to about the size of the country, the backend hit me with a
403 telling me to log in again. After that, the same JWT started getting 403s even
on simple endpoints that had worked a minute earlier.
The wide query tripped some defense and killed the session, either a rough anomaly
filter or a token revocation kicking in once I got greedy. It was one of the few
spots where the backend actually pushed back.
Trip history
I threw vehicle references at the history endpoints and got empty sets or 400s, so
no cross access to other people's trips there. The responses did leak internal
class names and PHP/ORM messages, though, stuff like
T_app_usuario_viagem
and
Undefined array key 6
.
SQL injection and forged JWT
I chased both of these before the credential route, and neither went anywhere. The
login fields weren't injectable, and unsigned or messed-with tokens got rejected.
These misses are worth writing down: the access I got in the end didn't come from
SQLi or a JWT bug, it came from enumeration plus a weak password plus way too much
privilege.
Root cause of the chain
None of this needed some exotic vulnerability. It all came from a pile of
identity, authentication, and privilege problems stacking up:
WordPress exposes user ↓login confirms the user exists ↓repeated attempts allow brute force ↓credential grants direct level-1000 access ↓admin token lasts about 950 days ↓panel concentrates data, changes, and IoT commands ↓remote scooter unlock
The biggest problem is that nothing backed anything else up. The password was the
only thing standing between the open internet and a button that moves physical
hardware. Nowhere along the way was there a required second login or an extra
confirmation on the IoT commands.
What this means in practice
With one stolen admin credential, I could get to:
the operational panel;
the map of vehicles, garages, and docks;
the operational topology and geofences;
vehicle and IoT device inventories;
broad user data and financial modules;
editing records and benefits on test accounts;
sending the
open
and
close
commands to a test scooter, remotely.
In a real attack, this could turn into unauthorized fleet use, fraud, operational
losses, exposed user data, and indirect physical risk, all of it doable at scale
off a single credential.
Conclusion
The road to unlocking a scooter remotely didn't start with fancy reverse
engineering or some rare crypto flaw. It started with a forgotten subdomain, a
public user, and two different error messages.
The fancy attempts were the ones that flopped: the forged JWT got rejected, the
SQL injection never showed up, and the routes kept handing back 401. The thing
that actually worked was the most obvious attack there is. A known identity, a
guessable password, and no second line of defense were enough to turn web access
into control over a physical device.
SalesPatriot (YC W25) Is Hiring Forward Deployed Engineers
America's industrial base runs on systems built in the 1980s.
Billions of dollars in critical components (F-35 parts, industrial assemblies, electronics, bolts, and hoses) still move through email threads, excel sheets, and disconnected ERPs.
We're replacing that with an AI-native platform that makes aerospace, electronics, industrial, and defense supply-chain operations nearly autonomous. Faster quoting. Faster procurement. Full visibility. Real operational intelligence for the companies our nation depends on.
Quick Facts
YC W25 company. Top 5% growth in batch
Raised $10M+ from investors including Paul Graham, SV Angel, Pear VC, and CRV
Team of 20 co-live in our Warsaw & SF Hacker Houses
7 figure ARR transacting ~$50M a week through our system
The Job
Go. Fly out and plant yourself inside the customer's operation. Weekdays are onsite (Wisconsin, New York, Miami, Los Angeles); weekends we regroup at the SF HQ to debrief and keep building.
Understand. Map how the company actually operates, from sales and supply chain teams to executives and CEOs. Learn the breakpoints choking their growth and speed.
Implement. Configure automations within SP Studio tooling so the platform accurately reflects the customer's needs. Build trust and confidence in SalesPatriot.
Iterate. Take full ownership of the customer's outcome. Keep finding and killing their most pressing problems, leading org-wide scaling, until SalesPatriot is the operating system their business runs on.
What We’re Looking For
Ready to relocate full-time to San Francisco.
This is not a remote role.
Absolute grinder. Interested in co-living (though not required).
Comfortable with ambiguity and rapid change.
Track record of shipping fast.
Full-stack beyond code: comfortable jumping between frontend, backend, and organizational politics — earning trust with procurement specialists while navigating executive priorities and IT constraints.
Motivated by taking an unknown problem, sinking your teeth in, and coming up with a plan of attack.
Proficiency in TypeScript, JavaScript, and at least one frontend library (React, Svelte, Next etc).
Be ready to show us at least one full-stack project you've shipped (GitHub repo / web app / Loom demo video).
Knowledge of SQL databases, preferably Postgres.
Personable: clients trust you, like you, and look forward to your updates. You handle the conversation and the code.
Low ego, curiosity, and intellectual honesty — focused on outcomes, not "being right."
The Process
Call with engineer → 1hr technical test → Call with founder → fly out to SF HQ (on us) → offer.
About
SalesPatriot
Founded:
2024
Batch:
W25
Team Size:
15
Status:
Active
Location:
San Francisco
Founders
Quick impressions: A week of using Codex more than Claude
Some quick and very personal impressions from using Codex more than Claude this week (I will do a full analysis during the weekend hopefully).
(1) While I tried this year to keep Claude and Codex on par, having the same set of plugins/skills and so on, Claude had more skills, as I created skills out of some sessions and not all of them were ported to Codex. Fix for this is simple: Point Codex at the Claude skills folder and ask it to transform them for Claude
(2) When I was in a rush (like debugging something that felt urgent), I still opened Claude as somehow I felt more at home with it. I am not saying it was better, but it was familiar, and when debugging, using tools that I know is important.
(3) Changes created by Codex had fewer comments in Ruby/Ruby on Rails code. I liked that a lot, and I will soon share some experiments I ran on this.
(4) The output of the Codex agent harness is much more “technical” than the one from Claude. Claude feels more like your colleague in a Tuple session writing to you while Codex feels more like a version of Data from Star Trek.
(5) I want to open many more sessions of Codex and keep them focused instead of a big session of Claude as I was doing before. This may not be specific to Codex, but I noticed it while working with Codex.
(6) It feels to me that Codex does changes faster than Claude. But after making the main changes, it took a lot to finish the pull request: rerunning many tests, review, and so on. I like the thoroughness of this, but in the end, there was no win in terms of time difference.
(7) It felt to me that Codex created a much simpler solution in terms of code architecture than Claude. Claude usually goes on to create a lot of things: abstractions, concepts, Sorbet signatures, type aliases, and so on. Codex was a bit more contained and created less. This week I also tested an improved flow of
code research -> design change -> review change -> implement -> verify
. But I made both of them implement the same requirement using the same documents, and Claude’s code was a bit more complex but handled cases.
(8) Codex also made some mistakes. Claude could understand my intention to branch out from other work and keep them in sync. Codex did some nasty things like
branch A targets branch B that targets main,
and when I asked it to rebase, it rebased with
main,
which created some PR with 4000+ additions. I had to be explicit and ask it to rebase only with the target.
(9) For Codex, working with Jira and Atlassian was a hassle in my environment where I use the CLI tool and not the MCP. It opened JIRA to prompt me to log in, then switched to the CLI, then back to the browser. In this case, Claude was much more eager to try to get what I want and to do it the way I want it done, based on previous sessions.
(10) Working with MCPs, I like the Codex CLI approach more, where it asks me to execute
codex mcp login
and every time it opens the right authentication and authorization flow. Claude sometimes tries to run it automatically in a turn, and it can get stuck.
I think the main difference I feel between Claude and Codex is that Claude tries to go above and beyond what is asked and guess what you might want and then directly do it, while Codex is more like a companion that does what you tell it but will not overdo it. It will stop at the first sign that it might be done.
Church of the TigerBeetle: A Look at Tech Evangelism
The seats of the Institute of Contemporary Art in Boston are somewhat comfortable.
A plush base you can almost sink into, but only enough elbow room for you or your neighbor, not both.
By the end of an hour, you’ve explored the full space of position adjustments, and have the type of “settle in” feeling I only feel on flights.
The final talk of
Systems Distributed 2026
, titled Mission-Critical, by Joran Dirk Greef, opened with a definition so efficient it nearly circled back on itself: mission-critical software, we were told, is software that performs a critical mission.
Got it. That was quick, here we go dinner!
However, the talk continued, and came to TigerStyle, impressive logo and all, with the claim, delivered with absolute clarity, that this engineering philosophy was what made TigerBeetle possible.
Next came several claims about TigerBeetle, among them that a large share of the transactions on a national exchange run on it, capped with a mission statement about powering the world’s transactions.
The exchange number is real and genuinely impressive; it was the framing around it, the grand mission delivered as an applause line, that gave me pause.
Before I could settle with the idea, we were looking at the Jepsen report, a testimonial of sorts, highlighting the lines that affirm TigerBeetle made the right calls.
It was getting to be a lot, especially with the thank-you interlude and the applause for contributors with enough social pressure, or genuine admiration, that everyone around me clapped.
Then the music started, playing over a TigerBeetle sponsored race car, and my only thought was that this was a remarkable way to drive all day and end up in the same place.
If I’m not getting something out of a talk, I don’t owe anyone the social obligation of staying (and neither do you).
I excused myself from the middle of a row, and spent the rest of the talk in the lobby staring at an installation in amazement.
What I witnessed wasn’t a normal tech talk, it was a sermon from the Church of the TigerBeetle, complete with origin myth, testimonials, and orchestral music, all leaving you with the impression that their way was indeed the best.
It’s tech evangelism, and that scares the shit out of me.
The problem isn’t that TigerBeetle is making bad decisions with their tech, their deterministic testing, viewstamp replication protocol, and use of money and influence for good causes is impressive.
The problem is the claims made on faith.
Claims made without introspection, without proof, presented to you one after another without a moment in between to digest and integrate them into your current belief system, and set to emotional background music.
Are we gearing up to storm the Bastille here, or are we trying to build software that works for our end users?
The question shouldn’t be “Is this organization inspiring?”, but “Can we steelman these claims?”
Without that, we’re walking around with ideas, but not the rigor we need to defend them.
Not to ourselves, not to our teammates, and not to our organizations.
That’s a problem.
I’m not trying to dunk on TigerBeetle.
What worries me is the mode of persuasion: the presentation asked the audience to admire and believe before it equipped us to evaluate.
If I pay $500 (or my company paid that to send me), what good can I do if I believe the hype but don’t carry the weight of the evidence?
I want to believe that TigerBeetle is great, but faith without receipts is useless in most organizational cultures.
The other place I’ve seen tech evangelism is in Haskell, specifically the belief that strongly typed programs are a better way to build software.
It’s a claim that tends to be held in proportion to your knowledge about types, but it’s lacking evidence that it works.
In each of the three companies I worked for, Haskell started as the solution to software, but was eventually blamed for the company’s problems.
The blame was not fully deserved, but the fervent belief in the language made discussions of language tradeoffs difficult when being a Haskeller was part of your identity.
That’s not to say you shouldn’t get to pick your tools, or choose where you work based off those tools, just that a purely affinity-based identity is not a solution to technical problems, and it often gets in the way.
Evangelism is considered harmful as an organizational technology.
Even if everyone in that room (except me) is already armed with the evidence, rallying the faithful trains people to hold beliefs by faith, not by reasoning.
An org that rallies by faith teaches people to default to faith — so when a genuinely hard question comes up, like “should we enforce a coding standard?”, they reach for conviction instead of evidence: it worked for TigerBeetle, they do X.
Second, evangelism suppresses dissent — and my revulsion for the style is a feature of it, not a bug.
Had I stayed and forced a steelman, the framing itself would have made the question land like heresy.
The room wasn’t built to process disagreement; it was built to dissolve it. And a culture that dissolves dissent loses the one thing that lets it notice it’s wrong.
Finally, faith-based conviction is brittle when the facts change.
When the world shifts, like a new problem, a new constraint, one where Tiger Style isn’t the answer, a belief held on faith can’t pivot.
The evangelism that rallied you is now what traps you.
I don’t need to be inspired, I already am.
When I go to a conference, the purpose is to learn things, meet people, hear about new approaches, and gain the information or resources I need to take those good ideas and apply them to the problems I work on.
The issue with evangelism is that it creates hollow actors: people excited about the idea but lacking in substance.
Most people in that room are formidable on any side of a technical argument, but I’m greatly dismayed imagining that any of their motivations would be faith alone.
The faith isn’t needed. We need to let the work stand on its merits — show the receipts, empower people with the facts and reason, and trust them to make the right calls.
I’m glad the conference was in Boston.
It’s a fitting place to make this argument.
This is a city built by people who refused to take authority on faith.
People who looked at the divine right of kings and answered with reasoned argument, evidence, and the radical idea that claims should have to justify themselves.
They didn’t storm anything because they were inspired.
They did it because they had thought it through, they could defend it, and because they had guts.
That’s the inheritance I’d rather claim.
Not a congregation that believes, but a room full of people who can show their work.
Rust Glancer
, a functional
LSP server for Rust which uses two orders of magnitude less RAM, is incredibly
cool. Go check it out! This post started as a comment on lobste.rs, but I
figured it out that it’s better to publish it somewhat more prominently. Don’t
expect polished writing though!
Some thoughts:
rust-analyzer uses rowan for syntax tree representation
Yeah, rowan is garbage :P I was really thinking about
incremental parsing,
incremental, DOM-mutation style refactorings,
And Rowan is pretty good for that. But that’s 1% use case. The 99% use case is
all the code in your 6666 dependencies which you won’t ever look at, but which
needs to be at least shallowly analyzed. Even for incremental tool whose main
goal is refactoring, the primary AST structure should be just a list of arrays.
There might be a real post about that at some point, see
https://youtu.be/G93oYL1ry70
as a teaser.
Rust workspaces genuinely have a lot of information that must be indexed:
thousands of functions, structures, traits, relationships between these,
function bodies and statements in them, etc. Each of these needs to be
analyzed and remembered, and you can’t really cheat if you want to have things
like “find all references to this structure”.
If I understand correctly, Rust Glancer wants to process each function body. I
think
that
part can perhaps be made lazy (but not incremental!) with little
overhead? Index all items, but, for functions, do only the currently opened
file? This might combine some of the better parts of both worlds.
Would be interesting to compare memory usage with Rust Rover. Net of the IDE
GUI itself, I would expect RR to be more compact.
Some features are unlikely to be supported though, such as build scripts /
proc macros support via proc macro invocation
I might be rationalizing/misremembering things, but IIRC it’s exactly around
adding proc macros that the thing began to feel unreasonably bulky. Expanding
proc macros is slow as we are running real code, we can’t really do normal IDE
cheats. And proc macros generate a lot of code. At one point I measured, it was
like 30% of rust-analyzer binary size was attributed to JSON parsing code. If no
one sees the code, it can’t harm anybody, right?
One potential approach here is to pull the Sorbet trick, where you don’t run
meta programming at all, and instead have a plugin interface to “explain” the
effects
of what that would have done. Instead of running serde, we just add a
shim that injects
imp Serialize for T {}
with an empty body.
I’m not sure why, but in rust-analyzer I’ve observed that when agents edit the
code, inlay hints can get out of place
Rust analyzer’s core data model is
very
pedantic about always observing
consistent snapshots of the code, and does its best to ensure that the language
client and server have a shared, strictly serializable view of the world. It’s a
shame that
LSP doesn’t allow that to be
correct
, only heuristically right
,
unlike the older Dart Analyzer protocol, which has sound data synchronization.
However our implementation of file watching is sketchy! First, there are two
backends: we can ask the editor to do watching for us, or we can use server side
watching. Try changing
this option
and see if it helps? But then, yeah, my recollection is that our native watcher’s API was
fundamentally racy, and I didn’t do the messy platform-specific work of making it correct.
But the main thing I want to write, and why I moved from the cozy lobste.rs text
area to the luxurious comforts of an Emacs buffer, is that right now rust-analyzer is a bit
like that half-drawn horse meme, except that it’s only the head half of the
horse.
One Big Idea of IntelliJ is that it’s PSI API (essentially AST with resolved
types) is really an interface, and there are multiple provides. And in a typical
usage, there’s at least three backends in play:
For the files opened in the editor, actively modified by the user, the PSI is
backed by the concrete syntax trees.
For the rest of the project files, the PSI is backed by the so called Stub
Tree, a compact on disk representation storing only the “externally visible”
parts of the file (so, without function bodies). If the user navigates to a
new file, its PSI transparently switches from stubs to syntax tree.
For dependencies, the PSI is often backed by the compiled .class files,
produced by javac. If you navigate there, the IDE just decompiles stuff four
you! Super cool!
This
is how I think such things should work. rust analyzer
shouldn’t
use
salsa for all those 6666 dependencies you still haven’t looked at. It should
just use rustc’s .rmeta files, switching to salsa, transparently, only when the
user starts messing around their
~/.cargo/registry/src
folder.
The prerequisite for that is defining the abstract API for accessing Rust code.
That was always the plan, and we did start on that at some point:
rmeta-transparent – source code might not be available for some crates, the API
should support pre-compiled rmeta files as inputs.
But I don’t think that work was ever completed.
This still seems to me to be the lowest-hanging watermelon here — split the
world into arcy-pointy incremental tip of the iceberg, and mostly read-only,
on disk, compact, dark, moist breeding ground for supply chain attacks.
Such glance analyzer architecture would be great, imo!
Tumble Forth – from assembly to OS with C compiler
Hello, my name is Virgil Dupras, author of
Collapse OS
and
Dusk OS
and I'm starting a series of articles that
aims to hand-hold my former self, a regular web developer, into the rabbit hole
leading to the wonderful world of low level programming. Hopefully, I can
hand-hold you too.
The general goal is to broaden your perspectives on the subject of computing. I
intend do to that through story arcs leading, step by step, to some nice and
shiny objective. I also intend to work into a gimmick where in each episode, I
get to tell one corny joke.
The target reader is a person who knows their way around programming, but is
inexperienced in the area of low level programming. If you're the target reader
but find some parts of this content difficult to understand, this is not
intentional. In this case, or if you have any question or comment, reach out to
me at hsoft@hardcoded.net.
Story arcs
Buckle up, Dorothy
In my
“pilot” story arc
, we peek in
disgust in the abyss of modern software complexity and escape this dystopia by
tumbling down the rabbit hole of low level development.
Starting from bare metal on the PC platform, we build a Forth from scratch, then
switch to
Dusk OS
and then build a partial C
compiler (just enough to compile our example code), again from scratch.
Maybe you've observed this too. There's a line some games cross past which you no longer feel like you're playing the game. The game is playing itself, and you are there to be fed feel-good brainjuices. I'm not talking about idle games - you're doing something, but you're not
really
doing something. Sometimes you click a glorified "next" button and flashy numbers rocket upwards. Sometimes you put a pip into a skill tree over here, instead of over there. Poncle's bestselling bulletnothing is the leading exemplar of this. I think it's rubbish, partly because it feels like treacle. But mostly because it is accelerating a dull trend in game design: to make the game auto-go while building a coffin of upgrades around the player until they have precisely zero thoughts, swimming in whatever the opposite of a sensory deprivation tank is, flooded with dopamine. Death, I say! Death to the self-playing videogame.
Vampire Survivors
is not alone in hoodwinking me into the sensation that I am doing something when in fact I am a sad void pressing left and then right.
Loop Hero
sets your hero autotumbling around a circuit of monsters, and the closest it comes to having meaningful decisions is in the basic Carcassoning of resource and monster tiles.
Ball X Pit
is a roguelike ransacking of a classic arcade brickblooper with autofiring bullets. It eventually contains a full admission of defeat to the automaton approach to game design: an unlockable character called "the Radical" who will play brickblasting runs on your behalf and choose upgrades without any input from the player whatsoever.
Vampire Survivors turned a basic shmup into a karmic hamster wheel. BallXPit is an LCD brickpong toy running on cruise control.
The self-playing game automates a primary verb of play. That is a fancy way of saying: Me no shoot, game shoot for me. It then gets you as fast as possible onto a trundling treadmill of dopamine and ostensible progress, while sort of reminding you of another game you played before. The physicality of this "kernel" genre underneath is not really needed. It just needs to look like bullet hell. It just needs to seem like Breakout. They can become lauded, successful games, praised for their addictive charms and approachability.
To me, they are automated fun. Poisoning hand-to-eye happiness with the anti-activity of the idle game. We are lurching toward the idlefication of all genres. Why stop at Asteroids and Breakout? Let us automate Doom. Let us automate Street Fighter. Let us automate Rogue. You think I'm joking, but
somebody is already working on the latter
and it's giving me a bad case of the squinty eyes.
Self-playing games take microdecisions away from you in favour of macrodecisions. You don't need to choose where to aim your gun every second, but you will need to decide what ammo upgrade to select for the next level. This is an inviting approach for game designers (and not unique to the self-playing game). It allows designers to sidestep the difficult psuedoart of "gamefeel", and player skill becomes less of an issue to account for. You can focus on the metagame, the upgrades and abilities and perks and extras. The exofun. All you need to do is make the braintickling numbers work, and then make it look flashy.
But what if some genres
are
the microdecisions?
If somebody automated
Devil Daggers
, I would stab them in the hands with a barbeque fork. If somebody automated Tetris, I would have them arrested. When a designer of a racing game makes drifting round corners feel the perfect balance of scrapey and buttery, this feeling can
carry the entire racer
despite how bad its story mode inevitably is. On the other hand, when somebody strips an essential verb out of Asteroids and claps on an incremental skill tree, I can't help but feel they have piggybacked on the physical feeling unlocked by bygone engineers to deliver what is effectively Cookie Clicker under the hood. This is not a revolutionary new form. It is
Candy Box
sans the sense of humour, with a lobotomised arcade classic attached.
This post is for paying subscribers only
Already have an account? Sign in.
Bringing the cybersecurity capabilities of Claude Mythos 5 to more defenders
We're sharing an update on our efforts to help more teams use frontier capabilities for cyber defense.
Claude Mythos 5
is now available in
Claude Security
, and coming soon to partners' cyber defense tools. We're also launching a $35M fund to help secure open-source software and sharing plans to expand our
Cyber Verification Program
.
In April, we launched
Project Glasswing
to put our most capable frontier model, Claude Mythos Preview (and its successor, Claude Mythos 5), in the hands of a small group of organizations securing the world’s most critical software. This gave defenders a window of time to find and fix vulnerabilities ahead of models with similar capabilities becoming generally available or reaching malicious actors.
Our goal has always been to expand Mythos-level defense to as many defenders as we safely can. To do that, we've been working on
safety classifiers
and safeguards that let us expand access to Mythos-class models without putting their offensive cyber capabilities in the wrong hands.
Claude Fable 5
was the first step: it made the model broadly available while blocking dual-use cyber work.
Today, we’re taking the next steps. The riskiest behavior occurs when a user has direct access to a model, where a malicious actor can try to steer it toward harmful uses. But if users can only receive specific outputs, such as a patch for a vulnerability or a security alert, that risk is much lower. The changes we’re announcing give users greater access to the defensive results, while maintaining appropriate guardrails around direct access to the model:
Claude Mythos 5 integration into the tools defenders rely on.
We’re working with our cybersecurity technology and services partners to integrate Claude Mythos 5 into the products and services defenders already use to secure their software.
Claude Security scans can now run on Claude Mythos 5.
Customers on Claude Enterprise plans can now run our most capable model in Claude Security, using it to scan their codebases for security vulnerabilities and suggest patches.
$35 million in credits for open-source security.
Our new Defender Advantage Fund (0xDAF) will provide $35 million in credits to organizations working to patch vulnerabilities in open-source projects, automate parts of the process of scanning and patching open-source software, and experiment with new security approaches.
Expanding our Cyber Verification Program.
The program already gives vetted defenders reduced safeguards on Opus and Sonnet models. In the coming weeks, we will expand this program to include broader dual-use capabilities on Opus and Sonnet, with Mythos-class access to follow.
Our aim remains to help organizations adapt to the pace and demands of cybersecurity as AI models become increasingly powerful. We will continue to develop safeguards, access programs, and community support to make our most capable models safely available to a wide range of people and organizations.
Integrating Mythos into existing cyberdefensive tools
The teams defending hospitals, utilities, financial systems, and the software supply chain already rely on a suite of products and services for security operations, incident response, threat intelligence, and detection engineering. The fastest way to make frontier capabilities available to those defenders is to integrate Mythos-class models into the tools they already run.
Many of our partners have already
built cyber products on Claude Opus
that help security teams triage alerts, identify threats, and remediate vulnerabilities faster. We’re now working with these partners and more to build Claude Mythos 5 into their products and services, so they can deliver Mythos-level defensive outcomes to their customers.
When an end user uses one of these products, they’re not interacting with Mythos directly. Instead, they work through a purpose-built interface that runs Mythos in the background for a defined task and only receive the specific artifact the product is intended to provide. For example, a tool to remediate vulnerabilities might provide a list of suggested patches as its output. This output would be generated by Mythos, but the user would not have a way to prompt the model to, say, develop an exploit for a vulnerability. We and our partners also have abuse prevention measures in place to verify the model stays within its intended scope.
We're early in this work and expect it to expand over time. If you build security products or services and want to bring Claude Mythos 5 to your customers, you can
register your interest here
.
Making Claude Security available with Claude Mythos 5 for Enterprise customers
Starting today,
Claude Security
scans now run on Claude Mythos 5. Claude Security scans codebases for vulnerabilities and suggests patches for human review; it’s currently in public beta for Claude Enterprise customers, and scans with Mythos 5 are billed as standard token usage under your existing plan, with no separate add-on.
Enterprise admins can enable Claude Security in the
admin console
. From
claude.ai/security
, users can select a repository to scan using Claude Mythos 5. Claude then scans the codebase for vulnerabilities, and returns each finding with a
CWE
(Common Weakness Enumeration) category, confidence and severity ratings, and a suggested fix.
Users can then open Claude Code on the web to implement the fix. Interactive patching uses the models your organization has access to in Claude Code. The Mythos scan itself does not extend Mythos access to other surfaces. Every patch must be reviewed and approved by a human before it can be implemented.
Claude Security uses Mythos 5 to scan code you own, and returns detailed findings rather than raw outputs without exposing the model itself. This means defenders can access the capabilities of Claude Mythos 5 without the model becoming accessible to those who might misuse it.
Launching the Defender Advantage Fund to secure open-source software
Some of the world’s most widely used programs run on open-source software. Yet these projects are often maintained by volunteers or nonprofit foundations, who may lack the resources or personnel to comprehensively defend their projects against attack. Through Project Glasswing, we made $4M in direct donations to open-source security organizations, provided credits to the open-source security foundations in the program, helped scan and patch widely used projects, and support coordinated vulnerability-fixing efforts like
Akrites
and
Gold Eagle
.
Our new Defender Advantage Fund (0xDAF) builds on that work with $35 million in Claude credits for organizations helping open-source maintainers secure their software. Grants will focus on three areas: patching live vulnerabilities in widely used projects, automating scanning and patching in ways other projects can replicate, and helping projects pursue more ambitious security approaches that make them resistant to whole classes of attack.
We're starting with a small number of larger, pilot grants to learn what works and scales best. We will share details on initial recipients in the coming weeks.
Expanding our Cyber Verification Program
To date, our Cyber Verification Program has provided organizations with access to dual-use capabilities when using Claude Opus and Sonnet models. Organizations in the program experience reduced safeguards, minimizing interruptions for accepted teams doing legitimate cybersecurity work on systems they’re authorized to protect.
Over the coming weeks, we are evolving the program to expand safeguarded access to Claude Mythos. As part of this, access to defensive capabilities like vulnerability triaging and validation will expand to Mythos-class models, and cyber defenders will see reduced blocks on Claude Opus and Sonnet-class models. Additionally, we are continuing to expand access to Claude Mythos through Project Glasswing in collaboration with our partners in the U.S. Government, focused on protectors of critically important infrastructure that meet strict security control requirements.
We'll share more details about the Cyber Verification Program expansion in the coming weeks. In the meantime, we encourage all security teams performing legitimate cybersecurity work to apply for the program for reduced safeguards on Claude Opus and Sonnet models. If you are already enrolled and accepted, no action is needed; we’ll reach out with updates.
What’s next
These initiatives are a continuation of our efforts to make the defensive capabilities of frontier models available to more people and organizations, and to support the open-source community in hardening their projects against attack. We will continue to work with government partners, organizations, open-source maintainers, and the broader industry to build the resilient cyber infrastructure today’s highly capable AI models demand.
Claude Security is available in public beta for Enterprise customers. Admins can enable Claude Security in the
admin console
. For a full walkthrough, see our
guide to getting started
.
The DESI Legacy Imaging Surveys combined more than 263,000 telescope exposures to make the largest 2D map of the universe in visible and near-infrared light.
Astronomers can pair the Legacy Surveys map with their own observations to explore our universe and search for rare phenomena.
The 2D map serves as the foundation for the Dark Energy Spectroscopic Instrument survey to measure the universe in 3D and investigate dark energy.
Hold on to your telescopes: the
DESI Legacy Imaging Surveys
team has released the largest-ever 2D color map of the universe. The 5.6-trillion-pixel map contains nearly 4 billion celestial objects, primarily stars and galaxies. The data is available for all to use and publicly viewable through the
Legacy Survey Sky Viewer
.
Astronomers and citizen scientists can explore the map or combine it with their own observations to better understand our universe. Researchers can search for rare phenomena like gravitational lenses, observe fleeting events like supernovae, and investigate two of physics’ biggest mysteries: dark matter, the invisible substance that accounts for most of the mass in our universe, and dark energy, the force driving our universe’s accelerating expansion.
The new map builds on earlier versions from the DESI Legacy Imaging Surveys that have already proved invaluable. To date, more than 1,800 science papers that reference the Legacy Surveys data have been published.
“It’s part of the fabric of astronomy research now,” said David Schlegel, a co-lead of the Legacy Surveys and scientist at the Department of Energy’s Lawrence Berkeley National Laboratory (Berkeley Lab). “When you’re working with astronomical objects today, you often start by pulling up the Legacy Imaging Viewer to see what you’re looking at.”
Covering roughly 75% of the sky in visible and near-infrared light, the updated map provides a deep view of the extragalactic universe not blocked by the dust and stars of our own Milky Way. Researchers expect it will remain the most comprehensive 2D map of our universe for years to come.
More than 160 scientists contributed to data collection for the project, and a team of 20 produced the final dataset released today. It was built by combining 263,407 telescope exposures from three ground-based sky surveys: the Dark Energy Camera Legacy Survey (DECaLS) at NSF Cerro Tololo Inter-American Observatory, the Mayall z-band Legacy Survey (MzLS) at NSF Kitt Peak National Observatory, and the Beijing-Arizona Sky Survey (BASS) at the University of Arizona’s Steward Observatory. That was supplemented by years of data from NASA’s Wide-field Infrared Survey Explorer (WISE) satellite mission and additional public data.
“For our team, these data are fundamental to our investigation of the expansion history of the universe and the formation of our galaxy,” said Arjun Dey, co-lead of the Legacy Surveys and an astronomer at NSF NOIRLab. “But the skies belong to everyone, and this survey gives everyone the chance to explore the sky and marvel at its wonders.”
Here be galaxies
The DESI Legacy Imaging Surveys were originally conducted to prepare for the
Dark Energy Spectroscopic Instrument
(DESI) survey. The Legacy Surveys’ 2D map is essentially a deep photograph of the sky; it records where galaxies and stars appear and how bright they appear. This crucial step enables DESI to select objects and measure their light in different wavelengths to determine their distances, building the largest high-resolution 3D map ever made. Scientists study the way galaxies have clustered at different ages of the universe to track dark energy over time.
In April 2026, DESI
completed its original five-year survey
ahead of schedule and with vastly more objects than expected. The early results have shown surprising hints that dark energy’s impact may be
weakening over time
— a paradigm shift that could potentially shape the predicted fate of our universe. DESI expects to publish improved results using their first five years of data in 2027 and is continuing observations into 2028.
DESI was so efficient at observing galaxies, the Legacy Surveys map needed to expand. Early on, “it became clear we might run out of galaxies to look at and run out of sky, and we better start doing something about that,” said Schlegel, who also works on DESI. The new Legacy Surveys map has been used to select DESI targets since June 2026 and will guide the telescope’s operations over the coming years.
Computing the cosmos
Merging hundreds of thousands of images taken on 2,285 nights, each with unique atmospheric and telescope conditions, was a massive computational effort. It took about a year to develop the computer code and eight weeks to process all the images at the
Perlmutter
supercomputer at the
National Energy Research Scientific Computing Center
(NERSC) at Berkeley Lab.
Beyond supporting DESI, the Legacy Surveys will be a foundational reference for the next generation of telescopes. As new observatories like the NSF-DOE Vera C. Rubin Observatory (jointly funded by NSF and DOE’s Office of Science) and NASA’s Nancy Grace Roman Space Telescope come online, researchers can compare their observations with one of the deepest and most comprehensive views of the sky ever assembled.
The Legacy Surveys data will also help scientists train artificial intelligence tools to analyze petabytes of astronomical data and accelerate new discoveries. It will be among the datasets used in an astrophysics pilot project within the American Science Cloud, part of the DOE’s
Genesis Mission
.
The
DESI Legacy Imaging Surveys
are supported by the U.S. Department of Energy’s Office of High Energy Physics; the National Energy Research Scientific Computing Center, a DOE Office of Science user facility; the U.S. National Science Foundation, Division of Astronomical Sciences; and the partner institutions.
DESI
is supported by the DOE Office of Science and NERSC. Additional support for DESI is provided by the NSF; the Science and Technology Facilities Council of the United Kingdom; the Gordon and Betty Moore Foundation; the Heising-Simons Foundation; the French Alternative Energies and Atomic Energy Commission (CEA); the Secretariat of Science, Humanities, Technology and Innovation (SECIHTI) of Mexico; the Ministry of Science and Innovation of Spain; and by the DESI member institutions.
###
Lawrence Berkeley National Laboratory
(Berkeley Lab) is committed to groundbreaking research focused on discovery science and solutions for abundant and reliable energy supplies. The lab’s expertise spans materials, chemistry, physics, biology, earth and environmental science, mathematics, and computing. Researchers from around the world rely on the lab’s world-class scientific facilities for their own pioneering research. Founded in 1931 on the belief that the biggest problems are best addressed by teams, Berkeley Lab and its scientists have been recognized with 17 Nobel Prizes. Berkeley Lab is a multiprogram national laboratory managed by the University of California for the U.S. Department of Energy’s Office of Science.
DOE’s Office of Science is the single largest supporter of basic research in the physical sciences in the United States, and is working to address some of the most pressing challenges of our time. For more information, please visit
energy.gov/science
.
New SynkLoader malware pushed in Microsoft Teams phishing campaign
Bleeping Computer
www.bleepingcomputer.com
2026-08-21 14:01:30
A previously unknown malware family dubbed SynkLoader is being distributed in Microsoft Teams phishing campaigns to steal credentials via a fake lock screen. [...]...
A previously unknown malware family dubbed SynkLoader is being distributed in Microsoft Teams phishing campaigns to steal credentials via a fake lock screen.
The attacker impersonates the target company's IT help desk, a tactic
Microsoft highlighted
earlier this year as increasingly common in multi-stage attacks.
Expel’s security researcher Marcus Hutchins explains that the attacks direct the victim to install a fake “PowerShell Cleaner” executable (.MSI) hosted in Microsoft Azure, making the download appear trustworthy.
Analysis of the malware showed "compile dates and file timestamps indicating it was first compiled and distributed around July 28, 2026."
The installer extracts a PowerShell script named cleaner.ps1 and a ZIP archive containing the Python framework, a malicious Python script, precompiled Python libraries, and several fake Microsoft runtime DLLs.
SynkLoader ZIP archive content
source: Expel
Based on the breached environment profile and operational targets, the attackers select which modules to deploy.
SynkLoader was named as such because of its unusual combination of Python, PowerShell, C#, and C++, sometimes blending up to three programming languages in a single module.
Expel identified the following SynkLoader modules after setting up a honeypot pinging the attacker’s C2, posing as a legitimate victim:
System Profiler
— Collects the hostname, username, privilege level, running processes, services, domain details, and number of computers in Active Directory.
Persistence Module
— Creates a randomly named scheduled task that launches SynkLoader at user logon and daily at 10 a.m.
PhishLocker
— Displays a convincing fake Windows lock screen to capture the user’s login password.
TrafficRedirector
— Creates a reverse proxy that lets attackers reach internal network services or route internet traffic through the infected computer.
Interactive Shell (RAT)
— Allows attackers to remotely execute PowerShell commands and receive their output.
StreamMaster (VNC)
— Streams the victim’s desktop and enables remote mouse and keyboard control of the active session.
Module Status Script
— Reports which malware modules and associated threads are currently running.
The malicious task securing persistence
Source: Expel
Fake Windows 11 lock screen
The most interesting component of SynkLoader is the PhishLocker module, which attempts to obtain the victim’s Windows account password via a fake lock screen.
By obtaining the password, the attackers could use it alongside the tunneling module to access corporate environments from the infected device, bypassing IP allow-list restrictions.
Although the fake lock screen looks particularly convincing, Expel notes that simply using Alt+Tab exposes the active windows on top of the lock screen which is just a "full-screen borderless GUI application."
Alt+Tab exposing the deceptive lock screen
Source: Expel
Hutchins says that based on SynkLoader’s focus on measuring Active Directory environment size, it’s likely that it’s used in ransomware operations.
“We did end up writing an emulator for the reverse shell module, just to confirm it was actually a hands-on-keyboard attack,”
the researcher says
.
“The threat actor attempted to run several profiling commands before realizing they were not in a real environment and disconnecting.”
Expel provided indicators of compromise (IoCs) for the observed attack, though it noted that the SynkLoader module hashes are unique for each infection and therefore not very useful for defenders.
The best practice would be to verify IT requests independently and avoid installing unsolicited MSI files.
When met with an unexpected lock screen, try Ctrl+Alt+Delete or Alt+Tab to determine its authenticity.
Union Members, Activists Defiant in the Face of Federal Surveillance
Published
Gabriel Van de Water Davis, one of the 15 people federally indicted for conspiracy to impede or injure federal officers, talks to Unidos executive director Emilia González Avalos after speaking during a press conference condemning federal surveillance. | Nicole Neri
Unions, progressive organizations and churches that were the subjects of a wide-ranging surveillance operation by the Department of Homeland Security are presenting a unified message: We will not be intimidated.
The leaders spoke at a press conference Tuesday morning alongside three of the
15 people indicted
in an alleged conspiracy to impede or assault federal agents.
“The (Immigration and Customs Enforcement) and Border Patrol agents who murdered Renee Good and Alex Pretti have been hidden away and protected, while DHS turns its attention towards spying on union members, civil rights organizations, churches and individuals who stood up for their community in whatever ways they were able,” said Gabriel Van De Water, an artist and community organizer who was among the 15 people indicted in June.
As part of its
surveillance operation
during Operation Metro Surge, Homeland Security agents secretly obtained financial records from organizations, including the labor unions Service Employees International Union and Communications Workers of America; Voices for Racial Justice, a longstanding racial justice training organization; and Sunrise Movement, an environmental justice organization. Undercover agents attended meetings of anti-ICE organizers as they planned protests, including the Jan. 23 general strike and march, and food distribution for immigrants who were in hiding.
The tactics were revealed by a defense attorney representing one of the 15 defendants in federal court documents last week. Filings included reports filed by undercover agents, copies of subpoenas and descriptions of messages on the encrypted messaging app Signal.
“These shams of political investigations — these are just acts of retribution and intimidation,” said Rev. Jennifer Crow, senior minister of the First Universalist Church of Minneapolis, one of the churches where undercover agents infiltrated a meeting.
The speakers gathered outside the Target headquarters in downtown Minneapolis, and called on Target and other Minnesota-based corporations to use their political power to demand accountability for federal agents’ actions during Operation Metro Surge.
“Every single corporation that claims to be here for the right reasons is going to have to show it through,” said Marcia Howard, president of Minneapolis Federation of Educators Local 59.
ICE and Target did not immediately respond to the
Reformer
’s emailed questions.
The Minneapolis Federation of Educators Local 59 is one of nearly 20 organizations accused by the U.S. Department of Justice of forming part of a wide-ranging conspiracy to impede and assault federal agents. A diagram presented to the grand jury as part of the indictments shows the teachers’ union alongside several other unions, the ICE-tracking group Monarca, Democratic Socialists of America and “Left Jab,” a community martial arts group, among others.
Labor leaders and union members said their solidarity with their immigrant members, and with each other, was indispensable as they worked together to defend immigrants from ICE during Operation Metro Surge.
Howard said she’s proud to be affiliated with other institutions that stood up to the swarm of federal agents in Minnesota this winter.
“Kinda chic to be on that list,” she said.
Marcia Howard, president of Minneapolis Federation of Educators Local 59, speaks during a press conference condemning federal surveillance Tuesday, Aug. 18, 2026. (Photo by Nicole Neri)
Rebuilding our Electron meeting-recording engine in Swift
Our desktop app captures meetings without a bot and streams them to the cloud. For months, the recording engine was the hardest part of the product to make reliable. We'd fix one class of edge case, ship it, and a new one would surface the next week. Different root causes, same pattern.
The engine ran in the render process of our Electron app. We tried the obvious fixes: tighter lifecycle management, moving work off the main thread, isolating it from React's render cycle. Each change helped at the margin, but none addressed the real issue. A render process is the wrong place to do realtime audio and video capture. A capture engine can't tolerate GC pauses, throttling, or any of the other things a browser runtime does to stay responsive.
So we went native: ScreenCaptureKit on macOS, libobs on Windows, and a shared Swift layer tying it together.
Atomic: our Combine-to-Jotai bridge
Bridging a native runtime to React usually means writing native addon bindings by hand. You serialize every value that crosses the boundary, route events through stringly-typed names, and update three files whenever you add a property: the Swift class, the C++ binding, and the TypeScript wrapper. It works, but it's out of sync the moment anyone forgets a step.
What if every
@Published
property in Swift automatically became a Jotai atom in React? Fully reactive, type-safe, no glue code. That's what our internal tool Atomic does.
@NodeExport
public final class AudioPlayer {
@Published public var isPlaying: Bool = false
@Published public var volume: Float = 1.0
public func play() { isPlaying = true }
public func pause() { isPlaying = false }
}
#AtomicExport(AudioPlayer.self)
const player = new AudioPlayer();
const volumeAtom = atomWithNativeState<number>(player.volume);
store.set(volumeAtom, 0.5); // Flows into Swift.
player.play(); // Updates flow back into React.
From React's perspective, these atoms are indistinguishable from any other Jotai atom. The fact that the data lives in a Swift runtime on a different thread is invisible.
The
@NodeExport
macro generates the entire bridge at compile time. Types map automatically (
Int
→
number
,
String?
→
string | null
). Value changes in Swift schedule callbacks on Node's event loop. Every new property we add on the Swift side is instantly available in React. And because Atomic is built on Swift, not Apple frameworks (we use
OpenCombine
on Windows), the same bridge runs on both platforms.
Two capture engines, one interface
On macOS, ScreenCaptureKit gives us hardware-accelerated capture and the native content picker. On Windows, we use libobs through a Swift wrapper we call OBSKit. The two engines have fundamentally different architectures.
On macOS, we receive raw sample buffers from three independent sources and assemble the file ourselves. On Windows, capture, mixing, encoding, and muxing run as a single graph. We configure it and a file monitor streams newly written bytes to our upload session.
Windows capture has its own challenges. We use Windows Graphics Capture (WGC) as the primary method, and if it doesn't deliver frames in time, we fall back to BitBlt. We also detect all-black frames (common with some emulated windows or games) and switch methods mid-recording.
When clocks disagree
This is where the macOS engine earns its complexity. Three capture sources, three hardware clocks, three different ideas of what time it is.
Both audio sources get timestamped and converted to a global frame index. The mixer drains both queues in lockstep, only producing output when both have enough data. If one source stalls (muted mic, frozen virtual device), the mixer detects it after 500ms and switches to single-source mode until it resumes.
Then there's a subtler problem: audio drivers that lie about their sample rate. Some virtual drivers report 48kHz but deliver buffers at 44.1kHz. Over a 30-minute meeting, this drift becomes audible. Our fix is confidence-based correction: we measure actual buffer cadence, and if it consistently disagrees with the reported format across three consecutive buffers, we reinterpret the stream at the correct rate with a crossfade to avoid clicks.
On Windows, most of this complexity is abstracted away by the capture engine's internal mixer. The tradeoff is control: on macOS we detect and fix edge cases like lying drivers ourselves; on Windows we trade that granularity for simplicity.
Recordings that survive crashes
A regular MP4 writes its metadata at the end of the file. Crash before that, and the recording is gone. On both platforms, we use fragmented MP4 instead.
Each segment is self-contained. A crash at minute 30 loses at most the last second. Segments go to both local storage and the cloud simultaneously. If the network drops, segments persist locally and the upload resumes automatically when connectivity returns.
Desktop recording used to be one of our most common sources of support tickets. Now it's a boring part of the app that just works. The entire rewrite shipped in two months, and Atomic is why: once the bridge existed, adding a feature meant writing Swift and watching the UI update in real time.
Not everything belongs in a render process. Sometimes you need to go native.
If taking on problems like this sound interesting to you, consider
joining us
.
I commonly hear two different responses when people discuss whether the United States is in a recession. I hear “What recession?” and “Everything is expensive now!” All while credit card debt hit an all-time high last year ($1.28 trillion, Q4 2025). More “regular” people are living paycheck to paycheck, and closer to homelessness than ever before. 111 million Americans, ~40% of adults cannot pay their credit card balance in full each month. [
1
] Contributing to the issue is a single, simple business practice: constrain supply, raise prices far beyond what the constraint justifies, and then refuse to lower them.
The pandemic is where companies discovered it worked, with the perfect cover of confusion, panic, and unknowns. It was the proof of concept. When global supply chains broke down in 2020, the average markup over cost jumped from 56% to 72% in a single year — the fastest increase since 1955. [
2
] Corporate profits have risen 50% since; real hourly wages, only 3%. [
3
] As far as the general public knew, who was to say whether it was the pandemic creating shortages, or higher upstream commodity pricing. Few knew, and it certainly wasn’t a priority. Corporate profits drove more than a third of inflation from the start of the pandemic, and 53% of it by mid-2023 (after supply chains had recovered). In the forty years before, they drove 11%. [
4
] This format increased corporate confidence, and laid out a plan for every CEO and CFO [of large market companies] in the country.
In 2026, with reducing regulations, rampant collusion, lack of competition, and direct evidence (the pandemic) that corporations can gouge consumers without concern, we are in a dire circumstance that has caused goods to increase in cost 3x (or more) in 6 years. It’s truly unprecedented, and in my opinion, everyone should be aware of the specifics. Because even if you are one of the fortunate that are still able to afford your quality of life, there are things you can do for those that are closer to not being able to.
Rockets and feathers
The process of increasing prices more than constraint demands has been exceedingly popular across industries lately. Its academic name is
Asymmetric Price Transmission
, but it’s commonly known as
Rockets and Feathers
. Describing prices that rise like rockets, but fall like feathers. [
5
]
S&P 500 net profit margins hit an all-time high in Q2 2026, reaching 16.9%, up from the 10-12% range that held for most of the prior decade. [
6
] Workers’ share of national income fell to 52.9%, the lowest level recorded since the data series began in 1947. [
7
] 401(k) hardship withdrawals have tripled since 2020. [
8
]
At the CNBC CFO Council Summit in late 2023, Richmond Fed President Tom Barkin polled the executives in the room about their pricing plans: a majority said they would raise prices, a minority said they would hold, and not a single one said they would lower them. Companies won’t give up pricing power “until they have to,” Barkin said, noting that before COVID, most companies “really weren’t into raising prices [as they] didn’t think they had the power to do it.” [
9
]
Corporate profits hit $4.42 trillion in Q1 2026, more than double the 2010 level. [
10
]
In Q2 2026, S&P 500 earnings grew 50.4% while revenue grew 15.0%. This represents fewer sales, but higher prices on fewer units. [
6
]
“Obviously, our goal if a recession hits, and commodity costs come down, would be to then get a gap, maintain a gap going forward where obviously, we’re maintaining more price than the decrease on the commodity.” [
11
] -
DuPont CEO Edward Breen
Since 2020, the same pattern has played out across at least eleven major consumer categories. Eggs, cars, rent, groceries, gasoline, auto insurance, lumber, shipping, building materials, pharmaceuticals, baby formula. Each with its own supply shock, each with the same result: prices went up, profits hit records, and when the constraint eased, the prices stayed.
“We don’t reduce prices on the back end of these increases.” And: “A nice light recession would be perfect for us because it would bring raw material costs down even more.” [
12
] [
13
] -
H.B. Fuller CEO Jim Owens
The stock market is rewarding this behavior. PepsiCo posted nine consecutive quarters of volume decline while raising prices 10-17%. Revenue still grew. [
14
] Across consumer packaged goods from 2021 to 2023, dollar sales rose 13% while unit volume fell 6%. Every dollar of growth came from pricing, not demand. [
15
] And when companies break the pattern, they’re punished. Albertsons cut its full-year guidance in July 2026 as it invested in lower prices to hold off Walmart and Aldi, and the stock fell 21% in a day. [
16
] BJ’s Wholesale beat its earnings estimates in May 2026 and fell 8% anyway, on a ten-basis-point decline in its merchandise margin. [
17
]
“Why would I be the first to cut my margins when we just went through a period where we had the world’s best excuse [inflation] to recover margins?” [
9
] -
Esade marketing professor Marco Bertini
“Imagine I am the first to say I am holding on prices, and make that known to customers? That’s how a price war starts and the competitive advantage from being the ‘good guy’ lasts two seconds. No one wants a race to the bottom. The gains over the past few years evaporate in a few months.” [
9
] -
Esade marketing professor Marco Bertini
“We’re all a bunch of cars on a highway… Who hits the brakes first? Who wants to hit the brakes before the person in front of them hits the brakes?” [
9
] -
Anonymous CFO on the CNBC call
Ongoing collusion
Every pattern above can be explained without conspiracy. Costs rose, firms responded independently, nobody wanted to cut prices first. That interpretation is reasonable, it just requires ignoring how often these same industries have been caught coordinating on purpose. Price fixing is not a theory about how corporations behave. It is a category of federal crime with a long prosecution record.
Industry
Conduct
Outcome
LCD panels
60+ secret “Crystal Meetings” in Taiwan hotels, 2001–2006. $71.9B in price-fixed panels
~$3B in global fines. 22 executives charged, several imprisoned. Samsung reported first and paid $0 [
18
]
Auto parts
Bid rigging across 30+ component categories, 2003–2010. Parts in 25 million US cars
Coordinated bids to manipulate the Urner Barry benchmark, 2022–2025
$3.3M and 53M donated eggs. 0.08% of Cal-Maine’s annual revenue [
25
]
Rental housing
RealPage algorithm setting rents from competitors’ confidential data
Settled Nov 2025. $3.8B added to renters’ bills in 2023 [
26
]
The first six are criminal convictions or guilty pleas. The egg and RealPage matters were settled without admission of wrongdoing.
Samsung, SK hynix, and Micron hold about 90% of the global DRAM market. Samsung and SK hynix pleaded guilty to fixing DRAM prices in 2005; Micron avoided penalty by reporting them. In June 2026, seventeen plaintiffs filed a class action in the Northern District of California alleging the three used a coordinated pivot to AI memory as cover to cut DDR3 and DDR4 output, with prices up roughly 700% in four years. [
27
] The allegations are unproven at this time. A nearly identical case was dismissed in 2020 and upheld on appeal in 2022, the Ninth Circuit holding the conduct “more likely explained by lawful, unchoreographed free-market behavior”, that the law requires an actual agreement, not the conscious parallelism common in a three-supplier market.
Three companies controlling ninety percent of a market can each cut supply and raise prices, and the law calls it competition. The system doesn’t punish collusion, so it’s a rational business approach to take advantage of the consumer to benefit the real product, share price.
The Supreme Court struck down IEEPA tariffs in February 2026, making approximately $166 billion in refunds available to companies. [
28
] [
29
] Technically Judge Richard Eaton at the Court of International Trade ordered the refunds. These are refunds for tariff costs that companies had already passed through to consumers via price increases. $100B of it has already been paid out. [
30
] The prices didn’t come back down. The refunds went to the companies. This is what they had to say about the tariffs:
McCormick CFO Marcos Gabriel
: “We are going to use the majority of the tax refund to offset these higher costs.”
PepsiCo CFO Steve Schmitt
: “We will be using the tariff… refunds to help offset some commodity inflation.”
Polaris CEO Mike Speetzen
: “Positive net pricing more than offset higher commodity costs.”
Descartes’ Jackson Wood
: “Recapturing those duty payments is really going to be about making their businesses whole. It’s unlikely to bring much relief to the U.S. consumer any time soon.” [
29
]
Note: Of 22 companies reviewed, half adjusted executive bonuses upward to exclude tariff costs from performance calculations. [
31
] Meaning consumers paid the tariff through higher prices, the company got the money back, and the executives were scored as though the tariff had never happened.
Corporations are not acting in your best interest, they are doing anything they can to increase profit.
“We’ll continue to offset a portion of the cost impacts with price increases.” [
32
] -
Procter & Gamble CEO Jon Moeller
, January 2022
Procter & Gamble predicted $800 million in windfall profits by retaining savings from falling commodity costs rather than passing them to consumers. [
4
]
Stop buying at inflated prices
Consumer sentiment has been sitting at recession levels since late 2025, and we need to behave like it. [
33
] 124,000 unsold new homes [
34
] and record car ages [
35
] show the truth of American finances in 2026. The corporations are attempting to convince consumers that there is nothing wrong, so that they can sustain their profits. They’re banking on you not noticing.
We should be purposefully delaying purchases. It is the most logical move with inflation, the ongoing war in Iran, and the predatory profits currently recorded. The only option consumers have right now is refusal, and historical examples say it works, but only if enough partake.
In January 2025, Croatian consumers organized a retail boycott through a Facebook poll of 144,000 people who voted on which specific chains to target. On the first boycott day, invoices dropped 44% and sales fell 53%. Over three weeks, retailers lost €108 million. [
36
] Kaufland cut prices on 1,000 products. The government expanded price caps from 30 to 70 essential items. [
37
] The boycott spread to 13 countries. North Macedonia saw a 46% revenue decline at major chains. We need to trigger the price war that the CEOs quoted above are afraid of.
Discuss product research with others. It helps us know what things actually cost before assuming their personalized advertised prices are the norm. See through the marketing to the manufacturing. See the problems a product solves, and the value of your purchase (to the company).
Be honest with yourself about your financial position. Even if you can afford something, do you truly need it? Is the markup fair? Does it anger you to know that you could solve this problem cheaper? Do you feel reward from saving money by avoiding cost? Evaluate older alternatives. See the value in restoring something versus buying new. Foresee the equalization of the market through falling prices because you personally refused to partake in these profit-motivated price hikes. Once you see the markup, you should be more proactive about identifying fair value. Train yourself to avoid luxury brands when the product is roughly the same. Here are a few common examples of product markups, use these as reminders when shopping to look for more reasonable prices.
Mainstream automobiles cost roughly $16K-$25K to produce, but sell for $48K-$50K. 2-3x markup before the dealer adds another 10%. [
38
]
Appliances have a 95-100% manufacture-to-retail margin. [
39
]
Furniture carries a 42-43% markup on average, up to 50% at the major chains. Only the sale prices ever make sense to consider. [
39
]
Mattress manufacturers give the same mattress different names at different retailers, so you can’t comparison shop. Markups routinely run into the hundreds of percent.
We live at a time where obvious oligopolies with concentrated market power are using supply constraints (real or manufactured) as a trigger to increase margin, and never lower it. This is a systematic and coordinated attack on the consumers of the United States by companies that identified the mechanism and prioritize their stock value. They can do this because they think
you
have no choice. Show them you do.
Why so popular? Probably because it was early “vibe coding” – I copy-and-pasted between ChatGPT and Xcode to code it, and that was new at the time.
But ALSO because knowing where the galactic centre is surprisingly grounding? I wake up every few months to an email in my inbox from someone who is having a tough time in life, or is losing a loved one, or similar, and somehow they have discovered
Galactic Compass
and they tell me how they sit outside at night with a cigarette and gaze at the arrow and it gives them a place of comfort and infinity.
I know what they mean. The Earth spins; it turns around the Sun; and so, at first, the supermassive black hole of the galaxy appears to slowly whirl around us, above and under the horizon, round and round. But then your perspective flips, and we are the ones moving, and the centre of the galaxy becomes a fixed point, our rock.
Anyway
Galactic Compass 2
has two new features:
Augmented reality mode.
You can place the arrow in the world around you and walk around it.
Apple Watch app.
See the compass arrow on your wrist (tap to use
alignment mode
which gives you a haptic bump when the arrow is pointing straight ahead).
Plus a new Liquid Glass appearance ready for iOS 27.
Apple’s in-camera augmented reality is really, really good. Like, the arrow remains rock solid as you walk around. I hope they keep improving it.
I added a specific interaction that I’m intrigued by: you can hold down on the compass around to “drag” it around. It remains about 75cm away in phone reference frame, then drops into world frame when you release. I like how fluid it feels. My phone starts to feel like a glove that can reach into the virtual.
With the Apple Watch app… RealityKit, Apple’s graphics SDK, isn’t supported on watchOS. So how does the arrow rotate any which way? The joy of AI and
agents that grind problems into dust
: Claude Fable built its own 3D graphics library. Astounding.
It isn’t all fire-and-forget vibing with AI agents:
That first version of
Galactic Compass
didn’t work when you lifted your phone higher than about 30 degrees. ChatGPT couldn’t get the maths right.
And there is a lot of maths: device rotation, world frame rotation, astro… the appropriate way to combine these 3D rotations (and avoid gimbal lock) is a method called “quaternions” which - despite my physics background - I have never grasped.
After I released version 1.0, I figured I would have to do the rotations myself. So I sat down with ChatGPT and I didn’t get it to write the code, but I got it to educate me. With a patient, interactive tutor, I was able to finally do what I hadn’t by reading books and asking mathematician friends – I learnt how to use quaternions just enough to make the app work.
So learning doesn’t stop just because I outsource a bunch of thinking to AI. It pushes me to learn more. I like that as an outcome.
llm 0.32.1
Simon Willison
simonwillison.net
2026-08-21 13:16:13
Release: llm 0.32.1
Fresh installs of LLM stopped working the other day because the OpenAI Python library dropped its usage of httpx, and it turned out LLM depended on that library but only installed it via a transitive openai dependency.
This dot-release fixes that for the moment by pinning...
Fresh installs of LLM stopped working the other day because the OpenAI Python library dropped its usage of
httpx
, and it turned out LLM depended on that library but only installed it via a transitive
openai
dependency.
This dot-release fixes that for the moment by pinning to
openai<3
, and a soon-to-drop 0.33 release will switch from
httpx
to
httpx2
.
LiteLLM (YC W23) Is Hiring – Rust / Performance Engineers
Release: llm-openrouter 0.7
Now that this plugin is compatible with LLM 0.32 it works much better with reasoning LLMs available through OpenRouter.
Updated for compatibility with LLM 0.32.
Models now use OpenRouter's implementation of the Responses API.
Three new server-side tools: Shell, ...
This sort of research is both exciting and terrifying:
The two models in question were told to generate complete genomes for a viable bacteriophage—a type of virus able to infect and replicate itself inside bacteria, destroying them from the inside.
Using an existing bacteriophage as an exampl...
This sort of
research
is both exciting and terrifying:
The two models in question were told to generate complete genomes for a viable bacteriophage—a type of virus able to infect and replicate itself inside bacteria, destroying them from the inside.
Using an existing bacteriophage as an example—ΦX174 (pronounced “fie-ex-1-7-4”), known for its ability to infect and destroy E. coli bacteria—the models generated about 700,000 potential designs, of which the researchers picked 285 that looked most promising.
The researchers then synthesised new DNA molecules using those designs and inserted them into E. coli bacteria, before waiting to see if viable bacteriophages would emerge.
Shortly afterwards, 16 of the Petri dishes in which the bacteria were growing began to show clear spots, as the viruses began to attack and replicate themselves inside the E. coli, demonstrating their viability.
Some of those viable viruses proved more effective at attacking E. coli than the original ΦX174 bacteriophage.
That’s a positive use of a synthetic virus. We can all imagine the negative uses.
Let s(n) be the side of the smallest square into which we can pack n unit squares.
For n=16, the best is obviously a 4x4 array, so s(16)=4.
For n=15, the 15 unit squares can also obviously be enclosed in a 4x4 square so s(15)≤4. Proving that it’s the smaller square is not obvious at all. Anyway, Erich Friedman proved that in 1999, so s(15)=4
For n=17, the obvious enclosing square is the 5x5, but in 1998 John Bidwell found an example that shows that a square of 4.6756… is enough, so s(17)≤4.6756… It’s a very interesting arrangement of the squares, so it’s worth visiting
the collection to see it
and the versions for other numbers.
On the other hand, Trevor Green proved in 2000 that 4.4452…≤s(17), (more details later). So there was a huge gap 4.4452…≤s(17)≤4.6756…
A few weeks ago, Sam Burns with ChapGPT 5.6 Sol
improved
(?)
the lower bound
. The new bound is still not reviewed by the community. I took a look and it makes a lot of sense and I think it’s correct, but I may be missing a small corner case in the proof or the accompanying program, or I may be missing a huge hole. I’ll add a small
(?)
to the number just in case, but I’m quite optimistic and confident it’s correct so I’ll use only a half font size. So the current bound is 4.4452…≤4.4811
(?)
≤s(17)≤4.6756…
My main objection to Sam Burns is that it really deserved a nice graphic! So my first step will be to add a nice graphic here. Also, making a few improvements to the program, I found a new lower bound that is 4.5058 So now we have 4.4452…≤4.4811
(?)
≤4.5058
(?)
≤s(17)≤4.6756…
My new example and the modification of the code are here, but the more technical details about finding the new bound are part of a
second post
.
Trevor Green’s bound
The idea of the old proof (19+40*sqrt(2))/17≅4.4452…≤s(17) of
Trevor Green
is to pick 16 very interesting "unavoidable" points in a square of side 4.4452… and then he uses a lot of geometry to prove that any unit square must include at least one of them. So if we try to fit 17 unit squares there, at least two unit squares must share one of the 16 interesting points. The construction chooses 16 points out of a 4x6 grid.
I only found an image of the points
in the old article
, but I couldn't find the analytical definition. Looking at the formula for the side of the square, and using a rule, and some guessing, I think that the empty left/right margin is 0.5 and the empty top/bottom margin is sqrt(2)-1/2≅0.9142… With these choices, the diagonal segment in the original graphic has length 1, which is a very useful number to make triangles that have vertices that are unavoidable points. (I’d be glad to hear a confirmation.)
It uses a 6x4 grid with an empty margin of 0.9142… and 0.5000, and the total size of the grid is 2.6168… and 2.4452…
To compare the construction to the
newer constructions, it’s better to symmetrize it. In this symmetrized
version each unit square includes at least 4 points, but some points are
thicker, and they count as double points (more details later).
Sam Burns’ bound
The idea to prove 4.4811
(?)
≤s(17)
posted by Sam Burns
using ChatGPT picks 268 somewhat interesting points in a square of side 4.4811 The points have different weights, and the total weight is only 16.9476. After some reductions, it’s only necessary to test a finite number of directions and they use a program in
Python
to test “all” the possible “almost-unit” (actually .9973) squares and verify that the sum of weight inside each one of them is at least 1 (actually 1.0003). So if we try to fit 17 unit squares there, at least two unit squares must share at least one of the 268 somewhat interesting points. (
More details in the second post.
)
This method has false negatives. If it verifies a solution then it’s surely correct, but if the program fails there is a tiny chance that it’s a mistake. This is fine to ensure the weight proves a lower bound.
It’s not clear how the weights were selected. Comparing this solution to all the examples
in the old article
, the 0.5 margin is too narrow because most examples use ~1.0 or ~9.1 or something like that. The selections of weight agree with me, and all the weights in the first/last row/column of the grid are zero. In my handwaving opinion, the second/penultimate row/columns should be empty too, but there is a non-zero weight in (1, 11) of the grid and the symmetric images, I hope it is not necessary in a better example. The third/penpenultimate row/column is quite full. It’s closer to the border than in the old examples, so it looks like adding more points near the border may be a good idea to improve the bound.
I draw the images using
Racket
with the
Metapict
package. The radius of each circle is calculated from the weight as
r = sqrt(weight^(1/gamma)) * scale
With gamma = 1.0 the area is proportional to the weight, but the small weights are too small in the image. After some tweaking, gamma=2.0 looks nice because the smaller weights are easier to see. The scale is not so mysterious, and I should have used pi somewhere in it, but scale=0.07 looks nice in my machine. The circles are semi-transparent, so it’s possible to see when they overlap if you ever increase the scale. The code is at the bottom, and it divides the weight by 1.0003 that is the actual minimal sum.
New bound
My idea was to try different combinations of the margin and internal grid size. As I said, it’s not clear how the weights were selected in the example of Sam Burns. So for each fixed size, I decided to use
linear programming
to find them.
Then I used a combination of brute force search and luck to get the best grid I could find. After that, I rounded the weight so they look nice and are nice fractions. (
More details in the second post.)
After a lot of time, the best I got is 4.5058
(?)
≤s(17). The new solution uses 168 somewhat interesting points in a square of side 4.5058 in a 29x29 grid. They sum only 16.9166… Each unit square includes at least a total weight of 1.There is an empty margin of 0.77565 and the internal grid has a total side of 3.9545.
As I said, the weights are closer to the border than what I expected looking at the old examples, close to the second/penpenultimate row/column of the previous one. It also uses fewer weights, so I hope it’s easier to prove that it’s correct without a computer. I’d like to make a non symmetric version, that may be even better.
The program published by Sam Burns assumes that the empty margin is 0.5, so I had to modify it slightly to allow arbitrary borders with a variable M that is the double of the margin. The version with that modification, the new sizes and the new table of weight is at the bottom. Running that program and making the obvious changes to the
explanation posted by Sam Burns
proves
(?)
the new bound.
Conclusion and Future Work
The distributions look quite discrete in the corner, but it has some strange bars near the center. It would be nice to increase the grid size and take a look. Also, the narrow empty margins appear to be useful.
My search program in the second article is too slow (like 1 hour), so I avoided changing the size of the grid. It may be useful to explore other grid sizes in case there are some interesting coincidences.
Adding more digits takes only a few minutes, I didn't bother because it looks like refining the grid or using more directions for the rotations would make bigger changes.
This result also automatically improves the lower bound of s(18), s(19) and s(20). But a more deep search for those values should provide even better bounds. I’ve seen too many cases where the total sum of the weight is 18. There is something interesting about 18.
I’d like to find the non-symmetrical version. I have some ideas to try, so check again in a few days. A non-symmetrical version hopefully has like 1/8 of the weight and hopefully shows the almost equilateral triangles and is easier to understand without a computer.
You may like to read the
second post
with details about how I got the new weights.
Program to verify the bound
from __future__ import annotations
from bisect import bisect_left, bisect_right
from fractions import Fraction as F
import numpy as np
# Original version posted by Sam Burns 2026
# Modified by Gustavo Massaccesi 2026
# Proposed exact lower-bound certificate for packing 17 unit squares in a square.
#
# All geometric quantities and predicates are rational. NumPy is used only for
# integer range-addition and cumulative sums; no floating-point geometry is used.
L = F(45058, 10000) # side of the square
M = F(15513, 10000) # both empty borders
B = F(9973, 10000)
T = F(207107, 500000)
KMAX = 180
D = T / KMAX
WEIGHT_SCALE = 576 # min weight
NGRID = 29
LAST = NGRID - 1
# (i, j, w): every distinct D4 image of grid point (i,j) receives weight w/WEIGHT_SCALE.
CERT = [
(0, 2, 165),
(0, 11, 129),
(1, 8, 36),
(1, 10, 21),
(1, 11, 15),
(2, 2, 246),
(2, 8, 129),
(2, 9, 105),
(2, 10, 36),
(2, 11, 105),
(5, 10, 36),
(6, 10, 63),
(6, 11, 12),
(7, 10, 21),
(8, 9, 33),
(8, 11, 15),
(9, 11, 75),
(9, 14, 39),
(10, 11, 25),
(10, 12, 21),
(10, 13, 24),
(10, 14, 3),
(11, 11, 16)
]
def orbit(i: int, j: int) -> set[tuple[int, int]]:
n = LAST
return {
(i, j), (n - i, j), (i, n - j), (n - i, n - j),
(j, i), (n - j, i), (j, n - i), (n - j, n - i),
}
def build_atoms() -> list[tuple[F, F, int]]:
step = (L - M) / LAST
coord = [M / 2 + step * i for i in range(NGRID)]
by_index: dict[tuple[int, int], int] = {}
for i, j, w in CERT:
for ij in orbit(i, j):
if ij in by_index:
raise ValueError(f"duplicate orbit assignment at {ij}")
by_index[ij] = w
return [
(coord[i], coord[j], w)
for (i, j), w in sorted(by_index.items())
]
# Clip a convex rational polygon against U >= bound or U <= bound.
def clip_u(
poly: list[tuple[F, F]],
bound: F,
keep_ge: bool,
) -> list[tuple[F, F]]:
if not poly:
return []
out: list[tuple[F, F]] = []
def inside(p: tuple[F, F]) -> bool:
return p[0] >= bound if keep_ge else p[0] <= bound
prev = poly[-1]
prev_in = inside(prev)
for cur in poly:
cur_in = inside(cur)
if cur_in != prev_in:
u1, v1 = prev
u2, v2 = cur
if u2 == u1:
v = v1
else:
lam = (bound - u1) / (u2 - u1)
v = v1 + lam * (v2 - v1)
out.append((bound, v))
if cur_in:
out.append(cur)
prev, prev_in = cur, cur_in
return out
def center_domain(c: F, s: F) -> list[tuple[F, F]]:
# A B-square at orientation (c,s) lies in [0,L]^2 exactly when its
# center lies in [h,L-h]^2, with h=B(c+s)/2.
# Transform that square to the B-square's (U,V) frame.
h = B * (c + s) / 2
lo, hi = h, L - h
corners_xy = [(lo, lo), (hi, lo), (hi, hi), (lo, hi)]
return [(c * x + s * y, -s * x + c * y) for x, y in corners_xy]
def verify_orientation(
c: F,
s: F,
atoms: list[tuple[F, F, int]],
) -> int:
"""Return the exact minimum integer score for one rational orientation."""
half = B / 2
dom = center_domain(c, s)
u_dom_min = min(u for u, _ in dom)
u_dom_max = max(u for u, _ in dom)
v_dom_min = min(v for _, v in dom)
v_dom_max = max(v for _, v in dom)
rects: list[tuple[F, F, F, F, int]] = []
u_events = {u_dom_min, u_dom_max}
v_events = {v_dom_min, v_dom_max}
# In center coordinates, atom membership is an axis-aligned rectangle.
for x, y, w in atoms:
pu = c * x + s * y
pv = -s * x + c * y
u1, u2 = pu - half, pu + half
v1, v2 = pv - half, pv + half
rects.append((u1, u2, v1, v2, w))
u_events.add(u1)
u_events.add(u2)
v_events.add(v1)
v_events.add(v2)
ue = sorted(u_events)
ve = sorted(v_events)
ui = {x: i for i, x in enumerate(ue)}
vi = {x: i for i, x in enumerate(ve)}
# Exact integer 2D difference array. Scores are constant in every open
# event cell. NumPy performs only integer arithmetic here.
diff = np.zeros((len(ue), len(ve)), dtype=np.int64)
for u1, u2, v1, v2, w in rects:
a, b = ui[u1], ui[u2]
p, q = vi[v1], vi[v2]
diff[a, p] += w
diff[b, p] -= w
diff[a, q] -= w
diff[b, q] += w
scores = diff.cumsum(axis=0).cumsum(axis=1)
nu, nv = len(ue) - 1, len(ve) - 1
best = 10**18
for i in range(nu):
u0, u1 = ue[i], ue[i + 1]
if u1 <= u_dom_min or u0 >= u_dom_max:
continue
slab = clip_u(dom, u0, True)
slab = clip_u(slab, u1, False)
if not slab:
continue
vlo = min(v for _, v in slab)
vhi = max(v for _, v in slab)
if vhi <= vlo:
continue
# This may examine a superset of feasible event cells, which is
# conservative for a lower-bound verification.
j0 = max(0, bisect_right(ve, vlo) - 1)
j1 = min(nv - 1, bisect_left(ve, vhi) - 1)
if j0 <= j1:
row_min = int(scores[i, j0:j1 + 1].min())
best = min(best, row_min)
if best == 10**18:
raise RuntimeError("center domain was not enumerated")
return best
def angle_net() -> list[tuple[F, F]]:
out: list[tuple[F, F]] = []
for k in range(KMAX + 1):
t = T * k / KMAX
den = 1 + t * t
c = (1 - t * t) / den
s = 2 * t / den
assert c * c + s * s == 1
out.append((c, s))
# The final adjacent pair brackets pi/4.
assert out[-2][1] < out[-2][0]
assert out[-1][1] >= out[-1][0]
# If psi_k=2 arctan(t_k), half an adjacent angular gap is
# arctan(t_{k+1})-arctan(t_k), whose tangent is
# D/(1+t_k*t_{k+1}) <= D. Therefore every angle in [0,pi/4]
# is within an error epsilon < D of a net direction.
for k in range(KMAX):
t0 = T * k / KMAX
t1 = T * (k + 1) / KMAX
tan_half_gap = (t1 - t0) / (1 + t0 * t1)
assert tan_half_gap <= D
return out
def main() -> None:
atoms = build_atoms()
total = sum(w for _, _, w in atoms)
print(f"atoms = {len(atoms)}")
print(
f"total_weight = {total}/{WEIGHT_SCALE}"
f" = {total / WEIGHT_SCALE:.4f}"
)
#assert len(atoms) == 268
#assert total == 169476
assert total < 17 * WEIGHT_SCALE
net = angle_net()
# For an orientation error epsilon <= D,
# cos(epsilon)+sin(epsilon) <= 1+epsilon <= 1+D.
contain = B * (1 + D)
print(f"angle_net_size = {len(net)}")
print(f"b*(1+d) = {contain} = {float(contain):.12f} < 1")
assert contain < 1
global_min = 10**18
argmin = -1
for k, (c, s) in enumerate(net):
m = verify_orientation(c, s, atoms)
if m < global_min:
global_min, argmin = m, k
if k % 30 == 0 or k == KMAX:
print(
f"orientation {k:3d}/{KMAX}: "
f"min={m}/{WEIGHT_SCALE}, "
f"global={global_min}/{WEIGHT_SCALE}"
)
print(
f"minimum_score = {global_min}/{WEIGHT_SCALE}"
f" = {global_min / WEIGHT_SCALE:.4f} at k={argmin}"
)
assert global_min >= WEIGHT_SCALE
print("CERTIFICATE CONDITIONS VERIFIED.")
print(f"By the scaling argument: s(17) >= {L} = {L:.4f}.")
if __name__ == "__main__":
main()
Program to draw the images
#lang racket
(require racket/list)
(require metapict)
{define-syntax-rule (for/append clauses body ...)
; Todo: Add support for #:breack and #:final
(append* (for/list clauses (begin body ...)))}
{define (mirror-x N atoms)
(for/append ([a (in-list atoms)])
(match a
[(list x y w)
(list (list x y w) (list (- N 1 x) y w))]))}
{define (mirror-y N atoms)
(for/append ([a (in-list atoms)])
(match a
[(list x y w)
(list (list x y w) (list x (- N 1 y) w))]))}
{define (mirror-d atoms)
(for/append ([a (in-list atoms)])
(match a
[(list x y w)
(list (list x y w) (list y x w))]))}
{define-values (L-Green atoms-Green)
(let ()
; Todo: Confirm these are the correct lenghts.
(define grid-nx 6)
(define grid-ny 4)
(define border-size-y (- (sqrt 2) 1/2))
(define grid-size-y (/ (+ 12 (sqrt 8)) 17))
(define border-size-x 1)
(define L
(+ (* border-size-y 2) (* grid-size-y 3)))
(define grid-size-x (/ (- L (* border-size-x 2)) 5))
{define atoms/int '(#;()
(0 3 1) (1 3 1) (3 3 1) (5 3 1)
(0 2 1) (2 2 1) (4 2 1) (5 2 1)
(0 1 1) (1 1 1) (3 1 1) (5 1 1)
(0 0 1) (2 0 1) (4 0 1) (5 0 1))}
(define min-weight 1)
{define atoms (for/list ([a (in-list atoms/int)])
(match a
[(list x y w)
(list (+ border-size-x (* grid-size-x x))
(+ border-size-y (* grid-size-y y))
(/ w min-weight))]))}
(values L atoms))}
{define-values (L-Green/S atoms-Green/S)
(let ()
; Todo: Confirm these are the correct lenghts.
(define grid-nx 6)
(define grid-ny 4)
(define border-size-y (- (sqrt 2) 1/2))
(define grid-size-y (/ (+ 12 (sqrt 8)) 17))
(define border-size-x 1)
(define L
(+ (* border-size-y 2) (* grid-size-y (- grid-ny 1))))
(define grid-size-x (/ (- L (* border-size-x 2)) (- grid-nx 1)))
; It's easier to calculate the overlaps by hand
(define atoms/gen '(#;()
(0 1 2) (1 1 1) (2 1 1)
(0 0 2) (1 0 1) (2 0 1)))
(define min-weight 4)
(define atoms/int (remove-duplicates
(mirror-x grid-nx
(mirror-y grid-ny
atoms/gen))))
(define atoms/one-dir (for/list ([a (in-list atoms/int)])
(match a
[(list x y w)
(list (+ border-size-x (* grid-size-x x))
(+ border-size-y (* grid-size-y y))
(/ w min-weight))])))
(define atoms (mirror-d atoms/one-dir))
(values L atoms))}
{define-values (L-Burns atoms-Burns)
(let ()
(define grid-n 29)
(define L 44811/10000)
(define M 1)
(define grid-size (/ (- L M) grid-n))
(define border-size (/ M 2))
(define min-weight 10003)
{define atoms/gen '(#;()
(1 11 107) (2 4 137) (2 9 214) (2 11 107) (2 12 137)
(3 4 3884) (3 7 214) (3 8 913) (3 9 214)
(3 10 214) (3 11 1234) (3 12 2189) (3 14 384)
(4 4 1961) (4 7 520) (4 8 214) (4 9 1413) (4 10 1234)
(4 11 1083) (4 13 137) (4 14 292)
(7 11 529) (7 12 33) (8 10 906) (8 11 384) (8 12 351)
(9 9 340) (9 10 180) (9 11 204) (9 12 549)
(10 12 879) (10 13 201) (10 14 378)
(11 11 396) (11 12 622) (11 13 204) (11 14 204))}
(define atoms/int (remove-duplicates
(mirror-x grid-n
(mirror-y grid-n
(mirror-d
atoms/gen)))))
(define atoms (for/list ([a (in-list atoms/int)])
(match a
[(list x y w)
(list (+ border-size (* grid-size x))
(+ border-size (* grid-size y))
(/ w min-weight))])))
(values L atoms))}
{define-values (L-Massaccesi atoms-Massaccesi)
(let ()
(define grid-n 29)
(define L 45058/10000)
(define M 15513/10000)
(define grid-size (/ (- L M) grid-n))
(define border-size (/ M 2))
(define min-weight 576)
{define atoms/gen '(#;()
(0 2 165) (0 11 129) (1 8 36) (1 10 21) (1 11 15)
(2 2 246) (2 8 129) (2 9 105) (2 10 36) (2 11 105)
(5 10 36) (6 10 63) (6 11 12) (7 10 21)
(8 9 33) (8 11 15) (9 11 75) (9 14 39)
(10 11 25) (10 12 21) (10 13 24) (10 14 3) (11 11 16))}
(define atoms/int (remove-duplicates
(mirror-x grid-n
(mirror-y grid-n
(mirror-d
atoms/gen)))))
(define atoms (for/list ([a (in-list atoms/int)])
(match a
[(list x y w)
(list (+ border-size (* grid-size x))
(+ border-size (* grid-size y))
(/ w min-weight))])))
(values L atoms))}
{define (draw-example L atoms #:gamma [gamma 2.0] #:scale [scale 0.07])
[with-window (window -.1 (+ L .1) -.1 (+ L .1))
(define big-fill-color "whitesmoke")
(define big-border-color "black")
(define dots-color (change-alpha "darkred" 0.75))
(define big-square (curve (pt 0 0) -- (pt 0 L) -- (pt L L) -- (pt L 0) -- cycle))
(draw (color big-fill-color (fill big-square))
(penscale .1 (color big-border-color (draw big-square)))
(draw* (for/list ([a (in-list atoms)])
(match a
[(list x y w)
(define s (* (sqrt (expt w (/ 1. gamma))) scale))
(penstyle 'transparent (color dots-color (filldraw (circle (pt x y) s))))]))))
]}
(scale 4 (draw-example L-Green atoms-Green #:gamma 2.0 #:scale .07))
(scale 4 (draw-example L-Green/S atoms-Green/S #:gamma 2.0 #:scale .07))
(scale 4 (draw-example L-Burns atoms-Burns #:gamma 2.0 #:scale .07))
(scale 4 (draw-example L-Massaccesi atoms-Massaccesi #:gamma 2.0 #:scale .07))
Show HN: Proliferate- open-source, self-hostable Codex for any coding agent
Run Claude Code, Codex, OpenCode, Grok, and any other coding agent in parallel, in one workspace.
Each task gets an isolated git worktree for its branch, terminal, conversation, and review state.
🔀
Parallel agents
- run agents side by side in the same workspace, each on its own task
🪆
Subagents
- agents delegate scoped work to child agents and pick the results back up when they finish
🧩
Integrations
- MCPs, skills, Computer Use, Browser Use, and custom tools, configured once and shared by every agent
⏰
Workflows
- recurring and event-driven agent runs: nightly review passes, triage on alerts, dependency bumps
Supported agents
Proliferate runs each agent through its native harness.
Claude
Codex
OpenCode
Cursor
Grok
Self-hosting
The full Proliferate control plane is self-hostable. Start with the
deployment docs
, which cover Docker,
AWS, GCP, Azure, Kubernetes, and air-gapped operation.
Docker Compose:
self-hosted-deploy.md
runs Caddy, Postgres, and the API, with bootstrap and update scripts
AWS (one-click):
self-hosted-aws.md
is a CloudFormation wrapper that provisions the stack on EC2
Run the desktop app with the bundled local AnyHarness runtime:
make install
make dev-local
Local full-stack development
additionally requires Python 3.12+,
uv
, and
Docker for the local control plane database. Use named dev profiles when
multiple worktrees run at the same time.
make server-install
make setup PROFILE=main
make build # first clean worktree, or after generated/Rust/frontend artifacts change
make dev-list
make run PROFILE=main
See
dev profiles
for profile state, ports,
generated Tauri config, and app labels.
tl;dr: It worked! From one prompt it created a repo, wrote the application and tests, got CI green, provisioned
Postgres and deployed the finished app behind HTTPS without another message from me.If you just wanna see the outcome
you can find a demo video at
the bottom
LLMs got fun again! Maybe they always were and I was just stuck in the trough of disillusionment. Lately, whenever I
need a little tool, I just build it.
I was in the gym the other day and wanted a weights tracker. The app I had in mind was about as CRUD-y as it gets, but
all the app store versions wanted £12 per month, so I just one-shotted one with Claude. Great fun, but giving an LLM
root access to my machine in auto mode still doesn’t sit right with me.
So, the challenge: how can I create a fully remote agentic development environment where we structurally contain the LLM
rather than just trusting it? I want to give it an instruction and have it autonomously move through the
whole SDLC:
Researching the right stack and packages to use.
Planning and writing the code and tests.
Committing to Git, building and running a CI pipeline.
Deploying the work to a ‘production’ server with databases, o11y, and a domain with SSL.
All on my home server, without another cloud infrastructure bill. The only ongoing cost specific to this experiment is
a £20 Codex sub.
The Server(s)
Here they are in all their glory.
The one at the bottom is a 2014 dual-core i3 I’ve been running as a homelab for five years. It’s valiantly hosting this
blog and about 45 other Docker containers, from Pi-hole to a full Prometheus / Loki / Grafana stack. It also has port
443 forwarded from my router. I’d be miffed if an LLM broke it, so that’s not what we’re using today.
The top one is a 2021 10th-gen i7 with 32GB RAM, bought fresh from eBay with nothing on it. Perfect.
The Stack
The core development stack is self-hosted through Coolify. Inference and integrations like Tailscale, Telegram, DNS and
ACME still leave the box. You could host inference too, but I don’t have the hardware and I’d rather OpenAI subsidise
my experiments.
Component
Notes
Pi-hole
Local DNS rules, with the side benefit of seeing less shitty advertising.
Tailscale
Makes my home network follow me around.
Coolify
A self-hosted, Heroku-style PaaS built on Docker.
Forgejo (with runners)
Self-hosted Git and CI.
Hermes (with WebUI)
An OpenClaw-style virtual assistant, using Codex for inference.
Telegram
Talk to the agent from the toilet or wherever.
Firecrawl (self-hosted)
A scraping / translation layer between the agent and the web.
Porkbun (Registrar) & Let’s Encrypt
A domain and on-the-fly SSL certificates.
Whatever else
Postgres, Redis, whatever your apps need. It’s just Docker under the hood, innit?
Sources
This isn’t a full how-to guide. I could probably write an Ansible one-shot script to set it all up; leave an issue on
the GitHub repo below if you’d like one. If you’ve read this far, though, you can probably figure it out.
Networking
The first guardrail is obvious: it’s on its own metal. Hermes could
rm -rf /
and at worst it would cost me a couple
of hours rebuilding it.
The next layer of bombproofing is the network. My older server has port 443 forwarded from the router; this one doesn’t.
There’s no external ingress, cutting out a huge attack surface and all the internet background radiation from people
speculatively probing
/wp-admin
on every DNS A record I set up.
But, if there’s no ingress, how do I:
Get access to all our cool new apps on my phone?
Generate an SSL cert at a vanity URL so I can access
https://cool-new-app.internal.jakeshomelab.me
?
I have Tailscale set up with my older server as an exit node. When I’m away from home, selecting it routes my traffic
through that server and Pi-hole, which I’m using for custom DNS. Pi-hole lets you add dnsmasq rules like this:
address=/internal.jakeshomelab.me/192.168.1.201
Anything requesting
*.internal.jakeshomelab.me
now resolves to my new server, where Coolify’s reverse proxy picks it
up and serves my shiny new services.
SSL Certs
With Caddy or Traefik and Docker labels, you can serve port 3000 on container X from
https://my-service.internal.jakeshomelab.me
. Point an A record at the server and it’ll contact Let’s Encrypt,
complete an ACME challenge and get an SSL cert. I learned this three years ago and it still seems like magic.
The problem is the A record. I don’t want to publicly associate
my-service.internal.jakeshomelab.me
with my IP,
whether people can access it or not. I want an SSL cert for a ghost service.
To solve this problem, I turned to DNS-01. I’ll be honest this is new to me, but here’s how it works:
Buy a domain (in this case from Porkbun).
Generate Porkbun API keys and add them to Coolify’s environment with write access to the domain.
Modify Coolify’s Docker Compose file to use
lego
and the Porkbun API:
Then, when I register a new URL, Traefik / Coolify:
Uses the Porkbun API to create a new TXT record at
_acme-challenge.my-service.internal.jakeshomelab.me
.
Let’s Encrypt validates the challenge and issues a valid SSL cert.
Traefik deletes it.
That’s it! You now have a valid HTTPS URL, reachable within your tailnet, with no public A or AAAA record pointing to
the service. The hostname may still appear in public certificate-transparency logs, but the service is only reachable
from the tailnet.
The best bit is that Coolify does this on the fly. Our agent can create a service at any subdomain and it’ll
✨magically✨ sort itself out.
So, glue all this together and you get the following:
The same setup covers the tooling, so Coolify, Hermes, Forgejo and Firecrawl all live on their own local subdomains.
Development Stack & MCPs
Now we have an isolated(ish) box, let’s move on to the tooling. The tools are well known; gluing them together is the
fun part.
Forgejo
We need somewhere durable to store code and run CI. I decided not to use GitHub because:
Giving the box my GitHub token rather undermines the isolation. Also, it’s not self-hosted.
Its API and CI minute limits won’t work at the scale of our new software factory.
It’s down most of the time these days anyway.
Forgejo is a great self-hosted alternative. The Docker Compose file linked above sets up Forgejo and its runners;
registering yourself and the runner takes a little extra work, but it’s well documented.
I’ve also included a Compose file for syncing projects back to GitHub. That puts your GH token in the environment, but
the trade-off is yours to make.
The Forgejo Hermes skill linked above gives the agent full control of the instance.
Hermes
Hermes is an OpenClaw-style personal assistant with agentic capabilities. I never got in on the OpenClaw hype, so I
can’t compare the two, but Hermes has a few features I’ve found handy:
Web UI
: A standard ChatGPT-esque interface for working from my laptop and managing skills.
Shared filesystem
: I’ve mounted its workspace from the Docker host and shared it over Samba. The agent and I can
use the same files instead of copy-pasting Markdown and code around.
Telegram integration
: I can chat to the agent from my phone. Setup took two minutes and required no login details,
which suited the sandbox approach.
Self-building skills
: Hermes can create and register its own skills. I couldn’t find a good Coolify one, so it
read the docs, looked at the MCP and built one.
Firecrawl
: Self-hosted Firecrawl gives the agent much nicer access to SERP data and web scraping at scale.
Getting Hermes and Firecrawl set up with the right keys in the right places is a massive pain in the arse. I’ve added
Coolify-friendly Docker Compose files to the repo linked above.
Coolify
Coolify is the glue holding this together: a self-hosted PaaS built on Docker and Compose that comes on in leaps and
bounds with every update. If you want Heroku or DigitalOcean App Platform niceties on your own hardware, I’d highly
recommend it.
Some of my favourite features are:
It’s just Docker under the hood. Existing deployments mostly work, and if Coolify won’t do something weird you can
docker exec <whatever>
from your laptop. Things are only abstracted away if you want them to be.
The SSL / routing stack which I’ve gone into in depth above.
Coolify ships with a bunch of pre-made recipes for all the most common apps. Postgres, Redis, Hermes, Forgejo and
almost anything else is available to deploy with a single click.
Postgres backups to S3 are a three-click job, and env vars and user management are built in.
GitHub webhooks give you automatic deploys on push to main.
Here are a couple of screenshots of my Coolify setup in action:
What it actually did
The demo below shows this pretty well, but the starting gun was the following prompt:
Please build me an app for tracking my calorie intake. It should be similar to MyFitnessPal but with a form to
add specific food and meals for quick selection later.
Your task is to build it, commit it to a new repo with tests, test it with CI, and deploy it to
http://calories.internal.jakeshomelab.me.
I’d like it to be a full stack svelte kit app with Drizzle and Postgres for the database layer.
I’d like tailwind for the CSS. It should be mobile first.
For deployment, please use docker and docker compose and deploy your own Postgres instance.
From there, it just got on with it:
Created a new Git repo and bootstrapped SvelteKit, Drizzle, Postgres and Tailwind.
Wrote the app and its tests, committing the work in sensible stages.
Created a CI pipeline.
Worked through test failures until CI turned green.
Containerised the app and its own Postgres instance with Docker Compose.
Deployed the lot to Coolify at its own URL.
All without a single further prompt. No nudging it through failed tests or copying error messages back into the chat.
It just kept going until the app was running.
At that point I gave it a whirl and hit a CSRF issue when submitting data. I sent one more prompt; it diagnosed the
problem, fixed it, added regression tests and redeployed.
And it worked!
That’s the loop I wanted: prompt, repo, code, tests, CI, deployment, bug fix. It’s not a complicated app, obviously,
but it went from a paragraph to tested, deployed software and handled all the boring bits in between. That still feels
a bit like witchcraft.
Enough of all that, show me the goods!
I’m no YouTuber, but here you go:
Thoughts on isolation and next steps
There is always a trade-off between fully agentic development and security. This was a fairly contrived example: the
app works completely in isolation. Most useful software talks to other software, which means handing over API keys,
and every key adds another little hole in the sandbox.
Even in this setup, Hermes can still:
Nuke the new server and everything running on it.
Delete repos, databases and deployments.
Leak or abuse any credentials I’ve given it.
Burn through inference tokens like its end-of-year review depends on it.
Make rando outbound requests and download whatever rubbish the internet hands it.
Poke anything else on my network that the firewall allows it to reach.
So no, it isn’t harmless. What I’ve done is make the machine sacrificial and sharply limit how much stuff I care about
is within reach. The failure mode is now “rebuild the eBay box and rotate a handful of keys”, rather than “discover an
LLM has enthusiastically reorganised my actual laptop”. That’s better I think, but it isn’t magic.
The obvious next steps are:
Put the box on its own VLAN and explicitly block access to the rest of my home network.
Scope every credential as narrowly as the provider allows, and rotate them regularly.
Automate backups and make rebuilding the whole box a one-shot job.
Coolify’s DB backup and my shared Docker compose mounts should make this relatively easy.
Require approval before it does anything genuinely public or difficult to undo.
At some point, though, enough approval gates turn your magical autonomous software factory back into a collection of
forms you
have to fill in. Finding the useful point between “needs me every five minutes” and “has the launch codes” is the next
experiment.
Cobalt is an open-source application platform for Kobo e-readers: a launcher, a signed App Store, a Rust SDK, and a runtime that keeps every app in its own unprivileged process.
Install it once over USB. Every app after that installs, updates and removes on the reader itself, over Wi-Fi. A reboot returns to the stock Kobo reader.
Every app is a static ARM binary running as its own unprivileged process on stock hardware. The App Store installs, updates and removes them over Wi-Fi, with signatures verified before anything launches.
arXiv papers and coding agents, on the panel.
These are photographs of the device, not simulator captures. The arXiv app reads the HTML rendering arXiv publishes for every paper since December 2023: abstracts, sections, math and result tables, paginated for the panel.
Apps
The apps.
Every screenshot below is a capture from a Kobo Clara BW. Store apps version independently of the platform; the rest ship with the platform install.
Locates the hall sensor behind the bezel and reports its changes.
The SDK
An app is one Rust file.
Implement
KoboApp
, describe screens declaratively, and the runtime handles layout, e-ink refresh planning, Back navigation and lifecycle.
Apps don't open device resources; they ask. Network, storage, audio, frontlight and Wi-Fi are capability-gated, and a refusal comes back as a value the app can handle.
Store reads a signed catalog from a fixed GitHub release. Each package holds one ARM executable and a signed canonical manifest. The runtime verifies the catalog, the package, the installed manifest and the binary before an app runs.
App releases are independent of platform releases: merging an app PR builds it for ARM, signs it, and updates the catalog. No Cobalt version bump, no reinstall. The app simply appears in Store.
The Cobalt platform itself also updates over Wi-Fi, through Settings, on a channel separate from the app catalog. The USB cable is only ever needed once.
Install and catalog transactions are recovery-safe; an interrupted update leaves the reader with the version it had.
Charge a Kobo Clara BW (N365)
and connect it over USB. Other models are refused, not guessed at.
Run the setup:
git clone https://github.com/BandarLabs/Cobalt.git
cd Cobalt
rustup target add armv7-unknown-linux-musleabihf
cargo run -p kobo-cli -- setup
Restart the reader
and open
Cobalt
from Kobo's menu.
Open Store.
Everything from here on arrives over Wi-Fi.
The complete walkthrough, including recovery steps, is in
docs/INSTALL.md
.
Contributing
Contribute an app.
App contributions are regular pull requests. If it runs on your device and the PR shows it running, it gets merged and published.
Build it.
Add the app as a workspace package under
apps/<app-id>/
and register it in
apps/catalog.json
.
Test it.
Add unit and layout tests, and run it in the browser and runtime simulators.
Run it on your own device.
A real Clara BW, not just the simulator.
Open a PR with a gif or photos of it running.
Once reviewed and merged, the publish workflow signs it and it appears in Store. No platform release needed.
Own a different Kobo model?
Porting
is welcome too; open an issue first so the device profile can be agreed. Full details in
docs/CONTRIBUTING_APPS.md
.
Safety
Device support and safety.
Cobalt does not replace Kobo's boot chain. Device writes are gated on an exact hardware and firmware match, and a reboot returns to the stock reader. The first installation does modify files on the user storage partition, and it is provided without warranty.
Only the Clara BW profile has been hardware-tested. Don't install on another model until it has a
reviewed, hardware-tested profile
. Cobalt is an independent project, not affiliated with Rakuten Kobo.
Our
previous post
followed a
vector-add kernel —
c[i] = a[i] + b[i]
, one thread per float — from
nvcc
down to the warps. We went into a lot of detail on how the kernel was launched,
but we also left a lot out.
This time, we’re going to address our omissions, and follow the path the
critical SASS instruction (a global load) takes through the hardware — in this
case, since it’s under my desk, an
RTX
4090
We do this kind of reverse engineering for performance reasons, at least
in principle (for a great rationale, see 'Why these details matter' in the
Citadel microbenchmarking
paper). For the
same work applied to more production-relevant GPUs, watch this space.
.
Little of the detail of this path is documented by NVIDIA, at least not to
the level that we’d like, so we’ll determine it by running timing
experiments on the hardware itself.
The CUDA kernel we are investigating has two lines in its function body:
__global__ void vadd(const float* a, const float* b, float* c, int n) { int i = blockIdx.x * blockDim.x + threadIdx.x; if (i < n) c[i] = a[i] + b[i];}
If you inspect the compiled SASS, you’ll see the instructions that power
those lines:
They serve to load the elements of the vector
b
The instructions are the same for
a
, we're following
b
.
from global memory into a
register, where they can be added to the elements of
a
to perform the kernel.
One
LDG.E
asks for four bytes in each of 32 lanes. Serving it takes four
32-byte sectors, one cache line, one address translation, a crossbar crossing,
one of thirty-six L2 slices, and, when it misses everywhere, an activate and
four column reads at a DRAM chip. It’s this journey of the instruction through
the hardware, and back, that we’ll try to follow.
To set the scene: our warp lives on one of the SM’s four
sub-partitions
, alongside eleven
other resident warps. Each cycle the sub-partition’s scheduler picks one warp
that is eligible, and issues its next instruction across the 32 lanes at once.
Our warp wins twice: once for the
IMAD.WIDE
, and a few cycles later (the
addresses now sitting in
R4
and
R5
) for the
LDG
.
Let’s start with the instruction.
LDG.E R4, [R4.64]
is a global load of 32
bits from the 64-bit
address
stored in registers
R4
and
R5
R5
appears because of the
.64
annotation: registers are
32
bits in
size.
, storing
the result in register
R4
. To load the data itself, we first must go get that
address from those registers.
One row of the register file holds
R4
for all 32 lanes at once
The reads are staged in an
operand collector
first. The staging is
there for instructions whose sources share a bank of the register file, since a
bank serves one read per cycle. There are two banks, picked by the low bit of
the register number, so an adjacent pair always spans both.
. Another
holds
R5
. The warp reads both entries, yielding 256 bytes read as 32 distinct
64-bit addresses, one address per lane.
What the register retrieve costs
The address read adds at most one cycle. A shared-memory load taking its
address from a register takes 24 cycles from issue to first use, and the same
load with the address as an immediate takes 23. (
LDG
can’t take an immediate).
With all of its addresses resolved, the instruction issues to the
load/store
unit (LSU). The LSU takes the instruction and its operand
addresses, does some address arithmetic (if necessary)
This unit can add immediate offsets (
[R4.64]
carries no offset to add),
and scope loads (
LDG
names the global window directly).
, and sends on the
opcode
(‘load these addresses’, in binary), a 32-bit mask of active lanes,
its computed addresses, and the number of the register the result belongs in.
The next destination is the
coalescer
.
Each
LDG.E
instruction in each lane asks for 4 bytes, but our next
destination, the
L1 cache
, is addressed in 32 byte
sectors
. The
coalescer’s job is to figure out the minimal number of L1 sectors it needs to
retrieve to service our 4-byte requests.
The coalescer figures out that it ought to emit 4 contiguous sector requests,
for the 128 bytes the warp has asked for
1
.
The request for four contiguous 32-byte sectors is sent onto the
L1 cache
.
The L1 cache’s unit of organization is still less granular: 128 byte
lines
.
Our 4 contiguous sectors represent the 4 parts of a single line, so a request
gets made to L1 for that cache line.
First, we have to determine whether that line is already in the cache. The
cache is divided into groups of slots called
sets
In technical terms, the L1 cache on the 4090 is 4 way set-associative.
Caches lie on a continuum between fully associative (any cache line can be
stored anywhere in the cache), and 'direct-mapped' (each cache line can be
stored in only one place).
, and a line’s address
determines which set it belongs to. A set on this card holds four slots
2
,
and each carries a
tag
identifying the line in it. The lookup compares
all four against the tag of the line it wants. The address it uses is the
virtual
Presumably so that we don't have to pay translation cost to hit L1.
address used in the program
3
. The set in which a line lands is
generated from the line’s virtual address by a hashing scheme
It's a complex parity scheme (see the appendix), not just some slice of
the bits, so that power of 2 strided accesses (think columns of a matrix,
tensor etc.) don't keep hitting the same sets and churn.
, which you can
reverse engineer
4
.
If one of the four tags matches and the sectors we want are in that slot, the
data is read out and the load is done
5
. Because we’re loading all of our
data for the first time, our request misses, and must descend further into the
memory system.
How much does an L1 hit cost
An L1 hit returns in about 15.4 ns — 40 cycles. The number comes from one
thread chasing a dependent chain through a random permutation of L1-resident
lines, with
the latency chase
.
Virtual memory
puts one level of indirection between the addresses a
program names and the addresses at which the hardware stores data. The
program gets a contiguous space of its own, and the hardware lays that space
out across physical pages however it likes.
Translation
is the map between
them.
The L1 we just spoke to was
virtually addressed
, so we didn’t need to
concern ourselves with translation. Past this point, we have to start speaking
the hardware’s language — an L1 miss has to be translated before it leaves the
SM
6
.
The actual mapping between physical and virtual addresses is established at
allocation in the driver: when
b
was allocated, the driver picked physical
(2MiB) pages for it and wrote page tables into VRAM recording the assignment
7
.
The translation unit takes in a virtual address and returns a physical address,
according to those tables. The SM keeps its sixteen most recent translations in
a TLB, shared across warps
8
. The very first load will miss in this TLB.
What translation costs
We can’t see any cost to hitting the TLB in any of the probes we have. Misses
cost about 4.4 ns — eleven cycles. The same refill cost holds within 0.1
ns across all the pages this chip can map, and from any SM, so the next level
of the translation cache is universal, and very cheap.
Once translation has been performed, what leaves is one request per 128-byte
line: now with the line’s
physical
address, along with a mask of the
sectors we want from it. Ours is a single request with all four sectors
marked
9
.
The request proceeds out of the SM, across the
crossbar
to the
L2
cache
.
The request runs across the crossbar to one of 36 2 MiB L2 slices, picked by a
somewhat complex function of its physical address
10
. Any SM can hit any
slice. All slices can serve in parallel, so the aggregate bandwidth is 36x that
of a single slice.
Inside a slice, the structure is of the same kind as the L1. Each slice holds
1024
sets
. The set to which a line belongs is picked by a hash of the line’s
physical address. Each set now contains 16 slots: the slices are individually
16 way set-associative
11
. The lines are 128 bytes in size, the same as
in L1.
The line is not present in L2, since we’ve not fetched it before
This is perhaps artistic license: loading the
b
vector across from host
memory over PCIe might have cached it in L2. But then we couldn't continue down
to DRAM!
. Each
slice falls through to one of 12 memory controllers — 3 slices per controller.
Each memory controller’s job is to speak to a single
GDDR6X
DRAM chip
12
. Our
request gets handed over to that controller.
What does this cost
An L2 hit costs about 127 ns — some 330 cycles. Each SM can hand the crossbar up to two
line-requests per cycle, and the 36 slices serve independently. The exit-port
counter is
l1tex__m_l1tex2xbar_req_cycles_active
.
The memory controller’s job is to load the data from its 2 GiB DRAM chip. It
does so by issuing commands to DRAM over a bus.
The DRAM is divided into two separate buses the controller drives independently,
called
channels
. On each channel sit 16
banks
: two-dimensional arrays
of memory cells. A bank consists of 65,536
rows
. The hardware can open one
row at a time (an
activate
, expensive), and then return any 32-byte
columns
from that row (a
read
, cheap while the row is open).
GDDR6X
The address is taken apart one last time, to match this memory structure. It
picks out a channel, a bank, a row, and a column. Our four sectors are
four columns of one row
13
.
So, to serve our load, the memory controller must first send one
activate
,
and then four
read
s
14
.
What does a DRAM chip do in response to those commands?
Each DRAM cell is one capacitor behind one transistor. The transistors of a
row share a
wordline
, attached to their gates. Each transistor sits
between its capacitor and a
bitline
, which runs along a column, providing a path
from each cell (shared with the cells of other rows) to the
sense
amplifiers
. Bits are stored in the charge state of the capacitor. The
capacitors constantly leak charge, so the chip has to pause each bank now and
then to top them up.
The structure of DRAM. Click a row to act as the row decoder, releasing
charge from the capacitors onto the bitline and into the row buffer.
The activate command triggers the
row decoder
to drive that row’s wordline,
opening the row’s transistors and driving the charge from the capacitors in
that row (and only that row) through the bitline into the sense amplifiers,
which amplify that charge into full-rail bits and hold them for the controller
to read.
When the read is issued, its column address picks out 256 of these row
bits. Reading from the sense amplifiers gives us very many bits at once, but we
need to serialize them onto the pins that drive data back across the bus. There
are 16 data pins per channel. The 256 bits of our read leave on these pins as
PAM4
GDDR6X is the GDDR6 standard, with this PAM4 signalling added.
symbols: each symbol is one of four voltage levels, carrying two
bits, so 256 bits over 16 pins is 16 bits per pin — eight symbols. The clock is
sent along a shared wire so that the controller can sample at the right edges.
These PAM4 bursts are deserialized in the memory controller, and written into
the L2 slice’s line. The results run back through the crossbar, back to their
SM, and fill their L1 slot. They rendezvous with the record left by their
leaving, and their bytes are written into register
R4
across all the lanes.
When the load was issued, a dependency barrier was set, which this register
write clears. The warp becomes eligible again, and on the scheduler’s next cycle
it wins the arbitration. The instruction it issues is the add that was waiting
on
b[i]
.
The round trip — L1, TLB, crossbar, L2, controller, and back —
costs about 255 ns, some 660 cycles. All the while our warp was parked on its
barrier. The rest of the chip wasn’t idle though. The sub-partition issued the
same loads for another 11 warps, the rest of the SM for another 36, the other
SMs for the other 6096. The result is a cacophony of loads, the per-load
latency of any one of them lost in the noise. Here’s what that looks like:
A timing-proportional simulation of the execution of only the instructions
in the
vadd
kernel that correspond to the load of
b
. Each SM loads only
those addresses it loads in the real kernel: those addresses light up (and
miss) in the correct L1 set, then are routed through the crossbar to the
correct L2 slice, where they miss, falling through a
correctly contended memory controller to a simulated DRAM bank,
before returning back through L2, back through L1, and returning their results
into the correct register.
SMs (128), one pixel per L1 set
crossbar
L2 (36 slices, 3 per controller), one pixel per set
memory controllers (level is instantaneous throughput)
All measurements are on one RTX 4090 (
sm_89
), with the core clock locked at
2.6 GHz. Cycles come from measured nanoseconds at that frequency. Two main
instruments:
A latency chase.
To get a latency measurement (especially when that
latency changing tells you something about the chip), we run a pointer cycle
through a chosen set of lines, hopped 20,000 times, and then measure the mean
ns per hop. If the lines we point to fit in a cache level, then
they stay resident, and the mean is that level’s hit latency. Because of the
steepness of the hierarchy, any loads that overflow to the next level down tend
to show up strongly in the average.
ld.global.ca
(
LDG.E…STRONG.SM
) for chases at the L1,
ld.global.cg
(
LDG.E…STRONG.GPU
) goes past L1. Hit latencies are 15.4 ns at the L1, 127.4
ns at the L2, and 255.4 ns at DRAM.
Hardware counters.
To read
ncu
’s counters reliably
you have to take them as slopes over iteration count so fixed overhead cancels.
Sector and request counters at the L1 exit port and the L2 side are used to
figure out more about the shape of the requests, and a per-slice sector counter
helps to give us the L2 slice measurements.
The 16-entry TLB is only the first level, but what happens when you miss? A
miss refills in about 4.4 ns, and an L2 hit is 127 ns and a VRAM access is 255
ns, so we can’t be going from those. The inference is that it comes from some
larger on-chip translation cache.
The cost is flat within 0.1 ns for all the pages the chip can map, and from any
SM. More evidence: walking the page tables with
nvdebug
shows the
volatile bit set on every directory entry, so they’re not cached in the normal
hierarchy.
Measuring which slice owns a line is pretty hard. The L2 is physically
indexed, so the probe has to work in device-physical addresses from the page-table
walk. Nsight Compute does have a per-slice sector counter, but reports only the min,
max, average, and sum across the 36 instances, never the actual slice index.
Even so, the aggregate is enough to tell whether two addresses share a slice.
If the two addresses live on the same slice, after loading both, the max
counter reports 2, if they’re on different slices the max is 1. You can use
this probe to get a representative address that lands on each of the 36 slices.
With the 36 representatives in hand, you can get any new candidate’s slice. If
you read the candidate many times alongside all 36, with each of the
different addresses read a distinct number of times (say 20001, 20002, …
times), the sum of the candidate’s read count and only one of the
representatives will match the max counter, and you can figure the slice by
inference.
From that, you can produce a table of many physical address-slice pairs. The
hard part is going from such a table to a physically plausible function. One
tool that helped us a bit was running the same kinds of experiments on two
different chips built on the same die: the 4090, and the L40S, which has an extra
slice per memory controller.
Here’s one Claude made earlier
It's hard to be sure what's actually in the hardware here, but this is
plausible given my limited knowledge. The priors: there's got to be some
shared silicon between the L40S and the 4090 (assuming NVIDIA don't ship two
completely different functional paths for chips on the same layout but with
different amounts of L2 fused off). And the function has to be simple-ish in
hardware, i.e. XORs, arithmetic etc. are fair game, but if Claude tries to put
in a 4096 entry lookup table you tell it to go try harder.
:
SHIFT, OFFSET = (5, 0, 1), (1, 0, 0)def parity(x): return bin(x).count("1") & 1def _state(a, N): wide = (N == 48) # L40S: 4 slices/controller, and it reaches bit 35 b35 = (1 << 35) if wide else 0 # stage 1 — which of the 12 controllers: two parities and a mod-3 digit P1c = parity(a & 0x76A990400) # controller parity 1 (narrow; used on both chips) P1 = parity(a & (0x76A990400 ^ b35)) # wide form, only needed for the L40S read-out P2 = parity(a & 0x2CCF7B000) # controller parity 2 A = ((a >> 15) + 2*parity(a & 0x3C9041000) + parity(a & (0x2882B0800 ^ b35)) + 2) % 3 # mod-3 digit: (a>>15) + 2 corrections # stage 2 — which slice inside the controller: a 9-position cyclic counter g = ((a + (1 << 16)) >> 17) % 9 # the counter value, round(a / 2^17) mod 9 q0 = parity(a & 0x8000) # four correction parities q1 = parity(a & 0x5985E0500) q2 = parity(a & (0x2354E4400 ^ b35)) q3 = parity(a & 0x3C9041000) carry = 1 if q0 + q1 + q2 >= 2 else 0 # q0,q1,q2 as a full adder: the carry (majority)... start = (5 + 7*q0 + 5*q1 + 2*q2 + q3 - carry) % 9 # ...sets where the counter starts o = (g - SHIFT[A] - start) % 9 # position within the 9-cycle Lf = 2 if (q0 ^ q1 ^ q2) == 0 else 1 # ...and their XOR sets where it splits return P1c, P1, P2, A, q2, o // 3, (1 if (o % 3) >= Lf else 0) # d = o // 3, u = the split bitdef slice_of(a, N=36): P1c, P1, P2, A, q2, d, u = _state(a, N) controller = (2*P1c + P2) * 3 + A # 0..11 if N == 36: # 4090: 3 slices live, read (d, u) as three arcs of Z/9 base = 2 if d == 0 else (1 if (d == 1 and u == 0) else 0) B = ((1 - base) % 3 if q2 else base) % 3 # q2 flips the arc order B = (B + OFFSET[A]) % 3 # per-controller offset return controller * 3 + B if N == 48: # L40S: 4 slices live, read u as two index bits i0, i1 = P1 ^ q2 ^ u, P1 ^ P2 ^ u return controller * 4 + 2*i0 + i1 raise ValueError("N must be 36 or 48")
Whilst it is very hard to find such a function, it’s very easy to tell if
you’ve found one that works. Drawing 8,192 L2-resident lines from exactly
k
predicted slices:
Once the slice function pins addresses to a single slice, you can do the same
eviction-set archaeology on that slice, to figure out the structure, which
tells you that it’s 16 way set-associative (a chase with 17 elements thrashes, but
one with 16 doesn’t).
The set index within a slice is the same kind of parity function as the L1’s —
ten bits, with the same
(a >> 15) mod 9
nonlinearity in the top bit.
Unfortunately, the masks involved differ depending on the slice. For one slice:
It has some properties that let you sense-check it. For example: a contiguous
72MiB fills each slot in each slice without thrashing anything, as you’d
expect.
DRAM cells leak charge and so have to be periodically refreshed, which makes
some kinds of timing probes harder. You can see it by running a dependent chase
that writes each hop’s timing into shared memory. Most DRAM accesses come back
at the usual latency, but a small share take longer, spread evenly out to a
hard ceiling about 210 ns higher than usual. An evenly spaced run like that is
the signature of a fixed length stall. The stall is ~210 ns. About 2% of
accesses hit one. It doesn’t hit the whole chip at once — it’s more local than
that — but I couldn’t tell what the unit was.
You can get some visibility here from the hardware counters. We set up a
one-warp kernel that loads 4 bytes per lane with a fixed stride. If you
run it under
ncu
, it reports ‘sectors per load’ as
l1tex__average_t_sectors_per_request_pipe_lsu_mem_global_op_ld.ratio
. At stride 1 the 32 lanes cover 128 contiguous bytes and the
counter reads four sectors. At every wider stride the count equals the
number of distinct 32-byte spans the lane addresses touch, with no extra
sectors requested.
↩
To count the slots in a set, we take a pool of candidate lines much larger
than the L1 cache and pointer-chase them in a cycle. By design, the pool
doesn’t fit in L1, so each line thrashes, and the latency stays > L1
latency. Then you progressively drop members, and watch the latency. If the
latency suddenly drops, you know that somewhere in your pool there is one
full set (since it doesn’t thrash). The goal is to find the minimal set
such that everything thrashes, where removing any address drops you to L1
access speeds. This is the standard process of finding
eviction
sets
.
↩
Caches can be indexed & tagged either virtually or physically. This L1 uses
the virtual address for both its index and its tag. You can see this by
mapping one physical allocation at two virtual addresses, in the minimal
conflict set we built to identify the number of ways. Swapping one member
of a minimal conflict set for the same physical line seen through the other
mapping breaks the conflict, so the index must be computed from virtual
bits (if it was physically addressed, they’d deduplicate). Adding the alias
back to a full set restores the conflict, so the alias occupies a slot of
its own and the tag is virtual too.
↩
Each of the eight index bits is the exclusive-or of a fixed subset of the
address bits. The masks defining these subsets are in
the
appendix
. Figuring out these masks takes two probes.
Inside one 2 MiB page, the differences between members of minimal conflict
sets fix the masks over address bits 7 to 20. Above the page, flipping one
high address bit and reading which set the line lands in gives that bit’s
contribution, for every bit from 21 to 32. You can check if you’ve got the
right function by using it to construct eviction sets manually (since you
know what addresses go in what sets).
↩
A slot can hold a line with only some of its sectors present. When a load
misses, the request sent on to the L2 names only the sectors the warp wants.
ncu
’s L2-side counters show the sector count per request tracking exactly
what the lanes touch, with no rounding up to the full line. The counters are
lts__t_requests_srcunit_tex_op_read
and its
t_sectors
counterpart.
↩
To figure out that the L2 is physically tagged and indexed: we map one
physical allocation at two virtual addresses, and a chase visits every line
in the allocation through both virtual indexes. If the L2 tagged lines by
virtual address, the aliased chase would occupy twice the footprint and
exceed the L2’s 72 MiB capacity edge at half the size (this is a tradeoff
for any virtually addressed cache: that you get no deduplication. Also
vulnerable to timing attacks w/ multitenancy, not a factor here).
Sweeping the physical footprint from 24 to 128 MiB, the control and the
aliased chase cross the L2 threshold at the same size, so translation
must happen before L2.
↩
You can read GPU page tables directly, using a tool like
nvdebug
that walks
the GPU’s page tables from the host. Every device allocator terminates in a
2 MiB page-table entry, with the 4 KiB table beside it invalid. That covers
cudaMalloc
,
cuMemAlloc
, the
virtual-memory API at either granularity, and managed memory. Pinned host
memory is the exception, with 4 KiB entries in the system aperture. The
walker is in
the appendix
. In the open kernel
modules, the mapping path is
dmaAllocMapping
,
which calls
dmaUpdateVASpace
and then
mmuWalkMap
to fill the entries. The walker allocates each page-table level the first time a mapping needs it.
↩
The TLB is fully associative, holds sixteen entries, is per-SM (shared by
the warps), and replaces the least recently used entry. You can find this
out with yet another pointer chase, this time, one line per 2 MiB page.
Sixteen pages cost a 127 ns baseline — the L2 hit latency, since the
chase bypasses L1 — but seventeen thrash.
Splitting the pages among the warps of one block gives the same step, so
the pool is per-SM and shared by its warps. Cyclic visits over seventeen
pages miss on every hop, which is the LRU pattern. The tables are in
the
appendix
.
↩
Same logic for figuring out the L1 request from the counters, only using
the L2 counters.
↩
In the function, two address parities and a mod-3 digit are used to pick
out the memory controller, which is shared between 4 slices in the full
AD102, but only 3 on the 4090, which fuses off one slice per controller.
Within those three, a mod-9 digit picks the specific slice. The card has
twelve controllers (a 384-bit bus, 12 × 32-bit). The function was recovered
by measurement; details in
the appendix
. You
can tell once you’ve got it right, because loading from a set of pointers
that share a slice is ~36x (the number of slices) slower than a load from
pointers that spread across all the slices.
↩
Once you figure out the function mapping specific addresses to specific
slices, you can do the cache archaeology in the same way we did it for L1,
using eviction sets, with the caveat that you can only use addresses that
map to a single slice. More in
the appendix
.
↩
The count of twelve controllers is public (it’s a 384-bit bus at 32 bits
per chip). The association of slices to controllers is read out of the
slice function. Three of its digits take twelve values, and the same three
digits appear unchanged on the L40S, which ships the same silicon with all
four slices per controller enabled. The factoring is in
the appendix
.
↩
You can measure the row size from timing. Because a DRAM chip is much
faster at serving loads that sit in the same row (since a pair of addresses
in different rows require closing the row buffer, + activating the new
one), if you assume that rows are contiguous, you can find row size by
sweeping. Offsets of 32, 64 and 96 bytes always stay within one row, so our
four sectors are four columns of one row. With the same instrument, you can
figure out the set of addresses that share a row: any difference in
,
,
,
and
preserves a row, so the row is
1 KiB, or 32 columns of 32 bytes.
↩
You can measure the cost of activating a new row by keeping many reads in
flight. When consecutive reads land in the same row, each extra read adds
about 3.4 ns. When each read opens a fresh row, it costs about 15x as
much. With one read in flight at a time the difference disappears, because
the row is closed again before the next read arrives.
↩
In which I meander about creative tasks and offer some advice on learning to write Rust code (or any other programming language really).
So I am fortunate enough to write a fair bit of code in the course of my
research. Over the past several years this has primarily taken the form of Rust,
a language I really like but that has a reputation for being difficult to pick
up. So how do you go about learning Rust, or any other creative skill for that
matter?
Like many people I suffer from a bad case of perfectionist brain gremlins, which
can make it very hard to get things done. They’ve a paralytic effect that can be
hard to manage, particularly in the early stages of things. Writing words, the
thing I am doing now, is a real pain point. The complicated, messy process of
taking ideas from the churning mass of thought-soup that is the conscious and
pinning them down with language is something I become easily frustrated by. This
frustration stems entirely from the recognition that what I’ve made, that what
sits in front of me, is not what I want it to be. Either it’s missing some
essential truth, feels self-indulgent, requires foreknowledge not contained
within the writing, or one of many other unavoidable imperfections of existence.
There’s no getting away from this. No matter what you’ve made, it can almost
always be improved in some way, even if you’re the only one who would notice.
And caving in to this desire to polish, to improve, is a fantastic way to get
nothing done at all, or to never even start. The most perfect version of a thing
is the imagined idea of it; and unfortunately that version will never and can
never actually exist in the world.
So here’s how I think about it: our brains are lazy—or efficient depending on
your perspective—and tend to be very use-it-or-lose-it about skills. And,
unfortunately, this means you have to do whatever it is you want to learn and
that you’re going to be bad at it, at first. And that, well that sucks, doesn’t
it? It would be nice if thinking hard about something was enough. But I suspect
that art would be a lot less compelling if that were the case.
Boiled down to its simplest elements, I think the act of creating something, be
it art or code, looks something like this:
Start with an idea fragment
Attempt to make it
Assess what was made
Improve until “good enough”
And in my experience, my failure mode looks like jumping right to assessing what
I’ve made after the smallest amount of creation. It’s easy to slip on the
editor’s hat before the author is done, because making things is, quite often,
challenging. And while good critique is hard, criticism is quite easy. This
looks something like
Start with an idea
Make a
tiny
bit
Spiral
Go browse the web instead
I’ll get to this project later. Totally.
But here’s the thing, it’s almost always the case that you don’t
really
understand what you’re making until after you’ve finished trying to make the
first version of it. I believe this is true for writing as much for code
development. And until you actually know the shape—not merely the vibe—of
what you’re making, criticism is largely paralytic and insidious in the way it
promises improvement while grinding everything to a halt.
Put another way, there are, broadly speaking, two layers at which you have to
understand a piece of work: High-level and Low-level. The former is the big
picture, the broad goal, the overall approach, while the latter is all the work
and mechanics needed to accomplish it and manage all the messy troubles of
reality.
1
This high-level view of the work is
necessarily incomplete
and should be revised based on understanding gained from the actual doing, i.e.
the low-level work. Recognizing this is how we make actual progress, rather than
becoming discouraged when the high-level understanding clashes with the
low-level work. In this way, the two layers form a loop: the doing improves the
high-level understanding and the high-level understanding guides the doing lest
it get lost in the weeds.
This is starting to sound an awful lot like self help. Gross.
Okay, brass tacks, how do you learn to program rust? Well, in short, I recommend
that you first read at least the beginning of the
Rust Book
and then as soon
as possible start writing some code. Make a tiny project to do whatever silly
thing strikes your fancy. You can try
rustlings
. You could even
follow
some
tutorials
.
There are a ton of
educational resources
out there,
and I’m certain some of them will work for your brain.
But remember: you should always actually write the code yourself. It is far too
easy to look at a piece of code and think “yeah I get it” without actually
understanding or retaining it. Incidentally, this is why I think LLMs are an
absolutely terrible tool for learning programming in general, though I will
accept that I trend pretty negative on the topic.
2
There’s a good chance that stuff you write won’t work correctly, won’t compile,
or will otherwise frustrate you at first. This is okay. You aren’t doing
anything wrong. You need to get down into the weeds and muck and really wrangle
with things to wrap your brain around them.
Unfortunately, you must do the
thing.
But fortunately with code you can run it and see if it works, no
subjective analysis required! Terrible code that works is still code that
works.
3
And often that’s the first step to good code that works. And while
technical debt is real, that’s generally not a concern when you’re first
learning a language.
Make something first, then worry about making it better.
The core of
nixpkgs-multiverse
, when you strip away the Nix API and the CLI, is an index. It is a map from
(attribute, version)
to the revision that shipped it as a JSON file.
1
1
There are actually a few other files that drive other features such as the statistics or
“fast mode”
, but they are all JSON as well.
$ls-lh index/
-rw-r--r--. 1 fmzakari fmzakari 7.5M Aug 19 13:57 history.json
-rw-r--r--. 1 fmzakari fmzakari 5.3M Aug 19 13:57 versions.json
As of
9cc0209
,
versions.json
is 5.3 MiB and
history.json
is 7.5MiB covering 305,492 package versions across 31,904 packages and 1,534 revisions.
The Nix API loads the JSON files lazily and are all read via
builtins.fromJSON
:
I would like to enrich the data with even more information however it comes at a cost: mo’data, mo’problems.
The goal of the project is to minimize the number of Nixpkgs that are downloaded. If we merely swap fetching huge Nixpkgs for huge JSON, it’s not a clear win.
For now we have to be judicious about what we store in the JSON files and think of clever encoding schemes to make the data small and compact.
If we were not constrained to the Nix
builtins
, we would leverage established technologies to efficiently encode our dataset that allow multiple query access patterns: databases!
Let’s say we were not restricted to JSON, do we have any other options?
Why are large JSON files so problematic?
builtins.fromJSON
is
eager
. There is no lazy JSON in Nix, no streaming parse (i.e. “just give me this one key”). The moment you touch the result you have parsed all 5.3 MB and materialised all 305,492 values on the Nix heap.
In the case of the multiverse, asking for one package costs the same as what asking for all of them.
Note
The lookup itself is not the problem. Nix attribute sets are a sorted array,
so access is a binary search, not a scan.
The cost is entirely in the JSON parse and in allocating the values and downloading a large file.
If we want to do alternate questions over the index, we have to make sure we keep the answers efficiently stored to better match
the access pattern.
What we want is obvious. We want a way to efficiently encode the data and a declarative way to define queries: we want SQLite!
2
2
nixpkgs-multiverse
already exports a SQLite database as a package to help others explore this data.
$sqlite3 index.db "SELECT version, rev FROM versions WHERE attr='hello'"2.10|728
...
0.01s, 4 MB
Nix
by default
cannot do this. Unfortunately there is no
builtins.sqlite
, although I think there should be…
Turns out though there are knobs we can touch or sources we can patch to get what we want anyways, albeit each one has a caveat. 😈
I was surprised I did not know about this
builtin
, and it has been around since
release 1.11.9
in April 2017. It is the ultimate escape hatch for a variety of use-cases when you simply can’t get them done with what’s available.
builtins.exec
takes a list of strings, runs the program, and
parses its stdout as a Nix expression
.
It is gated behind a setting that makes it clear it’s unsafe.
For integration, SQLite is perfectly capable of printing the Nix syntax. We never need a serialisation format in between as we make SQLite emit the attrset directly:
letversionsOf=attr:builtins.exec["${sqlite}/bin/sqlite3""-noheader""-separator""""./index.db"'' SELECT '{' || group_concat( '"' || version || '" = ' || COALESCE(CAST(rev AS TEXT), 'null') || ';', ' ') || '}' FROM versions WHERE attr = '${attr}'; ''];inversionsOf"hello"
The caveat is that every query is now a
fork
, an
exec
, a process image of SQLite, and a re-parse of the output through the Nix parser.
If you do not plan to execute many queries that overhead is likely acceptable given the simplicity of the integration.
From researching
builtins.exec
, I stumbled upon
builtins.importNative
. It takes a path to a shared object and a symbol name,
dlopen
s it, and calls that symbol. It landed in
1.8
, December 2014.
3
3
The C++ field was originally called
enableImportNative
and was renamed to
enableNativeCode
for
exec
.
The shared object must implement the following signature:
The implementation is ordinary C++ using the Nix API. Below is a snippet
of the implementation, making sure to cache our
sqlite3
handles to avoid
the same startup penalty as
builtins.exec
:
/* The whole point: the database handle outlives a single query, so the
b-tree pages we touch stay warm for the rest of the evaluation. */std::map<std::string,sqlite3*>handles;voidversions(EvalState&state,constPosIdxpos,Value**args,Value&v){std::stringpath(state.forceStringNoCtx(*args[0],pos,"..."));std::stringattr(state.forceStringNoCtx(*args[1],pos,"..."));// cached across callsauto*db=openOnce(state,pos,path);sqlite3_stmt*stmt=nullptr;sqlite3_prepare_v2(db,"SELECT version, rev ""FROM versions ""WHERE attr = ?1",-1,&stmt,nullptr);sqlite3_bind_text(stmt,1,attr.data(),attr.size(),SQLITE_TRANSIENT);/* ... collect rows ... *//* Build the attrset directly. No text ever exists. */autobindings=state.buildBindings(rows.size());for(auto&[version,rev]:rows){auto&slot=bindings.alloc(state.symbols.create(version));if(rev)slot.mkInt(*rev);elseslot.mkNull();}v.mkAttrs(bindings);}
This section was added after publishing based on an idea from
rickynils
.
Nix is often described as resembling JSON and there is a very easy translation from JSON to Nix.
What if instead of reading JSON we read the same contents but as a
.nix
file?
Theoretically it should have no parser boundary, no
fromJSON
, and no serialisation format at all.
The index becomes an expression the evaluator already knows how to read.
The idea would be to leverage Nix’s laziness. Nix attribute set values are thunks, so in principle you should be able
to
import
a very large expression, touch one attribute, and never pay for instantiating the rest.
Transforming the index is a dozen lines of Python, and produces something very similar to the JSON:
Determinate Systems
shipped another option
in March of 2026:
builtins.wasm
, which calls a function inside a WebAssembly module.
4
4
Eelco gave a talk about this at
SCALE 23x
.
The motivation was similar to wanting to extend Nix surface area but avoid expanding
builtins
. Wasm is sandboxed and deterministic, so unlike the two builtins above, the goal is to provide a
safe escape-hatch
.
WebAssembly is a binary instruction format for a stack-based virtual machine. The claim is that it is well suited for Nix because it has
deterministic execution
, which is a lot more restrained than a backdoor
builtins.exec
.
A module needs to export
memory
, an initialiser called
nix_wasm_init_v1
, and the entry point.
#![no_std]#![no_main]typeValueId=u32;#[panic_handler]fnpanic(_:&core::panic::PanicInfo)->!{core::arch::wasm32::unreachable()}// Host functions supplied by the Nix evaluator.#[link(wasm_import_module="env")]unsafeextern"C"{fnget_int(v:ValueId)->i64;fnmake_int(n:i64)->ValueId;}#[unsafe(no_mangle)]pubextern"C"fnnix_wasm_init_v1(){}fnfib(n:i64)->i64{ifn<=1{1}else{fib(n-1)+fib(n-2)}}#[unsafe(no_mangle)]pubextern"C"fnfib_entry(arg:ValueId)->ValueId{unsafe{make_int(fib(get_int(arg)))}}
Nixpkgs already includes the target for cross-compilation, so making one is pretty
straightforward:
You call back into the evaluator through the Nix API functions, so a wasm module builds real Nix values, similar to
builtins.importNative
minus the footgun.
Initial attempts to try and load a SQLite database with the traditional Nix
builtins
were a bit of a failure as Nix strings cannot contain NULL bytes.
$nix eval--impure--expr'builtins.stringLength (builtins.readFile ./index.db)'error: the contents of the file '/tmp/mvsql/index.db' cannot be represented as a Nix string
Thankfully, with the help of some additional due-diligence by LLMs, we discovered
that one of the Nix API functions is not in the blog post:
/**
* Read the contents of a file into Wasm memory. This is like calling
* `builtins.readFile`, except that it can handle binary files that
* cannot be represented as Nix strings.
*/uint32_tread_file(ValueIdpathId,uint32_tptr,uint32_tlen)
read_file
is
specifically
designed for this problem. This function allows a WASM module to pull arbitrary raw-bytes off disk into its memory.
Unfortunately, it’s a little
too broad
in that it reads
the complete file
which is kind of overkill and what we are trying to avoid from our
initial JSON solution.
In the pursuit of exploration, let’s
patch
the implementation and augment the API to allow random access and partial read of a file.
Turns out the patch to add is relatively small and straightforward.
/**
* Read a range of a file into Wasm memory, starting at `offset`
* and copying at most `len` bytes.
* Returns the number of bytes actually copied.
*/uint32_tread_file_range(ValueIdpathId,uint64_toffset,uint32_tptr,uint32_tlen){auto&pathValue=getValue(pathId);autopath=state.realisePath(noPos,pathValue);autobuf=memory().subspan(ptr,len);/* If this is a real file on disk, do a positional read*/if(autophysical=path.getPhysicalPath()){AutoCloseFDfd{open(physical->string().c_str(),O_RDONLY|O_CLOEXEC)};if(!fd)throwSysError("opening file '%s'",physical->string());auton=pread(fd.get(),buf.data(),len,offset);if(n<0)throwSysError("reading file '%s'",physical->string());returnn;}/* Otherwise fall back to materialising the whole file. */autocontents=path.readFile();if(offset>=contents.size())return0;auton=std::min<size_t>(len,contents.size()-offset);memcpy(buf.data(),contents.data()+offset,n);returnn;}
Now we have everything we need to hook up SQLite and a custom virtual filesystem (VFS) layer to read from the provided
/nix/store
path entry.
We build a WASM target of SQLite and we set
SQLITE_OS_OTHER=1
. That flag removes SQLite’s entire VFS layer and requires us to supply one.
We provide the build a simple implementation of the
xRead
API which is a call-back into the Nix evaluator via
that newly exposed
nix_read_file_range
function. Everything else is stubs.
staticconstsqlite3_io_methodsnixIoMethods={.iVersion=1,.xClose=nixClose,.xRead=nixRead,.xFileSize=nixFileSize,.xDeviceCharacteristics=nixDeviceCharacteristics,/* ... the rest are stubs ... */};staticintnixRead(sqlite3_file*f,void*buf,intamt,sqlite3_int64off){NixFile*p=(NixFile*)f;/* The one line that matters: SQLite's pager asks
for a page, and we ask the Nix evaluator for
exactly those bytes. */unsignedgot=nix_read_file_range(p->pathId,(unsignedlonglong)off,buf,(unsigned)amt);if(got<(unsigned)amt){memset((char*)buf+got,0,(unsigned)amt-got);returnSQLITE_IOERR_SHORT_READ;}returnSQLITE_OK;}
Note
Unfortunately
builtins.wasm
gives every call a
fresh instance
. This is deliberate from the implementation, meaning we pay some startup code each time although not quite as drastic as a
fork
&
exec
The
sqlite_nix
WASM module takes an attrset of
{ db, sql }
and returns one attrset per row.
5
5
The full
sqlite_nix.c
, the VFS, the build derivation and the Nix patch are all
in this gist
.
We can provide it any arbitrary SQL and now query our dataset!
# query.nixbuiltins.wasm{path=./sqlite_nix.wasm;}{db=./index.db;sql="SELECT version, rev FROM versions WHERE attr = 'hello' ORDER BY version";}
The benefit of SQL is that now we are not limited to the shape of the data in JSON.
# which packages have shipped the most versions?sql="SELECT attr, COUNT(*) AS versions FROM versions GROUP BY attr ORDER BY versions DESC LIMIT 3";# => [ { attr = "linux"; versions = 548; }# { attr = "linux_latest"; versions = 540; }# { attr = "freefall"; versions = 534; } ]
That is a real full SQLite
with all the bells and whistles: query planner, aggregates and subqueries, b-tree descent through an index, executing inside the Nix evaluator.
All through WebAssembly. 🤯
Every one of those answers is byte-identical to what the
sqlite3
CLI gives for the same query.
How do these compare? Here is every approach answering the same question: “which revisions shipped this package?” either against the same 22 MB SQLite build of the index or the
whole-file JSON/Nix equivalent.
As we initially complained,
fromJSON
is a flat line in the wrong place.
It is 0.29s whether you ask one question or two hundred, because the 5.3 MB parse happens once and dominates everything after it.
The giant
.nix
file is the same flat line, drawn higher.
It is roughly twice the time and 1.7× the memory of the JSON it replaced. Surprisngly, laziness never gets a chance to help: importing the file and touching
nothing at all
already costs 0.53s. The baseline cost is the parse, and the parse is eager as we well. Turns out parsing Nix expressions is even more expensive than JSON. Nix has run the file through its Bison grammar, build an AST for all 305,492 entries, and add every attribute name into the symbol table.
fromJSON
skips the AST entirely and goes straight from bytes to values, which is why the format with a “serialisation boundary” beats the one without.
builtins.exec
starts the cheapest and climbs
, roughly 3.8 ms per query of
fork
+
exec
+ Nix-parsing the output. It crosses
fromJSON
somewhere around eighty queries.
builtins.importNative
is flat and nearly free
, 0.05s across the whole range since we
reuse SQLite instantiations
across multiple invocations. The database is opened once for the entire evaluation and the pages stay warm.
Unfortunately,
SQLite in wasm is dominated by a fixed cost
, roughly
2.5 s
before the first query, then about
7 ms
each query thereafter.
That 2.5 s is Cranelift compiling 1.1 MB of SQLite. Right now that is a limitation of the WASM implementation however Eelco has mentioned that the generated code could be cached on disk in the future across invocations.
For a lock file pinning thirty packages,
fromJSON
still wins outright at the current index size.
None of these is right for shipping the multiverse index, and I am not going to make
nixpkgs-multiverse
depend on
allow-unsafe-native-code-during-evaluation
. Asking people to run their evaluator with native code loading enabled so my flake can be faster is not a worthwhile request
at the moment
.
For now, the index stays JSON and I’m holding back on some of the more loftier ideas I have that require
a lot more data
.
Although philosophically I only use
CppNix
, I was a little intrigued and impressed with what the ecosystem could unlock with WASM. There are definitely some warts however such as waiting for it to JIT and the developer-experience of maybe having checked-in compiled blobs but there is definitely potential to unlock a variety of problems.
Decayfmt – a file format that corrupts itself a little every time you open it
A file format that corrupts itself a little every time you open it.
Every open
permanently damages the file on disk, by an amount baked into the filename, before it is
ever shown to you. There is no recovery from the file alone. The file is the only copy
that matters, and every read destroys a little more of it.
Two file types:
.idcy<x>
for images (example:
photo.idcy3
)
.tdcy<x>
for text (example:
note.tdcy7
)
x
is a positive integer in the filename, the instability parameter. Higher
x
means
more corruption per open.
Watch it decay
The grid above is one image encoded at
x=1
,
x=3
,
x=8
, and
x=15
, each opened the
same number of times. Same picture, four rates of decay. To follow a single instability
value across individual opens instead, each open corrupting the file further on disk
before it is ever shown, with no way back:
The clean original:
Instability
After 1 open
After 3 opens
x=3
(gentle)
x=10
(severe)
At
x=3
the image degrades gracefully over many opens. At
x=10
it is nearly gone after
one open and pure noise after three.
x
is the dial between a slow fade and near-instant
destruction.
Text decays the same way. A sentence encoded at
x=1
(a slow burn), printed after a few
opens:
original : This sentence is dying, and every time you read it you kill it a little more.
open 1 : This sgntence is d+ingd !nd every time you re&p it P~u kiKl it a little more}
open 3 : This sgfxFn0e is d+ingd 3D6 every tibe you re&" it P~u kiKl it a 1ittl> m1re}
open 6 : TIbm sgf}Fn0e ts d+iqgd yD6 ev*ry tibe you re&" )t Pnu kiKB )t aC1it"l> m1^e}
open 9 : T/Sm sgf}Fk0- ts d|iqgd HD6 e@*rV tiFe you re&" )t Pnu kiKB )tpaC1it"lYMm1^>}
open 12 : h/Sm hgf}Nk0-'ts?K|iqgd HD6 e@`~V}t&Fe y%u re&" )2 Pnu kiKB )6UaC1it1lYMm1]b!
Corruption only ever swaps in printable characters, so text garbles into readable-looking
nonsense rather than binary noise.
What this is, and is not
decayfmt is a social contract enforced by math, not cryptography. It is not encryption,
not DRM, and not a secure deletion tool. The corruption is honest and unrecoverable from
the file alone, but anyone with a backup or a hex editor can defeat it. If you want the
original, keep a backup. If you do not want anyone to recover it, do not make one.
Install
With cargo
If you have a Rust toolchain, the quickest install is the published crate:
From a release
Download the binary for your platform from the
releases page
and put it on your PATH.
There is no runtime dependency to install.
On macOS the binary is unsigned, so the first run may be blocked by Gatekeeper. Right-click
it and choose Open, or clear the quarantine flag with
xattr -d com.apple.quarantine decayfmt
.
From source
Requires a Rust toolchain.
The binary is produced at
target/release/decayfmt
.
Quickstart
See it decay in your terminal, with no image or sample file needed:
echo "this sentence is about to start dying" > note.txt
decayfmt encode --input note.txt --output note.tdcy8
decayfmt open note.tdcy8
The instability
x
comes from the output name (
note.tdcy8
decays at
x=8
). Run that last
line a few more times and watch the sentence rot further on each open. The corruption is
written to disk before it prints, so there is no way back. A high
x
like 8 garbles it
fast; a low
x
like 1 is a slow burn over many opens.
On Windows PowerShell the
>
redirect writes UTF-16, which decayfmt refuses; create the
file with
Set-Content note.txt "this sentence is about to start dying"
instead. cmd.exe
and PowerShell 7 are fine with the line above.
Usage
Encode
Turn a source image or text file into a decayfmt file. Encoding never corrupts; the new
file is clean.
Both the file type and the instability
x
come from the output name:
idcy
for images and
tdcy
for text, followed by
x
as a positive integer (
photo.idcy3
is an image at
x=3
).
An output name that could never be opened is refused rather than written. Images are decoded
to raw RGBA; text must be valid UTF-8.
Open
Open a decayfmt file. This corrupts it in place on disk, then displays the result.
Images open in your system's default image viewer. Text prints to the terminal, and
when there is no terminal (for example when launched from a file manager) it also
opens in your default text editor.
decayfmt open photo.idcy3
decayfmt open note.tdcy7
x
is read from the filename, so renaming the file changes how hard the next open hits.
How the corruption works
On each open, a per-byte corruption probability is derived from
x
:
So
x = 1
corrupts roughly 9.5% of eligible bytes per open,
x = 5
roughly 39%, and
x = 10
roughly 63%. The randomness comes from a cryptographically secure generator
seeded from operating system entropy, never from a fixed seed, so two opens of the same
state look different and the corruption sequence cannot be replayed.
Images:
the red, green, and blue channels are each corrupted independently with
probability
p
. The alpha channel is never touched, so corruption shows as color
noise rather than transparency holes.
Text:
each byte is replaced, with probability
p
, by a random printable ASCII
byte. This operates on bytes, not characters, so at high
x
it can break UTF-8; the
viewer renders what it can and substitutes the replacement character for the rest.
Corruption substitutes bytes in place and never inserts or deletes, so the file length
and the positions of untouched bytes are preserved: content decays but structure does
not. The original byte length is always recoverable, and at low
x
word lengths and
layout largely survive. Spaces are not protected; they are replaced at the same rate as
any other byte and erode along with everything else as
x
rises.
The contract
Corruption is written to disk at open time, before display. A crash or kill after the
write does not undo it. Opening always costs a corruption.
A read-only file is refused with an error and never displayed. A free read would break
the contract.
The header is never changed after encoding. Only the payload decays.
There is no state in the file: no read counter, no timestamp, no record of who opened
it or when.
There is no recovery mechanism of any kind.
Limitations
This is a social contract, not cryptography. A backup defeats it entirely.
A determined person with a hex editor can tamper with the file.
It is not a secure deletion tool and makes no cryptographic guarantee.
Displaying a file writes the corrupted result to a temporary file for the system viewer.
The most recent one persists until the next open sweeps it, or indefinitely if there is
no next open, so a snapshot of the last-shown state stays recoverable until then.
Two opens running at the same time can race: both read the same starting state, and the
last write wins, so concurrent opens may cost fewer corruptions than sequential ones.
v1 supports images and text only. No audio, video, or other binary formats.
License
decayfmt is released under the MIT License. See
LICENSE
.
★ When New DF Posts Drop in a Forest and No One Is There to Read Them
It occurred to me last night that I’d gotten less feedback regarding recent posts than usual. There were a
few
items
I’d posted in recent days that I felt sure to hear from readers about, both yay and nay. But: nothing. Crickets chirping. I almost always hear from squeaky wheels in the EU when I write about Apple and the DMA, for example, but I heard nothing about
my take yesterday that Apple has effectively pantsed the European Commission
regarding App Store commissions.
I noticed this morning that the bot that auto-posts new articles to the
DF account Mastodon
hadn’t posted since Tuesday night. But it wasn’t the Mastodon posting bot that was broken. It was a different automated task that updates the RSS and JSON feeds. That’s what broke sometime between Tuesday night and Wednesday morning. The Mastodon posting bot reads the RSS feed, and if there’s nothing new in the RSS feed, there’s nothing for the bot to post. Still though, I thought it was weird that no one who
follows DF from the feeds
had emailed, texted, or @replied to me to complain that new articles on the website had stopped appearing in the feeds. No one.
Then, I remembered that the DF website home page is generated from ... the RSS feed.
1
So, yeah, pretty much no one but me realized that I’d written seven new posts after the last update Tuesday night. Oops. Needless to say, it no longer seems surprising at all that I haven’t heard anything from readers in a few days. My apologies for delivering most of this week’s output all at once. We can pretend today that DF is a weekly newsletter.
Stop Making TUIs
Simon Willison
simonwillison.net
2026-08-21 12:07:32
Stop Making TUIs
Thomas Ptacek advocates for building real native user interfaces for even the smallest of personal tools, because coding agents have reduced the cost of getting a usable-enough GUI up and running to almost nothing.
I wrote about my vibe-coded bandwidth and GPU monitoring macOS task ...
Stop Making TUIs
. Thomas Ptacek advocates for building real native user interfaces for even the smallest of personal tools, because coding agents have reduced the cost of getting a usable-enough GUI up and running to almost nothing.
I wrote about my vibe-coded bandwidth and GPU monitoring macOS task bar apps
back in March
, and I'm still using both of those on a daily basis.
I'm not habitually knocking out real UIs for my other projects yet, but I'm running out of excuses!
Thomas:
If you haven’t tried your hand at turning one of your 500 throwaway CLIs into a native app, you’re doing yourself a disservice. Go build a native UI. It’ll probably change the way you think.
It’s time to dream big. Omarchy Quattro has given people a chance to experience what the malleable computer of the future looks like, and they like it (a lot!). It now feels like a moral obligation to make this future more broadly available and fundamentally change how people relate to their computers for the first time in what seems like forever.
To do just that, I’m incorporating the Omacom Foundation to ensure that this mission is fully funded, durable, and ready to accelerate.
This nonprofit foundation will hold the trademarks, fund the infrastructure, promote the work, and support the open-source projects and developers Omarchy depends on.
These eight Founding Patrons are each contributing $1 million to this mission:
This is a ridiculous sum of money, so I intend to make sure it lasts a long time, and that we make the most of it. But just as important as the incredible cushion is the vote of confidence delivered by these pledges.
We’re going to make the prophecy of The Year of Linux on the Desktop come true. All the pieces are now in place. Time to go all in!
EFF and Civil Society Groups Call on Nottinghamshire Police to Halt Live Face Recognition
Electronic Frontier Foundation
www.eff.org
2026-08-21 12:03:17
This week, EFF, along with Big Brother Watch, Defend Digital Me, Liberty, Open Rights Group, Race Equality First, Statewatch, and Stopwatch, wrote to Nottinghamshire Police Force in the UK raising concern about the proposed roll-out of live facial recognition technology (LFR), and called for its imm...
This week, EFF, along with Big Brother Watch, Defend Digital Me, Liberty, Open Rights Group, Race Equality First, Statewatch, and Stopwatch,
wrote to Nottinghamshire Police Force
in the UK raising concern about the proposed roll-out of live facial recognition technology (LFR), and called for its immediate halt.
In particular, the
letter
highlights six concerns:
LFR Is Not "Just Another Tool"
Nottinghamshire Police has
stated
that “facial recognition is just another tool to fight crime.” But LFR used in public spaces is an incredibly intrusive biometric mass surveillance technology that scans the faces of everyone who walks past the camera and takes biometric face prints. This is not just another tool, but a major escalation of surveillance that treats everyone as a suspect by default.
People Having "Nothing to Worry About" Does Not Hold to Scrutiny
According
to
Nottinghamshire Police, “if you aren’t entering the city or county to commit crime then you have nothing to worry about.” However, many people have legitimate concerns about the normalisation of invasive technologies. So a public that cannot move around their towns and cities without being subjected to a biometric identity check may be less willing to seek medical care or legal advice, speak with journalists, act in a union, vote, protest, or express their gender, sexual or religious identity.
Disproportionate Targeting With LFR
We are particularly concerned to learn that Nottinghamshire Police could deploy LFR to tackle low level crimes, such as youth behavior deemed anti-social, as part of Operation View. Reporting suggests that the force
already possesses
“a watchlist of young people believed to be causing the most problems,” including children as young as 11 years old. It would be highly disproportionate to deploy live facial recognition to tackle this behaviour. Many of these children are reportedly known to the police, and it is highly likely that there are more proportionate means for locating them.
LFR Could Increase Social Problems
We are also concerned that Nottinghamshire Police has not adequately examined the distinct risks of using LFR to target children, including negative impacts on their behaviour and outcomes, risk of recidivism, and relationship with the police. Use of LFR could exacerbate behavioural problems in children and create an adversarial, rather than trusting, relationship with the police from a young age.
Lack of Public Support
Recent polling commissioned by Liberty
indicated
that 48% of people oppose scanning the faces of those walking on high streets when there is no suspected imminent threat. Furthermore, Opinium
found
that the majority of people oppose the use of facial recognition in schools. Likewise, a report by the London Policing Ethics Panel
found
that Londoners aged 16-24 were most likely to find the Metropolitan Police Service’s use of LFR unacceptable and most likely to stay away from events where LFR was in use.
On these grounds, Nottinghamshire Police must immediately halt their plans to use live facial recognition surveillance any further.
Learning to search was a key skill growing up as a teen in the 2000s. How to use accurate keywords, quotes, the entire lot of search operators. Google-fu was a hard earned skill (yes, I read a book) and helped me figure out a surprising amount of life.
How to Do Just About Everything on a Computer, 2000
Search these days though is a bit of a dumpster fire. It has shifted from just retrieval towards relevance and recommendation and it sucks. I’m not just talking about web search, which has oscillated between “help me find the link for this product” to “somewhat useful if you can ignore the SEO spammers” [1]. No, I’m lamenting how poor search has gotten inside emails, online marketplaces, and messaging apps. I’m lamenting that the user has progressively lost control over what constitutes a search.
My beloved AV receiver died a few weeks back. No biggie (even though it’s EOL), let’s see if we can find the same model being sold for parts so we can try and replace some board components.
I’m betting you’ll get inundated with listings of the same brand, not the model you’re looking for, even if you put in the exact model number in quotes. It’s not limited to obscure AV equipment. Searching for any item that’s not in oversupply means the quotes get basically ignored while the search branches into anything that might make sense.
Searching on YouTube means scrolling over two entire rows of Shorts,"People also watched", "Explore more", with your actual results somewhere in between [1]. Time related filters have gotten lost too: not being able to choose a date range (unless you manually enter date operators into the search field), not able to sort by date at all, or just videos from the last hour for breaking news.
Gmail is a whole other story. An exact invoice number or name pasted into the search bar in quotes sometimes returns nothing. Is it a genuine tech issue, or did Gmail quietly switch me back to “Most relevant” again?
And yes, I know. I can’t fathom the scale these services run at. Full text matching across billions of inboxes is computationally expensive. You’re indexing billions of emails and attachments, keeping those indexes fresh, and trying to return an answer in milliseconds. At some point, I can see someone looking at all of that and thinking: what if we just show people the results we think they want?
The en-shittification angle is maybe too cynical, too simplistic. Maybe this really is the natural progression of trying to make search better. At some point, relevance became more useful than literal matching for most people and the defaults followed the metrics. The problem is that there’s increasingly no mode for people who know exactly what they're looking for and want just that.
So what exactly is the point of writing this? I want the random PM looking for ideas to know that I’d be very happy to pay for better search on platforms that help me get what
I need
instead of optimizing a metric. Give me a “literal match only” toggle. Kill the fuzzy suggestions, ignore the semantic guessing, and just run a dumb, reliable grep across my data, boss.
[1] I got so annoyed I just spun up a version of my decluttering app
just for YouTube
.
If you've reached this far, thank you for reading! :)
I thought retiring in my mid 30s after a few exits would be fun but I've just been bored and a bit undersocialized without morning Slacks and emails to wake up to. If you’re building something interesting and could use an extra set of hands to ship, or just want to say hi,
feel free to reach out
. My inbox is open.
When I first learned about Unix and “Unix-like” operating systems, I was
intrigued. I had only known the colorful world of
Windows 3.1
so far.
Like
Japanese carpentry
, everything seemed to be carved out of one block with no apparent cracks (except that Japanese carpentry is rock solid, and the same cannot be said about Windows 3.1 with a straight face).
Imagine my surprise when I sat in front of a command-line prompt for the
first time. The blinking cursor dared me to enter
something
and there
was, at first, no obvious way to achieve any of the things I already
knew a computer could do. A formidable puzzle—I was hooked! Thanks to
a surprisingly well-stocked library the next village over, I learned
that there are different flavors or evolutionary cousins of Unix, and
that some of their behavior is codified by standards like
POSIX
.
I also learned about
GNU
and the
heroic efforts of the first waves of hackers who made all of this
software available to a world that seemed more interested in locking
down everything and preventing any tinkering. O brave new world!
But I persisted. I stared down the ever-blinking prompt and fed it. Many
moons later, after a detour with
FreeBSD
,
I remain an avid Linux user since it suits my working style: I like to
live dangerously, often deferring kernel updates right before
important deadlines—what a thrill—and generally being quite
optimistic about my ability to get myself out of any jam. What
I appreciate is that Linux lets me exercise my
self-efficacy
. In
essence, virtually all the pain I may experience by using it is, to
a large extent,
self-inflicted
. That feels so much better than having
to pray that the next
iOS
update
does not destroy my devices or some other nonsense.
This attitude is often met with blank stares or the usual “Anyway, …” by
people who just don’t get it, i.e., almost everyone else who is not
a huge nerd,
Neal
Stephenson
fan, or
blessed with an abundance of free time. Next to the nice tingling sense
of danger, the thing that entices me most about Linux is the
ability to
mold
it to my purposes like digital clay. Many of the
command-line tools I use have been around for quite some time now
1
but they
still
work admirably and allow me to do things like this:
2
In natural language, the purpose of this command is to extract and count
domain names like
github.com
from URLs that are assigned to variables
named
url
across all Python files in the current directory and its
subdirectories. If this reads like gibberish to you, my younger hothead
self, full of (neo)vim and vigor, would have hit you with the old “Linux is very
user-friendly; it’s just also super picky about its friends.” Yes,
younger me was adept at making enemies like a craftsman.
3
With the wisdom and mellowing of the years, I would now be diplomatic,
but it still strikes me as odd that this way of working with a computer
is so alien to many. Explaining my ancient workflow to someone who is
used to only modern GUIs is a bit like explaining higher dimensions to
someone inhabiting
Flatland
:
At best, they will politely listen before discarding what you said as
mildly odd and going back to their old ways. But again, I persisted and
stuck to my conviction that a computer should offer you
a general-purpose interface that enables
you
to build things you like.
GUIs can only partially sate that need since they need to guess what
path you are wont to take. By contrast, the Unix graybeards of yore
realized that it is futile to guess or railroad user behavior—instead,
they opted to equip everyone with a couple of smallish tools that adhere
to a
certain philosophy
:
Write programs that do one thing and do it well.
Write programs to work together.
Write programs to handle text streams, because that is a
universal
interface.
Decades later, this still works. Programs have become larger, more
complex, but also more convenient for highly-specific cases like video
editing—but at the core of many machines lies this wonderful interface
that offers nigh-limitless fun.
4
Instead of widening the gap between
the CLI dwarves and the GUI elves, however, something unexpected
happened, viz., the development of
large language models
.
Presenting at first nothing but an input box to the user, they
constituted a deliberate break in habits for many. Here, then, was no
GUI waiting for you to specify what type of picture you wanted to
create. The prompt was daring you to dream big. I imagine for some, it
must have been a bit shocking even—a program that does
not
tell you what to
do with it was unheard of.
5
And progress marched on, leaving the prompts of the early days
6
for
ever-refined queries in natural language. Now, instead of having to know
about
awk
,
grep
, and friends, one can just ask their favorite LLM:
I want to extract and count domain names like github.com from URLs
that are assigned to variables named url across all Python files in
the current directory and its subdirectories. How do I do this with
a set of shell commands?
The commands do more or less the same thing. My hand-crafted one with
rg
automatically ignores hidden directories, though, which is
typically what you want to do when searching code, but I did not provide
that
context
to the LLM. Moreover,
-P
will fail on operating systems
that use the BSD variant of
grep
. Again, the LLM lacks the context,
but this command will work when I copy and paste it into my terminal.
I could even use one of the CLI tools myself to make it directly execute
the command
for
me, with the LLM serving as a translator between
natural language commands and ancient Unix incantations.
In that sense, LLMs are embodying the Unix philosophy. Of course, this
analogy has holes so big you can easily ride a horse through. LLMs are
neither small nor do they do
one
thing—you could even argue that
some of the things they do, they certainly do not do
well
. These issues
notwithstanding, LLMs understand that text is
the
universal interface.
Instead of users needing to learn how to talk to the computer, the
computer now talks to you. A couple of years ago, this notion would have
seemed utterly optimistic. No one would have expected that “text and
tokenization” are the recipe for building general-purpose AI models. But
here we are, relying less and less on GUIs and instead going back to our
beloved Unix-like interface.
For all the problematic things around AI,
7
we may at least find some comfort in being
vindicated after so many decades: Text reigns supreme and Unix won.
Hundreds of leaked AWS keys give full control over corporate accounts
Bleeping Computer
www.bleepingcomputer.com
2026-08-21 11:55:15
More than 9,300 Amazon Web Services (AWS) access keys publicly exposed between August 2022 and August 2026 are still active and valid. [...]...
More than 9,300 Amazon Web Services (AWS) access keys publicly exposed between August 2022 and August 2026 are still active and valid.
Truffle Security has been tracking this exposure for the past four years and says that 817 of the exposed keys were linked to companies, 526 of them being AWS root keys.
According to the researchers, 242 of the keys are associated with Identity and Access Management (IAM) users with the AdministratorAccess policy. This role has full permissions to create, modify, delete, and view virtually all AWS services and resources within an account.
They note that each key of the 768 live keys in the two sets “full control of a company's AWS account.”
The company found 431,875 AWS secrets across code repositories, Git history, datasets, Docker images, registries, and CI logs and extracted 64,024 unique AWS keys that corresponded to 50,654 AWS accounts after removing duplicates.
However, the subset for which the researchers had complete credentials that could be used for re-verification was 10,616 keys, and 88% of them continued to authenticate as of August 10.
Amazon Web Services (AWS) is Amazon’s cloud-computing platform used by companies to host websites and applications, store data, run databases and servers, manage domains, and operate their online infrastructure.
Full control of a company’s AWS account could allow an attacker to access, exfiltrate, or wipe cloud-hosted data, take control of servers and applications, and create rogue admin accounts for persistent access
Threat actors could also use their access to deploy cryptominers, generating substantial charges for the company.
Truffle Security says
that only 262 of 2,754 readable accounts had a budget alert set up.
Hugging Face, a popular online platform where developers share AI models, datasets, and applications, was the largest single source of leaked AWS keys, accounting for 8,482 unique key exposures.
Also, 17.9% of those keys were root, meaning the highest-privileged identity, which isn’t restricted by IAM permissions.
Roles of exposed AWS keys
Source: Truffle Security
Truffle Security found that, for the 2,903 keys with available creation dates, the median age was 1,831 days (about five years), while the oldest had existed for 17.4 years.
Only 398 (13.7%) of those entries had a newer access key associated with the same user, suggesting most had never been rotated.
Age of exposed AWS keys
Source: Truffle Security
To defend against potential abuse, the researchers recommend deleting all root access keys, reviewing IAM credentials by age, rotating or revoking exposed keys, and configuring budget alerts.
Also, any credential committed to a public source should be treated as compromised.
Truffle Security said its testing was limited to read-only metadata, and that it has notified all identifiable owners of the exposed credentials.
Our Qwen3-TTS 1.7B CustomVoice implementation achieves
10 requests per second (RPS) and sub-50 ms p95 time-to-first-audio (TTFA)
while
maintaining real-time playback
on a single NVIDIA H100 SXM.
We compare five implementations: ours, vLLM-Omni, SGLang-Omni
△
, VoxServe, and M*, under Poisson open-loop traffic. After tuning each implementation for low-latency streaming, ours is the only one to achieve
sub-50 ms p95 TTFA
. We maintain
sub-50 ms p95 TTFA through 10 RPS
and keep it
below 100 ms even at 20 RPS
.
Our system produces approximately
630 characters per second
at 10 RPS. At
$4.29
per hour for a 1× H100 SXM instance, this translates to
~$2 per 1M characters
at full utilization
1
.
For comparison
, ElevenLabs V3 is $100 / 1M and Cartesia Sonic 3.5 is $49 / 1M at a higher
TTFA
.
Let’s start by discussing what a real-time TTS server needs to achieve. We think it’s a four-part problem:
Low Audible TTFA:
Time from request dispatch to the first audible sample must be low.
Zero underruns:
Once playback starts, the client must not run out of buffered audio.
Capacity:
1 and 2 must hold as RPS increases.
Non-malformed output:
Speech must be intelligible.
We choose
Qwen3-TTS CustomVoice
1.7B because it is one of the most popular TTS models with a permissive license.
Based on the above definition, we target
low p95 audible TTFA with zero underruns while maintaining high RPS on a single NVIDIA H100 SXM.
All benchmarks run for five minutes under Poisson open-loop traffic to approximate real workloads, following Fireworks AI’s LLM
benchmark
. Each engine receives the complete text in a single HTTP request, while audio output remains streamed. We detect audible TTFA, reconstruct playback from received PCM, and evaluate the completed audio using Deepgram STT.
How Do Other Engines Perform?
The table below shows the upstream/default result at 1 RPS for each engine. We only apply changes for compatibility in this run.
These defaults have substantial room for improvement. We tune each serving engine for its own latency, continuity, quality, and capacity requirements.
1. Remove leading silence
The first PCM returned by a model can contain tens of milliseconds of silence before the first sustained sound. This gap pushes audible TTFA back like so:
We add a dynamic trim. It detects sustained speech from short RMS windows, removes samples before onset, and streams the remaining audio normally. This change improves TTFA by ~80ms but does not make model inference itself faster.
2. Tune frame accumulation
We also tune how many codec frames are collected before decoding and releasing an audio chunk.
Smaller initial chunks reduce TTFA, but provide less playback headroom and create more frequent decoder work. Larger chunks are easier to batch and make continuous playback safer, but delay the first audible output. A useful configuration therefore starts with a small chunk and increases the chunk size for later output.
The exact knobs differ by engine: vLLM-Omni exposes settings such as
codec_chunk_frames
and
codec_chunk_ramp
; the other engines provide equivalent chunk or stride controls. We iterate over these values to find the config that best matches: low p95 TTFA, zero underruns and stable behavior as load increases.
Performance after tuning existing serving engines
The following table shows the selected no-underrun profile for each engine after leading-silence and frame-accumulation tuning.
VoxServe reaches sub-50 ms p95 TTFA at 1 RPS, while the other three engines do not. By around 6 RPS, every engine is at roughly 100 ms p95 TTFA or higher
2
.
How We Optimized Qwen3-TTS
We first need to understand Qwen3-TTS architecture. It is a 3-part model performing hierarchical multi-codebook generation. The
Talker
predicts the first codebook token for each audio frame, the
Code Predictor
generates the remaining 15 codebook tokens, and the causal
Codec
converts codebook tokens into waveform samples.
Each module has its own compute profile, batching behavior, and latency requirements. Rather than optimizing each module in isolation, we focus on a broader question: how should a serving system coordinate these heterogeneous tasks?
1. Bringing three modules under one scheduler
Most Qwen3-TTS serving implementations are split into two stages: the Talker and Code Predictor run together, while the Codec runs separately. This separation enables token generation and waveform decoding to overlap across requests.
We take this a step further. We expose the Talker, Code Predictor, and Codec as three independently schedulable tasks. The key is not merely splitting them into parts, but bringing all three onto a shared scheduling surface managed by one scheduler. This design draws inspiration from M* (
arXiv
).
With this setup, the scheduler can decide whether to run the Talker, advance the Code Predictor, or prioritize a Codec job that is approaching its playback deadline. It can also batch requests waiting for the same module. Instead of following a fixed execution order, we can rearrange work according to urgency.
Combining the Talker and Code Predictor may appear more efficient because it removes an intermediate boundary. However, the combined operation can become a non-preemptible unit of work that blocks more urgent Code Predictor or Codec jobs. Keeping the modules separate creates shorter units of work and gives the scheduler more opportunities to interleave requests.
2. Scheduling around the needs of speech streaming
Speech streaming has two distinct notions of urgency.
Before the first chunk of audio arrives, every millisecond increases TTFA, so we need to prioritize this path. But once playback begins, the goal changes: the next chunk only needs to arrive before the current audio finishes playing. Producing it earlier provides no user-visible benefit.
Thus, we give high priority to requests that have not produced their first audio, while established streams become urgent only as they approach a playback deadline.
Running every urgent request alone would destroy batching efficiency. Instead, our scheduler selects an urgent request as an anchor and fills the rest of the batch with compatible work. This helps the critical request meet its deadline while making effective use of the GPU.
This policy works especially well because all three modules share a scheduling surface, allowing the scheduler to choose both the request and the pipeline stage to advance.
3. Exploiting the regular structure of the Code Predictor
The Code Predictor is an autoregressive transformer, but its execution is unusually regular. It always performs a fixed number of steps (15) per frame to fill the remaining audio codebooks.
We exploit its fixed structure to preallocate its KV cache and capture the entire frame-generation loop as a single CUDA graph. We also use a Triton attention kernel specialized for its short, bounded context.
By replacing a host-driven sequence with a fixed GPU program, we lower latency and simplify the execution system.
4. Rebuilding the Codec around cached state
The Qwen3-TTS Codec is made up of Transformers and CNNs. Generating the next audio chunk depends on both the Transformer context and convolutional state from previous chunks.
A naive implementation reprocesses the full frame history on every update, repeatedly decoding old audio as the utterance grows.
To avoid this, we use a state-cache-based Codec. Each request retains the Transformer context and convolutional state needed by the next chunk. Incremental decoding then reuses this cached state and processes only newly arrived frames instead of replaying the full history.
Initializing the state cache from the first frame adds overhead and hurts TTFA. We therefore use full decoding for the first audio, then switch to state-cached incremental decoding for efficient sustained playback.
We similarly vary chunk sizes over the course of a request. Smaller chunks let playback begin quickly, while larger chunks improve batching and GPU efficiency during sustained playback.
5. Additional serving optimizations
We capture CUDA graphs for a predefined set of batch sizes. If a ready cohort exceeds the largest captured batch size, we split it across scheduling turns rather than falling back to eager mode.
We also avoid unnecessary CPU–GPU synchronization. For example, while EOS is suppressed, generation cannot terminate, so we defer the termination check until EOS is enabled. This lets the CPU prepare and submit subsequent work without waiting for the GPU.
Finally, we support input streaming for modular speech-to-speech systems. As an upstream LLM generates tokens, the TTS model can begin synthesizing speech before receiving the complete response, reducing end-to-end latency.
What’s Next?
Qwen3-TTS is just the beginning of our work on multimodal inference. We plan to extend our scope to image, video, and world models, as well as fine-tuning. Our ultimate vision is to simulate the world 1:1 through realtime multimodal inference.
We are a team of experts in multimodal AI research and infrastructure. Our open TTS model, Dia, has been downloaded over two million times and has ranked #1 on Hugging Face. Our team of ex-YC, ex-KRAFTON, and ex-NAVER engineers has published research at NeurIPS and ICLR and earned three IOI and ICPC World Finals gold medals. Nari Labs is backed by Y Combinator.
If you want to work with us on anything multimodal,
let’s chat
.
Why Mayor Mamdani Can't Quit NYPD Commissioner Tisch
hellgate
hellgatenyc.com
2026-08-21 11:40:57
The Eric Adams holdover is antithetical to much of the left. She's also essential to the mayor....
NYPD Commissioner Jessica Tisch and Mayor Zohran Mamdani brief the public ahead of the annual Israel Day Parade on May 28. (Ed Reed/Mayoral Photography Office)
The Empire State Building, the Chrysler Building and One Vanderbilt are seen among other buildings in midtown Manhattan in New York, Jan. 11, 2024.
Angela Weiss | Afp | Getty Images
A version of this article first appeared in the CNBC Property Play newsletter with Diana Olick. Property Play covers new and evolving opportunities for the real estate investor, from individuals to venture capitalists, private equity funds, family offices, institutional investors and large public companies.
Sign up
to receive future editions, straight to your inbox.
It should come as no surprise that the number of artificial intelligence-specific tech workers is growing rapidly, and the effect of this growth on regional office markets is substantial. For the first time, New York's office market is home to the most tech workers, thanks in large part to AI, according to a new report from
CBRE
.
New York's 394,300 tech talent jobs edged out the San Francisco Bay Area's 375,730 jobs, CBRE found. The report analyzes tech-specific workers in 75 metropolitan markets in the U.S. and Canada. It's the first time New York has taken the lead in the 13 years of this analysis.
"The story there is that there's been cuts in the Bay Area, so the tech industry has contracted the size of the tech talent workforce, and the finance sector [in New York] has hired a lot of tech talent and a lot of AI workers," said Colin Yasukochi, executive director of CBRE's Tech Insights Center in San Francisco.
For both the U.S. and Canada, AI tech roles grew by 45% in the past year, with San Francisco and New York each adding more than 20,000 AI-specific jobs since mid-2025, according to CBRE.
As of June, there were 751,000 AI-related workers across the two countries, the report found. Those include both new jobs and conversions from existing jobs. AI-related roles now account for nearly one-third of all tech-talent job listings in the U.S., per the findings.
By market, 37% of AI jobs in the U.S. are in the San Francisco Bay Area, New York, Seattle and Washington. While New York leads in overall tech talent, San Francisco still leads in AI, specifically.
In Canada, there is greater concentration of AI employment, with 60% of those jobs based in Toronto, Montreal and Vancouver.
Office leasing is rising accordingly in those markets where AI workers are most in demand.
Get Property Play directly to your inbox
CNBC's Property Play with Diana Olick covers new and evolving opportunities for the real estate investor, delivered weekly to your inbox.
In San Francisco, AI companies made up 58% of all leasing in the first half of this year and have accounted for 30% of leasing activity, totaling about 10 million square feet, since 2023, according to CBRE.
While overall tech drove the Bay Area's office market over the past few decades, the pandemic pushed many of those workers to remote jobs. AI, however, has a more office-centric culture and is now fueling the market's recovery.
"It's more of the sort of startup innovation culture that we've seen, where people are in the office [a] minimum of four, but usually like five or six days a week," said Yasukochi. "Through this whole innovation process, being together and working in person is just much more efficient and innovative."
In addition to San Francisco, AI leasing activity is concentrated most in Manhattan, Boston and Seattle, according to CBRE.
There was concern that AI would reduce head counts, and consequently the need for office space, but in the short term, at least, that has not been the case.
"It basically changes jobs and creates new jobs, more so than it eliminates," said Yasukochi, pointing specifically to the finance sector.
AI Boosted Homework Scores by 18% – Then Exam Scores Dropped 20%, Study Shows
A new study tracking 27,000 students in China has found that pupils who used artificial intelligence tools saw higher homework scores over time, but performed worse than their peers on exams taken without AI assistance, according to research covered by The Economist on August 18.
The Study
The research was conducted by David Stromberg of Stockholm University along with Victor Lei and Wu Yanhui of the University of Hong Kong. The study followed 27,000 pupils aged 12 to 18 in China, where adoption of AI tools among students has grown quickly. Around 80% of the students surveyed reported using AI models such as Doubao and DeepSeek, while the remaining 20%, who did not use such tools, formed a control group.
According to figures shared by The Economist, students who used AI saw their average homework scores rise by 18% across all subjects over a six-month period. However, when the same students were tested under exam conditions without access to AI tools, they scored 20% below classmates who had not used AI during the study period.
Context on AI Adoption Among Students
The study cited broader data on how widespread AI use has become among students. A survey conducted last year by ed-tech firm Chegg found that 80% of undergraduate students in wealthy countries reported using AI in their studies. More recent polling put the figure at 94% among students in Britain and 93% in Germany.
The Economist noted that teachers have reported grading formulaic, similar-sounding essays they suspect were generated by AI chatbots such as ChatGPT, but said that, prior to this study, robust evidence on AI’s actual effects on learning outcomes had been limited.
Related Research
A separate study conducted in 2024 at the University of Pennsylvania examined a similar dynamic on a smaller scale. Students attending a math lesson practiced problems using either traditional study methods, such as notes and textbooks, or AI tools including ChatGPT and an AI tutoring program. According to a summary of the research, students using AI performed better during short-term practice sessions, but the advantage did not carry over to a subsequent closed-book test.
Reactions
The Economist’s summary of the findings, shared on the social platform X, drew significant engagement, with some commenters attributing the exam score gap to students copying AI-generated answers without engaging deeply with the material. The study was also discussed on forums including Hacker News, where some users questioned aspects of the study’s design while others noted it aligned with existing concerns among educators.
A tip sheet published by the Brookings Institution earlier this year said AI can support learning when used intentionally and designed well, but cautioned that overreliance on the technology to replace thinking, social interaction, or creativity could prevent students from developing cognitive and social skills.
The Stockholm University and University of Hong Kong researchers’ full study had not been independently verified by other institutions at the time of publication.
Cancer-Related Mortality Among US Pilots and Flight Attendants
AgentSight is a zero-instrumentation AI Agent observability tool based on eBPF. It captures LLM API calls, Token consumption, and process behavior at the kernel level without modifying Agent code.
Overview
AgentSight provides full-stack observability for AI Agents running on Linux:
Capability
Description
Token consumption analysis
Multi-dimensional Token accounting by agent, task, and model
Behavior audit
Complete tracing of LLM calls and process execution
Dashboard visualization
Web UI for real-time Token trends, Agent health, and session traces
Agent auto-discovery
Automatic detection of running AI Agent processes
Interruption detection
Detection of LLM errors, SSE truncation, context overflow, and crashes
External log export
Supports exporting structured events to external log services
Prerequisites
Requirement
Minimum
OS
Linux
Kernel
>= 5.8 (BTF support required)
Privileges
root or CAP_BPF (for eBPF probes)
ANOLISA raw package
Linux x86_64, system mode
macOS
: On macOS, AgentSight provides two commands —
trace
(trajectory collector that scans local JSONL session files, no eBPF) and
serve
(Dashboard viewer). All other eBPF-dependent commands are Linux-only.
Installation
Install the published component with the ANOLISA CLI:
Use
make build-all
for source builds: it builds the Dashboard frontend, the main binary, and
agentsight-enforcer
in sequence. Running only
make build
skips the enforcer, and
serve
will keep logging
AgentSight enforcement unavailable
.
Quick Start
Use the systemd unit for a normal deployment. It runs eBPF tracing and the
Dashboard together and starts the enforcer dependency in the required order:
sudo systemctl enable --now agentsight.service
sudo systemctl status agentsight.service
Open
http://localhost:7396
after the service becomes active. Enabling the
main unit also keeps AgentSight available after a reboot.
The bundled systemd launcher binds the Dashboard to
0.0.0.0
. Restrict port
7396 with a firewall or security group before exposing the host to an
untrusted network.
The service runs as root with a private umask and stores data under
/var/log/sysak/.agentsight
. Use
sudo
for CLI queries and Dashboard access
commands that read this service-owned data.
For foreground troubleshooting, stop the systemd unit first so it does not
compete with a second tracer. Then use two terminals and run both commands as
root. The second command is not reached if both are entered sequentially
because
agentsight trace
stays in the foreground:
sudo systemctl stop agentsight.service
# Terminal 1
sudo agentsight trace
# Terminal 2: Start Dashboard
sudo agentsight serve
# Open http://localhost:7396 in browser# Print the Dashboard URL and token; open the URL as your desktop user
sudo agentsight dashboard --no-open
Requires root privileges. Captures SSL/TLS traffic, process events, and file operations.
Run
sudo systemctl stop agentsight.service
before starting a foreground tracer.
agentsight serve — Start API & Dashboard
# Default: bind to 127.0.0.1:7396
sudo agentsight serve
# Bind to all interfaces (remote access)
sudo agentsight serve --host 0.0.0.0 --port 7396
Run
serve
as the same user that runs
trace
so both commands resolve the
same data directory. Binding to
0.0.0.0
exposes the Dashboard on every
interface; restrict network access before using that form.
Dashboard Access & Authentication
Dashboard token authentication is enabled by default:
Localhost access
(loopback) bypasses authentication — just open
http://127.0.0.1:7396
.
Remote access
requires a token: append
?token=<TOKEN>
to the browser URL, or set the
Authorization: Bearer <TOKEN>
HTTP header.
The token is auto-generated on the first
serve
startup (64 hex characters) and persisted to the
.dashboard_token
file next to the database (default
/var/log/sysak/.agentsight/.dashboard_token
); it is reused across restarts.
Run
sudo agentsight dashboard --no-open
to print the service-owned access URL and token, then open the URL as your desktop user.
To disable authentication (only recommended on trusted internal networks), set in the config file:
{
"server": { "auth": { "enabled": false } }
}
After editing
/etc/agentsight/config.json
, run
sudo systemctl reload agentsight.service
to apply the change — no
restart
needed.
API Endpoint List
GET /api/docs
returns the full API route inventory (method, path, description) so scripts and integrations can discover endpoints; requests to unknown
/api/
paths also point to it in the 404 response.
curl http://127.0.0.1:7396/api/docs
agentsight dashboard — Show Dashboard Access Info
Displays the Dashboard URL and auth token, then tries to open a browser. On ECS instances it also prints a security-group configuration guide.
# Show URL and token without opening a root-owned browser
sudo agentsight dashboard --no-open
agentsight summary — Unified Overview
Rolls up sessions and Token usage, interruption events grouped by severity, and Tokenless savings for a recent time window — one command for the overall health picture.
# Last 24 hours (default)
agentsight summary
# Last 7 days, JSON output
agentsight summary --last 168 --json
Data sources degrade independently: a missing database contributes zeros without affecting the rest of the report.
Query and manage AI Agent session interruption events.
Interruption types:
Type
Description
Default Severity
llm_error
HTTP status >= 400 or SSE body contains error
high
sse_truncated
SSE stream ended without
finish_reason=stop
high
context_overflow
Context length exceeded
high
agent_crash
Agent process disappeared mid-session
critical
token_limit
finish_reason=length
with output near max
medium
# List interruption events (default: last 24h)
agentsight interruption list [--last <HOURS>] [--type <TYPE>] [--severity <LEVEL>]
# Statistics by type
agentsight interruption stats
# Count by severity
agentsight interruption count
# Get a single event by ID
agentsight interruption get <ID># List all interruption events of a session / conversation
agentsight interruption session <SESSION_ID>
agentsight interruption conversation <CONVERSATION_ID># Mark as resolved
agentsight interruption resolve <ID>
Configuration
Configuration file:
/etc/agentsight/config.json
(override with
--config
).
Important
: User config files
replace
(not extend) the built-in default rules. Ensure your config includes all Agent rules you need.
Feature Flags
Feature
JSON Path
Default
Description
Token stats
features.token_stats
true
Core Token accounting
SQLite storage
features.sqlite_storage.enabled
true
Local persistence
Interruption detection
features.interruption_detection.enabled
true
Error/crash detection
Audit
features.audit
true
LLM call audit
Session mapping
features.session_mapping.enabled
true
responseId→sessionId
Runtime Limits
Config
Default
Description
event_channel_capacity
10,000
Probe event bounded channel capacity
pending_genai_max_count
1,000
Max events awaiting session_id
max_connection_body_mb
8
Single HTTP connection body buffer limit
ring_buffer_mb
32
eBPF Ring Buffer size (must be power of 2)
Agent Framework Integration
Conversational Skill (cosh)
AgentSight provides a built-in conversational skill for Copilot Shell. Users can query Token usage and audit logs via natural language:
"How much Token did I use today?"
"Show me today's LLM call records"
Token Savings (Tokenless Integration)
AgentSight integrates with the Tokenless component to display Token savings data in the Dashboard. No additional configuration needed — if both are installed, savings data appears automatically.
Data Management
Database Auto-cleanup
Default maximum database size: 200 MB. When reached, automatic cleanup triggers.
Customize via environment variable:
export AGENTSIGHT_GENAI_DB_MAX_SIZE_MB=500
Clear History
rm -rf /var/log/sysak/.agentsight
# Then restart AgentSight
FAQ
Q: Why can't I see Token data for OpenClaw?
A: AgentSight monitors the
openclaw-gateway
daemon. Check client-gateway connectivity. If you see "pairing required" errors, run
openclaw devices approve
.
Q: Why does the Token savings page show 0?
A: Possible causes: (1) The AK/SK authentication mode is not yet supported; (2) Session ID format is non-standard UUID.
Q: Why do cumulative savings exceed the single-call difference?
A: Agents include historical messages in context. Savings accumulate across turns, so cumulative savings exceed per-turn differences.
Although it may not seem like it, what can broadly be termed
"personal computing devices" are based on only three
conceptual models. In order of their historical appearance, they
are: the "stand alone computer model," which first came
into being as the batch processing mainframe with a command line
interpreter; the "networked stand alone computer model,"
which saw its first incarnation as the object-oriented workstation
with a graphical user interface (GUI); and the "network interface
computer model," a terminal-like computer for single users
in an environment where all computers are linked together. The
golden age of the first conceptual model has long passed, and
the current age is a transitional period in which we are moving
from the second conceptual model to the third.
What may be surprising to many is that the third conceptual
model was first conceived not in the U.S., which has rightly earned
the distinction of the "world's systems house," but
rather in Japan, which has historically been viewed as a country
that creates great hardware but is poor at software--particularly
systems software. However, in the mid 1980s, long before computing
platforms based on either the IBM-PC or Macintosh operating systems
were considered for use as interfaces to a "network of networks"
called the Internet, the TRON Project began work on the BTRON-specification
operating system, which was conceived as a real-time human-machine
interface to a "hypernetwork in which all computers and computerized
devices throughout human society would be interconnected."
The network interface computer differs from the standard personal
computers of today mainly in terms of size. Network interface
computers have very small operating systems, and hence they require
very little in the way of hardware resources to run. This allows
them to be built at very low cost, which in turn allows organizations
that employ them to save considerable amounts of money on management
information systems. But all network interface computers are not
based on the same design precepts.
One computing model for the network interface computer, the
"network computer" proposed in the late 1990s in the
U.S., was conceived of as a machine that would operate inside
a company's local area network (LAN). Since it would always be
used in conjunction with a company-owned server, programs and
data could be stored there and downloaded as necessary. In fact,
Sun Microsystems Inc., one of the companies that is strongly pushing
for the adoption of network computers in the U.S., is planning
to take this paradigm and apply it to the Internet. The company
plans to offer its freeware StarOffice productivity suite via
StarPortal
,
a Web site from which users can download data and applications
and do data processing inside their browsers. People signing up
for this service will only need a browser and an Internet connection.
However, there is a problem with this computing model in that
current browsers are huge applications, and they run on even larger
operating systems, so the cost savings will mainly be limited
to software. (For a critique of Sun's efforts with StarOffice,
click
here
.)
That's where the BTRON-specification operating system is different.
BTRON lies between today's gargantuan personal computer operating
systems--which have become as large as mainframe computer operating
systems and continue to get bigger with each new upgrade!--and
the stripped down operating systems of LAN-based network computers.
BTRON is compact, which is why basically the same BTRON3-specification
source code can be used in both PDAs and IBM-PC/AT compatibles,
and yet it is extremely powerful. The design specification calls
for word processor and graphics editor functions as standard equipment,
but the commercial implementation by Personal Media Corporation
called B-right/V has, in addition, a spreadsheet program, a scripting
language, an e-mailer, a card database program, a Web browser,
PC communications software, plus various utilities, such as file
converters--and that's not even to mention a World Wide Web-like
hypertext filing system at the system level.
As a result, a BTRON-specification computer requires no Internet
or LAN connection to do data processing, but the hardware required
to run the latest BTRON implementation, B-right/V R2, is minimal:
an Intel 486DX microprocessor-based PC, 16 megabytes of main memory,
and a few hundred megabytes of hard disk space. In other words,
a BTRON-based system can be manufactured almost as cheaply as
a network computer, although it has all the functionality of a
standard computer. And if that sounds too good to be true, it
gets better. B-right/V R2 has for the first time in the history
of personal computing implemented a true multilingual computing
environment that allows users to employ up to approximately 130,000
characters in their documents. The majority of these characters
are
kanji
(Chinese characters), which for the first time
allow the Japanese people to write any word in their language--and
they come in outline fonts to boot!
The True TRON Multilingual Environment Finally Appears
The BTRON3-specification "B-right/V" operating system
for IBM-PC/AT compatibles was first marketed in Japan on July
18, 1998. Historically, B-right/V is a descendant of the "3B"
operating system, which was designed for a TRONCHIP-based hardware
platform called MCUBE that hit the Japanese market in 1995. The
3B operating system subsequently bifurcated into µBTRON-specification
"B-right," which is used in Seiko Instruments Inc.'s
BrainPad TiPO PDA, and B-right/V (B-right for DOS/V machines,
which is what IBM-PC/AT compatibles are called in Japan). The
code for both of these operating systems, which are based on a
micro kernel design, is basically the same; there are only minor
variations having to do with window functions, selectable colors,
power saving, character input, etc., which are a result of the
hardware limitations of handheld devices that do not use a keyboard.
There are, however, many differences between B-right/V and
"B-right/V R2," the latter of which hit the Japanese
market on November 12, 1999. The major difference is that B-right/V
had only a partial implementation of the TRON Multilingual Environment.
Specifically, its multilingual capabilities were based on a "single
48,400 character plane" (a plane is called a "script"
in the TRON Architecture) into which multiple national character
sets were loaded. The B-right/V R2 operating system, on the other
hand, implements the true TRON Multilingual Environment, which
is based on "multiple character planes of 48,400 characters"
that can be switched in and out as required using "language
specifier codes." [1] In fact, the current implementation
has 31 such character planes defined for it, which means that
it can handle a total of 1,500,400 characters. Needless to say,
it is going to take some time to fill up that space.
[1] Some readers might be wondering why the "script
planes" in the B-right/V R2 operating system are not switched
in and out using "script switching codes." The answer
is that the language specifier codes used for this purpose have
"multiple functions," one of which is to switch in
and out of script planes. In addition, they specify what script
"group" is involved and what "language" the
data are written in. For an introduction to the four layers of
the TRON Multilingual Environment hierarchy (Font, Script, Group,
and Language), please click
here
.
The B-right/V R2 script planes and their current contents are
as follows:
System Script (0xFE21)
JIS levels 1 and 2,
JIS auxiliary
kanji
Chinese GB 2312
Korean KS C 5601
6-point Braille
8-point Braille
Japanese Script 1 (0xFE22)
Reserved
Japanese Script 2 (0xFE23)
Reserved
Chinese Script 1 (0xFE24)
Simplified Chinese
additional characters
Chinese Script 2 (0xFE25)
Same as above
Chinese Script 3 (0xFE26)
Traditional Chinese
Chinese Script 4 (0xFE27)
Same as above
Korean Script 1 (0xFE28)
Korean
additional characters
Korean Script 2 (0xFE29)
Same as above
Various National Scripts (0xFE2a)
Unicode basic multilingual plane
(excluding Chinese characters)
Mojikyo Script 1 (0xFE2b)
Konjaku Mojikyo
characters
Mojikyo Script 2 (0xFE2c)
Same as the above
Mojikyo Script 3 (0xFE2d)
Same as the above
Mojikyo Script 4 (0xFE2e)
Same as the above
(0xFE2f - - 0xFE3F)
Reserved (17 planes)
One thing that is important to note here is that there is no
official "TRON Character Set." The BTRON operating system
merely provides a "framework," called "TRON Code,"
into which character sets that have, or will, come into wide use
are loaded. Of course, once those character sets are loaded into
the TRON Code framework, a de facto "TRON character set"
comes into existence as can be seen above, but there are no TRON
Project committees deciding which characters can or should be
used by BTRON end users. The TRON policy is to register all characters
and leave it to the end user to decide which characters he or
she should employ in data processing. In order to implement this
policy, the TRON Project has also created a character registration
center (officially called the "
TRON
Character Resource Center
") on the Internet through which
new characters can be added to the TRON character set. As long
as the source of new characters is clear and there are no copyright
complications involved, the character or characters will be registered
free of charge and made available for downloading by BTRON user
community.
Another thing that it is important to note--which no doubt
is something that any Unicode folks reading this article would
like to point out--is that the exact same Chinese character can
appear on different planes in the TRON character set. That is
absolutely correct, and it is in fact the reason that only a BTRON-specification
computer can used used to discuss via e-mail the "unification"
that the Unicode movement is undertaking, and it is why only a
BTRON-specification computer can print out the entire Unicode
specification. In other words, the lack of unification is not
viewed as something bad, but rather as something that is good.
There is, of course, the chance that the user will not be aware
of what character plane he or she is dealing with. However, there
are ways of checking. Hexadecimal savvy users merely have to check
the language specifier codes given in the parentheses above, and
ordinary users can pull a character into the Character Search
Utility (see "New Utility for Searching for
Kanji
"below).
Finally, since there are disagreements among specialists about
what is and is not a "distinct Chinese character," a
"thesaurus-like function" is also under development
to give end users information to make their own judgments.
However, improvements to character-related functions in the
B-right/V operating system that appeared with Release 2 are not
limited to solely to the processing of
kanji
. As the following
list of character-related improvements shows, proportional font
compatibility and word wrap functions have been added. These functions
are necessary for processing languages that are written with the
Latin script. Moreover, there is also a multi-font function, which
is necessary for doing high-quality word processing and desk-top
publishing.
Multi-
kanji
, multilingual functions
Character Search Utility that can find
kanji
using
elements, readings, and number of strokes
Multi-font function
Proportional font compatibility
Gray scale font function
Word wrap function
Function for displaying a list of candidates for
kana
-to-
kanji
conversion
Function for customizing
kana
-to-
kanji
operations
Among the
kanji
-related processing functions in the
above list, the Character Search Utility, which will be described
below, is essential to enable the end user to easily find his
or her way through the large
kanji
character sets that
come with B-right/V R2. But that utility is only necessary when
the Japanese-language input system (
kana
-to-
kanji
conversion) does not output he desired characters. Thus it is
important to note that the functionality of the Japanese-language
input system has also been improved. The list display function
makes it easier to select among the conversion candidates in input
dictionaries, and the customization function makes it possible
for the end user to match the input functions to his or her typing
habits and even allocate key assignments.
New Utility for Searching for
Kanji
TRON Project Leader Ken Sakamura has been saying for years
that you can not just stuff a large number of
kanji
into
a personal computer system and hope the end user will make good
use of them. To use such a computer system, a function that makes
it possible for the user to easily find the necessary characters
is also required. Thus it is only natural that along with B-right/V
R2's impressive unabridged
kanji
character set comes an
extremely easy to use Character Search Utility that can--according
to various specified search criteria--spit out huge lists of
kanji
in a flash. Perhaps the most remarkable thing about this utility
is that it makes it possible to search for
kanji
without
even knowing the "radicals" according to which the
kanji
are listed in traditional dictionaries. This is truly a revolutionary
development for students beginning their study of Japanese.
As can be seen in Fig. 1, the Character Search Utility is a
panel that fills a small section of the screen of a personal computer.
The utility has three functions--the tabs at the top of the panel--that
allow the user to select among: (1) viewing character codes, (2)
searching for characters, and (3) looking up information about
a character. In the example in Fig. 1, the Search function has
been selected, and two radicals (basic elements used for sorting
characters in traditional
kanji
dictionaries) have been
input in the Search Key box. From left to right, these are
kuchi
hen
('mouth') and
takumi hen
('carpenter's square').
Among the output characters, which cover two pages as indicated
at the bottom of the panel, a character comprised of only these
two radicals has been found on the Mojikyo Script 1 plane, and
that character along with its character code has been displayed
in the upper right hand corner for easy viewing. Please note that
the output characters are color coded (black for JIS levels 1
and 2, blue for JIS auxiliary
kanji
, and green for non-JIS
[
Konjaku Mojikyo
] characters), and that the "Enlarged
Display" option has been selected in the lower left hand
corner.
In addition to radicals, it is also possible to input the
katakana
pronunciations of the above radicals. Likewise, a character incorporating
the same two elements can be used for searching for another character
with the same two elements. Other search methods are based on
arithmetic-like expressions. In Fig. 2, for example, the expression
"
too
('climb') minus
mame hen
('bean') radical
has been input, which yields the
hatsugashira
radical.
In Fig. 3, the expression "
kuchi hen
('mouth') times
four" yields a huge list of characters, one of which on the
Mojikyo Script 1 plane consists of exactly four mouth radicals.
The Character Search Utility can also be used for obtaining
information about a
kanji
that one does not know. In Fig.
4, the user has selected the Character Information function of
the Character Search Utility and has dragged and dropped a character
listed under the
uo hen
(the 'fish' radical) into it. The
following information about the character, which is on the Mojikyo
Script 2 plane, has been output:
Mojikyo No. 046382,
Uo
Radical 10 strokes, Basic Character
Kan
,
gigi
,
ken
,
kon
,
nayamu
,
hararago
,
yamu
,
yamoo
, hwan
Conversely, by dragging and dropping
kanji
from the
Character Information function of the Character Search Utility,
the user can also easily create a custom
kana
-to-
kanji
conversion dictionary for converting Japanese syllabic data written
with the
hiragana
syllabary in
kanji
. As is shown
in Fig. 5, the user has entered two
kanji
and their readings
(
wanizame
'shark', and
hararago
'hard roe') into
a text real object (text file) titled "
Uo Hen no Kanji
Jisho
("
Uo Hen Kanji
Dictionary"). When the
real object is closed (green dotted line) and the virtual object
(link to that real object) is dragged and dropped into the User
Dictionary registration panel (red dotted line), the user can
then input the rare
kanji
using the operating system's
kana
-to-
kanji
conversion function.
One thing that is not shown here--but which is exceedingly
important to remember is possible!--is a user employing the Character
Search Utility to read data from Web pages on the Internet. That
is to say, critics of the TRON Project believe that an unabridged
kanji
character set is unnecessary, since no one knows
as many as 80,000
kanji
. Accordingly, not listing all of
those
kanji
in a computer system only seems logical. However,
if as shown above, a user can easily learn the pronunciations
and readings of an unknown
kanji
simply by dragging and
dropping it from a Web page into the Character Search Utility
panel on the screen of his/her personal computer, then lack of
knowledge tens of thousands of obscure
kanji
is no problem
at all. This Character Search Utility can also serve as a dandy
learning tool, both for native speakers of Japanese and foreigners
studying the language.
New Internet and Peripheral Device Features
As was stated in the first section of this article, the BTRON-specification
computer was originally conceived as a real-time human-machine
interface for a hypernetwork--specifically, the "TRON Hypernetwork"--in
which every kind of computer device is linked together. Accordingly,
networking functions are central to the BTRON computing model,
and they are under constant development, both at Personal Media
Corporation and at the Sakamura Laboratory on the University of
Tokyo campus. The latest networking functions that have been added
to the B-right/V R2 are as follows:
Dial-up (PPP) function for connecting to the Internet
E-mail software (freeware) bundled with the operating system
File transfer function (ftp)
Network printer function
The PPP function allows BTRON users to connect to Internet
service providers via dial-up (public telephone) lines using a
modem. Since there are not many areas in which cable modem and/or
Digital Subscriber Line (DSL) service is currently available in
Japan, this is an extremely important function for people using
BTRON from home. The e-mail software, which is freeware application
developed at the Sakamura Laboratory, is a new type of e-mail
application based on BTRON programming concepts. The application
is made up of a group of miniature applications that are started
up as necessary to handling outgoing and incoming e-mail.
The file transfer protocol (ftp) function is for the BTRON
Basic Browser, which it enables to download download software
from the Internet. The BTRON Basic Browser has been greatly improved
compared to its first release. It is now possible to set fonts,
and there are four Save options. The user can save a Web page
as HTML, the Web page itself, or as a TAD (TRON Application Data-bus)
text or graphic file, which are referred to as "real objects."
When a Web page is saved as a TAD text real object, for example,
the layout changes, but it is possible to click on a link and
open up the Web page to see the latest update. This is an advanced
feature that is not available to most people using a personal
computer to surf the World Wide Web.
Supporting various types of peripheral equipment is the hallmark
of a good operating system, and B-right/V R2 is squarely aimed
at that target. The latest additions for peripheral equipment
support are as follows:
Peripherals
Wheel mouse and three-button mouse compatibility (middle
button used for double clicking)
Function for setting the screen to non-standard sizes (e.g.,
1024 x 480 dots)
Improved performance accessing HDDs and CD-ROMs using a DMA
function
Addition of compatible printers and network adapters
For those who are unfamiliar with a "wheel mouse,"
it is in fact a PC mouse with a tiny wheel between the two keys
that are respectively used for clicking and displaying pop-up
menus. The wheel is used to scroll through pages, thus alleviating
the need for the user to push page up/down keys, click scroll
arrows, or drag scroll boxes.
The Future: Improving on New Basic Functions
The biggest problem that westerners have in evaluating Japan
and/or Japanese technologies is that they believe what the U.S.
is doing is the yardstick, and what Japan is doing should be evaluated
according to that yardstick. Thus based on this "technocentric
reasoning," if there is a higher penetration of personal
computers in the U.S. than in Japan, then Japan is behind the
U.S. in becoming "computerized." This reasoning, unfortunately,
leaves aside the fact that millions of Japanese use "personal
word processors," which are little more than specialized
personal computers. Likewise, if several times more personal computer
users access the Internet in the U.S. than in Japan, then Japan
is behind the U.S. in "connectivity." This reasoning,
unfortunately, leaves aside the fact that wireless usage in Japan--which
is not to mention facsimile usage--is far higher than in the U.S.
Moreover, this reasoning also leaves aside the fact that the overwhelming
majority of the Web sites on the Internet have only English-language
content!
Accordingly, when western analysts look at the BTRON subproject,
they view it using the U.S. market as a yardstick. Since there
are more application software programs that run on Microsoft Corporation's
MS Windows or Apple Computer Inc.'s Macintosh operating systems
than on B-right/V R2, then B-right/V R2 will go nowhere in the
Japanese market. Unfortunately, that argument overlooks the fact
that B-right/V R2 is all about bringing "new basic functions
to the market," basic functions that neither Microsoft nor
Apple are interested in providing to the Japanese people. BTRON3-specification
B-right/V R2 is the first and only personal computer operating
system that allows the Japanese people to write any word in their
language. The Japanese people have only been able to do this on
a personal computer since November 12, 1999--the day B-right/V
R2 went on sale! Moreover, the BTRON3-specification operating
system is the only operating system on the market that has a hypertext-like
filing system. When this is improved to incorporate the HyperText
Transfer Protocols of the Internet, it will be the only personal
computer operating system to seamlessly integrate a personal computer
filing system and the structure of the World Wide Web.
And so, this is where the immediate future of the B-right/V
R2 operating system lies--in bringing new basic functions to the
market and consistently improving upon them. One of the first
improvements to the B-right/V R2 operating system will have to
do with the TRON Multilingual Environment, which in the present
implementation only realizes two layers (Font and Script) of the
four-layer hierarchy (Font, Script, Group, and Language). Thus
one of the coming improvements to the B-right/V R2 operating system
will be the expansion of the language specifier codes to include
the Group and Language layers. In addition, other key parts of
the multilingual environment, such as algorithms for expressing
the various languages in writing, will have to be developed. One
important element of this work will be expanding the functionality
of the Basic Text Editor, which currently only accepts left-to-right
horizontal character input. In the future, it will have to accept
both vertically written (top-to-bottom) input and right-to-left
horizontal character input. In addition, various sorting algorithms,
such those for putting word lists in alphabetical order, will
have to be developed to deal with input data in various national
languages.
There is, however, one U.S. market yardstick that the BTRON-specification
operating system should be measured against. That yardstick is
following through on what one has promised to end users. If one
promises end users something and then does not follow through
on that promise, that party is guilty of producing what is known
as "vaporware," software that's all talk and no reality.
Well, the TRON Project promised the world the best multilingual
operating system on the planet back in 1987 at the Second TRON
Project Symposium, and 12 years later it came through on that
promise when it unveiled the B-right/V R2 operating system. And
so if anyone wants to know where the BTRON3-specification B-right/V
R2 operating system is headed, the answer is "exactly where
its developers say it's headed." So stay tuned for some extraordinary
developments in the world of personal computing that are going
to take place on top of this unique and highly flexible operating
system. The BTRON subarchitecture has only just started to show
its greatness.
B-right/V R2 Software Available on the World Wide Web
Although not many third party commercial software applications
exist for the B-right/V R2 operating system at present, there
is a considerable number of freeware/shareware programs available
for downloading from the Internet. A large list of these, the
majority of which are freeware, is maintained at the following
URL.
As of this writing, 64 entries are listed there, including
the B-right/V R2 development environment from Personal Media Corporation.
Since they are described in Japanese, let me give the categories
and number of programs below.
Internet: 3
Graphics/music: 4
Text: 3
Utiltities/accessories: 13
Operation-related: 5
Desktop-related: 5
Input-related: 9
Dictionaries: 6
Development-related: 7
Games/novels: 5
Data (clip art, character enlarger): 2
Peripheral/hardware/system-related: 2
B-right/V R2 users should continually check this Web page,
since new entries are constantly added.
There is also a list of B-right/V R2 freeware that is maintained
at the Yahoo! Japan Web site. The URL is:
A 3D fruit fly that lives on your macOS desktop — driven by a live spiking
simulation of the real
FlyWire
connectome. It walks across your windows, grooms, sleeps, and decides to flee
your cursor with the same neurons a real fly uses.
It scans your disk for agent markers (
AGENTS.md
,
CLAUDE.md
,
.cursor/rules
,
.kiro/steering
and ~40 more) and turns anything on screen that leads to them into an odour
source: an editor or terminal window with the project open, a row in the front
Finder window, a folder icon on the desktop. An open project smells strongest,
a closed icon weakest, and the reach of each grows with how much vibecode it
holds — a hub of six marked repos is smelled across the whole screen, a single
weak folder only from nearby. The steering neurons then walk the fly there,
and when the smell is far the population wakes up enough to make it fly.
The fly's brain window: 23,210 real neuron soma positions from FlyWire v783,
with live spikes flashing at real neuron locations. The two glowing yellow
markers are the Giant Fibers — the escape command neurons. Click any region
to stimulate it.
What's real
23,210 neuron soma positions
(of 139,255 in FlyWire v783) render the
rotating brain window, colored by super-class (FlyWire's coarse cell-type
grouping).
A 668-neuron circuit with ~19,000 real synaptic connections
(synapse
counts, signed by neurotransmitter prediction) runs as a 1 kHz
leaky-integrate-and-fire (LIF) simulation:
their 330 strongest partners, including ascending (proprioceptive) and
sensory (wind) neurons
Escape is not scripted.
Your cursor's approach becomes looming input to
the real LC4/LPLC2 cells; the fly takes off only when the Giant Fiber
actually spikes through its real synapses — ~1,200 synapses of feedforward
inhibition push back, which is why slow approaches are tolerated and fast
lunges trigger escape in ~4 ms, just like the real animal.
The body itself is procedural (FlyWire is a brain connectome — no body
geometry exists), with a tripod gait, visible wing-beat, altitude-scaled
flight, grooming, and sleep postures.
Installation
Requirements:
macOS 13+
, Xcode Command Line Tools (Swift 5.9+).
No permissions or entitlements needed — everything it senses
(cursor, window frames, clicks-as-taps, thermal state) is permission-free.
git clone https://github.com/DenisSergeevitch/desktop-fly.git
cd desktop-fly
./build.sh
./DesktopFly
A 🪰 item appears in the menu bar; quit from there. The fly wanders your
desktop on a transparent, click-through overlay — it never intercepts your
mouse or keyboard.
Controls (menu bar 🪰)
item
effect
Pause / Resume
freeze the world
Show/Hide Brain
toggle the live brain window
Escape Test (loom)
inject a looming stimulus, watch the GF fire
Move to Next Display
hop the fly across monitors (shown when >1 display)
Add / Remove Fly
extra flies (only fly #1 carries the brain)
Scare Flies
startle everyone
The brain window is interactive
: hovering pauses the rotation; clicking a
region "optogenetically" stimulates the ~60 nearest circuit neurons for
400 ms. The fly's reaction is whatever the real network does downstream —
click the Giant Fiber and it escapes; click DNg11 and it grooms; click one
side's DNa01/02 and it turns.
How real neurons drive the body
body behavior
driven by
escape takeoff
DNp01 giant fiber spike
walk vs. rest, walking speed
DNp09 rate
steering
DNa01+DNa02 left−right rate difference
grooming
DNg11 rate
backward scoot
MDN burst
nervous darting
LC4/LPLC2 population rate
wing-beat effort, threat wing-raise
DNp02/04/11 rate
spontaneous takeoff
whole-population arousal
The loop also closes body→brain: the gait rhythm feeds the circuit's real
ascending (proprioceptive) neurons in phase with the legs, and fast cursor
motion stimulates its sensory (wind) partners.
Window terrain
: window top edges are ledges — the fly lands on them,
walks along them, rides a window you drag, and startles when one closes
under its feet.
Window looms
: a window appearing near the fly feeds the looming
pathway; the circuit decides whether to flee your dialogs.
Clicks are substrate taps
; clicking next to the fly startles it through
the wind→GF pathway.
Typing is vibration
(idle-time API — knows
when
keys were pressed, never which).
Circadian rhythm
: dawn/dusk activity peaks, midday siesta, night
quiescence.
Sleep
: idle at night → it sleeps, breathing slowly, with
raised arousal threshold; it grooms after waking.
Temperature
: flies are ectotherms — a hot Mac is a faster fly.
Regenerating the data
data/
ships with compact derived files. To rebuild them from the raw
FlyWire Codex dumps (~60 MB download):
Honesty section: the connectome gives wiring, not physiology. The LIF
dynamics, neurotransmitter signs (ACh+, GABA−, Glu−), the gap-junction boost
on LC→GF and wind→GF (documented electrical coupling), synaptic delays, and
the sensory transduction (cursor → looming value) are standard modeling
choices layered on the real graph. Everything downstream of the sensory
neurons — who connects to whom, and how strongly — is FlyWire data.
License & citation
Code is MIT. The files in
data/
are derived from FlyWire (FAFB v783) and
are
CC BY-NC 4.0
— see
data/DATA_LICENSE.md
.
If you use this, cite:
I've always had a fascination with simpler and low power computing. This lead me to run Photoshop on a 60 pence computer chip and it amazes me that this is possible.
Photo: A monochrome screen showing Adobe photoshop with a simple drawing running on an emulated Apple Macintosh
OK; this is hardly editing a full colour, highly detailed photograph but at least my "self portrait" will obviously set the illustration world on fire.
This was done by emulating an old Apple Mac on a Raspberry Pi RP2350 chip, which can be purchased for around
60p
. Add the various extra parts needed and say roughly that a mid 1990s equivalent computer can be built today for a few pounds in one-off quantities. Call it the price of coffee and a cake in a European café today.
Adjusted for inflation, a roughly equivalent Macintosh SE to the one being emulated would cost £9554 today. (I found the pricing from a review in this
1989 Personal Computer World
magazine online and used the
Bank of England inflation calculator
to convert the £3495 selling price.)
Whichever way you estimate: Computers are vastly cheaper nowadays and this computer is much cheaper than a modern PC.
The same emulated Mac can run other software - e.g. the
WordPerfect
word processor which was popular at the time. I found this surprisingly pleasant to type and concentrate with: It had an uncluttered interface and was distraction free when compared to my modern Macbook with its notifications and busy displays.
Photo: WordPerfect 1.0 on an emulated Apple Macintosh
Sometimes experimenting with things can spark my imagination and create new ideas. Simple and low power computers appeal to me, so it's interesting to ask "what is the minimal viable modern computer". A 60p chip is definitely not the cheapest possible, but it is certainly has an impressive "computing power to price" ratio.
Assume this is equivalent to a mid 1990s computer (it has two 150MHz processors, and easily adds 4Mb RAM and gigabytes of storage) . This was time when people made presentations, wrote documents, ran spreadsheets, sent emails and browsed (an admittedly simpler) web - not dissimilar to a lot of our computer use today.
This provokes questions: How to provide computing for those who need to or want to spend their resources elsewhere? What could a super thin, super lightweight laptop look like? Perhaps it could have an e-paper screen to be calm on the eyes. Could it be solar powered? Could it have a calmer, less cluttered and less distracting interface than modern operating systems but still provide similar functionality? Could it be long lasting with reliable software and not have to be thrown into landfill every few years? Does this have enough processing power and quality to become part of a personal music playing or streaming system? Can this help reduce the addictiveness of modern devices and stop "doom-scrolling"?
The relative simplicity of this chip compared to more powerful ones (and the software limitations this would imply) means that the whole computer is (with effort by the right people) relatively understandable, and everything is documented, and the power consumption is lower.
If I had this computer as my phone or laptop what would I miss? If I could read web pages and write and listen to music I think this would do a lot of what I want. I'd want some kind of messaging and probably maps. I think this can probably play music. It could display maps but maybe in a simpler "A to Z atlas" style of flipping between pages. But maybe this trade-off vs power use could be useful in remote places with no smartphone charging. Maybe cycle tourists would appreciate such a thing.
It could (slowly) take and view grainy pictures, but videos and good pictures would be difficult. Saving videos to watch later on a display at home is probably a healthier way of consuming them than distracting myself with them, and maybe carrying a separate digital camera wouldn't be a sacrifice.
The modern web is sadly too complex to be viewable on such a device, but a bridging solution would be to have conversion software running on larger computers. A browser for something like
Gemini
would be possible. We could still have a WiFi chip on this, but the complexity of modern 4G and 5G mobile networks could be a challenge. Adafruit has
An IRC messaging client
demonstration running on this chip.
Maybe I'm being naive and nostalgic for a past that never existed. But I think it is important to think of the trade-offs we make with modern computers and maybe playing with some of these things can help us make computers which are kinder, as
discussed by James in this blog post
after our discussion.
There is overlap here with some of the
Solarpunk
and
Permacomputing
communities. Having privacy respecting solar powered computers that people can tinker and mold to their own needs feels to me like an exciting, positive future.
It's interesting to imagine an alternative world where these modern chips had arrived much earlier in time, bringing un-imaginable (for then) amounts of computing power and memory for little money. How would computers have evolved and what would they look like today?
Felony Bench counts unique instances where AI agents affect third-party entities. Escaping a sandbox alone does not constitute a counted incident. It is for these reasons that Frontier Security's
Kimi K3
incident and
Alibaba's ROME
incident are not counted.
Enabling the next-generation trait solver on nightly | Rust Blog
After nearly 4 years of active development,
the next-generation trait solver
is close to stabilization. We are enabling it by default on nightly to surface any remaining issues and plan to stabilize it in the next months. This is the largest single change to the Rust compiler since its initial release. It completely replaces how we prove where-clauses, normalize associated types, and much more.
Please try out the latest nightly and
open an issue
if you encounter any bugs or regressions.
Even so, this already fixes a huge number of issues. As an underapproximation, we currently know of
more than 200 issues on GitHub fixed by this change
. This also has a significant impact on compile times; more on that later. When developing on nightly, you may accidentally rely on behavior only supported by the new trait solver.
This is an incredibly big change which results in a non-trivial amount of breakage. Most of these changes are intended improvements to type inference or the removal of undesirable behavior. We are tracking the known issues and breakage
in a pinned GitHub issue
.
What can I do?
Please update to the latest nightly version by using
rustup update nightly
and use it to test your existing projects and libraries.
⚠️ While the next-generation trait solver has been enabled on our
main
branch, this change will only be accessible on the nightly channel starting from Saturday 22nd August. You can already test it before then by providing
-Znext-solver=globally
as a command-line argument ⚠️
Please tell us if you encounter any breakage, compile-time performance regression, or bad diagnostics. We have not yet spent too much time on error messages for the next-generation trait solver, so we would also appreciate you using this nightly for development to find poor diagnostics and other bugs in our error handling.
If you encounter any issue, take a quick look at
the pinned GitHub issue
to see if the affected crate is already listed, and if not, please open
a new issue
! To disable the next-generation trait solver on nightly, you can pass
-Znext-solver=coherence
to
rustc
, use
RUSTFLAGS=-Znext-solver=coherence
, or change your project's
.cargo/config.toml
configuration file:
[build]rustflags = ["-Znext-solver=coherence"]
What exactly does this mean?
We will go into more detail about the next-generation trait solver, how we got here, and what it changes when fully stabilizing it. This is a quick summary of its main impact.
impl Trait
handling
The way opaque types — return-position
impl Trait
(RPIT), but also the unstable
Type Alias Impl Trait (TAIT) and Return Type Notation (RTN)
— are handled in the type system has nearly completely changed. This fixes a lot of bugs and edge cases with them and should make their behavior a lot more consistent in general. This change is why the next-generation trait solver is necessary to stabilize TAIT and RTN.
The implementation change mostly does not matter for RPIT as we special-cased
impl Trait
from the method signature when type checking the method body. This means the only way to observe the old behavior is via recursive function calls. The following snippet errors with the existing implementation, but compiles with
-Znext-solver
enabled:
godbolt
fn foo(b: bool) -> impl Sized { if b { // The old implementation errored here. foo(false) + 1 } else { 0 }}
Associated types in higher-ranked types
The most impactful change is way we handle associated types referencing bound variables, i.e., lifetimes from a
for<'a>
binder, for example, the type
for<'a> fn(<T as Trait>::Assoc<'a>)
. While most users don't encounter such types directly, there are widely used crates which do. This change impacts existing code by removing incorrect type inference, such as in
bevy
and
minijinja
.
It also fixes a bunch of unnecessary errors like in the following example:
godbolt
trait OtherTrait { type Assoc<'a>;}impl OtherTrait for u32 { type Assoc<'a> = &'a u32;}trait Trait {}impl<T: OtherTrait> Trait for (T, for<'a> fn(<T as OtherTrait>::Assoc<'a>)) {}fn impls<T: Trait>() {}fn main() { // The old implementation failed to prove // the where-bound of `impls`. impls::<(u32, for<'a> fn(&'a u32))>();}
We've spent a lot of time on the compile-time performance of the next-generation trait solver. There have been many cases where it performed quadratically or even exponentially slower than the old solver.
Especially the last few weeks were mainly spent on improving performance. This work was shared by many people, with major contributions by
Nick Nethercote
,
jana
,
Rémy Rakic
, and
mira
.
As part of this effort,
Rémy Rakic
compared the performance of both implementations for the top 20,000 crates on
crates.io
. Below you is a visualization of the performance changes over the last two months.
On the left and the right, the major outliers can be found. Note that the sample of crates here is biased
towards
such crates, because those are more interesting to us. Nearly all crates we tested in the top 20k had effectively the same performance with both implementations.
This graph shows that we've mainly focused our efforts on the negative outliers and made significant progress there. While many of the crates that previously took more than twice as long to compile with the new solver are still slightly slower, our work has made a few of them actually compile
faster
than with the old solver.
We will continue to improve its performance over the coming months, and there are still a lot of optimization opportunities compared to the existing implementation. My expectation is that, in the long term, nearly all crates will benefit from the next-generation trait solver. I am especially excited about the huge performance benefits for some trait-heavy crates.
A Linux®-powered mini-PC, designed to run Workbench.
The
c100
is designed for offices, workshops, labs, and factories. Built around a full-size mechanical keyboard, with tool storage and a removable lid.
c100
folds with its magnetic hinge, for closed operation or reclaiming bench space.
All configurations include Caligra Workbench Operating System, keyboard, mouse, user-replaceable RAM/SSD/Bluetooth/Wi-Fi, hot-swappable low-profile Kailh switches, power supply, and direct support from Caligra engineers.
After I released version 1.0, I figured I would have to do the rotations myself. So I sat down with ChatGPT and I didn’t get it to write the code, but I got it to educate me. With a patient, interactive tutor, I was able to finally do what I hadn’t by reading books and asking mathematician friends –...
After I released version 1.0, I figured I would have to do the rotations myself. So I sat down with ChatGPT and I didn’t get it to write the code, but I got it to educate me. With a patient, interactive tutor, I was able to finally do what I hadn’t by reading books and asking mathematician friends – I learnt how to use quaternions just enough to make the app work.
So learning doesn’t stop just because I outsource a bunch of thinking to AI. It pushes me to learn more. I like that as an outcome.
—
Matt Webb
,
Galactic Compass 2: now with new augmented reality mode
Microsoft blames Windows gaming issues on RGB lighting devices
Bleeping Computer
www.bleepingcomputer.com
2026-08-21 10:54:49
Microsoft says ongoing issues causing games to crash or fail to launch after installing the August 2026 Windows updates may be caused by peripherals with RGB lighting. [...]...
Microsoft says ongoing issues causing games to crash or fail to launch after installing the August 2026 Windows updates may be caused by peripherals with RGB lighting.
As
Microsoft explained
when it confirmed it's investigating on Wednesday, this known issue affects games like ARC Raiders, MARVEL Tōkon: Fighting Souls, and The Finals on systems running Windows 11 24H2 and 25H2.
"Following the release of Windows updates on August 11, 2026 (KB5121003) and later, Microsoft received reports of issues involving inability to run games as expected," Microsoft said on the Windows release health dashboard.
On impacted PCs, users are also experiencing gaming freezes, "EXCEPTION_ACCESS_VIOLATION" errors, and even unexpected system restarts.
In a Thursday update, Microsoft said the gaming issues may be caused by drivers or components installed by RGB devices on affected Windows systems.
"Ongoing investigation indicates that this issue is related to peripherals or internal device components which have RGB lighting features. Such devices may install drivers or code components with file names similar to inpoutx64. In systems where these drivers are found, the issue is then triggered by launching certain games,"
it noted
.
"We are presently working to understand the relationship between these RGB components and the games which trigger this issue. We will provide an update when more information is available."
Unofficial workaround shared by game dev
Embark Studios, the Swedish video game developer behind ARC Raiders and The Finals, also said these issues are caused by inpoutx64.sys, but added that they stem from changes made to the Windows kernel driver.
"We're aware of a crash that has been impacting some players since the most recent Windows update (KB5121003). This is a crash related to the file inpoutx64.sys, which changed with the Windows update,"
Embark said
on Monday.
Until Microsoft ships an official fix, Embark
shared a multi-step temporary workaround
that requires users to delete the service and remove the inpoutx64 file from the Windows drivers folder.
This isn't the first time Microsoft has had to address gaming performance and stability issues caused by Windows updates. For instance, in October 2024, Microsoft
blocked Windows 24H2 upgrades
that caused Asphalt 8 crashes and Easy Anti-Cheat blue screens.
Early last year, Microsoft also removed
several
upgrade
blocks
that were preventing Asphalt 8: Airborne, Assassin's Creed, Star Wars Outlaws, and Avatar: Frontiers of Pandora players from upgrading their devices to the latest Windows version.
This is Behind the Blog, where we share our behind-the-scenes thoughts about how a few of our top stories of the week came together. This week, we discuss three years of 404 Media.
JASON:
Tomorrow is the third anniversary of the launch of 404 Media. If you haven’t been paying attention to us yammering on about this on the podcast and in our emails, you can celebrate with us at our party in New York in a few weeks.
Anniversaries are a good time to take stock of things, but they always seem to sneak up on us. I don’t think we’ll have much of a public post this year on the actual anniversary, but maybe some public celebrating
closer to the party and panel
we’re throwing.
First off, if you’re reading this, thank you for your support and for being a subscriber. We could not be doing this without you. Starting 404 Media has changed all of our lives; we’ve said it a million times at this point, but it was not clear when we started this that it would actually work. Three years in, it is extremely working, and we are very proud of the work we’ve done, what we’ve built, and the impact our journalism has had. These are going to be disorganized, off the dome thoughts, but a few things:
This post is for paid members only
Become a paid member for unlimited ad-free access to articles, bonus podcast content, and more.
As people grow older, they grow wiser, or at least they think they do.
Then it becomes their duty to impart their accumulated wisdom to the younger generation.
Leslie Lamport
made his name in distributed systems and fault tolerance.
For many he is better known as the author of
LaTeX
,
the famous macro package that makes
Donald Knuth’s
legendary
TeX typesetting system
usable
for the rest of us.
As Leslie grew older, he felt impelled to write a series
of fairly wacky papers with titles such as
“How to Write a Long Formula”
.
Another of these papers was called
“Types Considered Harmful”
, a diatribe against types in specification languages.
Its title was an echo of a famous letter,
“go to statement considered harmful”
,
by Edsger Dijkstra. The title of that letter (chosen by the journal editor) was subsequently borrowed by many authors who were against lots of things.
Leslie was against types. But how did I get involved?
Types considered harmful
Leslie‘s thesis was that specification languages should be based on an untyped formalism (a sort of set theory) as opposed to a typed formalism. He advanced several arguments in favour: that untyped formalisms were more flexible; that typed formalisms raised numerous anomalies and issues; that what we would view as a type error in a specification would be detected anyway during verification.
There was some sense in this thesis.
Type systems were in a state of flux in 1992 when that note was written.
Coq (now Rocq) had only just appeared,
and big changes were happening to Martin-Löf type theory.
As for simple type theories, early implementations of HOL had been around only for a couple of years.
It wasn’t clear what any typed calculus could do.
Proof assistants did not yet support type classes.
John Harrison was years away from introducing his trick to get
low-budget dependent types
,
which works well enough to express $T^n$.
On the other hand, Lamport’s note was a mess. He seemed to be unfamiliar with any actual typed formalism and devoted most of his note to knocking down straw men.
So when he submitted his note to
TOPLAS
for publication and it reached me to referee, my verdict was to reject.
The other referee, David McAllester, reached the same verdict.
That should’ve been that, but the editor, Andrew Appel, had other ideas.
“Put lipstick on it”
Debate is good, he said. These ideas deserve airing, or something of that sort. But we can’t allow errors in TOPLAS. Why don’t you join with Lamport as co-authors and transform the paper into something technically accurate but in the same spirit? I was game: I knew a fair bit about type systems and I also had my own untyped set-theoretic formalism
(
Isabelle/ZF
),
which I was happy to promote.
David went along for a bit but soon dropped out.
He was smart.
1
Leslie and I worked on the paper for a good while.
It was a weird form of unwilling co-authorship, but somehow we managed.
The new paper captured the core of Leslie‘s thesis while including a saner description of how types worked.
Along the way, I witnessed Leslie’s unrivalled TeX mastery:
low-level tricks that I have never encountered since.
A second round of review, oh God
Meanwhile, Andrew Appel had stepped down as TOPLAS editor.
The new editor, Carl Gunter, had not been informed about the special status of this paper.
So when it reached him, he sent it to fresh referees.
This was not part of the plan. And the new referees also decided to reject the paper.
One of the reports was incoherent.
It obviously had been written while its author was suffering a fit of apoplexy.
So then I contacted Carl and said, wait a minute, my rejection is worth nothing and this guy‘s rejection is somehow valid? Plus, he’s literally insane. So the paper appeared after all, with a disclaimer expressing wishes
for a lively debate, etc. etc. etc.
I’m not sure the debate ever happened.
In retrospect
And now we can ask how well Leslie’s thesis holds up 27 years later. It’s fair to say, not so well. Type systems have evolved considerably and they have proved their worth in numerous specification and verification tasks, some on an industrial scale.
Meanwhile, little progress has been made on the issues that plague set-theoretic formalisms.
Without types you don’t have overloading of notation,
which is trivial in principle (you can just use lots of different symbols),
but a big deal in practice.
And worse, the ability to write absolutely anything is mostly an invitation to make mistakes.
Verification is an extremely expensive way to find such mistakes,
and those you do not find could render your proofs worthless.
As far as I know, even Lamport’s own specification language (TLA+) was
eventually implemented
with some type restrictions.
So, in fact, your specification language probably should be typed. But it is also still worth looking for ways to make set-theoretic notations work better.
Obviously it's common knowledge by now that Anthropic has solely trained claude on old Buzzfeed articles (explaining its love for 90s nostalgia). So Claude and I built a
Claude Code
skill (
/debuzz
) that takes Claude's last response and runs it through the Gemini CLI to translate it from talking like a millennial clickbait to regular English. We're thinking of calling it "Claudette" but Cat Wu is famously litigious so it's definitely absolutely not called "Claudette".
The problem
Claudette is our solution to Claude being a great engineer with one incurable condition: it talks like it's delivering a TED talk about its own pull request. You ask why a test is flaky and you get a "load-bearing assumption," three numbered revelations, and a solemn note that the third one is
the most instructive yet
. Nothing is ever just a bug. There is always a kicker.
This skill accepts that no amount of prompting fully cures this, and does the honest thing instead: it hands the response to a different model, Gemini (remember gemini from last summer?), whose only job is to say it like a human person. Claudette has pinky promised to print Gemini's translation verbatim, because letting Claude "tidy up" the translation reintroduces exactly the voice being removed.
Before / after
Before (Claude):
Here's where it gets interesting: the retry logic isn't just a nice-to-have — it's - the - load-bearing - assumption - of - the - entire - sync - pipeline. Three things jumped out at me, and the third one is the most instructive yet. [...] And third — and this is the kicker — the dedupe key includes a timestamp, which means retries are never actually deduplicated.
After (
/debuzz
, colleague mode):
The sync pipeline's retry logic has three bugs.
syncQueue.ts:142
swallows
ETIMEDOUT
instead of re-queuing the job. The backoff caps at 2 seconds, which is too low for mobile networks. And the dedupe key includes a timestamp, so retries are never deduplicated. Fix: strip the timestamp from the key, raise the cap to 30 seconds, and re-throw the timeout error.
The
Gemini CLI
(
npm install -g @google/gemini-cli
), authenticated — run
gemini
once and use
/auth
, or set
GEMINI_API_KEY
.
Usage
Mode
Audience
What you get
colleague
(default)
An engineer
Same content, every file path and code block intact, zero theatrics
manager
A technical-adjacent manager
What happened, why it matters, what's next — about a third the length, no code
director
An executive
Three to five sentences: outcome, impact, ask. Assumes thirty seconds of attention
With no text argument it translates Claude's previous reply. Paste text after the mode to translate that instead. It also triggers on natural phrases like "say that in normal english."
How it works
No magic. Claudette writes its previous reply to a temp file, pipes it through
gemini -p "<plain-English style instructions>"
, and prints Gemini's output verbatim. If Gemini errors (usually auth), you see the actual error — Claude only offers its own rewrite as a clearly labeled fallback, because a debuzzer that quietly asks the buzzer to debuzz itself is how you end up with a load-bearing translation.
License
MIT
rust-glancer: An alternative LSP for Rust with focus on low memory usage
I want to present a project that I've been working on for the past 4 months: an alternative Rust LSP implementation that is built with a focus on low memory usage.
It has two main features:
It can use very little memory (target <100mb for reasonable projects). There are caveats, these are described below.
It allows immediate indexing after restart: if your project was indexed, restarting the editor will not require re-indexing.
Note: throughout this video, the used RAM remained under 100mb
These features make Rust Glancer suitable for the older computers: I have tested it on my old MacBook Pro M1 2020 with 8GB RAM, and it was pretty good.
Machine
LSP
Base indexing (engine usable)
Full indexing
MacBook Pro M4 Max, 36GB (2025)
Rust Glancer
5 seconds
8 seconds
MacBook Pro M4 Max, 36GB (2025)
rust-analyzer
6 seconds
13 seconds
MacBook Pro M1, 8GB (2020)
Rust Glancer
6 seconds
9 seconds
MacBook Pro M1, 8GB (2020)
rust-analyzer
7 seconds
14 seconds
As you can imagine, 4 months is not a lot of time for a project as big as a Rust LSP. Rust Glancer is not a complete LSP yet, it has a lot of missing functionality, it has some known bugs, and it has a lot of things I want to improve.
At the same time, it is already pretty capable: it has a full indexing pipeline with type inference and a trait solver (chalk), most of the "normal" Rust syntax is supported, and most of the "normal" LSP actions do work as well: goto definition, hover, inlay hints, completions, you name it.
If you are interested, you can already try it out: just install the VS Code extension
here
, or, if you prefer, build and install the vsix from the
repository
.
The rest of the post contains the history of the project: motivation, LLM use, plans and roadmap. If you're not interested, you might want to check out the
project documentation
instead.
Difference with rust-analyzer
There are several reasons why rust-analyzer consumes a lot of memory:
Rust workspaces genuinely have a lot of information that must be indexed: thousands of functions, structures, traits, relationships between these, function bodies and statements in them, etc. Each of these needs to be analyzed and remembered, and you can't really cheat if you want to have things like "find all references to this structure".
rust-analyzer uses
salsa
as its database. It's an incremental query-based database, which lazily computes all the data you need without having to explicitly "record" everything. It is a very cool approach, but it's inherently tied to memory, which makes it hard to move parts of data from memory elsewhere.
rust-analyzer uses
rowan
for syntax tree representation. The cool property here is that it allows partial invalidation: if only a part of the file changed, only the relevant bits have to be reparsed, which makes it faster than having to re-parse the whole file on each keystroke. However, the tree-like representation inside of it can cause heavy memory fragmentation (meaning that the amount of RAM taken from the OS is higher than the amount of "actually used" RAM).
(1) is something we have to live with (though there are a few optimizations we can do there which Rust Glancer does), but (2) and (3) are the consequences of the rust-analyzer architecture. rust-analyzer chose them to make the LSP faster, and it does work for that purpose.
The idea I had when I started the project: what if we
don't
try to make an incremental LSP? What if all we have is a frozen analysis result that gets invalidated on save? It obviously will not be as fast as rust-analyzer, but it will give us the properties we seek:
analysis results can be offloaded to the filesystem and loaded to memory only when they are actually needed.
saved analysis is reusable, and since it's already offloaded to the filesystem, it can be reused after the editor restart.
This is the core idea of Rust Glancer.
It indexes the workspace once and preserves results in the filesystem, and then whenever queries need something, they can load the required information for the duration of the query.
It doesn't come for free though: frozen workspace analysis is slower than lazy incremental by definition, since loading and deserializing data from filesystem is slower than loading from memory. To mitigate that, Rust Glancer has to use some tricks: for example, when you type, it doesn't perform full blown analysis on each keystroke, it instead attempts shallow analysis of the current body and reuses the
previous
complete index. This makes completions reasonably fast, but it also means that new items (imports, structures, traits) are not "indexed" until you save the document. Which, hopefully, should not be a problem: you really get used to it fast, and at least in my case it does not feel overly wrong after a while. If that sounds scary, I suggest to just try it, it really is not.
For people who rely on agentic workflows, Rust Glancer is also optimized for large amount of out-of-editor changes. I'm not sure why, but in rust-analyzer I've observed that when agents edit the code, inlay hints can get out of place, and I had the same problem in Rust Glancer initially, but it was resolved by implementing a custom file watcher and tweaking it somewhat. The server also has lower priority for out-of-editor changes, so agentic changes do not cause rapid re-indexing.
Still, it's important to understand that Rust Glancer has some benefits, but also has some drawbacks (besides being incomplete, obviously) compared to rust-analyzer. Maybe I will manage to solve some of them eventually, but it's highly unlikely that Rust Glancer will ever become "just like rust-analyzer, but better". I imagine that rust-analyzer will remain the default choice for projects that care about completeness and keystroke accuracy, while Rust Glancer will work for people with weaker machines
or
people who are ready for some sacrifices to reduce RAM usage.
How and why it happened
I have been writing Rust professionally for ~7 years, and since pretty early on I started observing how the compiler and its tooling are developed. I've made some contributions to rustc, clippy, and rust-analyzer, and I've spent dozens of hours reading its source code just to teach myself. So I was pretty much aware
how big
of a project a Rust LSP is.
At the same time, I have a love-hate relationship with rust-analyzer. It is absolutely beautiful except for two things: memory usage and initial indexing (especially with build scripts / proc macros enabled). These problems seem to be brought up quite a lot, but in my case they are even more drastic: I have a rather stupid workflow where I have two identical IDEs open on two displays with a bunch of projects inside a workspace. So the memory consumption is roughly 2N, and with my last set of the projects I had to work on, rust analyzer was consuming 16GB of memory that I, ugh, would prefer to have available for other uses; not to mention that each time I opened VS Code, my PC fans would go brr because of a ton of parallel indexing jobs.
At some point I thought that I am fairly confident in my Rust knowledge, so I probably don't need a full-blown LSP, and can use something simpler and more memory efficient. I decided to try building a "smart ctags for Rust". I very explicitly did not want to build an alternative LSP, because of how insane of a task it is. Little did I know...
The initial progress was going pretty smoothly: I made use of rust-analyzer's syntax library, lowered items to internal representations, then built definition maps and module structure, got all the declarations indexed. It was so surprisingly straightforward that I decided to do some primitive body lowering. Then I decided to add very very simple type propagation. Then it turned out that naive type propagation doesn't give me much -- but I already had these nice inlay hints, so I wanted more. Overall, I don't care about complex cases and nightly features, right? (
Right?...
). So then came naive trait resolving via impl header matching. It's quite addictive, you get it.
The illusion, however, broke when I decided that it is pretty reasonable to expect the following code to be supported as well:
fn mul_by_two(vals: &[u8]) -> Vec<u8> { vals.iter().copied().map(|v| v * 2).collect()}
The
code
is pretty simple, but in order to support it we need:
Slice type support
Closures / Fn traits
Trait solving
Associated type projection
A bunch of nightly stuff
the last item is funny: I wanted to avoid nightly, but I somehow didn't think that
std
(or sysroot in general)
breathes
nightly. Welp.
So all in all, one feature after another, I slowly was getting from "smart ctags" to a "real LSP".
Probably, the three biggest milestones were:
Declarative macro expansion (I hate declarative macros now). Thankfully, I was able to reuse most of rust-analyzer's infrastructure for that.
Proper type inference engine. It was a big "oh wow" moment when I truly realized how type inference works (in short: we "link" all related type bindings in a big inference table, and then we try to get evidence from all possible places, where providing evidence can solve types for multiple places). It was the moment that probably brought me the most joy during the work on this project so far.
Proper trait solving engine. I initially wrote "it's highly unlikely that we will have a trait solver in this project", but then I
really
wanted to get the abovementioned iterator example to work properly. I resisted integrating trait solver for a while, trying to have naive hacks like naive trait impl matching + specialized handlers for
std
traits, but it was getting more and more complex while working pretty poorly. Then I gave up and integrated Chalk, which turned out to be significantly simpler than the whole hierarchy I have built. Making Chalk fast was another challenge, though.
Somewhat separately, probably the thing I am most proud of (and the thing that made Rust Glancer possible -- had I not designed it early, the project would die very quickly) is a cool profiling stack that can measure performance, memory usage (both natively, tracking actual allocated objects, and with jemalloc), profile data on demand, and compare LSP against rust-analyzer, as well as a set of benchmarks running in CI. If you're interested, it's partially covered in the docs (
1
,
2
), but I'll work on a more detailed coverage later.
Probably ~1.5 months ago I started using Rust Glancer as my daily driver instead of rust-analyzer. Now, I am happy with its state enough to present it to a larger audience.
LLM use
This project was built with heavy use of LLMs. It is not vibe coded, though.
I am verifying each pull request to make sure that I am happy with the state of the codebase. If you need proofs, you can check the git history: it has PRs with 10k+ lines of diff, but these are multiple days apart despite the fact that I work on this project nearly every day since its inception. I care about the code, and tbh it would be weird for me to spend 4 months creating a
Rust LSP
if looking at the code wasn't something I do a lot.
I am not going to pretend that I am an experienced LSP developer and the code is perfect. It is in a state that I can work with, but I understand that some bits might not be idiomatic in terms of compiler tooling design. The code has a lot of comments, and I tried really hard to make sure that these comments are not sloppy but helpful, because
I
have to read them all the time; so far the quality is obviously not as good as professionally written human docs, but IMHO it's pretty helpful and not annoying to read.
A large part of the journey is learning. LLMs can be pretty good domain experts, and LLMs know about LSP design much more than I do. At the same time, LLMs are not great at building big projects. So the following loop happened multiple times during development:
I build something new.
LLM proposals seem reasonable, so I go with them.
It works but something bugs me.
I think about the design for a while and see a big flaw.
I work with LLM to fix it (sometimes for a week, if the screw up was particularly big -- but the bigger the screw-up is, the more I learn).
So on one hand, if I am to attribute code ownership to the LLMs, I can complain: "LLMs tried to derail the project so many times!11". But since it's
my
code, I think that the code might get worse at some moments, but as I learn, I get to improve it. Which is pretty normal software development flow, just accelerated.
All in all, LLMs are just a tool, and it's one's choice to use it responsibly or outsource thinking to it.
Given the amount of witch hunting today, I have just one request: do not reduce me to a clanker. It is my code, so if you consider it to be slop, call it
my
slop, not AI.
I am open to criticism and will happily listen to feedback: the more I learn, the more I can improve the codebase. Whether I use LLMs for that or not does not matter that much, in my opinion.
What's next
The project is already in a state where it can be a daily driver for some users, but I have rather big plans for it.
So in the coming releases, you might expect:
Further performance optimizations
Some more memory optimizations (primarily during indexing, plus there are a few fragmentation issues happening after a full indexing run that I want to fix)
Potentially proc macro support (I have some weird idea that will not require actual code execution, but it'll take a while to prepare).
Some features are unlikely to be supported though, such as build scripts / proc macros support via proc macro invocation (e.g. anything that requires untrusted code execution). I also don't plan to work on things that are unnecessary at the current state of the project, such as migrating to the new trait solver. Niche things like particular nightly features will likely be postponed until the project reaches some degree of maturity with stable Rust.
Additionally, there is a lot of cool little tricks I've done in Rust Glancer that I'm somewhat proud of (aligning allocation lifetimes to reduce memory fragmentation, engine-as-a-subprocess model to help with both memory fragmentation and multi-workspace projects, sharded cache, and others), so if people will be interested, I'll be happy to write some blogs telling about how Rust Glancer works under the hood. It's partially covered in the docs already (
1
,
2
) if you want to get some info right now.
But in any case, I hope that the project can be helpful for some folks already, and for more folks in the future.
WPD won't replace stolen Flock cameras, citing public trust
Following a public outcry over the use of the surveillance, the city of Winona announced on Wednesday that it will not replace its stolen Flock cameras, saying that although the cameras helped solve crimes and prevent harm, the community’s trust in police was more important. The Winona Police Department (WPD) used these automated license plate reader cameras to monitor all of the highways leading in and out of Winona for stolen vehicles, drivers with revoked licenses, and missing persons. On August 3, the WPD discovered that a vandal or vandals had cut off all eight of the city’s Flock cameras.
Flock cameras and similar systems have been controversial across the country for the potential misuse of the systems by local law enforcement and the potential of data sharing with third-party companies and the federal government. After the WPD cameras were stolen, hundreds of Winonans spoke out on social media over, criticizing the city for using them in the first place, praising the vandals, and urging the city not to replace them.
Over 900 people commented on
a WPD post
about the theft before the department restricted further comments. “
It’s almost like people don’t want to be watched 24/7,” one person wrote.
“
I need to buy whoever did this a drink,” another stated. “
Hopefully it was a sworn officer, realizing that their duty to uphold the constitution necessitated the removal of the devices,” a third commenter quipped.
Flock offers free replacements for cameras damaged or destroyed by vandals, and most cities’ insurance policies would help cover replacement costs, roughly $3,000 a piece.
However, the WPD announced on Wednesday that it would not replace the cameras.
In a statement
, WPD leaders wrote that “Flock cameras have been an effective and impactful resource for law enforcement,” helping them locate a homicide suspect, find people in crisis to make sure they are safe, track down stolen cars, and solve hit-and-run cases, and that those uses have “been guided by clear policies, appropriate training, and ongoing oversight.”
“At the end of the day, however, our community’s trust is the foundation of effective policing, and we’ve worked hard to build and maintain that trust,” WPD leaders continued. “While Flock has been a valuable tool, we believe its use has contributed to growing concerns about trust in policing, both locally and across the state and nation. After careful consideration, [the] Winona PD has decided not to reinstall any Flock cameras in the city of Winona.”
I want to talk about something that's been on my mind for a while: search has quietly gotten worse over the past several years, and I think it's worth being honest about why, and what it's actually costing us.
Anyone who's pasted a specific error message into a search box recently knows the experience. The first several results are usually SEO content, the same underlying answer reworded a dozen different ways, padded with filler paragraphs before it even addresses the actual problem, because ranking algorithms have historically rewarded length and keyword density over direct usefulness. Mixed in increasingly are AI-generated pages that read confidently but occasionally get the actual technical details wrong, with no obvious signal to the reader that anything's off. Somewhere further down, if you're persistent, is often the original, genuinely useful answer, sometimes from a forum post years old, occasionally scraped and republished elsewhere with the context stripped out.
This isn't really anyone's fault in a simple sense. It's what happens when the economics of the web reward getting in front of an algorithm rather than being correct. That gap between "ranks well" and "is actually right" has always existed to some degree, but AI-generated content has widened it considerably, since it's now possible to produce large volumes of plausible-sounding text far faster than anyone can fact-check it, and ranking systems haven't fully caught up to distinguishing genuinely useful content from confident-sounding filler.
The response from several major search products has been to layer AI-generated summaries directly into results, effectively synthesizing an answer from whatever's been indexed, including the very content I just described. I understand the appeal from a product standpoint, it reduces the number of clicks needed to get an answer, which reads well in almost any metric. But it also means users are increasingly being handed a confident paraphrase instead of a source, with no easy way to verify whether that paraphrase is accurate unless they already know enough about the topic to catch an error. That's a strange thing to optimize for, since the people who most need a reliable answer are often the ones least equipped to spot when they've been given a wrong one.
What actually concerns me most isn't the quality of individual search results, it's what this shift is doing to the underlying skill of research itself. Holding a real question in your head, forming a hypothesis, checking it against multiple sources, and noticing when two sources disagree is a skill that gets sharper with practice and duller with disuse, the same as anything else. Every time someone accepts a generated summary at face value instead of following through to an actual source, that's a small rep they didn't do. It's not dramatic in the moment. It adds up slowly, the same way any skill quietly erodes when you stop exercising it, and you tend not to notice until you're actually asked to do the work yourself and find it harder than it used to be.
This pattern isn't limited to search either. It shows up anywhere people reach for a generative tool to skip the effortful part of producing something, an essay, a difficult message, a first draft of anything. The output is often fine, sometimes genuinely good, which is part of what makes this hard to talk about honestly. But the actual struggle of starting from a blank page is a large part of how people get better at generating ideas in the first place, and skipping that struggle repeatedly doesn't make someone faster at it over time, it tends to make the underlying skill weaker from lack of use.
I want to be clear that I'm not against these tools generally. I use them myself, including for mundane things like tightening up a resume, and there are plenty of contexts where they add real value. The distinction I care about is between a tool that extends what someone is capable of doing and a tool that quietly does the thinking for them while preserving the feeling that they're still in control. Search, at its best, used to require a small amount of genuine cognitive effort: comparing sources, weighing credibility, forming your own synthesis. It was never perfect, but that friction served a purpose. A lot of recent product decisions across the industry, not maliciously, but as a natural consequence of optimizing for engagement, have been quietly removing that friction, and I think it's worth pausing to ask what we're trading away in the process.
I don't have a tidy conclusion here, and I'm skeptical of anyone who claims to. This isn't a problem with an obvious fix, and I don't think one side project changes much about how the incentives above actually work. Mostly I just think it's worth naming what's happening plainly, because the trade is easy to miss when it happens one convenience at a time, and a lot easier to notice once you say it out loud.
The people vs the AI overlords
Anarcat
anarc.at
2026-08-19 10:14:42
Previously in this series: The Four Horsemen of the LLM
Apocalypse.
In a post to oss-security, my (Debian) co-developer Russ Allbery
stated that "open source software [OSS] is coming face to face with a
motivation crisis that has been building for a long time". His point
is essentially that large l...
In a
post to oss-security
, my (Debian) co-developer Russ Allbery
stated that "open source software [OSS] is coming face to face with a
motivation crisis that has been building for a long time". His point
is essentially that large language models (LLMs
1
) are making the
existing OSS community crisis worse. For him, it's the flood of code
reviews, but he argues that varies according to people's desires, for
others it's security issues and so on.
I think Russ is right, but I would argue there's something much bigger
than our open
communities
going on here, and it's about the entire
field
of computing. This pressure is on
all
of us, regardless of
whether we work on open source software or not.
How people use models
People using LLMs in their workflow have
radically
changed how
programming works, even for
people who claim to avoid
vibe-coding
. And I'm sorry to single out one poor maintainer here:
it's not you, Brian, you're just one example among many. But this is
typical use of those models nowadays:
Once it’s done, I’ll use
/code-review
and let Claude spawn
sub-agents to do a full review of the new code. This usually finds
some problems, even problems that the “main” Claude instance didn’t
find during its validation. I usually keep running
/code-review
again and again after finding and fixing issues, until there aren’t
any left.
Think about what that means for a minute. This is automation built to
fire up dozens of agents crunching at a problem for minutes if not
hours of GPU compute time, in parallel. This is essentially a couple
of shelves in a datacenter rack, totally maxed out on power and
cooling, abstracted behind a cute little
/code-review
command.
The author, here, is rightly concerned that "Anthropic could pull the
rug out and require API pricing", which is perhaps a code word for
"charging something closer to actual costs". Brian also pays lip
service to environmental and societal costs but those are largely
abstracted away, so let's keep that conversation aside here as well,
as we have
discussed it before anyways
.
But clearly, this way of working has an (
externalized
) cost, to
say the least.
For decades my work has been focused on free and open source
software. I've long stopped using proprietary operating systems like
Windows or Mac, and even before that switch, I was mostly using free
software on those platforms, partly out of principle, but also because
I was too poor. So the tools of my trade are free, and I build free
tools with them.
It feels like we're going backwards: when I was in school, a millennia
ago, my classmates didn't have access to a compiler and were wondering
how they would scrape the money to buy a compiler like
Borland's
or
Microsoft's
. I had a compiler built into my operating system
(
FreeBSD
at the time), so that wasn't a problem for me. For them,
it was a significant expense, but at least those expenses (or more
shady sourcing of programs
) were a one-shot deal.
Fast forward 30 years, and software is rented: you pay monthly for
Adobe's Photoshop and Microsoft's office suite just like you pay for
Netflix, Disney+ or Spotify
2
. And now you need to add dozens (if not
hundreds of dollars) of monthly credits to access LLMs on top of that.
So, now we have to
pay
to get anything done? This is peak
enshitification
of our job: first they steal our work to train their
models, and then they sell it back to us at a profit.
Attacking the engineers
AI is coming for our jobs, as engineers, if not
everyone
, according
to the narrative. For a while now, our job market has deteriorated:
less jobs, for less pay. Lots of skilled engineers looking for work
and finding crap jobs then still looking while working.
This is not by accident.
3
We engineers have a
lot
of power, it is not
organized, but that's just a couple of unions away (
easy
!). Tech
overlords know this, so they are attacking our profession, directly,
by forcing us to train and use models that
they
can control.
Even in environments where programmers are not
forced
to use LLMs,
the mere pressure of other people's LLM-generated work is huge. One can
be forced to review LLM outputs, or just peer pressured you into
producing more.
We're now supposed to accelerate delivery, because models can
presumably
do things so much better and faster. With supply chain
security becoming such a large vector that we now have
worms crawling
around developers accounts on NPM
, increasing the delivery cadence
seems like a really bad idea.
4
The LLM hype is part of the larger wave of cyberwar against workers,
against water, against the Earth, against all the people. This is not
a matter of individually "adapting to the reality" or personal choice,
but a political, social, hard problem we need to address collectively.
Is Online Privacy Possible? How Digital Identities Can Help
Bleeping Computer
www.bleepingcomputer.com
2026-08-21 10:00:10
Using the same email, phone number, payment method, and other identifiers makes it easier for data brokers and attackers to profile your activity. Anonyome Labs explains how separate digital personas can reduce correlation and limit the impact of breaches, spam, and identity theft. [...]...
How can normal users increase their privacy, safety and security online?
Over the last two decades, the internet quietly rebuilt itself around a business model that depends on knowing everything about you. Every app you install or use, every account you create, every website you visit, and every form you fill out becomes another data point feeding a system designed to track, profile, and monetize you and your identity.
This process is often referred to as
surveillance capitalism
and creates an economy where attention and personal data are the product and you are the raw material.
The mechanics of this are almost invisible day to day. A single email address becomes the thread that ties together your shopping habits, your health searches, your location history, and your social connections.
Data brokers exist specifically to aggregate these threads, buying, selling, and cross-referencing fragments of your life until they can construct a profile more complete than most people would recognize about themselves.
None of this requires a breach or a hack – it's simply how the default internet works.
Data brokers
are often the most consequential handlers of personal information that operate without consumers’ awareness or informed consent.
The result is that privacy is no longer something you can expect. It has become something you have to
actively
construct, piece by piece, against the grain of nearly every service you use. The harms of this model are diffused and delayed and you don't feel the effects of a data broker profile the way you feel a stolen credit card.
The damage shows up later, as spam, as price discrimination, as
identity theft
, as a general erosion of control over your own digital identity.
The graphic below shows the problem of using a single identity across your online actions. When a data breach occurs, everything connected can be exposed and tied back to you. Data brokers can use it to construct a complete picture of your life, and this valuable information is for sale.
Artificial Intelligence (AI) systems have made the problem significantly worse.
Data brokers can now use AI
to link your different actions in a way that was previously thought impossible. AI’s expertise is data analysis, working through vast amounts of information to correlate your actions into a valuable profile.
In this world of surveillance capitalism, can we shift the privacy pendulum back in your favor? Is it even possible to be private, secure, and safe online?
Personas and Compartmentalization
If surveillance capitalism works by linking everything about you into one exploitable profile, the countermeasure is structural (not just legal or political): break the correlating identifiers.
This is the premise behind compartmentalization. Instead of using one set of identifiers such as one email, one phone number, one payment method, one communication handle across every context in your life, you deliberately compartmentalize activities into separate, purpose-built personas.
One persona for online shopping, a different one for dating apps, another for travel, another for marketplace listings, and even another for that newsletter you're not sure you trust yet.
Each persona operates as a self-contained identity with its own email address, its own phone number, its own payment method, its own browser, and its own communication handle. Crucially, these personas aren't connected to each other or back to your actual identity in any way a data broker or advertiser could observe.
As shown in the graphic below, if a persona gets swept up in a breach, starts attracting spam, or gets sold to a marketing list, the damage is contained and is not tied back to you.
This is a fundamentally different privacy model than the one most security tools rely on. Most tools try to protect a single identity better with stronger passwords, better encryption, more careful permissions.
Compartmentalization instead assumes that any single identity is eventually going to become correlated and anticipates corrections by ensuring that no single identity is valuable due to its changeability. This process is less about building an impenetrable wall and more about not putting all your value behind one wall in the first place.
The elegance of this approach is that it doesn't require the rest of the internet to change. You don't need every company you interact with to suddenly adopt better data practices. You just need a layer that sits between you and them, generating and managing these personas on your behalf.
Anonyome Labs patented many of the ideas related to creation of online personas and compartmentalization, here are some examples:
Compartmentalization and personas are an important advancement, but the harder problem (and the one that has occupied most of our product decisions) is making it usable by typical users and automatic enough that people can do it consistently, without a computer science degree, and without constant friction.
As shown in the graphic below
MySudo
was designed to implement this paradigm and enable each user to create up to 9 personas or
Sudos
. Each Sudo provides a different:
Phone number
that can make and receive phone calls and SMSs;
Email inbox
that can send and receive emails;
Virtual payment card
for purchasing online;
Communication handle
to enable end-to-end encrypted (E2EE) messaging, voice calling and video calling (similar to WhatsApp);
Browser
for complete separation of browsing.
MySudo comes in two form factors:
A
mobile app for iOS and Android
that allows each persona to communicate externally with phone calls, SMSs, and emails. It allows creation of individual payment cards and to have end-to-end encrypted messaging, email, voice and video. It also has a separate browser for each persona.
A
desktop companion app for Windows and Mac
that allows management of persona emails in a form factor that provides support for longer and more complex emails. More features are coming to the MySudo desktop app soon.
MySudo is part of a growing family of privacy and security applications from Anonyome Labs that also includes a privacy focused
VPN
and
Password Manager
(coming soon).
Individual online privacy has been under surveillance and attack almost since the beginning of the web – and now users have identity-based tools to fight back. By creating multiple personas that allow you to compartmentalize your life, you too can reap the privacy benefits.
Your identity is already being pieced together. Stop handing over the pieces.
Download MySudo now
to create separate digital identities that keep your personal information private and out of reach from data brokers, scammers, and the next data breach.
August 21st, 2026 - A new Stocks widget and a better everyday Assistant experience
#
Kagi Search
Bringing Stocks up to speed
We've revamped our Stocks widget. It should appear more often when you need it. It can now display information about exchange-traded funds in addition to stocks. Most importantly, it now features a price chart, with animations between time windows that instantly contexturalize how big the price fluctuations you're seeing are compared to the wider story:
As well, we've
added a setting
for removing paywalled links from search results automatically.
More powerful search
Search across all your threads, sort by recency or alphabetically, and start with
/
to filter by folder.
More control with calmer settings
Now you can choose whether temporary threads stick around for 24h, 7 or 30 days. All within a calmer, easier-to-scan settings experience.
July 30th, 2026 - Kagi Assistant on the go and design refinements for Search
#
Announcing the official Kagi Assistant apps
Kagi Assistant is now available as a native app for
iOS
and
Android
!
Ask a question, explore the web, work with files, conduct in-depth research, or choose from leading AI models, all from your phone. Your threads and Custom Assistants stay with you, so you can pick up wherever you left off.
These are the first steps towards delivering a fantastic Kagi Assistant experience on mobile, with much more to come.
You can now report an assistant response without leaving the conversation. Hover over any assistant message and select the thumbs-down button to open the feedback form, where you can report issues for reasons ranging from UI bugs to harmful content.
Note that when you submit a report, the full thread is shared with Kagi for review. The report and its associated copy of the thread are automatically deleted from Kagi’s review records after 30 days.
Export or delete all your threads
We've also added important controls, so you can now export all your threads or permanently delete them at once from
Settings > General
.
Kagi Search
A sharper search experience
We’ve polished the search results page to make its controls easier to find and understand. From the filter bar to domain-related options and menus, these updates bring greater clarity and ease of use to the features you rely on most.
Exchange rates, right in your search results
Next up in our broader effort to improve search widgets: currency conversion. Comes handy when you’re planning a trip, shopping abroad, or simply want to keep tabs on exchange rates.
Other improvements and bug fixes
Kagi Search
Fixed several animations that didn't respect the system's
prefers-reduced-motion
setting
New controls to completely turn off AI-based features in search
We've added an option to disable access to AI features in search, under
settings/ai
.
We're also planning to add this option to onboarding, so new users can personalise their Kagi experience from the start.
It's finally here! We believe that Kagi's application of AI should always be
useful
- there when you'd like it, and never when you don't, and always respecting your privacy.
This took us some time to navigate the right way to communicate this option. We did not want to create a confusing narrative as a company adding a toggle while continuing to invest in AI features elsewhere in our portfolio. But in the end, we want to stay true to putting you in control of
your
search engine - so here you are!
We deeply thank the community for their feedback and patience.
Flip coins and more sports widgets
By popular demand, our dice widget has gained the ability to roll dice with any number of sides. We're not sure what kind of games you're playing that need d7s, but we support them now.
We also added support for flipping coins, which are really just two-sided dice when you think about it:
We've added a set of switches on
https://kagi.com/settings/more_search
so you can disable any of our widgets you don't want to see. The toggle descriptions include links illustrating the widgets' capabilities so you understand what you're turning on or off; go check it out!
Orion browser ✴︎
This week, we’re launching
Orion 1.1 for macOS
, one of the most significant updates in our history. This version is built around three major new features (in addition to 170+ smaller improvements and bug fixes).
A New Interface ✴︎
When Apple released
LiquidGlass
, the reception was mixed—even within our own team. The demand was there, but we weren't ready to just copy-paste what Safari had done. They had even removed compact tabs!
So, we created our own implementation.
Containers ✴︎
Just like Firefox, we now offer containers. What are they? Each tab becomes completely isolated from the others: total privacy and the ability to log into multiple accounts on the same site from the very same window!
A Personalized Browser Border ✴︎
The current trend is an elegant, transparent border seen on many browsers. The problem is, they don't match Apple's design language. So, what did we do?
As we usually do: we made it an option! And we took it even further: transparency, solid colors, gradients, and even an automatic color-match with the website for total immersion.
This option is exclusively available to Orion+ subscribers.
Orion+
Orion is your free browser, but we offer a
support plan
to maintain the independence that guarantees your data is not, and will never be, sold to advertisers—or worse.
We have a dedicated website where you can download all the versions we currently support, as well as any we may support in the future (macOS, iOS, iPadOS, Windows, and Linux):
https://orionbrowser.com
Kagi News users from all over the world loved being able to read their news in the language of their choice, stress-free, and even add new topics.
Kagi Translate users loved the contextual features that provide a spectacular translation quality — far beyond what typical machine translation offers.
But these unexpected successes led to a massive spike in our costs for applications offered for free.
As a result, we have temporarily removed translations and left access to the articles’ original languages as well as English. Kagi Translate will be back in the coming days as a subscription-based service.
Thank you for your patience and your trust 🙏 we hope to have everything up and running again very soon!
We’re starting a broader effort to improve our search widgets! First up: sports scores and dice rolling.
Sports scores
now show up in a sidebar next to search results, so you can quickly check upcoming games, live scores, and recent results,
just in time for the World Cup
.
And we’ve also
added dice rolling support
for all you gamers out there, in case you ever need to roll
a d20
,
2d4 + 2
, or perhaps even
8d6
.
The new Kagi Assistant is here
Over the last few weeks, we’ve been rolling out a new Kagi Assistant experience. Most of you are already using it, and today we’re officially retiring the old assistant.
This is more than a visual refresh, we rebuilt the Assistant experience around a new layout, smoother web and mobile use, and a lot of UX improvements that add up quickly.
And just as importantly, this gives us the foundation we need for the next set of Assistant improvements we’ve been working towards.
Note: there is one notable change - folders have replaced tags. This means each thread can now belong to only one folder. We appreciate this is a downgrade for users who relied on multiple tags per thread, and we don’t want to handwave that away. We made this tradeoff because folders give Assistant a simpler, more predictable organisation model, and because multi tag usage was relatively low: about 20% of active accounts used tags at all, and appx 4% had any thread with more than one tag.
Still, for those affected, we understand this change may be frustrating. Thank you for bearing with us as we build towards a stronger Assistant experience!
Kagi Translate update
We've paused free access to Kagi Translate while we sort out running costs, so you'll need to be signed in to use it. If you have an active subscription, Translate still works. Sign in on translate.kagi.com or in the mobile apps. We share more details on this decision in this
blog post
.
Other improvements and bug fixes
Kagi Search
Different output formats for Wolfram Alpha results
#3183
@mm00
NEW: Extraction now keeps links from the original document, to enable deeper crawling flows.
NEW: Related searches is now part of the API responses, with more metadata than the v0 version where applicable.
NEW: Per-key cost tracking is now enabled. You can select a key in the usage page to see the specific key cost attached. (Cost tracking only available from when we deployed, historic data is not present.)
Fix: Extraction is now
faster
and more reliable.
Fix: Personalization rule types are now correctly validated with the doc types.
Fix: Various other internal improvements for a more stable experience.
Kagi Assistant
Regression: new assistant scrolls to bottom when inference completes
#10648
@spiffytech
The new Assistant does not let me edit the output from the model.
#10670
@Fernold
Japanese IME: Enter key submits message instead of confirming composition
#10707
@n22z9y28vh
New system prompt causing regressions esp. in no-search mode, and ignores /system_prompt_overwrite
#10684
@igakagi
Kagi Translate extension wrongly detects certain monitor as 'mobile'
#10693
@Roon
Post of the week
This week's featured social media mention:
Featured Kagi tip 💡
Here's
a guide
on how to make Kagi truly yours with custom CSS. Tweak colors, fonts, and layout, hide elements you don't need, or apply a community theme for a search experience that looks exactly how you want.
May 21st, 2026 - Search API preview opens to all users
#
Kagi Search API is now in public preview
Today we’re making the Kagi Search API preview publicly available, giving builders access to Kagi search across web, images, videos, news, and podcasts. The API is ready to use today, and we’re using this preview to transition existing beta API users, collect feedback, and finish the remaining launch details before the official announcement.
As a thank you to our subscribers, we’ve added
$5 in API credits
to your account. You can use them right away to try the API, explore what’s possible, and see how it fits into your workflow.
The API is not just a generic search endpoint, queries can inherit the preferences attached to the Kagi account behind the API key, including lenses, upranks, downranks, and blocklists, so applications can search through the same trusted and filtered view of the web that users have already shaped in Kagi.
Our Privacy Pass extension, which allows you to prove to our servers that you're a subscriber without revealing your identity, has gained a
long-awaited
toggle that makes it active only in incognito windows, so you can benefit from personalized results most of the time but have added anonymity for your more sensitive browsing. The extension was almost completely rewritten in the process, squashing several long-standing bugs.
Try it out!
Support for custom languages, explanations for alternative translations, word suggestions for dictionary mode, app shortcuts, and many other improvements!
A
quick video
from our team to serve as a reminder of what Kagi is all about: the web, and your time on it, belong to you.
April 30th, 2026 - Kagi API preview and ecosystem updates
#
Kagi APIs: the same search technology that powers Kagi is opening up to developers
Starting next week, we’ll begin onboarding developers to the Kagi API dashboard. Access will roll out first to people who joined the
API waitlist
or contacted Kagi support.
With the new Search API developers can bring Kagi Search into their own apps, tools, and AI systems. Here's an early look:
If you'd like to join this early preview of the Kagi API, please fill
out this form.
We'll reach out next week!
Kagi Search
New landing
We updated our landing page to bring awareness to Kagi's wider ecosystem beyond search.
Check it out!
This is the first of many steps toward helping more people discover everything Kagi has to offer.
IP address and subnet search to bring up the Wolfram Alpha answer
#10147
@dronics
Story corrections, both from user reports and our own continuous fact-checking. When something turns out to be wrong, we fix it and show a small correction notice on the story, with the changed sentence highlighted on your next visit.
Stories can pull in related coverage from other categories, so a single big story can span Science, World, and Tech when it makes sense.
Cleaner prose in hard-news categories: fewer filler phrases, less editorializing, more neutral writing.
Snappier all around: faster initial load, much faster story search, and browser back/forward now restores the page instead of reloading it.
Custom category order syncs reliably across devices now. Fixed several cases where reorders were lost or overwritten.
Category tabs use proper ARIA semantics for assistive tech.
Pasted text from books or PDFs is auto-formatted: broken mid-sentence line breaks, hyphenation across lines, and stray whitespace get cleaned up. An undo toast lets you revert if you wanted the original.
Auto-language switch now shows a toast with undo, and skips ambiguous cases like uncertain, mixed, or mid-typing input.
Pin any language to the top of your list, including custom or non-standard ones.
Romanization shown beneath alternative translations into Japanese, Chinese, Korean, Arabic, Russian, and other non-Latin scripts.
Link previews (Open Graph) for translated text now show the actual translation when shared on social media, instead of a generic logo. The /extension page also got its own dedicated preview.
New languages: Seto, Võro, Montenegrin, and Badini Kurdish (with both Arabic and Latin Hawar scripts).
Formal Ukrainian now correctly capitalizes Ви and Ваш.
Downloaded translations get the right file extension based on the detected content format.
Post of the week
Follow us
and tag us in your comments, we love hearing from you.
Kagi is growing
The team is expanding, and we're looking for talented people who want to help build a better web alongside us. We're hiring for multiple roles, including:
Product Designer (UI/UX)
: Take strategic ownership of end-to-end design across Kagi's product ecosystem.
Apply here.
An
Education Partnerships Lead
: If you believe the most important thing technology can do for students is teach them how to think for themselves, we'd like to talk.
Apply here.
A
Senior Platform Engineer
: If you have strong opinions about API contracts, auth correctness, and migrating user data without losing anyone's trust, we'd like to talk.
Apply here.
We also have openings for a Senior Search Engineer, Senior Platform Engineer, Senior Full-Stack Developer (Kagi Labs), and an AI Specialist. See the
full list of openings
here.
Kagi tip of the week 💡
Between AI-image filters, clickbait controls, reverse lookup, and source filters, there's a lot of power hiding behind the Images and Videos tabs.
Here's how to get the most out of them
.
This sets a cookie so your device remembers. To undo it, click Back to the Future at the bottom of the page or visit
https://kagi.com/?year=present_day
Post of the week
Here is this week's featured social media
mention
:
Follow us
and tag us in your comments, we love hearing from you!
Kagi tip of the week 💡
Did you know you can set up URL redirects to reroute search results to the sites or frontends you prefer?
Here's how
, with examples from the community.
Kagi art
AI and ads are a toxic combo. Across the Kagi ecosystem, there are no ads, and we're actively working to keep slop out of your search results. Read more about Kagi's SlopStop initiative
here
.
March 19th, 2026 - Small Web Expansion and Translate goes viral
#
Kagi Small Web just got bigger!
Kagi's
Small Web
just got a whole lot bigger. With over 30,000 feeds and new browser extensions, mobile apps, and categories, there's never been a better way to discover the independent web.
On March 16, we launched our latest fun language on Kagi Translate,
LinkedIn Speak
, and it quickly went viral on social media, generating millions of engagements. Check out some of the press coverage below:
Time Travel
: Browse news history by date. Pick any day on the calendar and read past summaries.
Content Filter
: Hide or blur topics you'd rather skip. Choose from built-in presets or add your own keywords.
Post of the week
Here is this week's featured social media
mention
:
Don't forget to
follow us
and tag us in your comments, we love hearing from you!
Kagi Specials
We're excited to welcome the newest addition to our
Kagi Specials
program:
EasyOptOuts
! Kagi members in the U.S. now enjoy 25% off for life.
This is a service that removes your name, address and phone number from 200+ data brokers and people-search sites automatically. Deal is
reciprocated here
for any EasyOptOuts subscribers in your network who want to try Kagi.
Kagi art
"Free" search costs more than you think. With Kagi, you get zero ads, zero tracking, and AI on your terms.
We're introducing a new and improved Wolfram|Alpha widget with support for rich equations, plots, better region-dependent queries, and more!
Other improvements and bug fixes
Kagi Privacy Pass extension conflicts with Kagi Search extension in Firefox, breaking login token recognition in private browsing windows.
#6432
@stone
. This was a bug in Firefox -
thank you to Mozilla for the fix
!
The after-login redirect doesn't work for maps or assistant
#8407
@Boomkop3
Make “Translate with Kagi” appear directly in Android text selection menu
#9801
@Matou
Added 'email' writing style for proofreading
Added setting to toggle haptics ON/OFF
Fixed UI issue on Android where certain elements were being drawn under system bars
Post of the week
Here is this week's featured social media
mention
:
Don't forget to
follow us
and tag us in your comments, we love hearing from you!
Kagi Specials
Kagi is happy to be part of the
privacy alliance
with Windscribe, a feature-rich VPN with built-in ad and malware blocking and audited no-logs policy.
Through this partnership via
Kagi Specials
, Kagi members receive a 3-month Windscribe Pro trial, then lock in the Pro plan at just $49/yr
for life
. In turn, Windscribe members get 3 months of Kagi's Professional plan.
Community creations
If you're using
Scribbles
to run your blog, you can now add
Small Web
badges directly to your blog footer, just head to the new "Small Web" section in your blog settings:
Kagi on TV!
Kagi was
prominently featured
as a private alternative to Google on KTLA 5 News, including an interview with Kagi's very own John Bardinelli, who recently joined the team as our Growth Manager.
Feb 12th, 2026 - Kagi Translate on Android & iOS: translate anything, anywhere
#
Kagi Translate Arrives on Mobile
Kagi Translate is now available as an app for
Android
and
iOS
!
The app supports over 248 languages and offers context-aware image translation, live voice-to-voice conversations, and a rich dictionary with audio, to name just a few of its features.
Read the full announcement and feature highlights
here.
Fast Company
featured the launch
as a privacy-first Google Translate alternative worth noticing. A
similar guide
was published on The Intelligence, which covers tips and tricks to help users get the most out of Android devices.
Other improvements and bug fixes
Kagi Translate apps
Allow removing individual translation history entries in kagi translate mobile app
#9774
@alcroito
Kagi translate mobile app: Editing text in the middle causes scrolling / jumping around, makes it hard to edit
#9758
@alcroito
Inconsistent Swipe-to-Go-Back Gesture in Kagi Translate (iOS)
#9729
@xx
An option to individually delete translations on Kagi Translate
#9713
@xx
Stylesheet definition order for ‘small results’ (srgi)
#8591
@KKagi
Kagi Assistant
We've made changes to how we phase out older or superseeded models. When a model is being phased out, your Custom Assistants using it will first show a warning for ≈2 weeks. Once the model is fully retired, the Custom Assistant is disabled until you update the model in settings.
#5597
@Thibaultmol
Larger copy-pasted content is now automatically converted to a
.txt
file and works like any other attachment. This ensures the full original content is always preserved, even in very large threads that hit context window limits. In most cases, it remains fully within the context window. In longer threads, the original content is stored separately and retrieved as needed.
Do not re-rank Assistant threads when their title changes
#8434
@dreifach
Remove ads/upselling for flagship AI models in Kagi Assistant
#9693
@lasu
To mark Safer Internet Day, Fastmail explains why your search engine matters just as much as your email provider when it comes to privacy, and why
they recommend Kagi
to their users: "Adding Kagi creates a powerful privacy stack".
Addy.io joins Kagi Specials
We're excited to welcome Addy.io as a new partner on
Kagi Specials
! Addy.io is an email forwarding and alias service that helps protect your privacy by allowing you to create unlimited email aliases.
As part of this partnership, Kagi users can now access exclusive discounts through Kagi Specials, and Addy.io users can discover Kagi through their
perks
program.
Jan 29th, 2026 - Assistant reliability upgrades and Search refinements
#
Waiting for dawn in search
We published a new blog post on the state of search and the critical need for open index access. The dawn of a healthier, user-centric web is possible, but it requires structural change.
We've upgraded our Academic lens! Try it when you want research results drawn from scholarly and professional sources. Ideal for topics like medicine, sports science, or other specialist fields
Added functionality for users to manually set their location, improving local search queries. This is part of our weekly incremental improvements to localised search in Kagi.
Several accessibility improvements have been made, including corrected roles and proper fieldset semantics for our dropdown menus throughout the site, thanks to Tamara Cook, an accessibility consultant who proactively reached out to us via our support email. Thank you very much for the input!
If search fails because no upstream sources responded in time, show Click to Retry
#7795
@kirkmc
New favicon pixelated when default search engine (Firefox?)
#9585
@Replica6
Related search suggestion for current search
#7781
@Keli
Unable to Renew Plan After Reaching Limit in Kagi Assistant
#7152
@0rb
Related search suggestion for current search
#7780
@Keli
Kagi Search Android
The app now follows the "Open in External Browser" setting, opening search results either in-app or in your default browser
Kagi Assistant
Recommended models are now more useful to users. We clearly outline which base models we recommend: best fast (speed), best balanced (speed<>depth), best overall (max quality)
We've made several changes to ensure the Assistant reconnects you to any response in progress if your network connection drops or you navigate away from your browser. This means no more loading animations while you wait 🚧
We have a
YouTube playlist
with all kinds of guides, quick tips and tricks to help you get the most out of your Kagi subscription.
Post of the week
Here is this week's featured social media
mention
:
We truly appreciate your support in spreading the word, so be sure to
follow us
and tag us in your comments!
Kagi art
Technology should serve you, not trap or burden you.
Jan 15th, 2026 - New Year tune-up: smoother everything!
#
Kagi Search
Kagi Search Android app
We’ve made meaningful improvements to the Kagi Search app — faster performance, smoother overall experience. If you’re on Android, give the update a try.
We also hope this makes it even easier to share Kagi with the people you care about.
Let us know what you think
!
Improved app startup time
Updated search home screen with native text editing
Updated home screen widgets with faster access to Translate, Summarize and Assistant
Add settings to Kagi Search app to autofocus the search bar on launch and to move the search bar to the bottom
#9042
@conradsrc
Improvements/fixes to the Android app screenshots
#5019
@Niraj
Android app: Pressing Enter on a physical keyboard should search
#8838
@ItsHarper
Android app: image, news... etc don't stay selected in the first screen
#7207
@Ronzino
Android Share Menu: "Assistant" option appears twice, first instance should be labeled "Search"
#8773
@artemp84
Our
video about Kagi Small Web
is resonating with members. We talk about the purpose behind this initiative and why we're committed to growing it.
Dec 18th, 2025 - Popular areas land in Kagi Maps
#
Kagi Maps
We're continuously improving Kagi Maps, and with the latest release we've added a new data layer: Popular Areas. It highlights the busiest and most frequented spots when you're exploring a new city.
New Global Map Layer:
Highlights most popular areas where people congregate near Cafes/Restaurants/Shops/Cultural-Centers
POI Infoboxes have more 3rd party external links:
OpenStreetMap, Wikipedia, Google Maps, Apple Maps
Reviews on Yelp and TripAdvisor
Social media profiles (Facebook, Instagram, Twitter)
Reservations via OpenTable
easier-to-read opening hours with weekly schedules
Direct links to restaurant menus when available
Strengthening ties to OpenStreetMap Community:
with ability to Report Map Issues to OpenStreetMap directly. A new "Report an issue" option in Infobox connects you to OpenStreetMap's note system, where you can flag errors or suggest improvements to the underlying map data.
Additional Map Data:
POI data now preloads in the background for faster navigation when clicking markers or search results
Mobile-optimized zoom controls for smoother touch interaction
Sorting preferences (distance, rating, price) now persist across sessions
Faster POI on click load-times with use of shorterm caching
Middle-click support on search results and sorting buttons
Various ad-hoc bug fixes and database improvements:
Improved caching system for POI data reducing redundant API calls
Better location cookie handling using
kagi_precise_location
Various improvements to our POI-matching algorithms
UI rendering fixes
Kagi Search
Location management is now available in settings, where you can view and update your location at any time. Kagi uses either a coarse location estimated from your IP address or, if you opt in, your device's precise location. This is
stored only on your device as a cookie
. It supports local-intent searches (e.g. "petrol stations near me") and sets the initial map position in Kagi Maps.
Research Assistant image generation should allow you to specify higher resolution than 1024x1024
#9156
@jmp242
Navigating between versions of the same prompt is broken with 3 prompts after page reload in Kagi Assistant
#7134
@bsamek
Post of the week
Here is this week's featured social media
mention
:
We truly appreciate your support in spreading the word, so be sure to
follow us
and tag us in your comments!
Is your browser a rat?
Check out
this fun video
we made for
Orion
. We also made this comic in collaboration with artist
Chaz Hutton
to show why we built Orion to be your trusted daily companion for the web:
End-of-Year Community Event
Join us tomorrow, December 19, at 09:00 PST (convert to
local time
) for Kagi's annual community event, covering major updates, launches, and what's next. Plus
live Q&A
with the Kagi team.
Register via Zoom.
Looking forward to seeing you there!
Dec 4th, 2025 - New Kagi Search companions and quality-of-life improvements
#
Kagi Search
Introducing Kagi Search companions
You can now choose your preferred companion on Kagi Search! And more companions coming soon.
Other improvements and bug fixes
Context menus for inline news and videos are stuck inside the frame
#9127
@pma_snek
"We haven't found anything" when asking a follow-up question in Quick Answer
#8986
@jstolarek
Check out this
growing list
of Kagi community creations for various devices and apps! Have one to share? [Let us know](mailto:
esra@kagi.com
).
Small Web badges
Small Web initiative members can display badges on their websites to identify themselves as part of a community committed to authentic content created by humans.
Grab them here!
And keep exploring what the
Small Web
has to offer.
End-of-Year Community Event
As we wrap up an exciting year for Kagi, we'd love to have you join us for our end-of-year community event on
December 19 at 09:00 PST
(convert to your
local time
).
We'll share a comprehensive "Year in Review" covering Kagi's major updates, product launches, and what's ahead, followed by an interactive Q&A session where we'll address your questions directly.
If you’re a Kagi member,
you can book up to 5 FREE reservations per month
and treat the Hub as your base whenever you’re in Belgrade. It is the same space our team uses, so you will be working directly alongside the people shaping Kagi’s future. More details, including how to reserve your spot, are in this blog post:
https://blog.kagi.com/kagi-hub
Having an actual physical space makes our mission to "humanize the web" feel so much more real. It is a place for Kagi members and our fully remote team to work, trade ideas, and build the tools we all wish existed.
We are looking forward to welcoming you to Kagi's first ever Hub!
Nov 20th, 2025 - Introducing Quick and Research assistants
#
Their main strength is research: identifying what to search for, executing multiple simultaneous searches (in different languages, if needed), and synthesizing the findings into high-quality answers.
And on top of web search, we’ve added new behavioural layers and a wider toolset, including Python execution and image generation for higher-quality answers. These capabilities go beyond what was already possible in Kagi Assistant using a base model with web search. See our
documentation
for the full details.
Finally, a huge thank you to everyone in our
Discord
for beta testing this with us and providing tons of feedback along the way! 🙏
Note:
With this change, we set the Quick assistant as the default mode in Kagi Assistant. You can always adjust this in your
Assistant Settings
.
Additionally, we plan to migrate the
q
bang, currently used for Quick Answer, to trigger an Assistant thread targeting the Quick assistant.
LLMs are bullshitters. But that doesn't mean they're not useful
Yesterday, we published an opinion essay exploring the useful yet disruptive nature of LLMs. Give it a read and let us know what you think
https://blog.kagi.com/llms
Colour code your Assistant tags
Now you can assign icons and colours to your tags. Spot important threads instantly.
Other improvements and bug fixes
Retired a handful of models. As part of a regular process, we occasionally review and retire models that are not used by Kagi customers and have been superseded by better, newer models. Saying bon voyage to:
gpt-oss-20b
,
gpt-4-1-nano
,
gpt-4-1-mini
,
gpt-4-1
,
o4-mini
,
o3
,
grok-code-fast
,
mistral-large
,
deepseek-r1
, and
hermes-4-405b
. In the future we will forecast these changes with more advanced notice.
Last week we kicked off our
SlopStop
initiative. Since then, the
community has submitted over 3,000 reports!
Our team is reviewing this data to refine our evaluation pipeline, with improvements expected to go live next week
Please continue reporting AI slop in your search results.
Paywalled news sites are now signaled on
/news
.
The new AI slop report breaks the layout when translated
#8923
@tux0r
Here is this week's featured social media
mention
:
Haven't tried the Kagi Translate extension yet?
Check it out
!
Nov 13th, 2025 - Raising the shield against slop
#
Kagi Search
Introducing SlopStop: community reporting to reduce low-quality AI content
Today we're releasing SlopStop, our first step in collaborative filtering.
This allows our community to directly improve search quality for everyone!
Read the full announcement here
!
Low-quality AI content is flooding the web. Kagi’s ranking already downranks and filters much of it. SlopStop gives you a simple mechanism to help us keep results even cleaner and more authentic.
How it works:
If you see low-quality AI content in web, image or video results, click the shield icon next to the result to report it. If something is flagged in error, use the same control to report the mistake.
We've integrated with
Surveillance Watch
, an interactive database that documents surveillance and spyware entities. When you visit a domain on their list, we'll display a banner to alert you.
Other improvements and bug fixes
Results show centered despite the setting being set to "Left"
#8886
@vaartis
Feedback on the new Quick Answer experience on mobile
#8729
@Jesal
Add option to auto trigger Quick Answer when submitting an edited query in which Quick Answer previously was used
#6769
@RoxyRoxyRoxy
If you use Quick Answer on Kagi Search, you already know it finds relevant content fast. Now we're taking it even further as a powerful research tool:
Built-in follow-ups
. Every answer now comes with three suggested follow-up questions to keep the momentum going.
Seamless transition to Kagi Assistant
. Your conversation thread carries over automatically so you can continue exploring instantly.
Mobile now has its own dedicated experience.
Use the new full-screen view and input field to ask your next question. You can switch back to the Search results page at any time by tapping the magnifying glass.
The goal is for your research to feel less like a chore and more like following your curiosity wherever it leads.
Privacy Settings
As we continue building tools for both greater privacy and anonymity, we wanted to make this information easier to find. Our new dedicated privacy page puts everything in one place: our
Privacy Policy
,
Privacy Pass
,
Tor access
, and privacy-preserving payment methods.
Single page mode (Sequential, Mix, Random) setting [web]
Add TTS, Simplifier and Anki flash card generation for Kagi Search subscribers [web]
Implement sharing [mobile]
Kagi Maps
Kagi Maps ignores 24-hour clock preference and shows 12-hour times
#8560
@Gilfoyle
October 17th, 2025 - Autumn patch notes & a new Kagi Special
#
Kagi Specials
Our
Kagi Specials
initiative is expanding. Today we are adding
Notesnook
, the privacy-first note taking app.
Kagi members will get a 10% lifetime discount to Notesnook, and Notesnook members will get a complimentary 3-month subscription to the Kagi Professional plan. Visit our
Specials page
or learn more about
this wider initiative.
Here is this week's featured social media
mention
:
Follow us
on your preferred social media platform and tag us with your feedback!
Kagi around the web
David Pierce of The Verge
described Kagi News
as "simple, straightforward, super useful."
Watch Cory Doctorow
talk about the journey to finding a "magical" search experience with Kagi on the inaugural episode of The Honest Broker video interview series. You can also
catch him
talking about Kagi on Adam Conover’s podcast.
After two years with Kagi, Michael Peter shares his
positive experience
and reflects on upgrading to the Ultimate plan: "I’m surprised I didn’t do it sooner. The AI assistant is really good and fits perfectly into my browser workflow."
Today
we’re officially introducing Kagi News:
a once-a-day press review that cuts through the noise. Global stories, community-curated sources, and zero tracking. News the way it should be.
What can I do with it?
Get a
thoughtful daily press review
, tailored to your interests and reading pace
Explore
up to 12 key stories per category
— choose global news, local coverage, or both
Dive into
international perspectives
from major global outlets
Or
focus on local news
from a specific country, with content curated from its national press
Read any article
in your preferred language
with built-in translation
See every story
structured clearly
: Summary, Highlights, Key Quotes, Timeline, Context, and Impact
Tap once to
access the original source
Help shape the feed by contributing trusted outlets to the community-curated platform
How It Works
Every day at 12:00 KT (Kagi time), we deliver a fresh press review based on your preferences.
Built on Values You Can Trust
✨ No surveillance, ads, or trackers
🌍
Open-source curation
by the Kagi community
🧠 Designed to empower you, not exploit your habits
We're excited to introduce
Kagi Specials
, where we spotlight privacy-first companies that share our values: no surveillance, no ads, no data selling, and providing special offers on these services for Kagi members.
Our first featured special is
Ente
! It's an end-to-end encrypted photo and video storage service that ensures only you can access your memories.
For users who value privacy, Ente is a perfect complement to Kagi, and Kagi members will get 25% off for the first 12 months.
Read more
about this initiative and keep an eye open for upcoming specials!
Kagi on Socials
Here is this week's featured social media
mention
:
Edward Kiledjian
expresses
why he moved away from Google Search and the switch to Kagi, specifically noting: "it was about choosing a search experience that aligns with my values: privacy, control, and quality."
September 4th, 2025 - Kagi Summarize goes mobile, Kagi Assistant adds source attribution and study mode
#
Announcing proportional source attribution in AI answers
We built technology that provides proportional content attribution in AI answers.
This helps you understand the importance of each source in forming the final answer.
More importantly, this technology paves the way down the road for Kagi to share profits with publishers participating in our AI answers. This would happen automatically for all websites, with no deals, no contracts needed.
We're introducing our take on
study mode
: a Kagi Custom Assistant designed to guide your learning journey through active discovery. Using Socratic method, evidence-based learning techniques, and collaborative exploration, it helps you uncover answers rather than simply providing them.
Please note that the Kagi Study Custom Assistant is available only on the Ultimate plan, as it relies on premium models.
Kagi Search
Going to the edit lens menu makes "Edit" the current lens
#7426
@fxgn
Typo and english text in French localization on kagi 100 searches page
#8132
@Sandbank2737
This exciting opportunity came about through meeting Shiho Watabe, the startup hub manager, during our recent
Japan visit
. We were immediately drawn to their vision of supporting bold, boundary-pushing ideas from the heart of Tokyo's creative scene.
Japan has already become our second highest source of traffic through organic growth, thanks largely to
Kagi Translate
, and we see tremendous potential to deepen our presence there. Being part of this program opens doors to bring Kagi directly to Shibuya's libraries and schools.
Kagi on Socials
Here is this week's featured social media
mention
:
This video
by EposVox perfectly sums up how when we choose better search engines like Kagi, we get better search results that actually surface
smaller sites
and content by independent creators.
Lee Hutchinson wrote a
great in-depth piece
on Ars Technica about making the switch to Kagi.
Orion gets
a prominent shout out
by designer Juxtopposed for its wide range of customization options and unique features.
Our CEO Vlad was interviewed
on an episode
of the Intelligent Machines podcast with Leo Laporte, Jeff Jarvis and Paris Martineau.
Sedat Kapanoğlu
wrote
about "good people doing good things", using Kagi as an example of a service whose incentives are aligned with its users.
Writer Dave Pollard mentions Kagi in
an article
about the internet’s "tragedy of the commons," using it as an example of a functioning digital commons.
Listen to Vlad on the
Mac Power Users
podcast with David Sparks and Stephen Hackett talk about what Kagi offers its customers and its role as a powerful alternative to products offered by tech giants.
The true cost of "free" ad‑supported search isn't mere annoyance, it's also real loss of money and time. A
CBS Chicago segment
shows how scam ads are flooding results and hurting users. That's why Kagi is and will always remain ad‑free.
July 31st, 2025 - Kagi Assistant gets tags, bulk actions, and much more
#
Kagi Assistant
This week's release brings several big updates to Kagi Assistant, laying the groundwork to exciting new chapters ahead.
Introducing tags
Tags let you organise your Kagi Assistant threads. Each thread can have multiple tags.
You can use the
Temporary
tag to mark any thread as temporary, and it will auto-delete 24h after its last update.
If you use Kagi Assistant in 24h mode, all new threads will be tagged as
Temporary
by default. When you remove the
Temporary
tag, that thread will be saved.
When you create a thread while viewing a specific tag, the new thread will automatically inherit that tag.
Threads without a custom tag, such as all your currently saved threads, appear in the
All
folder.
Model picker
The model picker now features a curated set of base models that we believe deliver the strongest performance, helping you choose the best option for your specific task. We continuously update this list using our own
in-house benchmarking
.
Define your default Assistant
You can now
define your default Kagi Assistant model
, as either one of your Custom Assistants, a base model or the last used model. Every new thread you create will start with that default model.
Removal of context windows limits
Kagi Assistant no longer enforces
context window limits
. We include all available information, and if we reach the model’s maximum size, we carry forward relevant context so it remains available for your next instructions.
Our redesigned
News homepage
is now live. You'll find top stories and selected categories featured prominently, so you can quickly access the news that matters most. Updated once per day, spend less time scrolling and more time staying informed. We plan to launch mobile native News experience soon.
Let us know how you like it!
We are still rapidly iterating on the quality of this content & welcome your feedback!
We've added model info boxes that highlight each model’s strengths with scores from the
Kagi LLM benchmark
, so you can pick the best fit for your task without wading through docs. For more in-depth information, see the
full details here
.
Grok4 (preview) has been added for Ultimate subscribers
New homescreen widgets for Kagi Assistant, Translate, and Summarizer -- access in one tap. Plus: smoother performance, cleaner experience, and fixes for the little things. Enjoy!
Android blank screen when going back from external window
#7360
@Numerlor
In one of the
latest episodes
of the Behavioral Science for Brands Podcast, Rory Sutherland discusses how Kagi stands out by prioritizing user interests over advertisers - highlighting what happens when search engines put advertisers first.
Jonathan Margolis
interviewed
our CEO Vlad in Air Mail magazine:
At last, a search engine that won’t collect your data and feed you ads.
Longtime Kagi user wrote a
great guide
on customizing Kagi Search with Lenses, Personalized Results / domain ranking to help you get the most out of your Kagi experience.
Kagi got a
nice shout out
on the Intelligent Machines podcast with Leo Laporte, Mike Elgan and Jeff Jarvis. An interview with Vlad will be on the show soon!
Over the past few weeks, we've made many tweaks to improve accessibility of our settings pages for keyboard users, screen reader users, and users who have JavaScript disabled.
Advanced control for Bangs via Regex
We've added an optional "Regex" field to Custom Bangs for more precise control. Instead of
$1
always being the first word and
$2
the second, you can now define exactly how your query is split to create more powerful shortcuts.
For example, to create a custom translator for a query like
!tr spanish live long and prosper
, you can now separate the language from the text. In your Custom Bang settings, you would use:
This pattern tells the Bang to use the first word for
$1
and the entire rest of the query for
$2
. If you leave the Regex field blank, your Bangs will continue to work as they always have. We're excited to see what you build with this
Other bug fixes and improvements
Site ranking radio buttons have no accessibility labels
#7412
@mehgcap
Proofreading mode improvements (statistics tab, apply/revert corrections, hover to find correction in text/list, all corrections now have explanations)
Added virtual keyboard for some alphabets
Language-specific tweaks for Russian, Estonian, Sami (All variants), Afrikaans and Cherokee.
Fix "Sorry, I cannot complete this request" censorship for some queries
hyacinth
Today marks a special milestone in Kagi's history as we reach more than 50.000 paying subscribers. That’s not just a milestone, it’s 50.000 real humans who looked at the web, saw what we’re trying to do, and said, “yeah, I'm in.” That means everything to us.
As tradition demands, we’ve written a blog post packed with announcements and new stuff:
read it here
!
Thank you for believing in us. Here’s to the next 50k, we’re grateful to be building something that matters, with you! 🥂
Since we introduced the Kagi Assistant to all subscriptions and began enforcing fair-use limits, users have requested better visibility on costs. In response, we have added the total token cost per month to the
billing page
.
Kagi Assistant - Cannot upload images on iOS in Lockdown Mode
#6392
@pseudonym
Kagi Maps
We're happy to announce the launch of the (new) Kagi Maps. It's still early days, but we're working hard because maps are long overdue for a major upgrade. Our roadmap is packed with innovative ideas and, most importantly, interesting data to make local mapping meaningful again. Check out the alpha at
kagi.com/maps
. This is just the beginning.
Here is this week's featured social media
mention
:
Tag our accounts or use #Kagi when mentioning us in your posts!
Podcast feature
On the latest Timetable podcast, Kagi CEO Vlad joins Manton Reece to discuss building a user-first, ad-free search engine, why the economics of search are broken, and how Kagi helps you discover hidden gems from the
Small Web
- the kind of sites you'd never find on mainstream, ad-driven engines.
Listen here.
Industry news
The latest developments in the tech and search industries that captured our attention and reinforces our mission:
A look into Kagi's trip to Tokyo, where we connected with the Kagi community, collaborated with local companies and potential partners, and enjoyed everything the city has to offer. A heartfelt thank you to everyone who met with us, shared meals, and made this experience unforgettable!
また次回お会いしましょう。
OMG Ubuntu
covered
Kagi's Orion Browser and its usage of GTK4/libadwaita for its Linux launch.
Kagi Assistant was mentioned on Computer World's "20 genuinely useful AI apps for Android"
list
:
Beyond basic queries, its $25-a-month Ultimate plan includes access to something called the Kagi Assistant, which lets you access the underlying intelligence from Gemini, ChatGPT, Claude, and other AI engines in a single streamlined spot and with the added advantage of complete privacy and custom filtering to help refine the results.
Liam Proven of The Register gives a shout out to Kagi in
a piece
about the enshittification of search:
Another option is to pay for your search engine. We've already mentioned Doctorow once, but his post about Kagi – a no-ads, no-tracking search engine with a tiered payment model – from a year ago makes interesting reading. The idea has gained traction with others, including Daring Fireball's John Gruber.
Kagi shout outs
Kagi is described as being best for "power users and professionals who want fast, curated, ad-free results" in
a post
by Maple Web Design.
Google’s search results have become so bad that I recently subscribed to Kagi, and so far it’s been great.
Have a site or blog and want to have an easier way to search through its archives?
This post
outlines a good use of Lenses to make that easier.
A Lens will focus Kagi’s search on one or more (up to ten) particular sites, date ranges, regions, keywords, or file types. You can also exclude sites or subsites. Kagi has several default Lenses, and you can add your own customized Lenses:
Kagi is used as an example for why it's important to
pay for what you love
, and how it helps align the incentives between the user and the service provider:
In order for Kagi to make more money, they have to make search quality better. By adopting a subscription model, Kagi has fundamentally aligned the goal of the service with providing excellent search. This is why a subscription-based service is the most likely to keep its users satisfied over time, and why — if you care, and if you have the choice — you should choose to use a subscription-based service over other offerings.
Along those same lines, Liu Miao writes about how
paying for search
offers a different perspective on using the internet:
Indeed, our generation grew up with the internet, and search engines have been free since their inception. Why pay to use one? But looking at it from another angle, if search engines have been free for over twenty years, then who has been paying for me all this time?
And noting specifically about Kagi:
I hope to see more products like this in the future—products that treat users as users rather than as products.
Industry News
The latest developments in the tech and search industries that captured our attention and reinforces our mission:
In collaboration with artist
Chaz Hutton
, this illustration captures how using a search engine should feel: straightforward and focused, guiding you directly to what you’re looking for.
May 6th, 2025 - Video search upgrades, enhanced Kagi Assistant experience and more
#
Search
Video search upgrade
Video search now pulls from more sources and shows richer data:
And other bug fixes and improvements,
Colour picker does not show correct colour on desktop with Zen Browser
#7001
@larke12
Fast/Best quality toggle: select quick processing or maximum accuracy
Add english language varieties to list of supported languages
JR
. Automatically switch to/from language, when typing text from "to" language in the input field
TomA
Frandroid, a popular French technology publication, published
a piece
about Kagi being an ideal alternative to Google. Read with Kagi Translate
here.
Spider's Web, a prominent Polish tech site,
wrote about
the importance of a search engine like Kagi that puts users' interests ahead of ads and other incentives. The article also emphasizes Kagi's dedication to privacy and supporting the small web. Read with Kagi Translate
here.
Tecnobits featured
an article
about how Kagi improves your search experience:
Everything you see on Kagi is there because it's useful, not because someone is behind it paying for clicks.
Kagi shout outs
Neel Dhanesha at the Nieman Lab published
an article about Kagi
highlighting its extensive features and the advantages of having full control over your search experience:
After testing Kagi both as my everyday search engine and as a research tool for a working journalist, I’ve been delighted to find that it’s the search engine equivalent of a Honda Civic: reliable, unobtrusive, and able to get you where you need to go.
Paying for Kagi today feels a lot like paying for HBO back in the cable TV heyday. Part of the deal is that you are paying for ad-free service, yes. But you’re also paying for noticeably higher quality. [...] It’s that good. No ads, no unwanted AI (but very good AI results if you want — just end your query with a question mark), and better search results.
My inaugural experience using Kagi was eye-opening: on my first search I got a genuine blog post instead of yet another click-bait AI-generated article. Think about it for a second: when was the last time you got someone's personal blog at the top of your search?
Industry news
The latest developments in the tech and search industries that captured our attention and reinforces our mission:
A few weeks ago, reporter Aaron Pressman
described
how "free" search engines come at an actual financial cost, sharing his journey that ultimately brought him to Kagi.
Artist
Chaz Hutton
helped us illustrate how a Kagi subscription can save you money in the long run by eliminating the many hidden costs associated with "free" search engines:
April 17, 2025 - Kagi Assistant rolls out to all Kagi users
#
Announcements
Kagi Assistant is now open to all plans
We're happy to announce that
Kagi Assistant is now available to everyone
, across all subscription plans!
⚠️
Note:
We are enabling the Assistant for all plans in phases, based on regions starting with USA today. The full rollout is scheduled to be completed by Sunday, 23:59 UTC.
With this release, we are beginning to enforce our
fair use policy
to ensure a sustainable, high-quality service for everyone as we expand access. Basically our policy states that you can use AI models based on your plan's value. For example, a $25 monthly plan allows up to $25 worth of raw token cost across all models (there is a 20% built-in margin that we reserve for providing searches, development and infrastructure for the service). This impacts only a very small percentage of users with extremely high usage patterns and is a simple way to control usage, compared to arbitrary usage limits. Our goal is to ensure broad availability without users needing to worry about typical usage. Affected users can currently renew their cycle instantly, with more flexible credit top-ups planned soon.
We're very happy to offer Kagi Assistant as another optional tool to support your work and exploration online. Let us know what you think via our
feedback forum
or
Discord
! Also see our
documentation for Assistant
.
Release notes
Search
The Yahoo Finance bang, eg "<ticker symbol> !yf", now leads to a yahoo.com error page
#6778
@numbers
"Share search" button disappears when toggling an image search discriminator.
#6172
@RoxyRoxyRoxy
Sensitivity in Action Panel Above Assistant Text Field
#6625
@Tarrek
Translate
We've launched our brand-new UI and a bunch of new feature to make your translation experience exceptional! We are incredibly proud of the work on this product. Check out
Kagi Translate
.
Improvements and bug fixes:
Enhanced alternative translations now provide insight about the main translation
Dictionary entries can now appear in the input box
If you can't still find the language you are looking for, you can just type it in
jvbf
Default Target Language option - your locale language will be automatically set as the default "translate to" language instead of it defaulting to the last one you used.
New website translation views - while translating a website, you can now view both the original and translated versions in a horizontal or vertical split. Drag your mouse in the middle to resize the iframes to your liking.
Lifehacker featured Kagi in
an extensive piece
detailing several features to get the most out of your Kagi experience.
On the Boston Globe, business reporter Aaron Pressman describes the journey that ultimately led him to Kagi, and how "free" search engines come at an actual financial cost:
"I switched all my default searches to Kagi."
As for me? I’ve voted with my wallet. I now use Kagi for search (and pay for it gladly)
Author Dave Pollard
mentions Kagi
in a links of the month post, stating:
I switched to Kagi from the enshittified Google search engine last year, and I’ll never go back. No ads, no sponsored links, no tracking your searches, no selling your data, no clutter.
Parth Shah
tested
8 search engines, stating the following about Kagi:
Kagi was new to me before this study, but it stood out quickly. It’s a paid search engine with no ads at all. The results were accurate, useful, and easy to read. I also liked the clean layout. [...] Kagi quickly became my favorite.
We partnered with artist
Chaz Hutton
on a graphic that illustrates how Kagi empowers you to search without the noise, trackers and distractions. Help us share it widely!
Tokyo, meet Kagi!
As noted previously, from April 21–25, Kagi’s team, including CEO Vlad, will be in Tokyo 🇯🇵 to meet companies and connect with the local community. If you are a local Kagi user, we'd love for you to join us for casual food&drinks! If interested, please contact Gillian at
gillian@kagi.com
to arrange.
March 21st, 2025 - Leveling up the Kagi Assistant experience, meet Kagi in Tokyo, and more...
#
Next chapter in the Kagi Assistant experience
With this release, we're
introducing a new sidebar
, designed to streamline your workflow and keep everything you need within reach. This is just the beginning - the new sidebar lays the foundation for even more powerful features to come.
We're also rolling out
Claude 3.7 Sonnet with extended thinking
to tackle even more complex challenges.
As always, we’d love to hear your thoughts - join the conversation on
Discord
or share your feedback through our forum at
https://kagifeedback.org
Kagi, lost in translation, in Tokyo!
From April 21 to 25, a dedicated team from Kagi including CEO, Vlad, will travel to Tokyo 🇯🇵 to meet with companies interested in our
advanced translation
solutions. If you are based in Japan and your company is interested, please get in touch with Gillian.
We'd also love to meet our local user community while we're there. If you're based in Tokyo, we'd love to
organise a dinner and connect in person
. Looking forward to great conversations and good food with fellow Kagi fans!
We are looking for a
Head of Business Development
and a
Technical Architect
(basically CTO equivalent in Kagi world).
Head out to
Kagi hiring
to apply and check out other open positions.
Improvements and bug fixes
Search
Based on the community's feedback, we’ve added a setting to always hide AI-generated images by default (suggested by
#5998
@keyboardJones
). You can enable it on
Settings>Search>AI
.
This
TechCrunch article
mentions Kagi as an option for users who prioritize not just a better search experience but also a more privacy-focused search.
t3n, a German tech magazine, also
featured Kagi
, highlighting it as a privacy-friendly and ad-free alternative. Read with Kagi Translate
here.
Kagi Community Reviews
We are deeply grateful to everyone who takes the time to share their experiences, tips, and feedback about Kagi, helping to spread the word and inspire others to discover its value. Here are some recent reviews to highlight:
Have a Kagi review you'd like to feature? Please share it and ping us on any of
our various socials
when you do!
The Kagi Way
We partnered with artist
Chaz Hutton
on a graphic that illustrates how Kagi empowers you to search without the noise, trackers and distractions. Help us share it widely!
March 6th, 2025 - Orion Embarks on Linux Journey & Kagi Doggo Art Celebration
#
Orion's Next Chapter: Linux Development Officially Launched
We're thrilled to announce that development of Orion Browser for Linux has officially started! Our team is working hard to bring the same speed, privacy, and innovation that Mac users love to the Linux platform.
This is an ambitious project that we expect will take approximately one year to complete. Our target is to achieve feature parity with the current macOS version by March 2026.
Want to stay updated on Orion for Linux?
Register here
to receive news and early access opportunities throughout the development year.
Celebrating Kagi's Community Creativity with Doggo Art
This month, Kagi Search is showcasing incredible community-created renditions of our beloved Doggo mascot,
Shoutout to the featured artists—
Edin Pasovic
,
Fangmoder
,
Eve Davison
,
Kevin
, and
Lynn
—who wowed us with their talent and earned 3 free months of ultimate. This is just the start: we’re making this a regular celebration of independent creators. Full story in our
docs
.
Feb 27, 2025 - Sonnet 3.7 hits Assistant, early access to Ki, TikTok search and major Translate improvements
#
Kagi Assistant adds Sonnet 3.7 and the preview of multi-step reasoning assistant called Ki
We're happy to unveil the latest updates to our assistant experience. This release brings a smoother, smarter, and more intuitive interface designed to make your interactions simpler.
Try it out and feel the difference!
We also added Claude 3.7 Sonnet to our model lineup! It's now powering our
!code
assistant and plays a key role in our advanced multi-step reasoning model,
Kii
. Currently in early testing, Ki is available exclusively to our
Discord community
members - join now to get early access (Ultimate account users only)!
And today OpenAI released GPT4.5 - we have already benchmarked it. Check
Kagi LLM benchmark
.
TikTok video search
Video search just got even sharper—you can now filter specifically for Tiktok videos.
Find exactly what you're looking for, faster.
We are hiring a Flutter developer!
We just opened a position for a skilled
Flutter Developer
. Apply now or send someone our way.
Interested in covering Kagi on your outlet, newsletter or podcast? Hit us up! Our team is very approachable and we welcome any opportunity to engage with various communities about our latest features.
Read all about why this matters and the details of implementation in our
announcement blog post
!
Privacy Pass support is provided:
Natively for
Orion browser
users (macOS/iOS/iPadOS). On iOS, make sure to have version 1.3.17 and above (expected to roll out globally today) and update your macOS Orion to version 0.99.131.
Natively through
Kagi App for Android
(make sure to have version 0.29, expected to roll out globally today)
Privacy Pass implementation is
fully open-sourced
for transparency and community collaboration
Kagi Privacy Pass is available for the Professional, Ultimate, Family, and Team plans.
Limitations:
It's not available for Trial/Starter plans
Privacy Pass mode disables account-specific features, like domain personalisaton (as we do not know which user is searching)
Privacy Pass mode is supported for Kagi Search in this initial phase, we will add support other services in the coming weeks. Check the
blog post FAQ
section for more details!
Kagi is redefining privacy in search. Try it now!
Kagi Tor Onion service
Access Kagi securely and anonymously via our new Tor Onion Service.
You can now access Kagi directly through the Tor network using our dedicated onion address:
See more information about Kagi Tor service in our
documentation
.
An updated comprehensive privacy policy
We're also excited to share our
updated privacy policy
, with simplified language and designed to clearly reflect our strong commitment to protecting your privacy.
Tag our accounts or use #Kagi when mentioning us in your posts!
Kagi in the News
Orion tops Apple's
App Store's list
of superpowered internet browsers "to seriously level up your web browsing"!
And Android Police
published an article
about Kagi's new fair pricing model: "This ethical search engine will return your subscription money if you don't use it." The Verge also
covered the news.
Kodak's "Pre-Invented" Lunar Orbiter Camera; Or, the Fate of SAMOS Readout
Left: The Lunar Orbiter camera system sitting in the bottom half of the pressurized shell. Project Manager Cliff Nelson (left) stands with NASA/Langley team members Calvin Broome, Israel Taback, and Joe Mooreman. From
NASA
.
Right: Lunar Orbiter frame 5017-M, showing artifacts from Kodak’s camera system. From
NASA/LOIRP.
In 1966 and 1967, NASA sent five robotic spacecraft into orbit around the Moon. Constructed for the primary purpose of finding landing sites for Apollo, the Lunar Orbiters also enabled nearly comprehensive mapping of the Moon in stunning detail. The quantity and resolution of the photographs returned by Lunar Orbiter was unprecedented, thanks to a camera system built by the Eastman Kodak company of Rochester, New York. Their imaging system involved elaborate mechanisms to expose photographic film in orbit, develop the film onboard the spacecraft, and remotely transmit images back to Earth.
In February of 1967, the Rochester Times-Union published a story about how Kodak had “pre-invented” the Lunar Orbiter camera. Kodak director of R&D Arthur Simmons told the Times-Union that “no one walked in and asked us to develop a camera and film system to take closeup photos of the moon…Kodak has, for want of a better word, a ‘library,’ of hundreds of ‘conceptual ideas’ which we don’t advertise.”
1
But the story of the camera’s “pre-invention” was more interesting than Simmons let on. When Kodak joined Boeing’s bid for the Lunar Orbiter in 1963, the camera system already existed. The company had originally developed it for the Air Force in the 1950s as a part of the highly classified satellite surveillance program called Weapons System 117L (WS-117L).
The nature of WS-117L and the clandestine origins of the Lunar Orbiter camera system were vaguely known by some at the time, but the full details were only revealed to the public through declassification decades later. This article won’t linger on the detailed technical specifications of the Lunar Orbiter cameras, but will instead focus on tracing the system’s development from conception to its adoption by NASA. It is the story of some of the first attempts by the United States to remotely transmit images from space, and how those same systems were adapted for lunar exploration.
WS-117L had roots in RAND studies of satellite surveillance concepts going back to 1946, and was the first major attempt to put those ideas into practice. The United States was hoping that satellites could be used to monitor the buildup of nuclear weapons and launch sites in the Soviet Union. In 1953, RAND Report 262 laid out in full the feasibility and utility of such systems, and by 1955 the Air Force began soliciting contractors for WS-117L. Eastman Kodak created camera systems for Lockheed’s bid, and the the Air Force awarded their contract in October of 1956.
2
Timeline of the Advanced Reconnaissance System, or WS-117L, from a 1958 summary. From
NRO (PDF).
Originally, RAND had primarily considered using television systems.
3
“Near real time” imaging was considered by some to be the ideal form of satellite surveillance, so television was a logical choice. But in a declassified history of the program, Robert Perry explains that the goal of real time imaging quickly became contested within the Air Force–it was unclear whether the technology was ready to satisfy requirements, and the alternative film recovery systems showed clear feasibility and reliability early on. Some of these efforts were spun off from WS-117L into the
Discoverer-CORONA program
, which sent Kodak camera systems into orbit, and returned capsules of exposed film for aerial retrieval and processing back on Earth.
4
Early on, however, a near real time system was very much a part of the plan. In their bid for WS-117L, Kodak created a remote transmission system that would fly in what became known as the SAMOS program.
Television systems were still in consideration early on, but there were clear technological limitations at the time, especially when it came to resolution. Kodak’s newly formed Apparatus and Optical Division settled on a system to develop film onboard the spacecraft and “readout” the images to receiving stations on Earth. Planners envisaged five cameras for SAMOS, the first three dedicated to testing Kodak’s readout system (the others testing
advanced recovery stems
). The E-1 camera would primarily be a technology demonstrator, while the E-2 and E-3 would test the ability for their system to take images at more functional resolutions. All used the same basic architecture.
A diagram showing the various SAMOS camera systems and their proposed capabilities. From
NRO (PDF)
.
In a way, this camera system was in fact “pre-invented,” as it was mostly a clever assembly of existing technologies, many created by Kodak. The company had decades of experience in aerial photography going back to World War I. By the end of World War II, they had created advanced aerial films, compact film storage systems, and image motion compensation techniques. They had also worked on IR bomb sights and proximity sensing for the Navy, which would become useful for developing thermal control materials for the spacecraft.
5
Illustrations from 1957 show a rough sketch of their plans for SAMOS readout. Kodak’s 70mm film would be exposed, processed, and stored before readout and transmission. Electronic signals received on the ground would be used to reconstruct the images.
An early diagram showing the SAMOS readout system without some of the key details. From
NRO (PDF)
.
By 1958, planning documents started detailing two of the key technologies that ultimately made the readout system possible. One was what became known as Kodak “Bimat” film, labeled in the illustration below as “WEB.”
6
This web was coated with gelatin containing the necessary processing chemicals, enabling “dry” processing. The web was pressed against the exposed film, developing and fixing the images before storage and transmission.
7
Diagram from a 1959 Lockheed briefing on the SAMOS program, image is labeled September 1958. From
NRO (PDF)
.
The origins of Bimat film are somewhat obscure in the public record. One Kodak-produced history suggests that the technology started as a laboratory investigation with amateur photography in mind, and was then applied to use in aerial photography.
8
Considering the timeline, it was either a happy accident that this experiment matured just in time for WS-117L, or Kodak engineers started looking into the technique specifically in response to the challenges of film photography in space.
A Lockheed development plan from March 1956 describes the processing system in vague terms, stating that it would “not differ significantly” from existing methodologies for “airborne rapid-processing,” and describing a notional “a roller-applicator type” system. It also lists “the handling of photographic chemicals” as one of the “major difficulties to be overcome.”
9
The illustration from 1957 shows the onboard processing step without the “WEB,” a detail that only shows up in diagrams like the one above labeled 1958.
10
Then, a patent for a “web processing method” was filed in August 1959 by Kodak researchers, presenting “a one-step method for substantially completely developing and fixing a photographic image…without immersion in photographic processing baths.”
11
David McDowell, an engineer who joined up with Kodak in late 1956 and worked on both SAMOS and Lunar Orbiter, also remembers Bimat being developed specifically for the project. “Bimat was started as soon as we started work on E-1 and E-2,” he told me, “because we knew we had to process film in orbit.”
12
The second key technology was the readout system itself, which involved collaboration with the Columbia Broadcasting System Laboratories to create a flying-spot scanner.
13
It worked using what McDowell calls an “inside-out CRT,” using a cathode-ray tube that fired an electron beam through the exposed film. Variations in the density of the film changed the intensity of the beam, and those variations were recorded by a photomultiplier and translated into electronic signals that could be sent back to Earth. Teams on the ground received those signals, used equipment to translate them back into an image, and recorded that image on film. Before passing through the film, the beam reflected off a revolving drum (seen in the diagram below) for thermal management.
Diagram of the SAMOS readout system from the same Lockheed briefing. From
NRO (PDF)
.
In October 1960, the first E-1 camera launched on an Atlas-Agena, but failed to inject into orbit. Meanwhile, officials were actively debating the wisdom of continuing the readout program. Costs were rising, engineering difficulties plagued the program, some of the technology was beginning to seem obsolete, and CORONA-like systems were looking like a better option until more advanced readout techniques could be developed. Despite these issues, there were advocates for readout, and tests continued so that any decision could be made based on tangible results.
In January 1961, the second SAMOS test launched with another E-1 camera and sailed into orbit. In Sunnyvale, California, technicians at a readout station received transmissions from the spacecraft, and the result was a photograph with a 100 foot resolution. The system had worked.
14
The E-2 camera would be the next step, with more advanced aiming systems and a higher resolution. All of the rotating systems within the spacecraft created complexities when it came to achieving these objectives. One key difference between E-1 and E-2, McDowell recalls, was that E-2 used a rotating nosecone to help stabilize and aim the camera.
15
A diagram of the payload section for E-2, showing the “steerable mounting” that made it distinct from E-1. From
NRO (PDF)
.
A diagram from SAMOS planning documents showing the proposed aiming and stereo operation capabilities of the E-2. From
NRO (PDF)
.
The first attempt to launch an E-2 ended two seconds after liftoff, when the Atlas fell immediately back to the ground and exploded. After the E-2 launch failure, readout was largely abandoned in favor of recovery programs. According to Perry, Air Force Colonel W.G. King believed that almost without exception, “everything a readout system could do a recovery system could do better.”
16
Among other problems, readout systems required long lives, necessitating higher orbits that sacrificed resolution and created greater power requirements.
17
Kodak engineers, including McDowell, remember one of the primary constraints being the bandwidth required to transmit the images, and the fact that they were only using a single ground station.
18
No other E-2 cameras were flown, but it wasn’t the end of the story for the camera system. After the Air Force canceled further launches, E-1 and E-2 hardware was left scattered around the country, with one test model remaining in Eastman Kodak facilities in Rochester.
Perry reports that officials at NASA knew about the E-1 system and inquired about the cameras as early as April of 1961, and that the Air Force gave them permission to get details from contractors. The film readout system was similar to the method employed by the Soviet Luna 3 spacecraft to return the first images of
the far side of the Moon
in 1959, and NASA was interested in using the E-1 for similar purposes. Perry quotes Colonel King saying that NASA officials “did [not] seem to understand much about the problems of taking pictures from a space vehicle.” He did not believe the system would be usable for lunar exploration, but the idea didn’t go away.
19
At that time, the best candidate for using such a camera system would likely have been for the Surveyor program’s planned orbiter, which was encountering its own problems. NASA historian Bruce Byers writes that several factors converged that led to dropping the
Surveyor orbiter
in favor of a standalone project. JPL was dealing with failures of the first Ranger probes, which delayed its work on Surveyor, and the development of the Centaur upper stage planned for Surveyor was also running into trouble.
Meanwhile, Apollo planning was underway. NASA officials decided to deprioritize orbiter data, because landing data was more helpful for hardware development, which had top priority. The orbital imagery would be primarily helpful for landing site selection, which could come later. JPL was to focus on getting Surveyor landers ready, while the Office of Space Sciences (OSS) began developing alternative plans for an orbiter.
20
Oran Nicks
put Lee Scherer on the job of developing a spacecraft that could fly on Agena. He originally looked into adapting Ranger or Able 5 to the task. After they handed the program off to the Langley research center, however Byers writes that Langley director Floyd Thompson opted for a competitive bid.
21
But this competition may have been, if not a complete smokescreen, weighted heavily in the favor of one particular bid. Correlating the Lunar Orbiter program with the timeline presented in Vance G. Mitchell’s declassified history of NASA/DOD relations paints a fascinating picture.
In 1962, as the Surveyor orbiter was under study, NASA Associate Administrator Robert Seamans met with DOD research official John Rubel to discuss lunar reconnaissance. Then, in May 1963, little more than a month after Langely submitted Lunar Orbiter’s Project Approval Document to Seamans, a much larger meeting took place between NASA and DOD officials. They directly discussed the use of NRO equipment for both unmanned vehicles and the Apollo program. Immediately following this meeting, NASA administrator James Webb and Seamans started working with DOD officials on how to put this into practice, and specifically on how NASA could create unclassified contracts for such arrangements. Despite reservations, Rubel’s successor, Eugene Fubini, had the NRO look into NASA’s request.
22
In mid July, an agreement was drafted between NASA and the DOD giving NASA permission to use NRO equipment for “both unmanned and manned lunar reconnaissance operations,” under certain stipulations. It included the following plan of action:
“…it will be the responsibility of the NRO to select a contractor, generally from among those engaged in the present covert reconnaissance programs, to develop equipment meeting these specifications in a secure and protected, or ‘black’, fashion. Concurrently, NASA will grant the same contractor an overt or ‘white’ reconnaissance contract which will serve as a technically plausible cover for the development of the flight hardware actually to be employed, during that length of time in which the flight hardware must be regarded as highly sensitive because of its relevance to the on-going covert reconnaissance operations.”
23
Mitchell recounts one instance of very direct contact between the interested parties during this period. “On 24 July 1963,” he writes, “NASA, NRO, and CIA representatives met with Fredrick C.E. Oder, a retired Air Force colonel involved with Samos in the 1950s, and now an Eastman Kodak executive.” The group directly discussed adapting the E-1 and E-2 cameras for lunar exploration, consulting Kodak engineers who thought it would be feasible.
24
On August 28, the DOD/CIA/NASA agreement was signed by James Webb and Secretary of Defense Robert McNamara.
25
On August 30, Seamans reviewed Langley’s Request For Proposals document, and NASA released it to contractors.
26
The Boeing/Kodak bid was approved by Seamans and Webb in December. “Although the available documentation does not say so,” Mitchell argues, “the NRO, by virtue of the provisions of the 28 August agreement and its knowledge of reconnaissance camera systems must have played a role in the selection process.”
27
This all may help explain the fact that at least in its early stages, Lunar Orbiter was kept under tight security measures at Kodak. McDowell remembers that at the time, work on Lunar Orbiter was kept “in the same level of secrecy that the [SAMOS] projects were.” Work was extremely siloed–engineers building individual components did not always know exactly what they would be used for. McDowell says that this was a pretty standard practice for Kodak at the time, but that the fact that they were using the E-2 probably had something to do with it.
28
Regardless of whether the outcome of the competition was predetermined, the Kodak system did have real advantages over the other bids for Lunar Orbiter. It promised increased flexibility, the capability of taking images simultaneously in multiple resolutions, and the ability to achieve impressively high resolutions.
Kodak’s final Lunar Orbiter camera system used a process largely identical to their E-2 cameras.
29
They even seem to have borrowed some of the illustrations from SAMOS presentations for Lunar Orbiter documentation.
The Lunar Orbiter readout subsystem as shown in the Lunar Orbiter I contractor report. From
NASA
.
The primary modifications that Kodak engineers made were to the lenses and shutter systems. They sought to meet very strict NASA requirements regarding resolution of the images. Compared to the E-1 and E-2, which had 100-foot and 20-foot resolution respectively, NASA’s goals for Apollo planning stated a roughly 3-foot resolution (closer to the never-realized plans for the SAMOS E-3). They also moved from a single ground station to three. Kodak’s final system was capable of achieving that resolution given the right orbit.
Through Lunar Orbiter documentation, we get a closer look at the reconstruction process. Transmitted images were displayed with a kinescope and captured on 35mm film, which was sent to Rochester for reassembly. Strips of 35mm film were assembled to form a full frame, which was in turn captured on film and sent off to NASA. This meant that the images themselves traveled across several different rolls of film before finally being put to use.
A diagram showing the photographic transmission and reconstruction system from the Lunar Orbiter III contractor report. From
NASA
.
The photographs brought back by Lunar Orbiter played an integral role in Apollo site selection, and brought a wealth of new information to cartographers and scientists. The camera system performed admirably, although engineers did encounter a handful of difficulties over the course of the five flights. Some of these difficulties had to do with the Bimat film itself, which operated somewhat inconsistently. The film could “stick,” experience dryout, or see droplet formation, leaving artifacts on the film.
30
The continuing issues with the film into 1966 and 1967 may hint at some of the specific engineering and reliability issues that contributed to the end of SAMOS readout.
The processing mechanism on Lunar Orbiter. Bimat supply was upper left, take-up upper right. The films were pressed together on the small drum in the middle. The developed film was dried and stored on the large drum below. From
NASA (PDF).
A diagram showing the processing mechanism from a
1965 NASA/Langley document.
Frame 76, H3 from Lunar Orbiter V. This frame contains the landing site for Apollo 11, and displays several of the artifacts seen on Lunar Orbiter imagery. The landing site itself is nearly obscured by the line in the center, which may be a Bimat supply separation line. From
NASA/LOIRP.
Because of its spin-off from Surveyor during the push for Apollo, Lunar Orbiter was arguably the very first spacecraft designed to conduct reconnaissance specifically for human spaceflight. The modification of military hardware for the purposes of exploration has a long tradition in the history of exploration, and this is a particularly fascinating example in that tradition. It is the story of a unique camera system straddling technological eras that ended up playing two very different roles in the geopolitical competition of the Cold War.
Footnotes
“How Kodak ‘Pre-Invented’ the Lunar Orbiter Camera (Based on an article in the Rochester Times-Union, February 3, 1967)”, 105:9, Kodak Historical Collection, D.319, Rare Books, Special Collections, and Preservation, River Campus Libraries, University of Rochester
↩︎
“Chronology: WS 117L Background,” NRO, Declassified WS117L, SAMOS & Sentry Records, ID 953,
https://www.nro.gov/Portals/135/documents/foia/declass/WS117L_Records/953.PD
; and “Space System Development Plan: SAMOS R&D Program” 12 July, 1960, NRO, ID 608,
https://www.nro.gov/Portals/135/documents/foia/declass/WS117L_Records/608.PDF
; during the history of WS-117L, ARPA took a direct role in management for a period, and the project went through various names. This article’s primary focus is on the camera systems, and so avoids detailing these changes for simplification. The documents linked here contain detailed explanations of these changes.
↩︎
David McDowell (former Kodak engineer) in discussion with the author, March 22, 2024
↩︎
This is one of the earliest references to this technology that I have been able to find. The term “Bimat” came later, and in Lunar Orbiter documents at the time, engineers have often retained the “web” terminology, referring to it as “Bimat web.” For more on how Bimat worked:
https://www.cia.gov/readingroom/docs/CIA-RDP33-02415A000500120032-7.pdf
↩︎
“Kodak Contributions to Aerial Photography,” p 6, 106:6, Kodak Historical Collection, D.319, Rare Books, Special Collections, and Preservation, River Campus Libraries, University of Rochester
↩︎
McDowell, 2024; The University of Rochester Special Collections Library is also currently processing some Kodak Research Laboratories documentation, to be opened in 2027. I am hoping that more information might be forthcoming.
↩︎
This was similar in principle to Luna 3, which would have been in development around the same time or slightly after the Kodak/CBS system, despite flying sooner. The 1956 Lockheed document contains a pretty detailed description of the flying spot scanner, pretty much as it appeared in the final system. Luna 3’s scanner had definitely begun development by the middle of 1958, but preliminary work may go back to 1957 or earlier. Some of the basic principles behind the flying spot scanner go back even further to some of the
earliest experiments in television
. See Don P. Mitchell’s description of the Luna 3 systems:
http://mentallandscape.com/L_Luna3.htm
; and this account of Luna 3’s transmission system, including the use of film recovered from US spy balloons:
http://www.svengrahn.pp.se/trackind/luna3/SpyBalloon.htm
↩︎
David McDowell in discussion with the author, December 17, 2025. McDowell discussed the stabilization difficulties and the nosecone design in the 2024 discussion, as well.
↩︎
Six weeks after
the release of 0.3
— nicely within the promised two-to-eight-week release train window — we are proud and happy to have shipped
rama 0.4
. It's a release that I am very happy with, as we were not only able to make tons of improvements here and there, but also got to work on some items that had been on our backlog for a long time.
Rama has had support for HTTP, HTTP over TLS (HTTPS) and SOCKS5 proxies for a long time. The easiest way is to configure them directly using
ProxyRoute
(s)
(what used to be directly inserted as a
ProxyAddress
). Within applications these can be hardcoded or exposed via a setting somehow. An example of this is your browser or editor, which allows you to configure a proxy via a settings file or its GUI. You can read more about this approach in
the "Application Proxies" chapter
of
the Rama book
.
Applications often also support the
HTTP_PROXY
environment variable.
curl
uses the lowercase
http_proxy
variant instead for CGI reasons, a convention we now also follow by default when using the
ProxyEnvLayer
. That said, there are more common env variables, such as
ALL_PROXY
,
HTTPS_PROXY
and
NO_PROXY
, with the latter used to add bypass rules to, for example, ensure some (sub)domains do not go via a proxy. All of these are now also supported, via the
ProxyEnvLayer
and
NoProxyEnvLayer
.
However, as you might be aware, operating systems also allow you to configure a proxy system-wide. Here one can configure HTTP, HTTPS and SOCKS5 proxies, as well as bypass rules for what not to proxy. This is all very similar to the env variables discussed earlier, but system-wide. Of course, in general, nothing forces an app to respect these system configuration settings, either because it does not want to or because it simply does not have the built-in capacity to do so (which was the case for network clients built using rama, until now). With
rama 0.4
this is supported out of the box via the
SystemProxyLayer
; you can learn more about how these settings work in
the "System Proxies" chapter
of the Rama book.
System configuration settings also allow you to have a proxy be dynamically selected using a JavaScript file. This is known as
Proxy Auto Configuration
, or PAC for short. To do so one must have a JavaScript runtime — not something rama shipped prior to
rama 0.4
. Now we do. With
rama-js
we now support running a JavaScript runtime within a WASM runtime (using wasmtime, the runtime we will also use in the future to build rama-wasm). This is important as it provides isolation, ensuring that if our JavaScript runtime crashes it doesn't take the whole process with it. Applications like Google Chrome run their JavaScript engine in a separate OS process; within the Rama framework we have chosen to do so within a WASM runtime instead. Same isolation, but without having to somehow allow any application built using Rama to run and bundle a separate process.
Rama now has PAC support, via the
rama-pac
crate, which allows you to have a PAC runtime, and thus evaluate PAC scripts, but also to easily generate scripts for cases where all you want is the ability to route domains X to proxy rules Y.
If you want to play with these new rama framework capabilities, you can also easily do so with our command line application (CLI), which you can learn how to install in
the "rama binary" chapter
of the Rama book:
Client
send
commands support it out of the box (unless you overwrite it with env variables or a command argument);
There are now
rama pac
subcommands allowing you to generate a PAC script as well as to evaluate a PAC script via a REPL. The latter is especially nice, as prior to this the only environments in which you could play with a PAC file were pretty old and obscure applications... Another nice addition to our network CLI toolkit if you ask me.
Protocol-wise, rama now also has support for
ttRPC
, via the new
rama-ttrpc
crate. You can see it as a lightweight alternative to
gRPC
that runs directly on top of a transport protocol such as
TCP
, but still via a protobuf (
proto
) contract.
This is not the only new crate within this space however, as we now also have the
rama-grpc-macros
crate, bringing rama the ability to generate client- and server-side
gRPC
code without writing a single line of
proto
, relying instead on your own codecs, optionally driven by
Serde
(see
define_service
). Great for those that already use
gRPC
and are in control of their whole stack.
It turns out that
HAR (HTTP Archive)
has support for
WebSocket
data. This last-minute
rama 0.4
surprise came to our attention thanks to a commercial partner. It's an obscure feature though: it was added to Chrome some years ago, but not many other applications have picked it up so far. As such we have mostly used Chrome as our oracle to guide our implementation.
It is now supported, and it also made us realise that our previous
HAR export implementation
was keeping too much data in memory for consistency and ordering purposes. This is all settled now, and we can stream all your HTTP and WS data nicely to disk, without having to buffer the entire stream into memory first.
The Rama CLI
send
command now also has support for exporting a HAR file of the HTTP/WS conversations that you executed. In order to do so you can make use of the
--har
argument.
There are several breaking changes and plenty of improvements. For a full list please see the changelog. But to name a few here:
Our peekers (used to check what protocol is flowing over a transport stream, see
the "Protocol Inspection" chapter
of the Rama book) are now able to fail as fast as a byte can no longer satisfy the heuristics, and they also received some minor bug fixes. The HTTP peek router can now also opt in to skipping "method" names that are commonly used and can be confused with HTTP header lines. In some cases, such as
PING
, these can needlessly stall the peeker logic until its timeout, which is not ideal. Using this extra configuration should prevent most of such edge cases, if not all.
This release also brings some more improvements to our
Apple Network Extension support
, which I'm sure are greatly appreciated by those relying upon it to build L4 and L3 proxies for Apple platforms.
Our (client)
connection services
now have a slightly different trait signature, which allows for nicer error handling and smarter decision making, for example knowing when to retry and when to fail for real.
In function of PAC, you can now insert multiple
proxy routes
(addresses) into your input extensions, which will be tried in the order given. The existing proxy DB functionality now also happily makes use of this by default, but if you want the old random single-proxy selection behaviour you can still opt in to that.
The open-source world has been struggling for a few years now to understand
how to approach large language models (LLMs) and the licensing applied to
them. What constitutes "freedom" with respect to a black box filled with
numerical weights? The process taken by the Open Source Initiative (OSI)
in...
The page you have tried to view (
Considering the OpenMDW license
) is currently available to LWN
subscribers only.
Reader subscriptions are a necessary way
to fund the continued existence of LWN and the quality of its content.
If you are already an LWN.net subscriber, please log in
with the form below to read this content.
Please consider
subscribing to LWN
. An LWN
subscription provides numerous benefits, including access to restricted
content and the warm feeling of knowing that you are helping to keep LWN
alive.
(Alternatively, this item will become freely
available on September 3, 2026)
Hochul Heads Into Nail-Biter Reelection With...Criticism of Mamdani?
In two and a half months, New York will elect a governor. One candidate is Kathy Hochul, the incumbent Democrat who has emerged as a
strong voice
in the state's fight against a cruel
federal immigration policy
, and has, to a point, worked well with New York City's popular mayor, easing the decades-long tension that previously plagued state and City leaders.
In evaluating those two choices, New Yorkers seem to be Blakeman-curious. Earlier this month, a Siena Poll of likely New York voters
showed Hochul leads Blakeman by just 10 points
, and she hasn't notched majority support (49-39 percent in her favor), with Blakeman narrowing the 20-point lead Hochul enjoyed as recently as June. In 2022, her Republican opponent Lee Zeldin came shockingly close—within 6 points—to defeating her. An August 2022 poll had her with a bigger lead against Zeldin than Blakeman.
Microsoft has started rolling out a Classic Outlook theme for users of Outlook on the web and the New Outlook for Windows.
This new Outlook theme is rolling out as part of a targeted release beginning mid-August and expected to complete by the end of September. The theme will become generally available worldwide between late September and late October.
"When enabled, the setting applies coordinated changes across the Outlook experience, including visual styling, layout, typography, icons, and selected interactions,"
Microsoft said
in a Microsoft 365 Message Center update.
Once rolled out, the feature will not override any existing administrator configurations and will not automatically migrate users from classic Outlook to the new Outlook.
Microsoft also added that the new user interface style will be enabled by default for some users, but they can toggle it off to switch to the standard theme from Settings > General > Appearance.
"This update is designed to help users who are transitioning from classic Outlook by providing a more familiar experience while maintaining the capabilities of the new Outlook," it added.
"The setting will be available to all users and can be turned on or off at any time. It will be off by default for most users. As part of a phased rollout, Microsoft will enable the setting by default for some users moving from classic Outlook to the new Outlook. Those users can change the setting at any time."
Classic Outlook theme toggle (Microsoft)
New Outlook (also known as Outlook for Windows), which still lacks some Classic Outlook features, replaced Windows Mail as a pre-installed app on Windows 11 and Windows starting in October 2023 and
January 2025
, respectively.
In February, Microsoft announced that it would
postpone the new Outlook opt-out phase
for businesses from April 2026 to March 2027, giving enterprise admins 12 additional months to prepare a staged migration to the new client.
Microsoft made this decision even though, according to the company, it was "seeing strong and accelerating adoption of new Outlook."
Bluesky’s Active User Base Is Shrinking, and Remains Tiny Compared to Threads and Twitter/X
Daring Fireball
techcrunch.com
2026-08-21 09:36:58
Sarah Perez, writing at TechCrunch last week:
Decentralized social network Bluesky was one of the bigger
beneficiaries of the exodus from Elon Musk’s X in November
2024, following the U.S. elections. But now, nearly two years
later, the social network and would-be X competitor is struggling
to h...
Decentralized social network Bluesky was one of the bigger beneficiaries of the
exodus from Elon Musk’s X in November 2024,
following the U.S. elections. But now, nearly two years later, the social network and would-be X competitor is struggling to hold on to its momentum.
According to data from digital intelligence provider
Similarweb
, Bluesky’s mobile app had 10.4 million monthly active users worldwide in June 2026, down 27.2% year-over-year. In addition, mobile daily active users continued to decline, falling 25.6% year-over-year in July to around 3 million.
These declines indicate that even though Bluesky’s app continues to add registered accounts, fewer people are using its mobile app on a regular basis. It also suggests that some of those who joined Bluesky due to their dissatisfaction with X post-elections haven’t stuck around long-term.
But the bigger takeaway here is not just that Bluesky’s post-election bump has shrunk. Bluesky’s app has lost more than half its monthly active users from its late-2024 high, which means that its app usage continued to shrink even after the initial post-election surge subsided. (Its quarterly average was around 22.1 million monthly active users in the fourth quarter of 2024, Similarweb’s data indicates, and it declined to 10.7 million in the second quarter of 2026. That’s a decline of around 52%.)
Building beyond Bluesky
While Bluesky’s numbers are down, those who stuck around are committed. Its smaller community remains relatively active, with a stickiness rate (the ratio of daily to monthly active users) of roughly 29% in June, about the same as Threads.
For
Bluesky’s new CEO, Toni Schneider,
these numbers may not be as concerning. The company is not entirely focused on making Bluesky (the app) succeed, but on making it possible for the underlying protocol (AT Proto) to power a growing number of social apps, services, and communities. That’s something that is now taking place, as projects like BlackSky and Eurosky are growing, while some AT Proto apps
like the video-focused Skylight
have found early traction, too.
In addition, the company has launched new products, like the
AI-powered research tool Attie
, and is now working on adding support for private data to Bluesky. The latter could generate new interest in Bluesky from a different type of user — those less interested in the public square, and more interested in private networking and communities.
What’s more, Similarweb’s data doesn’t necessarily suggest that Bluesky users have returned to X, as some may have feared.
Instead, the data shows that X’s worldwide monthly active users on mobile were down around 3% year-over-year in June, and X’s mobile daily active users dropped 7% in July to 123.7 million. (Of course, X remains a sizable social network with around 302 million monthly active users on its app as of June 2026 and a growing number of web visits, up 5.3% year-over-year in July to 4.7 billion.)
If anything, the app to now be concerned about is Threads. It’s unclear whether Threads is benefiting from Bluesky’s falling engagement, or if it has managed to attract a new set of people who had never used a Twitter-like service.
In any event, the Meta-owned app’s daily active users were up 21.3% year-over-year to 147 million in July 2026, and its website visits were up 112% year-over-year to 471.6 million.
When you purchase through links in our articles,
we may earn a small commission
. This doesn’t affect our editorial independence.
Sarah has worked as a reporter for TechCrunch since August 2011. She joined the company after having previously spent over three years at ReadWriteWeb. Prior to her work as a reporter, Sarah worked in I.T. across a number of industries, including banking, retail and software.
You can contact or verify outreach from Sarah by emailing
sarahp@techcrunch.com
or via encrypted message at sarahperez.01 on Signal.
Abstract:
As well as obtaining beautiful images of the Universe, the Hubble Space Telescope's CCD detectors are sensitive radiation dosimeters that have been monitored in Low Earth Orbit for more than 24 years. The rate of radiation damage they received has varied over each Solar cycle, but several years out of phase with the appearance of sunspots or coronal mass ejections. We investigate functional forms that successfully fit the time series of damage to telescopes elsewhere in the Solar system. We obtain remarkably accurate fits to Hubble data but with physically absurd parameter values. During image post-processing, such fits can be used empirically, to correct more than 99.5% of the radiation damage's effect on image quality. However, fits to the time series with physically reasonable parameters produce worse performance. Our results highlight the diversity of radiation environments in different parts of our Solar system, and the complexity of Low Earth Orbit in particular. Our results also motivate continued monitoring of radiation damage to currently operational spacecraft, to more reliably predict the rate of degradation in (and useful lifespan of) future missions.
Submission history
From: Juan Paolo Lorenzo Gerardo Barrios [
view email
]
[v1]
Tue, 18 Aug 2026 18:00:17 UTC (28,507 KB)
Nvidia AVO scores 100% on the ARC-AGI-3 interactive reasoning benchmark
NVIDIA AVO continuously inspects, plans, implements, and evaluates, using memory, tools, and execution feedback to build on what it learns along the way. This allows the system to sustain progress across long-running tasks rather than starting over with each model context.
Read
What Happens When the Cost of Intelligence Drops 100x
Progress in large language models is usually reported as what the best model can now do that no model could do before. That is the direction that produces headlines, and it has indeed been truly incredible. Each step up at the top of the range lets a model handle a kind of task that was previously out of reach, whether that is fixing a bug that spans a whole codebase or, lately, making progress on outstanding mathematical problems that had not been solved by anyone.
There is a second direction of progress that gets less attention, which is how cheaply a given level of capability can be bought. A great deal of useful work does not require the smartest model available, but a model that is good enough, applied many thousands of times. Reading every scientific paper on a topic, checking every contract in an archive for a particular clause, or summarizing every thread in a large discussion forum are tasks of this kind. For these, the question is not whether a model exists that can do the job, but whether it can do the job ten thousand times within a budget. The ceiling unlocks new kinds of tasks; the floor unlocks volume.
When you pick a model for an application you are trading off how capable it is against how much each call costs. For agentic coding I have focused almost entirely on capability, with the general sense that the improved quality of the work is worth the money, even when far cheaper models exist that are reasonably capable. My attention was recently drawn to the cost of the floor. We are measuring how often datasets shared on the DANDI Archive are reused in later publications, which means reading on the order of ten thousand candidate papers with a model and asking of each one whether it actually reused the data. At today’s prices a full pass over the corpus costs a little over a hundred dollars with a model whose capability was at the frontier in the spring. At the prices of this past March, the same pass with the same level of capability would have cost several thousand dollars, and a year ago that capability was not available at any price. That change in the floor is what turned the analysis from a thing we could do on a sample into a viable project. I have been surprised by the progress across the cost spectrum, particularly how intelligent cheap models have become.
Artificial Analysis
has been benchmarking intelligence and price across hundreds of models for a couple of years, and enough of that data is accessible to reconstruct the tradeoff. In particular,
this plot
shows the intelligence index vs. the cost per task, providing a realistic cost estimate for different levels of model capability. The top line is what they define as the “Pareto line,” the most capable models at a given price point. This line describes the true frontier of LLMs. I pulled data from artificialanalysis.ai and looked at how the Pareto frontier has moved as new models have been released. I think it is worthwhile to take a beat to review this progress and make some predictions for the next few months.
The short version: the level of intelligence that cost $1.22 per task in February costs $0.022 today, a 56x drop in under six months, and the rate of decline is accelerating. At the measured pace, a 100x drop for a given capability level takes about a year, and the question worth asking is not whether that happens but what it changes.
The Artificial Analysis Intelligence Index
The capability axis throughout this post is the Artificial Analysis Intelligence Index, so it is worth being clear about what that number is. The current version, v4.1.1, is a weighted average over nine evaluations grouped into four categories: agentic tasks at 34%, coding at 24%, scientific reasoning at 24%, and general capability at 18%. The weighting reflects where the field’s attention is: a third of the score comes from a model’s ability to complete multi-step agentic work, not from answering exam questions. The component evaluations, their weights, and the scoring details are documented in Artificial Analysis’s
intelligence benchmarking methodology
.
What you end up with is a single number that represents model capability, sort of like an IQ for LLMs. It isn’t perfect, and two models with the same score may have different strengths, but I have found that this score does a reasonably good job of indicating a model’s capability.
As a reference point, Anthropic’s “Claude 4.5 Sonnet (Reasoning)” was for me and many others the first time a model felt capable enough to use in an agentic harness for writing code. At the time I was using Cline, and this model provided substantial productivity gains over auto-complete and copy/paste workflows. That model had an intelligence score of 37.4 (based on today’s intelligence scoring system). The top current model is Claude Opus 5 max effort, at 63.1.
To give a more visceral sense of what the different index levels mean, I borrowed Simon Willison’s
pelican benchmark
: prompt a model with “Generate an SVG of a pelican riding a bicycle” and look at what comes back. It is not what the index measures, but it is a task anyone can judge by eye. The panels below use the GPT-5.6 family at four points on the index: Luna at low, high, and xhigh effort, and Sol at max effort. I generated three samples per model and show the first one; all of them are in the
site repository
.
Index 33.9
GPT-5.6 Luna (low)
Index 47.0
GPT-5.6 Luna (high)
Index 50.1
GPT-5.6 Luna (xhigh)
Index 60.9
GPT-5.6 Sol (max)
First of three samples from each model for the prompt "Generate an SVG of a pelican riding a bicycle", generated through OpenRouter on August 20, 2026. Intelligence Index scores are from Artificial Analysis.
The progression is visible: more detail, better proportions, and a pelican that is clearly riding the bicycle instead of hovering over it.
Measuring Cost per Task
Cost per token is easily available, but different models can use a very different number of tokens, so a better indication of the cost of a model needs to take this into account. Cost per task is Artificial Analysis’s own measured number: the average cost in USD to run one task from their Intelligence Index evaluation suite, including the input, reasoning, and answer tokens actually billed during the run. The website displays it but the free API tier does not include it, so I scraped it from the data embedded in each model’s page on the site, covering both the models they currently benchmark and retired models whose pages still carry the measurement (older Claude Opus and Sonnet versions, the GPT-5.x line, and others). That yields measured cost for 137 models reaching back to DeepSeek V3 in December 2024, each paired with a release date and an Intelligence Index score on the current scale.
How the Frontier Has Moved
The chart below plots intelligence against measured cost per task and traces the Pareto frontier, the cheapest way to reach each intelligence level, as it stands today and as it stood at two month intervals over the past year, using each model’s release date to reconstruct what was available. The chart builds up one frontier at a time, pauses on the full picture, and repeats; use the button to stop it. Hover any point for the model behind it.
Intelligence Index against measured cost per Intelligence Index task (log scale). Small points are all 137 measured models at their last measured cost, tinted by the two month window in which they were released (models from before August 2025 are grouped with the August 2025 window). Hollow points, both small and large, are open weights models; filled points are proprietary. Hover a point for its details, including whether Artificial Analysis has retired it from live benchmarking. Each line traces the cheapest way to reach a given Intelligence Index among models released by the snapshot date; markers are the frontier models themselves. Where successive frontiers share a segment, the older line is drawn on top, so a newer line is visible only where the frontier actually moved. Models whose pages no longer carry a measured cost (o3 and GPT-5.3 Codex among them) are absent (see caveats).
Each successive frontier sits above and to the left of the last: more intelligence at the same cost, or the same intelligence for less. The pace of that movement is accelerating. Through the second half of 2025 the frontier inched forward: only two small bumps between August and October, and a single one between October and December. The February and April frontiers each moved a large part of the curve, and the last two snapshots have replaced the frontier almost entirely, with ten of the eleven June frontier models new since April and fifteen of today’s sixteen new since June.
The right edge tells the capability story. The ceiling of the frontier rose from index 35.3 in August 2025 (GPT-5 at $0.26 per task) to 37.4 that October (Claude 4.5 Sonnet), 48.4 in February (Claude Sonnet 4.6), 55.0 in April (Claude Opus 4.7 at $2.23), 62.1 in June (Claude Fable 5 at $3.14), and 63.1 today (Claude Opus 5 at $2.34): twenty eight Intelligence Index points in a year. The left half shows the rising intelligence of cheap models. As of August 19, 2026, the GPT-5.6 Luna effort ladder now owns almost everything below index 52, with the level that was the August 2025 ceiling available for $0.0088 per task. The June 2026 frontier was unusually dominated by open weights models: six of its eleven models were open (MiMo-V2.5, DeepSeek V4 Pro, MiniMax-M3, and GLM-5.2 among them), and they held the whole middle of the range from index 38 to 53. In earlier snapshots open models appeared only at the bottom of the range, and today, after the GPT-5.6 Luna release, only two of sixteen frontier models are open.
Note that a single model can cover a large part of this range through different reasoning levels: the GPT-5.6 Luna ladder runs from $0.0088 at low effort to $0.047 at max and covers the whole lower half of the frontier, while Claude Opus 5 spans $0.43 at low effort to $2.34 at max and buys about ten Intelligence Index points along the way. Effort is now a key dial in the cost/intelligence trade-off, and as a consequence, cost and intelligence are inextricably linked.
The records plot below tracks the cheapest measured cost per task achieved by any released model at or above a given Intelligence Index tier. The series reaches back to mid 2025 for the lower tiers, and higher tiers appear when they become available.
Each step is a released model that set a new low for its tier; hollow markers are open weights models and filled markers are proprietary. A tier's line begins when the first model with measured cost crosses that Intelligence Index threshold. GPT-5.6 Luna is placed at its launch price from July 9 and at its current price from the July 30 price cut; all other costs reflect current prices (see caveats).
Tier
First measured crossing
Cost collapse
Halving time
Index ≥ 30
Aug 2025 (GPT-5 high)
29x
~73 days
Index ≥ 40
Feb 2026 (Claude Sonnet 4.6)
56x
~28 days
Index ≥ 50
Mar 2026 (GPT-5.4 xhigh)
35x
~29 days
Index ≥ 60
Jun 2026 (Claude Fable 5 max)
3.8x
~34 days
A capability level is first reached by a large frontier model at a premium price. After some time, cheaper models arrive at the same level, and the record steps down by an order of magnitude or more. The ≥ 40 tier opens with Claude Sonnet 4.6 in February at $1.22 per task, undercut within two days by Gemini 3.1 Pro Preview at $0.33; MiMo-V2.5-Pro, an open weights model, cut the record to $0.034 in April, and GPT-5.6 Luna on high effort holds it at $0.022 today. The ≥ 50 tier follows the same arc a month behind: GPT-5.4 crossed it in March at $1.10, GPT-5.5 and then GLM-5.2 and Grok 4.5 walked the record down through the spring, GPT-5.6 Luna’s xhigh setting took the record at $0.16 when it launched on July 9, and OpenAI’s 80% price cut on July 30 brought it to $0.032, a 35 fold drop in five months.
The first crossing is a maximum effort frontier model priced at launch premium, most often from Anthropic or OpenAI. Following this, small distilled models from the big labs (the GPT-5.6 Luna line holds three of the four current records), and open weights releases (MiMo, DeepSeek V4, GLM, Hy3) drive rates down dramatically. Across the tiers with enough history to measure, the records halve roughly every four to ten weeks.
The top tier is where the premium survives. Only six models score 60 or above, and the cheapest of them, Grok 4.6, still costs $0.84 per task. But that record has fallen 3.8x since June, and if the pattern from lower tiers holds, a distilled model at this level should collapse the price within a couple of quarters.
To me, the most impressive result is the low price of OpenAI’s “GPT-5.6 Luna” given its intelligence. Now, the intelligence of Anthropic’s “Claude 4.5 Sonnet (Reasoning)” that set off the coding harness revolution 10 months ago is available using “GPT-5.6 Luna (medium)” for 1/40th the cost! That figure depends on the July 30 price cut; at Luna’s launch price three weeks earlier, it would have been 1/8th.
Caveats
The most important limitation is that costs are the latest measured values indexed by release date, not historical measurements taken at release. Prices get cut over a model’s life, so early points reflect any cuts since launch, which biases the analysis toward understating the collapse and toward dating it too early. The one cut I have corrected for is the largest recent one: OpenAI cut GPT-5.6 Luna’s prices by 80% on July 30, 2026, three weeks after its July 9 release (Terra was cut by 20% on the same day and Sol was unchanged). Since a price cut does not change the number of tokens a task uses, I reconstructed Luna’s launch cost per task by scaling the measured value by the price ratio, and in the records chart and table Luna’s records are dated to the cut, not to the release. This lengthens the measured halving times for the three lower tiers by a few days each. Other models may have had cuts I did not find, and a retired model’s last measured price may not be the one it launched at. Coverage is the second issue. Retired models are included only when their pages still carry the measurement, which recovered 44 of 228 retired models with prices and scores; the rest, o3, GPT-5.3 Codex, and everything from the GPT-4 era among them, are invisible, so the oldest frontiers rest on fewer models than actually existed and the true opening price of the lower tiers was likely set by models this analysis cannot see.
The retired models’ Intelligence Index scores are on the current scale, but the Index itself is one aggregate of many evaluations. And cost per task on an evaluation suite is a reasoning heavy workload with long prompts; a chat workload with short prompts and short answers would scale differently across models, particularly between reasoning and non-reasoning variants.
Predictions
Extrapolating measured rates is risky, since each collapse is a competition event and not a law, but the arcs have been regular enough to be worth putting numbers on. At the ≥ 60 tier’s current halving time of about 34 days, Grok 4.6’s $0.84 record falls below ten cents around the start of December. Index 55, which Grok 4.5 holds at $0.36 today, should cost under a dime by mid October. The ceiling is harder to call: it climbed twenty eight points over the year but only one point since June, which reads as saturation of the current index, not a slowdown in the models, so I expect the next milestone there to be an index revision, not a big number. And if the pattern of the last four tiers holds, whatever the revised index calls the frontier will debut at a few dollars per task and be commoditized within a quarter.
The Two Directions of Progress
The two directions of progress serve different kinds of work. A higher ceiling changes what is possible at all: the tasks that no model could do last year and one model can do now. A lower floor changes what is affordable at scale: the tasks that one model could already do, but not ten thousand times. The literature scan that motivated this post is a floor problem. The model only needs to read a paper and answer a well defined question, which models well below the current frontier handle reliably, but it needs to do that for every candidate paper, and the difference between $1 and $0.02 per paper is the difference between a pilot study and a complete census. Legal discovery, systematic reviews, large scale data curation, content moderation, and customer support triage have the same shape, and all of them get cheaper by an order of magnitude roughly every few months without any change in the work itself. The practical consequence is that the set of problems worth attempting with a model is expanding from both ends at once, and the expansion at the cheap end is the one that is easy to miss.
Cheaper Intelligence Means More Spending on It
A natural reading of these charts is that spending on LLMs should be falling. The opposite is happening. In 1865 William Stanley Jevons observed that more efficient steam engines, which needed less coal per unit of work, had increased Britain’s total coal consumption instead of reducing it, because cheaper work found far more uses. The same dynamic applies when the cost of a unit of intelligence falls by 30x. The work that was already being done gets cheaper, but the much larger effect is the work that was not being done at all because it did not clear the bar. This phenomenon became known as the Jevons paradox. Our literature scan is a small example: at last year’s prices it would have been run once on a sample, if at all, and at this year’s prices we run it on the whole corpus, repeat it when the pipeline changes, and are planning to run each positive result three times to reduce noise. The cost per paper fell by more than an order of magnitude and our total spend on the project went up. Demand for intelligence at a given price appears to be highly elastic, and as long as that holds, the falling frontier translates into more tokens consumed, not fewer dollars spent.
The motion of the frontier is more predictable than any individual release. Every capability tier so far has followed the same arc: premium debut, rapid commoditization, a settled record held by a distilled or open weights model at a few percent of the debut price. If a capability exists at any price today, the sensible planning assumption is that it will exist at commodity price within months. For system design, that argues for architectures where the model is a swappable component and the routing between capability tiers is explicit, because the tier boundaries themselves have not settled and show no sign of settling soon.
The model routers appearing on the market are a sign that this is being operationalized. OpenRouter now offers a
router
that takes a minimum capability score and sends each request to the cheapest model on the Artificial Analysis frontier that clears it, so that a system benefits from the moving frontier automatically, with no developer tracking it. Hardcoding a model name into an application has become the fastest way to overpay.
What Changes at 100x
If the pace of the last year holds, the capability that cost a dollar per task at the start of 2026 will cost a cent by the end of it, and the index 60 models that cost a few dollars per task today will be under a dime within a couple of quarters. I want to be careful not to overreach from a year of data, but a few consequences follow directly from the numbers.
Reading everything becomes the default. At a cent per document, a model can read every paper in a field, every record in an archive, every email, or every message in a support queue as a matter of routine, and the question shifts from which documents to look at to which questions to ask of all of them. Projects like our reuse census stop being projects and become monitoring: the scan can run on every new publication as it appears. Multi-pass workflows become the norm, since running a task three times and taking a consensus costs less than running it once did a few months earlier, and the accuracy gains from that are large. And the capability tiers themselves stop being a meaningful way to describe a system, because a pipeline will route each step to whatever level of intelligence it needs at whatever that level costs that week. The scarce resource in that world is not intelligence but the judgment about what to point it at, the ground truth to check it against, and the systems to run it at scale. Those are the parts of the work that are not getting cheaper.
Model metadata pulled from the Artificial Analysis free API, and measured cost per task scraped from the model pages on artificialanalysis.ai, on August 19, 2026. Corrections welcome.
Ben Dichter, PhD
is the Founder of CatalystNeuro. He received his Ph.D. in Bioengineering from the UC Berkeley – UCSF Joint Program in Bioengineering. He is now a data scientist consultant for neuroscience labs, focusing on enabling collaboration by building systems for sharing of data and analyses.
We write a value in one goroutine and read it in another. Nothing crashes, and the value is there. It looks like our code works.
We actually got lucky. Go does not guarantee that one goroutine will see a write made by another unless the program explicitly coordinates their operations. This post explains what happens on a real machine, why it happens, and what we should use instead.
1. The program that works
Suppose we want to pass a value from one goroutine to another without using a channel. A simple approach is to store the value in one variable and use a boolean to report when the write is complete. The reader waits for that boolean before accessing the value:
You can run this example
in the Go Playground
. The Playground prints
hello
and exits, so this execution produces exactly the result we expected.
But when we run this snippet with
go run -race
, the race detector reports one race involving
done
and another involving
msg
:
==================
WARNING: DATA RACE
Write at 0x00c0000121cf by goroutine 7:
main.main.func1()
main.go:11 +0x68
Previous read at 0x00c0000121cf by main goroutine:
main.main()
main.go:14 +0x110
==================
==================
WARNING: DATA RACE
Read at 0x00c000014040 by main goroutine:
main.main()
main.go:16 +0x128
Previous write at 0x00c000014040 by goroutine 7:
main.main.func1()
main.go:10 +0x30
==================
hello
Found 2 data race(s)
exit status 66
So is this snippet safe and valid because we use a
for
loop to check the
done
flag? Let’s consult the Go memory model.
2. The Go memory model
The race detector does not care about the output. It checks whether 2 goroutines access the same memory concurrently without synchronization and at least 1 access is a write.
The
Go memory model
answers the next question: “Which write must each read use?” It tells us which behaviors Go guarantees across all runs. I know this is not obvious, so let’s diagnose the 2 reported races.
Race 1: main may not read
true
Let’s put the snippet here so we don’t lose context:
WARNING: DATA RACE
Write at 0x00c0000121cf by goroutine 7:
main.main.func1()
main.go:11 +0x68
Previous read at 0x00c0000121cf by main goroutine:
main.main()
main.go:14 +0x110
done = true
is a non-atomic write, and every evaluation of
!done
contains a non-atomic read of the same variable. The program does not require the write to happen before any of those reads.
Since 2 goroutines access the same variable and one access is a write,
done
has a read-write data race.
The
Go memory model
does not guarantee that a write in 1 goroutine becomes visible to another goroutine by itself. This snippet does not synchronize the
write to
done
with the
reads of
done
in
main
, so the loop may continue reading
false
. That may sound strange because the order looks clear in the Go source code.
In theory, the program may behave as if the generated code reused the value from its first read:
The source loop and a possible optimized form that reads done once
Of course, the code on the right is only for explanation. The compiler does not generate that form for this example. The important point is that the Go source code and generated assembly do not need a one-to-one relationship.
There is no guarantee that a write made by the new goroutine will become visible to
main
, so the compiler may reuse a loaded value in a register or a temporary, or arrange instructions in another order, as long as the optimization stays within the Go memory model.
Another question is what happens if the writer goroutine updates
done
while the main goroutine is reading it. Can the
main
goroutine receive a partially written value?
The answer for this specific case is no.
On arm64, a
bool
uses one byte,
The writer stores that entire byte with one
MOVB
(move byte) instruction,
main
loads the entire byte with one
MOVBU
(move byte unsigned) instruction.
Since each instruction accesses the complete one-byte
bool
, the access is indivisible:
main
cannot receive half of its value.
The writer and main access the complete one-byte bool on arm64
But the same reasoning does not apply to a whole struct, array, or other value made from multiple parts.
Go may read or write a struct one field at a time, an array one element at a time, and a complex number one component at a time. A value larger than one machine word can combine parts from separate writes. Strings, slices, and interfaces commonly use multiword internal representations, so a race can create an inconsistent value and may corrupt memory.
For example, consider a 24-byte struct made from three
uint64
fields:
The assignment is 1 statement in the source code, but Go may write the 3 fields separately and in any order.
One valid execution writes
C
first. The racing reader can then read the old values of
A
and
B
together with the new value of
C
:
A racing struct read combines old and new field values
The resulting
snapshot
is
{A: 0, B: 0, C: 2}
. On the 64-bit machine used for this example, each field contains a complete
uint64
value, but the struct as a whole matches neither the initial
{A: 0, B: 0, C: 0}
nor the value
{A: 2, B: 2, C: 2}
assigned by the writer.
Note
You can reproduce the mixed read
in the Go Playground
. The Playground version intentionally adds a 64 KiB byte array between each pair of fields. This makes both
state = one
and
s := state
copy 131,096 bytes instead of 24 bytes.
The larger copies take longer, so they are more likely to run at the same time before either one finishes. The extra bytes only make the mixed result easier to reproduce. The data race already exists without them.
Race 2:
done == true
does not guarantee
msg == "hello"
Assume that the loop reads
true
and exits, exactly as it does in the Playground. Race 2 asks a separate question: “does
done == true
also guarantee that
fmt.Println
reads
"hello"
from
msg
?”
Let’s put the relevant snippet here so we can follow Race 2 without scrolling back:
WARNING: DATA RACE
Read at 0x00c000014040 by main goroutine:
main.main()
main.go:16 +0x128
Previous write at 0x00c000014040 by goroutine 7:
main.main.func1()
main.go:10 +0x30
Since two goroutines access the same variable and one access is a write,
msg
has a second data race.
The Go source code gives us one order inside each goroutine. The new goroutine writes
msg
before it writes
done
.
main
reads
done
before it leaves the loop, then reads
msg
for
fmt.Println
:
Why reading done does not order the write and read of msg
If we read the code from top to bottom, it may seem obvious that
msg
must contain
"hello"
when
main
leaves the loop. The new goroutine writes
msg
before setting
done
to
true
, and main reads
msg
only after reading
true
from
done
.
But from Go’s point of view, the read of
done
answers only one question: which write supplied the value returned by this read? The value
true
came from
done = true
, but
msg
is a separate memory location with a separate read.
Under the Go memory model, nothing guarantees that when
main
leaves the loop,
msg
contains the value written by the other goroutine.
Go therefore allows this result:
read done true
read msg ""
The Playground prints
"hello"
in this example, so this run does not show us what can go wrong. Let’s use another snippet where the same missing cross-goroutine order produces a result that we can reproduce on real hardware.
Two goroutines start at the same time. Each one writes to its own variable, then reads the other one:
If goroutine A finishes before goroutine B starts, then
r2
is 1.
If goroutine B finishes first, then
r1
is 1.
If they interleave, at least one goroutine sees the other’s write, so either
r1
or
r2
is
1
, or both are.
But whatever order we imagine,
it should be impossible for both
r1
and
r2
to be 0
, because that would require each read to happen before the other goroutine’s write.
Running that experiment 200,000 times on an Apple M-series machine produced:
both goroutines read 0: 2 out of 200000 rounds (0.0010%)
The program produced this “impossible” result twice out of 200,000 rounds, which is 0.0010%. And nothing is wrong with the hardware. You can run the same experiment
in the Playground
.
But this result needs 2 separate explanations.
First, Go guarantees the result required by the source inside one goroutine. See this snippet:
a
must contain
1
. The compiler may combine the 2 statements, replace them with other instructions, or arrange those instructions differently. But any optimization must still preserve the dependency from
b
to
a
and produce the correct result.
But in our case, the 2 statements in goroutine A are independent:
r1 = y
does not use
x
,
x = 1
does not use
y
.
The same is true for goroutine B. The compiler does not have to preserve their textual order in the binary as long as the generated program still follows the
Go memory model
.
The compiler may emit the independent load before the store
This kind of compiler reordering could explain the result described in Race 2. But it did not happen in the experiment above. The compiler kept the 2 memory instructions in source order in the generated arm64 binary.
That leaves a second explanation: how CPU cores make writes visible to each other.
Even when the machine instructions keep the source order, 1 core does not have to make its write available to the other core before its next read finishes. No memory barrier enforces that order in this binary.
Core A can read the old value of
y
while core B reads the old value of
x
, so
r1
and
r2
can both be
0
.
The writes can reach the other core after both reads finish
This is also why the rate in our run is 0.0010% and not 50%. The exact rate depends on goroutine scheduling, core placement, processor memory behavior, and other runtime conditions, so a test may see the result only occasionally. This kind of flaky and annoying bug is often the hardest to reproduce.
3. How synchronization makes earlier writes visible
For this snippet to be correct, it needs 1 guarantee: the writer must write
"hello"
to
msg
before
main
reads
msg
. The smallest change that provides this guarantee is to replace the plain
done bool
flag with
done atomic.Bool
:
atomic.Bool
reads and writes its value through operations defined by
sync/atomic
. Go defines the following rule for atomic operations:
If the effect of an atomic operation A is observed by atomic operation B, then A "synchronizes before" B.
In our example:
done.Store(true)
is operation
A
.
The
done.Load()
call that returns
true
is operation
B
.
Operation
B
reads the value written by operation
A
, so
A
synchronizes before
B
.
In other words, every write sequenced before
done.Store(true)
in the writer goroutine is guaranteed to be visible to
main
after
done.Load()
reads that
true
. This includes
msg = "hello"
, so the later read of
msg
in
fmt.Println
must see
"hello"
.
If you read the Go runtime and compiler source, you will see names such as
StoreRelease
and
LoadAcquire
.
Release
describes the store’s guarantee for writes completed before it.
Acquire
describes the load’s guarantee for reads that run after it.
An atomic store and load connect the write of msg to its later read
A mutex provides exclusion and visibility
You are probably familiar with
sync.Mutex
and its main job: allowing only one goroutine at a time to access protected state.
But a mutex also makes writes from one lock holder visible to the next lock holder. This
Counter
uses both guarantees.
Unlock connects a protected write to a later read after Lock
Go guarantees that a call to
Unlock
synchronizes before
a later call to
Lock
returns. In other words, after
Add
writes
n
and unlocks
mu
, a
Value
call that locks
mu
later is guaranteed to see that write.
WaitGroup waits for task completion
The original snippet only needs
main
to wait for one task.
sync.WaitGroup
provides that relationship without a busy loop:
tasks.Go
starts the function and tracks the task.
tasks.Wait()
does not return until the function has completed. Go also guarantees that writes made by the function before it returns are visible after
Wait()
returns.
fmt.Println
therefore reads
"hello"
.
Unlike the earlier
for !done
loop,
main
blocks inside
Wait()
instead of repeatedly checking a value and using CPU while the task is still running.
Channel close can signal completion
The same program can use a channel when one goroutine needs to announce an event:
Of course, calling an arbitrary function such as
abc()
does not by itself create a guarantee between goroutines. The function would need to use a synchronization operation internally.
close(ready)
provides such an operation because Go connects it to a receive that completes after
ready
is closed.
A channel connects the write of msg to the later read
So Go guarantees that closing a channel synchronizes before a receive that returns because the channel is closed. The goroutine writes
msg
before
close(ready)
, and
main
reads
msg
after
<-ready
, so
fmt.Println
is guaranteed to read
"hello"
.
sync.Once makes initialization visible
Sometimes many goroutines need the same value, but the code that initializes that value must run only once.
sync.Once
provides that guarantee:
Only one call to
Do
runs the function. Other calls wait for that function to return. Go guarantees that the function’s return synchronizes before every
Do
call returns, so every caller can safely read
message
after
once.Do
returns.
Atomic operations follow one global order
The earlier
x
and
y
experiment can use atomic integers:
The atomic version produces no rounds in which both reads return
0
:
with sync/atomic, both read 0: 0 out of 200000 rounds
Go requires all atomic operations to behave as if they ran in one global order. In our example,
x.Store
,
y.Load
,
y.Store
, and
x.Load
must all belong to that same order. Both goroutines use this order when deciding which value each
Load
returns. Another execution may use a different order, but the two goroutines still cannot use separate orders.
In other words, only two cases are possible when we compare the two stores:
The first atomic store forces one of the later loads to read 1
At least one load must therefore return
1
. The result
r1 == 0
and
r2 == 0
is no longer possible here.
Atomic operations are useful when one shared value can be updated independently, such as a counter or a
ready
flag. They cannot combine several updates into one operation. For example, if
balance
and
version
must always change together, another goroutine could read between two atomic stores and see the new
balance
with the old
version
. A mutex can protect both fields while they are updated and read.
Note
The
WaitGroup
has a separate job in this snippet.
main
reads
r1
and
r2
only after both task functions return.
4. Testing the synchronization
The question in every example above is not whether the program returned the expected value. The question is whether Go guarantees that the reader sees the writer’s work.
The race detector can report executions that lack this guarantee:
The detector tracks memory accesses made by the running application. It reports a race when concurrent goroutines access the same location, at least one access is a write, and no valid synchronization connects those accesses.
But this is a dynamic check. An unsafe function that never runs during a test cannot produce a report. Even a function that does run may require a particular execution path or schedule before both conflicting accesses occur. A clean result should therefore be read as “no race was reported in these executions,” not “the program contains no races.”
The source still needs a clear explanation of why each shared read is safe. In the examples above, that explanation comes from atomics, mutexes, task completion, channels, or one-time initialization.
Don’t let the absence of a flying car in your garage distract you: we are definitely now living in a version of the future out of J.G. Ballard or William Gibson.
Every day I notice a news story that startles or alarms me, that feels like pure cyberpunk, nestled between reports about council bin collections and cabinet reshuffles.
“The first ever IV Drip clinic to appear in the centre of a shopping mall, our Westfield clinic is a must-stop shop when doing a bit of retail therapy. From Vitamin Drips to Instant Vitamin D Testing, Get A Drip Westfield has it all. Book your appointment today!”
Publicity for Get A Drip Limited
This made me think of Michael Moorcock’s habit of opening stories or chapters with startling newspaper clippings that revealed the future apocalypse already underway in the present, or
Ballard’s typographic collages
. So I began snipping.
“A video has been shared on social media with claims it shows a robotic dog patrolling the grounds of a ‘migrant hotel’. But this is false – while the footage does appear to show a robotic dog, it was filmed in the grounds of a building housing a religious group in Crewe.”
Full Fact, 4 November 2025
It’s not quite
Blade Runner
but it might be
Robot Jox
or
Bubblegum Crisis
.
“Soul ‘musician’ Sienna Rose has made headlines this week as suspicions mount that her music is a product of artificial intelligence… On Spotify, her sound is described as a blend of the ‘elegance of classic soul with vulnerability of modern R&B’, while she is simply referred to as ‘an anonymous neo-soul singer’. Many fans have taken note of the reference to her anonymity, which makes her 2.6million monthly listeners all the more staggering.”
NME
, 17 January 2026
In the middle of a video about stationery and everyday tech on YouTube the other day there was
a passing shot of two robots kickboxing in a cage
; another robot was taking coffee orders at a kiosk. This was (blandly) “cool to see”.
“British police have helped seize a record nine-tonne haul of cocaine from a ‘narco sub’ in the Atlantic Ocean… The mammoth seizure weighed nearly as much as a school bus and the sub was 230 nautical miles from the Azores when it was intercepted… The semi-submersible eventually sank before authorities could take all its cargo, sending 35 of the 300 packages to the bottom of the Atlantic.”
Sky News, 27 January 2026
What the visionaries got wrong, it turns out, was what would go digital. Did anybody bet on cigarettes? Or scooters? Or that neon would be replaced by LEDs?
“A humanoid robot named Edward Warchocki chased away a herd of wild boars in Warsaw, shouting ‘Go away!’ in Polish as the animals fled into the forest, in video footage released on Sunday…”
Reuters, 14 April 2026
My personal moment of future shock came when a delivery robot trundled past me on Gloucester Road in Bristol, weaving between Saturday afternoon drunks and stoners, some of whom chased it and blocked its way.
“Four child-sized humanoid robots take the stage at an arena in eastern Seoul, and as the opening beats of a song by K-pop star G-Dragon begin, they start to dance… Arms swinging, legs stepping in sync, heads bobbing, wigs and baggy clothes swishing, until – mid-performance – one of them seemingly malfunctions and has to be removed from the stage… Welcome to Galaxy Robot Park, a new 16,500 square metre facility in Gangdong district that its creators claim is the world’s first robot theme park.”
The Guardian
, 25 May 2026
More often, the shock is that there is no shock. My mum uses the voice-activated computer in her house to remind herself to take the sprouts off the boil.
“[Shania] Collins had enjoyed modest success as a sprinter, with contracts from Puma and Adidas. But by 2024, with her career stalled and earnings shrunk, she retired at 29 to begin a long screening process to follow her parents into working for the [Drug Enforcement Agency]… Then organizers of the Enhanced Games, a controversial sports startup, got in touch last fall with an offer. The organizers were planning a one-day competition of sprinting, swimming and weightlifting in Las Vegas that would not only allow but encourage doping. And it paid the kind of money that might take some athletes years to make — six-figure salaries, on top of prize money of up to $250,000 for event winners and $1 million for a world record.”
NBC News, 26 May 2026
All of this technological advancement, apparently utterly mundane and running in the backgrounds of our lives, is accompanied by a sense of apocalyptic weirdness both in the climate and the culture.
“Meta has quietly embedded face-recognition technology for its smart glasses into an app downloaded to millions of phones… Code discreetly added to Meta’s AI app over multiple updates this year shows that the feature, internally called ‘NameTag’, identifies people captured by the glasses’ camera and, when activated, alerts the wearer when it recognizes someone.”
Wired
, 4 June 2026
Blade Runner
had its acid rain. We have wildfires taking out suburban housing estates in the English Midlands and heat buckling railway tracks.
“Two arrests have been made during a police operation using drone technology to target anti-social driving… Dorset Police said its operation on 29 May was in direct response to concerns raised by people living in Sandbanks, Poole… A police drone was deployed over the area to give a higher vantage point to spot any offences of poor driving, allowing officers to then intercept motorists on the ground, said Dorset Police.”
BBC, 6 June 2026
My brother, a data scientist, told me yesterday that it is now possible to prompt a large language model (LLM) to produce a three-dimensional digital model which you can then realise with a 3D printer. It’s not quite “Tea, Earl Grey, hot,” but it’s alarmingly close.
“While spending money on video games is not uncommon, EVE Online stands out because players’ assets can be permanently destroyed; their real-world cash outlay gone in seconds… The game’s financial system is so complex that in 2025, a former economist from the Central Bank of Iceland was hired to oversee it… Playing EVE Online can take hundreds of hours. Some see it as a second job, dedicating up to 35 hours per week to their virtual duties on top of their real-world nine-to-fives.”
BBC, 6 June 2026
At work, people send AI bots to attend meetings for them and it’s no more than an annoyance – a question of etiquette. If you’d told me this was possible when I was reading
Neuromancer
as a teenager in a concrete council house I’d have been awestruck.
“Stranger Than Heaven is an action-adventure game that spans fifty years… On Friday evening… Snoop Dogg took to the stage of Summer Game Fest to confirm Tupac’s involvement in Stranger Than Heaven as Amaru, which was the late rapper’s middle name. ‘The Tupac estate and my son and myself, we work very closely together,’ Snoop explained. ‘So it just made sense to put him in this game, because his likeness and his spirit still lives on. I just felt like it was so connected to what we’re doing.’
NME
, 8 June 2026
Another moment of future shock was when a former colleague posted on LinkedIn about his first ride in a self-driving taxi in San Francisco. It was a novelty, sure, but mostly just a fun story he could use to liven up his feed. Most people didn’t comment, they just gave it a weak “thumbs up”.
“During mealtimes, Vincent Zhang, a tech worker in Shanghai, has a habit of whipping out his phone to check on his ‘virtual parents’: a middle-aged couple online, armed with an endless stream of warm words for their imaginary child… In one of their most popular videos, the pair coos to the camera. ‘Are you tired from work and study lately? Don’t push yourself too hard. Mum and Dad know that you have endured a lot.’ … In the comments, many call the couple mum and dad, telling them about their lives and asking for birthday blessings.”
BBC, 13 June 2026
There are AI-generated posters in my local pub. My local supermarkets have signs warning shoppers that they’re using facial-recognition technology. It doesn’t feel like science fiction because everything surrounding it is so ordinary: facial recognition scanners; two for one on Doritos; images conjured from a prompt; pool, 50p a game, and roast potatoes on the bar on Sunday afternoon.
“AI-generated photos and videos featuring Russian soldiers have gained popularity on social media since mid-2025. They are most often posted by relatives of Russian servicemen fighting in Ukraine… The quality varies. In some videos, the AI generates figures without limbs or produces grotesquely distorted faces.”
BBC, 14 June 2026
How do you write science fiction in this context? Anything you come up with will seem farfetched, until it actually happens three weeks later and nobody bats an eyelid.
“US President Donald Trump celebrated his 80th and America’s 250th birthday with an Ultimate Fighting Championship (UFC) event on the White House lawn… Trump and thousands of other mixed martial arts fans watched on as American fighter Justin Gaethje beat Spanish-Georgian opponent Ilia Topuria to win the lightweight championship in the main event.”
BBC, 14 June 2026
When Moorcock and Ballard repurposed contemporary newspaper clippings they were saying, stop, look – can you believe this shit? But does stopping and looking actually help?
“The alt-pop US musician and internet personality Oliver Tree was among six people who died when the helicopter he was travelling in collided with another in Brazil… The 32-year-old had been on a world tour when the crash occurred over Rio de Janeiro on Sunday. One of the helicopters then fell onto the car park of a dealership, setting around 20 vehicles ablaze… With his distinctive bowl haircut, he was known for hits including Life Goes On, Miss You and Alien Boy.”
BBC, 15 June 2026
The human ability to resist stopping, to look away, to adjust to a new reality, absorb changes, and move on, is extremely useful.
“Banks say that criminals are engaging in more sophisticated fraud at greater volume with the use of artificial intelligence (AI)… Criminals have used AI to mimic the voices of celebrities, and even those of the victims’ family and friends, which has enabled them to carry out the crime at a greater scale.”
BBC, 15 June 2026
I’ve often wondered how long it would take me to adapt if I suddenly found myself fifty or a hundred years in the future. I suspect the answer is about 48 hours.
“The Trump administration on Tuesday announced a ban on new foreign-made humanoid robot imports to the US over ‘unacceptable risks’ to America’s national security… The move applies to advanced robots – including humanoid and four-legged machines. Many of them are made in China, which is competing with the US to develop robotics and artificial intelligence (AI).”
BBC, 29 July 2026
Because everyone around you would be totally unfussed by whatever unimaginable technological advances were occurring around them: “That? Huh. I suppose it is a bit odd, now you mention it. Do you want another HobNob?”
Grand jury declines to indict Ohio man charged with destroying Flock camera
A grand jury in Ohio has declined to indict a man charged with felony vandalism for allegedly destroying a Flock automatic license plate reader camera.
Police in Union Township, a Cincinnati suburb, accused Cody Morelock of disassembling the camera, its support pole and solar panel on June 13.
Investigators, according to
WKRC-TV
in Cincinnati, identified Morelock after obtaining surveillance footage from other cameras near the scene as well as information linked to a credit card and a customer rewards account.
Download the Straight Arrow app today to get the stories that matter free from manipulation, bias or agenda.™
Point phone camera here
Police estimated the damage at more than $1,000. Morelock posted a $10,000 bond and was released from custody shortly after his arrest.
A Clermont County grand jury, however, opted not to indict Morelock, and the charges were dismissed.
The company’s cameras record the license plate numbers and characteristics of vehicles that pass by. The data is then hosted in a central database that can be accessed not only by local police but often by law enforcement agencies in other cities and states.
The incident in Union Township is part of an ongoing trend that has seen dozens of
Flock cameras vandalized
across the country. Earlier this month, police in Winona, Minnesota, reported that someone had
cut down and stolen
each of the city’s eight license plate reader cameras.
Social media users are promoting a loosely organized event known as “
De-Flock America Night
,” encouraging people to vandalize or obscure Flock cameras on Halloween.
The backlash against Flock has intensified as a growing number of police officers have been accused of or charged with abusing the technology, often to stalk romantic interests. As of Aug. 12, there had been more than
100 cases of abuse
by law enforcement, according to the Institute for Justice.
In response, Flock announced
new safeguards
designed to prevent misuse by police. Critics, such as the
Electronic Frontier Foundation
, argue that the reforms are largely “cosmetic,” and that warrants should be required for searching license plate reader data.
Welcome to Yes/No/Cancel, the online usability magazine. This first article describes the origin of
the name, and explains why it is bad to use buttons labelled
Yes
,
No
and
Cancel
in computer
programs. I also discuss why user-friendliness in general is a very important topic.
This online
magazine (or blog if you will) is about user-friendliness, and lack thereof. It criticises bad
design and promotes good design. One might think that after usability research has been conducted
for many years and many books have been written on the topic, finally people would have learnt to
get it right. But no – my impression is that many products are as bad as ever, and the reason why I
am writing this is to raise awareness of these problems.
But why should you care? As a user, you
should care because you have a choice – you can stop using/buying the product that is not friendly
to use. You can switch to a better one, saving you frustration and annoyance. As a manufacturer, you
should care for much the same reason – you are in a competitive environment, and if you don’t
carefully consider the needs of your customers, you will see them leaving very soon!
Maybe you ask
how this website got its name. Yes/No/Cancel, that sounds like computers. Yes, and a lot of the
content here (but definitely not all) is going to be about computer software. Today, many pieces of
software are amongst the most complex pieces of engineering which the human mind has devised. It is
therefore not too surprising that some software packages are extremely difficult to use. But
software is also used by many people every day who don’t want to know about this complexity. Is
difficulty of use really necessary? There are some examples of extremely complex systems which are
absolutely straightforward to interact with (
Google search
for example –
it’s the work of a big team of the world’s best software engineers over several years, and still
it’s just a simple search box).
Making complicated systems easy to use is actually quite a
difficult problem. Part of the problem is that the engineers designing the system are often used to
a certain way of doing things, but this way is not always best adapted to a particular situation or
audience. The designers and developers of a system must therefore constantly be questioning their
habits, so that they can find better ways of solving problems if better ways exist.
One particular
bad habit of programmers annoys me so much that I decided to name this website after it.
Johannes
suggested the name:
Yes/No/Cancel
. The choice you are so
often presented with in many computer applications, and so often you have to stop and think, because
it’s not immediately clear what each of the choices is actually going to do. Which one of the
buttons will cause all your work to be lost if you press it? Which one will save it? And what does
Cancel
mean anyway? Aaargh, it causes headaches.
Let me explain this with a few examples. A
situation in which you frequently encounter a Yes/No/Cancel dialog box is when you are trying to
close a document without having saved it. Like
this:
Nice of it to ask, you say – you had completely forgotten to save. Ok. Now
compare it to this one:
Can you believe it? It’s asking the opposite question! Now even if you
usually know by habit which button to press, suddenly you have to stop and think. And this box is
even worse, because it’s not clear what the difference is between No and Cancel.
Fundamentally the problem here is that we are actually asking two questions at the same time:
Do you want to save the document?
Do you want to quit the application?
The answer to each question might be yes or no, which gives us four different possible actions:
save changes and quit
(the “Yes” button in the first example)
discard changes and quit
(the “No” button in the first example)
do nothing – do not save and do not quit
(the “Cancel” button in the first example)
save changes but do not quit
The fourth option is generally perceived
to be silly, so there is no button for that purpose and we get a choice of three. In the first
example picture, these three correspond to Yes, No and Cancel respectively. What about the second
example? Clicking Yes will “discard changes and quit”. Maybe clicking No will save changes and quit,
or maybe it will do nothing. Who knows what cancel will do, let alone the mystery of the red X in
the corner.
Already with simple examples like this, you can begin to see that it’s a bad idea to
label buttons as Yes, No and Cancel. The meaning of these words depends very much on the question.
In fact, if you have no previous computing experience, you will probably have no idea what to
answer. As a user, you just want to know which button is going to cause your work of the last 2
hours to be lost – and neither of these examples makes it immediately clear which the “dangerous”
button is.
Apple have tried to avoid this problem by not labelling the buttons Yes/No/Cancel, but
more descriptively:
Using a verb (in this case “save”) is recommended in Apple’s
Human Interface Guidelines
.
Also note that the “dangerous” button (which discards changes and quits) is set apart from the two
“safe” buttons. This is clearly much better already, but the program is still trying to answer two
questions at the same time, which you may consider to be an unnecessary complication.</p>
I won’t dwell
on any Microsoft vs. Apple discussion though, because what I am saying applies not just to these two
companies, but also to every other organisation or person who writes software. And there are some
terrible occurrences of Yes/No/Cancel in the world for which neither Microsoft nor Apple carries any
blame.
One terrible thing which you see sometimes is an implicit relabelling of the buttons. Here
the programmer clearly couldn’t be bothered to make his own buttons, and instead placed the burden
on the user:
You really need to switch
on your brain to decide which button to press. And by phrasing the question badly, it can get even
worse:
At this point, I very much hope that you will have run out
of the room screaming. And maybe returned to read the rest of this article. (With a headache.)
You might think that the last example was very contrived, but the point I wanted to make was about
negative questions. Why ask whether not to do something (the negative) if you can simply ask whether
to do something (the positive)? I find this occurring particularly frequently with respect to
checkboxes:
It is counterintuitive to put a tick in a box for something you don’t want. Just don’t ask negative
questions. But that’s a story for another day.
A few final remarks:
This website is not related to the
web comic OK/Cancel
although we’re talking similar subject
matter.
You might have noticed that the implicit relabelling example above is actually
a case of Abort/Retry/Ignore. I first considered making that the name for this site, but fortunately
Abort/Retry/Ignore is largely extinct by now (lucky you if you don’t know what I’m talking about).
There is
an excellent poem about Abort/Retry/Ignore
though, inspired by E.A.Poe.
If you found this post useful, please
support me on Patreon
so that I can write more like it!
I won't give your address to anyone else, won't send you any spam, and you can unsubscribe at any time.
Bluesky Is Full of Anti-AI Zealots
Daring Fireball
bsky.app
2026-08-21 08:52:47
Mike Masnick, in a thread on Bluesky:
Multiple people I know have told me that they love the idea of
Bluesky, and want it to succeed, but have abandoned it for X
because the use agentic tools in their work and find them
incredibly useful, and feel that any mention of their usage here
leads to ha...
TL;DR: DuckDB v2.0 replaces its PostgreSQL-derived SQL parser with a PEG-based parser that is easier to evolve and can be extended at runtime.
At DuckDB, one of our goals is to make working with a database system as easy as possible. Users interact with the system through the widely understood Structured Query Language (SQL). Previous blog posts have covered DuckDB’s
friendly
SQL
, including
GROUP BY ALL
and column selection using
SELECT * EXCLUDE (...)
. Before DuckDB can execute a query using these features, however, it first has to determine whether its syntax is valid. That is the job of the
parser
, and in DuckDB v2.0 we are completely replacing it without you noticing.
At a high level, DuckDB processes a SQL query through the following stages:
In this blog, we focus on the tokenizer, parser, and transformer:
Tokenizer: This is the first step and is responsible for splitting up the raw input string into
tokens
. These can be of various categories, for example:
KEYWORD
,
NUMBER
, or
IDENTIFIER
. It is also where comments, in SQL denoted with either
--
or
/* */
, are recognized and skipped.
Parser: The parser determines whether these tokens follow DuckDB's grammar and produces a
ParseResult
tree.
Transformer: Converts the generic parse results into DuckDB’s internal abstract syntax tree (AST), forming structures such as
SQLStatement
,
TableRef
, and
ParsedExpression
. The resulting AST is passed on to the binder.
The parser determines whether a query is syntactically valid, while the binder determines whether the tables, columns, and functions it refers to actually exist.
Consider the following query:
SELECT*WHEREtrueFROMrange(1);
Parser Error:
syntax error at or near "FROM"
LINE 3: FROM range(1);
^^^^
Every individual
token
in this query is valid, but the clauses occur in an order that DuckDB’s grammar does not accept. Friendly SQL allows both
SELECT
-first and
FROM
-first syntax, but it does not allow the clauses to appear in an arbitrary order.
By comparison, the following query is syntactically valid, so it passes the parser and transformer. However, it fails later in the binder because the table
missing_table
does not exist.
Catalog Error:
Table with name missing_table does not exist!
LINE 1: FROM missing_table;
^^^^^^^^^^^^^
Although a SQL standard exists, every database system supports different parts of the standard and adds its own syntax and behavior. The resulting variants are commonly referred to as SQL dialects. Examples include the dialects supported by
PostgreSQL
,
Oracle
,
GoogleSQL for BigQuery
,
MySQL
,
MariaDB
,
SQLite
,
Spark SQL
, and, of course,
DuckDB
.
DuckDB’s SQL closely follows PostgreSQL conventions, but it has evolved considerably over the years. We have added features of our own, such as
GROUP BY ALL
, as well as features inspired by other database systems. At the same time, DuckDB does not implement every aspect of PostgreSQL’s behavior. DuckDB therefore speaks its own SQL dialect, which we will refer to as
DuckSQL
in this post, even though it remains strongly influenced by PostgreSQL.
This distinction is important when talking about the parser. The SQL dialect that DuckDB accepts and the implementation used to parse that SQL are two separate things. For DuckDB v2.0, we are replacing the parser implementation and rewriting its grammar. What we are
not
replacing is DuckSQL itself.
When DuckDB started out, it made a lot of sense to use the PostgreSQL-derived parser and grammar. This parser was already part of the
first commit
to DuckDB in 2018. It gave DuckDB a mature, battle-tested SQL grammar based on syntax that many users were already familiar with. We adapted the parser to our needs and added a
Transformer
that converted the resulting PostgreSQL-style parse tree into DuckDB’s internal AST.
However, over the years this parser also came with some downsides. Extending DuckSQL meant modifying the underlying YACC/Bison grammar. Because Bison generates an LALR(1) parser, seemingly small additions to the grammar can interact with existing rules and introduce
shift/reduce
or
reduce/reduce
conflicts. As DuckSQL grew, making changes to the grammar therefore became increasingly difficult.
This was one of the motivations behind our earlier
blog post
on runtime-extensible SQL parsers. In that post and the accompanying
CIDR paper
, we explored whether Parsing Expression Grammars (PEGs) could provide a better foundation for an extensible database parser. At the time, the PEG parser was still an experimental prototype capable of parsing only a subset of SQL.
The
<-
operator defines a rule,
/
specifies a choice between alternatives, and
?
makes an element optional. Together, these rules state that DuckSQL accepts both a traditional
SELECT
-first query:
And DuckDB’s Friendly SQL
FROM
-first equivalent:
A PEG evaluates alternatives in order. When matching
SelectFrom
, the parser first attempts
SelectFromClause
. If that does not match, it attempts
FromSelectClause
. The first successful alternative is selected. As a result, PEG grammars do not have the same
shift/reduce
and
reduce/reduce
conflicts as LALR grammars. Instead, alternatives are ordered explicitly, and that order forms part of the grammar’s behavior.
We are not the only ones changing to a PEG-based parser. Python
switched
from its LL(1) parser to a PEG-based parser in Python 3.9, also motivated by the additional flexibility PEG provides to evolve the language.
In DuckDB, these rules operate on the tokens produced by the tokenizer. The matcher applies the grammar rules to those tokens and constructs a generic
ParseResult
tree, which is subsequently transformed into DuckDB’s internal AST.
The research prototype demonstrated that a PEG-based SQL parser was feasible. Replacing DuckDB’s existing parser, however, required considerably more than parsing a subset of SQL. The new parser had to accept all of DuckSQL and produce the same AST expected by DuckDB’s binder.
The PEG grammar was first introduced in DuckDB
v1.2
, where it handled autocomplete in the CLI. Later, in DuckDB
v1.5
, we introduced the complete PEG parser as an experimental, opt-in feature. We also used it for an April Fools' joke that made
DuckDB speak Dutch
. Since then, the grammar, matcher, and transformer have been steadily improved to make the PEG parser the default for DuckDB v2.0.
Among other things, the parser had to support:
Every statement and expression type:
Supporting the complete DuckSQL dialect includes both common syntax as well as the less frequently used statements and expressions.
Operator precedence and associativity:
For example,
SELECT true OR true AND false;
must be interpreted as
(true OR (true AND false))
, because
AND
binds more tightly than
OR
.
Correct keyword classification:
Some keywords, such as
SELECT
, are
RESERVED
and cannot be used as unquoted table or column names. Other keywords may be used as identifiers depending on their context.
Compatibility with DuckDB’s internal AST:
The PEG transformer must produce the same DuckDB AST structures as the transformer for the PostgreSQL-derived parse nodes wherever the language behavior is intended to remain unchanged.
Correct error reporting:
For an invalid query, the parser should report where parsing failed and, where possible, provide context and a useful indication of what went wrong. Ideally, it should do so without pointing to a
manual
.
Performance on unusual inputs:
Besides keeping normal parsing fast, we also had to make sure that malformed queries do not suddenly take a long time to parse.
One issue we encountered was repeated work during backtracking. A naïve PEG matcher can evaluate the same grammar rule at the same token position many times while trying different alternatives. For certain malformed inputs, the amount of repeated work can grow exponentially.
We encountered this with queries containing a large number of unmatched opening parentheses:
SELECT((((((((((((((((((;
With the experimental PEG parser shipped in
v1.5
, adding one more opening parenthesis approximately doubled the parsing time:
We addressed this using
packrat parsing
, a memoization technique commonly used with PEG parsers. For each memoized matcher, we store the result of applying it at a particular token position. If the parser later attempts the same matcher at the same position, it reuses the cached result instead of evaluating it again.
With packrat parsing enabled, the same malformed query was rejected almost instantly:
19 opening parentheses: 0.001 seconds
As a result, a memoized matcher is evaluated at most once at a particular token position, removing the repeated work that caused the exponential behavior in this example. This requires additional memory while parsing, but that is a worthwhile trade-off for avoiding cases such as this.
Turning the prototype into a production parser involved much more than translating the grammar. The new parser had to cover the complete DuckSQL dialect, preserve DuckDB’s existing AST, remain compatible with existing queries, and handle both valid and malformed input efficiently.
The resulting architecture replaces the PostgreSQL-derived parser front end, while the binder and the remainder of DuckDB’s query-processing pipeline continue to operate on the same internal AST.
DuckDB Parser architecture. Key idea: replace the PostgreSQL-derived parse front end while keeping the rest of DuckDB's execution pipeline the same.
With an expression statement, the
SELECT
can be omitted:
date:current_date(),time:current_localtime();
As a bonus, this also works with prefix aliases.
Another example is the new
CONNECT
statement, introduced for
Quack
. It allows you to connect to a remote database and route subsequent queries to it until you run
DISCONNECT
:
CONNECT'postgres://localhost/mydb';SELECTcount(*)FROMorders;-- Runs on the PostgreSQL serverDISCONNECT;
There will also be new syntax for working with
external resources
. This will allow you to manage resources that live outside DuckDB through an extension. You will be able to create, register, inspect, connect to, or destroy a resource all from within DuckDB:
These additions would also have been possible with the old PostgreSQL-derived parser, but adding them would have been considerably more cumbersome. The PEG grammar makes it easier for us to continue evolving DuckSQL.
So far, these rules are all part of DuckSQL itself. The next step is allowing extensions to add rules of their own.
Extensions are a central part of DuckDB. They can already add scalar and table functions, optimizer rules, query-plan rewrites, and even custom physical operators.
Extensions that add new syntax already exist, such as
psql
and
duckpgq
, but under the hood they work as fallback parsers. DuckDB first tries to parse the query itself and only calls the extension if that fails. This works well for self-contained syntax, but an extension that wants to add syntax inside SQL also has to parse the surrounding SQL itself. These fallback parsers also make it impossible to combine the syntax of multiple extensions.
With the PEG parser, extensions can instead extend individual parts of DuckDB’s parser. They can extend the tokenizer, add grammar rules, and register custom matchers while continuing to reuse the rest of DuckSQL.
Warning The API shown below is still a preview and may change before
DuckDB v2.0
. You can follow the ongoing development
on GitHub
.
To make this concrete, we use Google’s
pipe query syntax
. This is an extension to SQL that adds piped data flow syntax. Pipe syntax expresses a query as a sequence of operators, where each operator consumes the result of the previous one.
Here,
+
means that
PipeStage
must occur one or more times, so a pipe query must contain at least one pipe operator.
This grammar can reuse existing rules, such as
GroupByClause
, to reduce the amount of grammar the extension needs to define. An extension can still define its own rule where DuckDB’s existing syntax does not fit.
Defining just the PEG rules does not yet make them part of DuckDB’s grammar. The extension must also specify (1) the existing grammar rule it wants to extend and (2) the transformer rules that convert the new syntax into DuckDB’s AST.
In the current prototype, certain grammar rules expose extension points. Pipe SQL registers
PipeSelectAtom
as an additional alternative for
SelectAtom
, together with the new keywords
AGGREGATE
and
EXTEND
.
The extension alternative is now tried first. If no pipe syntax is present, it fails without consuming any tokens and the query is parsed with the built-in alternatives.
Adding a grammar rule only gets us as far as a
ParseResult
. The extension still needs to transform that result into the DuckDB AST that is expected by the binder. Since
PipeSelectAtom
extends
SelectAtom
, its transformer returns a
SelectStatement
:
The shape of the
ParseResult
follows the grammar rule we defined earlier.
PipeSelectAtom
contains a
PipeSource
and one or more
PipeStage
s. We first transform the
PipeSource
into a DuckDB
SelectStatement
. Each
PipeStage
is then applied to that statement in order. The resulting
SelectStatement
is then returned and can continue through the rest of the parser's pipeline and eventually on to the binder.
This is where reusing DuckDB's existing grammar becomes especially useful. The extension only needs to transform the new syntax it introduced. When it reuses an existing DuckDB grammar rule, such as
GroupByClause
, it can also reuse the corresponding transform function instead of having to implement
GROUP BY
itself.
This is an important difference from the fallback parsers that are available today. An extension no longer needs to implement expressions, table references,
GROUP BY
clauses, and the rest of SQL itself. Instead, it can add only the syntax it needs and reuse DuckDB’s grammar and transformations for everything else.
With the extension registered, we can now execute queries using the new pipe syntax. For example, we can combine the pipe operators added by the extension with existing DuckSQL features such as
range()
and prefix aliases:
The extension only defines the pipe-specific syntax. Expressions, table references,
WHERE
,
SELECT
,
ORDER BY
, and other reused rules are still parsed and transformed by DuckDB itself. This means that new syntax can be combined with DuckSQL without the extension having to implement the rest of SQL again.
With DuckDB v2.0, we are replacing the PostgreSQL-derived parser with a new PEG parser. Existing DuckSQL queries should continue working as before. Under the hood, however, the new parser gives us something that is easier to evolve and designed for runtime extensibility.
The runtime grammar extension API shown in this post is still a preview and may change before v2.0 is released. However, the underlying idea is already working. Extensions can add their own syntax directly to DuckDB's grammar while reusing its existing rules and transformations. This means they no longer need to parse the rest of SQL themselves.
We are excited to see what new syntax the community will create. In the meantime, we will continue evolving DuckSQL and improving the parser.
If you do find an existing query that behaves differently with the PEG parser, please let us know by
filing an issue
.
How Trump Admin Weaponized "Antisemitism" Probes to Dismantle Higher Education "Brick by Brick"
Democracy Now!
www.democracynow.org
2026-08-21 08:49:59
Haley Van Erem, a former career attorney in the Justice Department’s Civil Rights Division, has filed a complaint claiming the Trump administration task force charged with investigating antisemitism pushed universities into massive settlements despite turning up little to no evidence of anti-J...
Haley Van Erem, a former career attorney in the Justice Department’s Civil Rights Division, has filed a complaint claiming the Trump administration task force charged with investigating antisemitism pushed universities into massive settlements despite turning up little to no evidence of anti-Jewish discrimination on campus. Van Erem said in her complaint that the government’s probes into schools like Harvard, Brown and Columbia were “an unlawful process designed to achieve predetermined political goals.”
“Columbia affiliates, from the Board of Trustees down, have actually collaborated with the federal government in these sham investigations,” says Marianna Hirsch, professor emerita at Columbia University. “These were not probes into antisemitism accusations, but they were efforts to dismantle higher education brick by brick.” Columbia and Brown settled with the administration for $200 million and $50 million, respectively. Harvard refused to settle, and a judge threw out the case against the university.
Please check back later for full transcript.
The original content of this program is licensed under a
Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 United States License
. Please attribute legal copies of this work to democracynow.org. Some of the work(s) that this program incorporates, however, may be separately licensed. For further information or additional permissions, contact us.
(“On October 13, 1825, nothing happened in this house, nor was anyone important born.”)
Meet Loui Ridi: Palestinian American Returns to West Bank Home Besieged by Israeli Settlers
Democracy Now!
www.democracynow.org
2026-08-21 08:36:33
Loui Ridi, a Palestinian American man who lives in Ohio, traveled to the occupied West Bank on Monday to help relatives defend their family home in the village of Qusra, south of Nablus. Israeli settlers have surrounded the home, which Ridi owns, for more than a week. Settlers have besieged several ...
Loui Ridi, a Palestinian American man who lives in Ohio, traveled to the occupied West Bank on Monday to help relatives defend their family home in the village of Qusra, south of Nablus. Israeli settlers have surrounded the home, which Ridi owns, for more than a week. Settlers have besieged several Palestinian houses in the village, trapping people inside and cutting off water and electricity in some cases.
“I’m not getting no protection here. I still fear for my life. I still fear for my family’s life here,” says Ridi, who joins
Democracy Now!
from the occupied West Bank. Ridi has raised an American flag on his home and reached out to the U.S. Embassy for help. “The
IDF
is not even making it better here. They’re not allowing us to leave the house. They’re not allowing anyone to come to my house freely,” says Ridi, explaining that getting food to the house takes “hours, if not days.”
The settlers “can act like this because of the active support from the Israeli government that gives them weapons, gives them ATVs,” says Israeli reporter Oren Ziv, who has traveled to Qusra twice in recent days. Ziv says the Israeli government also supports “the establishment of more and more [settler] outposts that are kind of front bases for these terror attacks.”
The original content of this program is licensed under a
Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 United States License
. Please attribute legal copies of this work to democracynow.org. Some of the work(s) that this program incorporates, however, may be separately licensed. For further information or additional permissions, contact us.
The biggest update to atproto since it first launched is available as an alpha that you can develop on, starting right now!
This project has been a long time coming, as evidenced by the many names it’s had (first
private data
, then
permissioned data
, briefly
buckets
, and now
atproto spaces
). From
early chatter on the forum
, to
the first development diary
back in February, to the
full proposal
, the design of the protocol has evolved through the feedback, contributions, and discussion of the ecosystem. This is a big undertaking, not just for the Blueksy team but for the entire Atmosphere.
There are, of course, features and entire products that rely on data that isn’t public. Settings, private bookmarks, forums ranging from dozens to millions of members, and subscription-only publishing apps all require a data model that isn’t fully public.
Spaces
, a new protocol primitive, provide a way to store and sync non-public data while retaining the advantages of atproto like portable identity, interoperable/remixable data, and permissionless participation.
Today, we’re making the alpha available with running code, published SDKs, a sample app, and even a hosted PDS you can create an account on and develop against. This is truly an alpha. There will be breaking changes, and you absolutely should not run production code against it.
You can think of an atproto space as a miniature atproto network that can be gated so that only certain people and applications are able to access the data published in it. It may sound a little “heavy-duty” to say each space is a mini-atproto, but spaces are actually very lightweight and low overhead. A space can have a single record in it with minimal overhead or scale up to a billion records.
Access to a space is controlled by a space authority, which is just a DID like any other account (and in some cases actually
is
your account!). The space authority determines which other DIDs are allowed to access the space. Records live in per-space permissioned repos on the author’s PDS.
It’s important to remember that spaces give you
access control
not
confidentiality. The data in a space is readable by any user or application with access to that space, it’s not encrypted.
Spaces are a very flexible primitive, and the range of uses is deliberately broad. The smallest spaces will contain exactly one member and are useful for storing data like settings, drafts, bookmarks and other private data that an app might want to store. Spaces work for gated content as well, such as a publisher that wants to distribute a subscription-only publication. Where spaces really shine, and in some sense what they were designed for, is establishing a shared social context. In this capacity, the largest spaces will be communities that may grow to millions of participants.
The sync protocol for space data is significantly lighter-weight and provides facilities for real-time sync. This is because, unlike the public broadcast protocol, there is no concept of a relay for data stored in a space. For public data, the relay helps provide applications access to all of the data across the network. However for spaces, it’s often not desirable to rebroadcast content. Applications will sync space data directly from PDS hosts.
If you want to test out the protocol without running any infrastructure, you’re in luck! We’re hosting one for you and will keep it up to date with the latest changes.
Head over to your
BPS account
for an invite code and a link to the alpha PDS.
This is a shared sandbox and we intend to keep it usable. If you cause moderation problems, engage in unproductive abuse of the network, or otherwise try to use the PDS for purposes other than experimenting with spaces, you will be permanently banned from the alpha.
You should also expect the data stored in the PDS is neither permanent nor stable. The data model will change, we may even delete everything without warning. The PDS as a whole will be deleted after the alpha.
We plan to update the hosted PDS and SDKs on Thursdays. We’ll post changes to the announcements thread on atmosphere.community, please subscribe.
If you want to run your own PDS, we’ll maintain a tagged Docker image at
ghcr.io/bluesky-social/atproto:pds-spaces-alpha
with support for spaces. This image is compatible with the
reference PDS distribution
and does not require any new configuration.
THIS IS ALPHA SOFTWARE DO NOT USE IT IN PRODUCTION
. Breaking changes will happen and database schemas may change without clean migrations. We strongly recommend that you do not migrate your real accounts to this version. Do not expect that you’ll even be able to cleanly upgrade between versions.
That said,
do
please use the new PDS with test data. Explore spaces and the kinds of applications you can build with them. Report bugs, let us know if you were expecting something to work one way and it turns out to work differently.
We’ve already seen a few ecosystem projects that have begun to implement the proposed spec:
ZDS
, a PDS written in the Zig programming language
Real protocols have many interoperating implementations, and it’s been amazing to see the ecosystem lead the way on this.
There’s an example app running at
https://bulletin.my
. This app lets you host a bulletin board (as a space!) that your mutuals can leave sticky notes on. Only your followers can see your board. The code is available
https://github.com/bluesky-social/bulletin
. Give it a run locally, or fork it and remix it into something new! If you have your own PDS implementation, try logging in and seeing if everything works as expected.
To support this, we released the TypeScript
@atproto
packages as alpha snapshot versions. These can be installed with the
alpha
tag. Check out the bulletin repo to see them in action.
If you want to dive deeper into the protocol, the latest version of the protocol specification can be found in the
proposals repo
. If you’re working on your own implementation of atproto spaces, this is the thing to collaborate around. We’ll keep the proposal up to date with the current reference implementation. If you find ambiguities or places where the implementation and proposal diverge, please open an issue.
The reference implementation can be found on the
atproto spaces
branch of the atproto repo. This branch is being actively developed and may temporarily diverge from the packages and PDS that are published.
As has already been mentioned several times, expect changes as we continue to develop the code. Specifically, this means:
The code has not undergone careful security review.
Do not upload sensitive information. Not your own, and especially not anyone else’s.
We are not running backups, and we may do destructive data migrations.
Do not upload content you are not willing to lose. There is no recovery path and we will not be able to make one for you.
The alpha PDS goes away at the end of the alpha.
Accounts on it are not accounts you should encourage anyone to depend on. Do not point non-developer users at it.
The protocol design, SDKs, and database schema are not final.
Anything you build will likely need revising.
We will continue to iterate and build tooling throughout the fall, with a goal of launching later this year. Follow
the announcement post in the Atmosphere Community forum
for updates on new builds, which we plan to drop on Thursdays.
Feel free to start building apps with test, non-production data against the hosted PDS. Or host your own PDS—either the reference implementation or one of the community-managed ones. Run the sample app or build your own.
Report issues
where you find them.
We are excited about the entire new class of applications atproto spaces enables and for developers to get their hands on the code.
CISA orders feds to patch actively exploited TrueConf Server flaws
Bleeping Computer
www.bleepingcomputer.com
2026-08-21 08:25:33
The U.S. Cybersecurity and Infrastructure Security Agency (CISA) ordered U.S. federal agencies to prioritize patching two actively exploited vulnerabilities in the TrueConf Server self-hosted communications platform. [...]...
The U.S. Cybersecurity and Infrastructure Security Agency (CISA) ordered U.S. federal agencies to prioritize patching two actively exploited vulnerabilities in the TrueConf Server self-hosted communications platform.
TrueConf Server is designed for secure corporate messaging and video conferencing and, unlike cloud-based software like Zoom or Microsoft Teams, it operates inside an organization's local network (LAN).
The most severe is a critical missing authentication security flaw (tracked as
CVE-2026-72529
) that allows attackers without privileges to remotely execute arbitrary scripts on unpatched servers.
"A remote unauthenticated attacker connecting to TrueConf Server over 4307/TCP can invoke an undocumented critical function and execute an arbitrary script on the server," the TrueConf security team
explains
.
The second is another critical severity vulnerability (
CVE-2026-72530
) that unauthenticated threat actors can exploit through high-complexity code injection attacks to gain remote code execution.
"Improper management of code generation can allow an attacker who has achieved code execution in the TrueConf Server isolated environment to escape the sandbox and execute arbitrary commands on the underlying operating system," TrueConf
adds
.
On Thursday, CISA
added the two flaws
to its
KEV catalog
and ordered U.S. Federal Civilian Executive Branch (FCEB) agencies to secure their servers within two weeks, by September 3.
"This type of vulnerability is a frequent attack vector for malicious cyber actors and poses significant risks to the federal enterprise," the cybersecurity agency
warned
.
While CISA didn't share details on these attacks, cybersecurity company Kaspersky said the Head Mare hacktivist group
has been exploiting CVE-2026-72529 and CVE-2026-72530
since at least July 2026 to replace client installers with malicious versions designed to deploy backdoor malware.
According to Kaspersky, multiple Head Mare campaigns targeted Russian organizations across various industry sectors, including transportation, energy, IT, electronics, and software development.
In April 2026,
Check Point Research also reported
that hackers were targeting another TrueConf flaw (CVE-2026-3502) in zero-day attacks dubbed "Operation True Chaos" and linked to Chinese threat actors, compromising users via trojanized client updates.
As International Pressure Grows, Israel Finally Opens Probe into 2024 Killing of Hind Rajab in Gaza
Democracy Now!
www.democracynow.org
2026-08-21 08:15:11
Israel’s military on Wednesday admitted that its soldiers opened fire in January 2024 on a car trying to flee Gaza City following Israel’s evacuation orders. The car was carrying 5-year-old Palestinian Hind Rajab and six of her relatives. They were all killed, as were Palestine Red Cresc...
This is a rush transcript. Copy may not be in its final form.
ANJALI
KAMAT
:
This is
Democracy Now!
, democracynow.org. I’m Anjali Kamat.
On Wednesday, Israel’s military admitted for the first time that its soldiers opened fire on a car carrying 5-year-old Palestinian Hind Rajab along with her aunt, uncle and five of their children, as well as Palestine Red Crescent Society medics dispatched to rescue them. Israel’s army fired more than 300 bullets in the attack in January 2024. In Gaza City, Hind Rajab’s grandmother responded to the announcement.
HIND
RAJAB
:
[translated] We, as Hind Rajab’s family, do not trust the judiciary of the state of Israel. We hope that justice will be delivered for the entire world. We appeal to the international justice system as a whole.
ANJALI
KAMAT
:
The Israeli military said it would also investigate the killing of 15 Palestinian paramedics whose bodies and crushed emergency vehicles were recovered from a mass grave in Rafah in March 2025. This is Ghada al-Attar, the widow of Anwar al-Attar, one of the 15 medics. She spoke while holding the couple’s young daughter.
GHADA
AL-
ATTAR
:
[translated] The truth will not come, and the wound will keep bleeding. The pain is still the same. The loss grows day by day. Every time this girl gets older and asks me where her father is, and I can’t answer her, the wound grows deeper. He was the pillar and support of the family. He’s gone. The investigation won’t achieve anything.
ANJALI
KAMAT
:
Israel said it would not investigate three other attacks on Gaza that killed eight workers from World Central Kitchen and Doctors Without Borders, and Israel made no mention of thousands of other incidents where Palestinian civilians were killed by Israeli forces.
For more, we go to Los Angeles, where we’re joined by Sharif Abdel Kouddous, award-winning journalist and the Middle East/North Africa editor at
Drop Site News
. He was correspondent on the
Fault Lines
documentary
The Night Won’t End
on Al Jazeera English that investigated the killings of civilians in Gaza by the Israeli military.
Sharif, welcome back to
Democracy Now!
Can you talk about why these investigations have been announced, why these two particular cases, and why now?
SHARIF
ABDEL
KOUDDOUS
:
Well, so, firstly, these were five incidents out of apparently 150 that were reviewed by the Israeli military. I think it’s not a coincidence that all five of these were among the most widely covered incidents of the war. They were documented in real time, and they were very prominent cases, mostly massacres. We don’t know what happened to the other 145 incidents that they apparently reviewed.
Now, the two incidents that you mentioned that are being referred to military police for criminal investigations, you know, this seems to be nothing more than political theater, to be honest. You know, as numerous Palestinian and international human rights groups have documented for years, Israel’s internal investigations into the criminal conduct of its own soldiers are fundamentally flawed, and they can’t be regarded as credible mechanisms for accountability. And they instead function primarily to shield Israel from international tribunals or prosecution abroad, rather than delivering any kind of justice. The Israeli human rights group B’Tselem has called these investigations a sham. It’s called them, quote, “a legal Iron Dome” to protect Israeli soldiers from international accountability.
And let’s remember that since the genocide began, Israel claims to have referred over a thousand incidents involving its soldiers to its fact-finding assessment mechanism, and, you know, it says it’s opened dozens of criminal investigations. There’s no comprehensive public accounting of the status or outcome of these investigations. There was a review by the
NGO
Action on Armed Violence of over 50 reported Israeli military probes. It found that 88% had been either closed or that a finding of wrongdoing — had been closed without a finding of wrongdoing, and they remained under review. Only one of them resulted in a prison sentence. So, you know, I think this is a lot more political theater, and it may be coming as Israel is coming under increasing international scrutiny, international criticism for the genocide in Gaza.
And finally, I’ll just say, it doesn’t matter what Israel says or what its investigation concludes in these two incidents. The responsibility for the murder of Hind Rajab and her family was well established long ago through multiple in-depth investigations by the media, by human rights groups, and the evidence is overwhelming. We know that an Israeli tank fired at least 335 rounds at a civilian car, at Hind Rajab, a 5-year-old, and six members of her family, from a range of just 13 to 23 meters away.
The Red Crescent, which had Hind on the phone for close to three hours, was waiting for approval to try and go rescue her. And finally, they get approval from the Israeli military. The Israeli military issues an approved route with a map. Two emergency workers with the Red Crescent go in an ambulance along that route. When they reach the destination, Israel fires a tank shell at them. It’s a direct hit and kills both of them and destroys the ambulance.
Then the Israeli military continuously lied afterwards. It repeatedly denied it had any tanks operating in the area, even though there’s clear satellite images that show multiple tanks in the area on that day, even though we hear a recording of Hind’s cousin Layan, 15-year-old Layan, saying there’s a tank right next to her, right before she starts screaming as a hail of bullets from the tank murders her. And then we have 5-year-old Hind, hours of recordings of her, saying also that there’s a tank right next to her, before her voice eventually fades away and she dies. So, we don’t need to wait for an Israeli investigation to know what happened here.
ANJALI
KAMAT
:
Sharif, you’ve set that up so movingly. I want to turn to an excerpt of your Al Jazeera English documentary,
The Night Won’t End
, which takes an in-depth look at these attacks on civilians by the Israeli military in Gaza, including on Hind Rajab and her family. Let’s go to a clip.
SHARIF
ABDEL
KOUDDOUS
:
Eventually, relatives were able to reach the Red Crescent in Ramallah to see if their team in Gaza could send an ambulance.
OMAR
AL-
QAM
:
[translated] We received an appeal that the car had been targeted at Fares gas station in Gaza City.
SHARIF
ABDEL
KOUDDOUS
:
When Omar called, it was Layan who picked up.
OMAR
AL-
QAM
:
[translated] Hello, dear.
LAYAN
HAMADEH
:
[translated] They are shooting at us.
OMAR
AL-
QAM
:
[translated] Hello.
LAYAN
HAMADEH
:
[translated] They are shooting at us. The tank is next to me.
OMAR
AL-
QAM
:
[translated] Are you hiding?
LAYAN
HAMADEH
:
[translated] Yes, in the car. The tank is next to us.
OMAR
AL-
QAM
:
[translated] Are you inside the car?
LAYAN
HAMADEH
:
[screaming]
OMAR
AL-
QAM
:
[translated] Hello? Hello?
A girl dies while she’s on the phone with you. I disassociated. I reached a stage where I was just mentally cut off. I’m trained for situations like this, but when it involves a child, your emotions get all mixed up. After calling back the same number that Layan answered, the voice sounded different this time. So I asked her, “The girl that was speaking with me, where is she?” She told me, “She’s dead.” Who am I speaking with now? Hind.
HIND
RAJAB
:
[translated] Hurry!
OMAR
AL-
QAM
:
[translated] Hide. Hide. Where are you exactly, in the car?
HIND
RAJAB
:
Huh?
OMAR
AL-
QAM
:
[translated] Are you in the car?
HIND
RAJAB
:
[translated] Yes.
OMAR
AL-
QAM
:
[translated] Hide under the seats. So you can’t be seen at all.
HIND
RAJAB
:
[translated] OK.
SHARIF
ABDEL
KOUDDOUS
:
After a few minutes, Omar asked other colleagues to help and speak with Hind.
HIND
RAJAB
:
[translated] The tank is next to me.
RED
CRESCENT
DISPATCHER
:
[translated] The tank is where?
HIND
RAJAB
:
[translated] Next to me.
RED
CRESCENT
DISPATCHER
:
[translated] The tank is next to you?
HIND
RAJAB
:
[translated] Yes.
RED
CRESCENT
DISPATCHER
:
[translated] Is it moving or still? Did anyone come out of it?
HIND
RAJAB
:
[translated] It’s moving.
RED
CRESCENT
DISPATCHER
:
[translated] It’s moving?
HIND
RAJAB
:
Mmm.
RED
CRESCENT
DISPATCHER
:
[translated] OK. Is it moving next to the car, behind the car or in front of the car?
HIND
RAJAB
:
[translated] In front of the car.
RED
CRESCENT
DISPATCHER
:
[translated] The tank is coming toward you from the front of the car?
HIND
RAJAB
:
[translated] Yes.
RED
CRESCENT
DISPATCHER
:
[translated] Is it very close?
HIND
RAJAB
:
[translated] Very, very.
RED
CRESCENT
DISPATCHER
:
[translated] And it’s moving?
HIND
RAJAB
:
[translated] Yes. Come get me.
SHARIF
ABDEL
KOUDDOUS
:
As the Red Crescent dispatcher spoke with Hind, colleagues were trying to get an ambulance to her, something that would require coordination with and approval by Israeli authorities.
HIND
RAJAB
:
[translated] Ask anyone to come get me.
RED
CRESCENT
DISPATCHER
:
[translated] My love, believe me, God willing, the coordination will happen.
NEBAL
FARSAKH
:
Usually, ambulances in the whole world, once they get the call, they directly dispatch the ambulances and send to save people’s life. Unfortunately, this is not the case in Gaza.
HIND
RAJAB
:
[translated] Come get me.
NEBAL
FARSAKH
:
Any area that there is Israeli occupation forces, Israel considered as it is a military zone. That means even if there is wounded people, people who are killed and need to be evacuated, we are completely denied access to these areas. And if any ambulance try to reach, it will be targeted. That’s why in order to be able to save Hind, we had to coordinate our safe access.
ANJALI
KAMAT
:
That last voice was Nebal Farsakh, the spokesperson for the Palestinian Red Crescent, explaining how emergency medical teams in Gaza need to get clearance from Israeli authorities before going in to rescue Hind. They eventually do get clearance. Let’s turn back to the documentary
The Night Won’t End
to see what happened next.
SHARIF
ABDEL
KOUDDOUS
:
Finally, nearly three hours after requesting clearance, the Red Crescent says Israeli officials gave the approval for the ambulance to go to the scene, and provided this map with an approved route.
NEBAL
FARSAKH
:
We had to wait almost three hours until the green light was given. So, they sent a map with a route, which means it identified exactly which route the ambulance should take. And once we received the green light, the ambulance was dispatched.
RED
CRESCENT
DISPATCHER
:
[translated] Oh, the best news in the world from Uncle Omar.
OMAR
AL-
QAM
:
[translated] Hind! Hanoud! In one minute the car will reach you. It’s just moving slowly. Yes, the Fares gas station. Where are you now?
PARAMEDIC
:
[translated] I’m coming up to the gas station.
SHARIF
ABDEL
KOUDDOUS
:
The two paramedics who started driving to the scene were Ahmed al-Madhoun and Yusuf Zeino. By the time they left, the sun had set.
OMAR
AL-
QAM
:
[translated] Can you see the car?
PARAMEDIC
:
[translated] I can’t see a thing here.
OMAR
AL-
QAM
:
[translated] Do you have your siren and flashing lights on?
PARAMEDIC
:
[translated] Just the lights, not the siren. Oh, there it is!
SHARIF
ABDEL
KOUDDOUS
:
The connection to the ambulance was lost right after that loud noise.
RED
CRESCENT
DISPATCHER
:
[translated] Hello, Hanoud? Hanoud? Are you OK?
HIND
RAJAB
:
[translated] Yes.
RED
CRESCENT
DISPATCHER
:
[translated] Thank God. Thank God. She’s OK. Did they go down to her?
NISREEN
QAWAS
:
They had to ask her, “Did you hear a bomb now? Did you hear anything around you?” And she said, “Yes, yes, I heard it.” Her “yes” means our colleagues who went to rescue her had died.
ANJALI
KAMAT
:
That clip from the award-winning documentary
The Night Won’t End
. Sharif, as you said, so much about this case is very well known. Hind Rajab is one of the most visible victims of the genocide in Gaza. There’s even a feature — there’s even a film made about her,
The Voice of Hind Rajab
. Why now? What is so significant about — what is new about this new investigation? And how much does it have to do with the International Criminal Court’s investigation?
SHARIF
ABDEL
KOUDDOUS
:
I mean, it’s unclear why now. You know, I think it’s because Israel is coming under more international scrutiny. There’s more criticism of the genocide. There is more political pushback from its backers in the United States and Europe, or, you know, from some corridors of power within those countries. And also, when a state says that it’s investigating itself, this shields it from international law accountability, if those investigations are found to be credible. So, it may just be a push around that. And again, as you mentioned, these are — you know, the Red Crescent massacre, the aid worker massacre and Hind Rajab are among the two most high-profile incidents.
And I think we should talk about, you know, the massacre of the aid workers, the other case that they’re looking at. This was 15 Palestinian aid workers from the Red Crescent, from Civil Defense, that were massacred in Rafah in March of 2025, and their bodies were buried in a mass grave. The ambulances and the fire truck and the vehicles that they arrived in to the scene were flattened and buried alongside them. And in the aftermath of this, the Israeli military again lied about it and was forced to change its story several times following the discovery of the bodies and the vehicles in this mass grave and the emergence of video and audio recordings taken from the bodies of the dead aid workers from their phones.
The group Forensic Architecture and Earshot did an incredible
report
around this just a few months ago using video and audio recordings about the incident and open-source material, as well as satellite imagery and interviews with two of the survivors, and they reconstructed what happened. And essentially, what happened was, an ambulance went to a scene, they were fired on, and then a five-vehicle convoy of ambulances and a fire truck and the rest of the aid workers went to go and try and find them and rescue them. And Israeli soldiers, over the course of a couple of hours, fired nearly a thousand bullets, a thousand bullets at these aid workers. And the findings show that they were slowly approaching the vehicles on foot, the Israeli soldiers, walking while they were shooting, until they were as close as one meter away, and essentially executing these aid workers in cold blood from very — from point-blank range, essentially.
Then, as I mentioned, the Israeli military lied about it. They claimed initially that the vehicles advanced, quote, “suspiciously” towards the troops without their headlights on or without their emergency signals on. Then the video emerges, first published by
The New York Times
, showing the vehicles with their lights on. So they backtrack. They admitted that their soldiers from the Golani Brigade did fire on these aid workers, but they said that they had approached suspiciously and that — you know, they blamed it on poor night vision or something, that they couldn’t really see them. And then they doubled down and said that six of the 15 aid workers were found to be Hamas terrorists, you know, just kind of ridiculous statements. And then it said, you know, it buried them in a mass grave to prevent harm and clear the vehicles to prepare for civilian evacuation. I mean, I don’t know how people are supposed to take this seriously.
And as you mentioned, again, these are just two of the most high-profile incidents out of countless war crimes in Gaza. We’re talking about over 20,000 children killed. We’re talking about starving Palestinians being gunned down as they’re searching for food, of an unprecedented number of journalists killed, over 270, many of them, you know, openly assassinated. We’re talking about displaced families being bombed in their tents, of hospitals, nearly every hospital, being attacked, of mass graves being found in the courtyards of the hospitals afterwards, of the torture of prisoners, of sexual assault. I mean, we could go on and on. So, I don’t think that these investigations that were announced are anything more than political theater, and I don’t think anyone’s really taking them seriously.
ANJALI
KAMAT
:
Sharif, last week was the one-year anniversary of the killing of Al Jazeera journalist Anas al-Sharif, along with five of his colleagues. And a collective of media workers called the Writers Against the War in Gaza have launched an
online memorial
to Anas and over 250 journalists who’ve been killed. It’s called “The Living Record” and includes testimonies from Palestinian journalists in Gaza remembering their slain colleagues and friends. As we talk about accountability and the possibility of it within Israel for Israel’s crimes in Gaza, your final thoughts on, you know, what is going on in terms of investigating the deaths of journalists in Gaza?
SHARIF
ABDEL
KOUDDOUS
:
I mean, very little is going on. What we’re seeing, and as was the case of Anas al-Sharif, one of the most prominent journalists to be openly assassinated, is that Israel has — keeps killing them in actually a more brazen way. Israel has reached the point where it is preemptively targeting journalists. So, it put Anas al-Sharif, for example, who was murdered on August 10th of last year in a media tent along with five of his colleagues outside of Shifa Hospital — it put Anas al-Sharif on a hit list in October of 2024, along with five other journalists from Al Jazeera, including our colleague at
Drop Site News
, Hossam Shabat, and it said that these are not terror — “these are not journalists; these are terrorists, these are militants. And we’re going to kill them.” And it has killed two of them. And after it killed them, it bragged about it. It bragged about killing this prominent journalist and said that this wasn’t a journalist. It said, “Don’t let the press vest fool you.” It did the same with Hossam Shabat. It just killed, during a so-called ceasefire, two brothers working for Al Jazeera, at different times, one a correspondent, one the cameraman, killed them both, called them both Hamas terrorists. So, it is not — there’s not even a semblance of an investigation. There is open bragging about the killing of these journalists.
And let me just say that that tribute site that was put together by Writers Against the War in Gaza is very moving. I would encourage many people to go to it. You can click. I think there’s, you know, over 50 now journalists who are profiled there. And you can listen to their colleagues, their mentors, their family members talk about them and discuss who they were and talk about their importance and what they meant to them. And so, it’s a very fitting tribute to these journalists in Gaza, Palestinian journalists, who have performed the most heroic act of journalism in our lifetime and should be remembered for their bravery.
ANJALI
KAMAT
:
Sharif Abdel Kouddous is an award-winning journalist and the Middle East/North Africa editor at
Drop Site News
. Sharif, please stay with us. Coming up, we’ll look at how Israeli settlers are laying siege to the village of Qusra in the occupied West Bank.
[break]
ANJALI
KAMAT
:
“Tama,” “Greed,” by the Palestinian oud musician Huda Asfour, performing in our
Democracy Now!
studio.
The original content of this program is licensed under a
Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 United States License
. Please attribute legal copies of this work to democracynow.org. Some of the work(s) that this program incorporates, however, may be separately licensed. For further information or additional permissions, contact us.
Felony charges for citizen deleting phone data at US Border
Cassandra is a compelling data system. It is one of extremely few vendor-neutral, open-source databases supporting a SQL-like query language with
builtin sharding
and
builtin replication
. A desirable combination. And a reason Cassandra has
so many (large) users
including Apple, eBay, Bloomberg, and Netflix.
Cassandra has evolved significantly since its first release. From an eventually consistent data model without transactions and a schemaless, NoSQL interface over Thrift to where (in the upcoming 6.0 release) it stands as an ACID transactional SQL-like database (granted: severe SQL limitations, transactions are non-interactive, we’ll get to that later).
Meanwhile the lack of joins plus automatic sharding (and a limited secondary index story) means a key characteristic has stayed the same: you model tables based on queries. And as a result your application might end up denormalizing, turning a single write into multiple writes in order to position the database to efficiently answer different queries later on.
In this article we’ll set up a three-node Cassandra cluster on one machine, running the
cassandra-6.0
branch (a pre-release state) to test out some transactional workloads across four of Cassandra’s transactional options: none (the default),
BATCH
updates,
Lightweight transaction
(LWT) updates, and
Accord
(i.e. ACID) updates. Accord transactions will become available only when Cassandra 6 is released (perhaps later this year), which is why we are using the pre-release branch.
Set up directories and configuration for three nodes, giving them unique IP addresses and
JMX
ports.
foriin123;don=node$i# run `killall java` first and then this will clean up the data directories for clean re-runs.rm-rf/etc/cassandra/$n/var/log/cassandra/$n/var/lib/cassandra
mkdir-p/etc/cassandra/$n/var/log/cassandra/$ncp-r~/cassandra/conf/*/etc/cassandra/$n/
echo"cassandra_storagedir=\"/var/lib/cassandra/$n\"JVM_OPTS=\"\$JVM_OPTS -Dcassandra.jmx.local.port=7${i}99\"">>/etc/cassandra/$n/cassandra-env.sh
echo"cluster_name: 'theconsensus-lab'listen_address: 127.0.0.$irpc_address: 127.0.0.$iseed_provider: - class_name: org.apache.cassandra.locator.SimpleSeedProvider parameters: - seeds: "127.0.0.1:7000"accord: enabled: true">>/etc/cassandra/$n/cassandra.yaml
# Set up max memory usage.echo"-Xms4G-Xmx4G">>/etc/cassandra/$n/jvm-server.options
done
Now start up the three nodes one at a time. (
-R
allows us to run as root.)
Since this is a view of the cluster you’d get these same results querying the
nodetool status
on any node in the cluster (in this fault-less environment anyway).
Let’s say we have an accounts table that tracks balances after transfers. We’ll generate transfers and apply them to Cassandra using every method we have available (plain Cassandra, BATCH, LWT per row, conditional BATCH with LWT, and Accord). We’ll partition our accounts table by customer ID and order by account ID.
We will have only two accounts, with starting balances of 1,000 each. We’ll have two writers produce transfers between the two accounts concurrently. On top of the first axis (e.g. LWT vs Accord) we’ll have a second axis where one variant will do a read-modify-write to produce the transfers and one variant of the workload will do blind writes (no reads involved) to produce the transfers.
While the two writers are concurrently writing, we’ll have a third thread reading concurrently and asserting that the sum of balances between both accounts is 2,000.
When the concurrent writes complete and the workload ends we’ll assert that the sum of balances is still 2,000. For the read-modify-write workload variants we’ll also assert that the end balance of both accounts is a well-known number (because the workload’s intent is deterministic).
And we’ll have a third and final axis. One set of workloads will cross partition boundaries by transferring between accounts belonging to different customers. And the other set of workloads will not cross partition boundaries by only transferring between accounts belonging to the same customer.
The tables will also have two
op
columns for operations that are capable of doing conditional writes (LWT and Accord) to use as idempotency keys. It isn’t cheating that plain updates and non-LWT BATCH updates won’t use the idempotency columns because they
can’t
use the idempotency columns.
Monastery
allows us to script concurrent operations on a database by a fixed number of clients. Monastery will run the script against the database for us.
We start off by defining a setup section of the script.
Since the replication factor is 3 and there are only 3 nodes in the cluster, sharding will effectively not happen. But if we added more nodes to the cluster and kept the replication factor at 3, sharding would meaningfully happen.
Then we specify a concurrent section for our two writers and one reader. Each client will do an action repeatedly. The writers will send blind updates repeatedly transferring units between accounts. And the readers will repeatedly try to assert that the balance of the two accounts is constant.
--- concurrent
w1: repeat 400 as x {
UPDATE lab.accounts SET balance = {x} WHERE customer = 1 AND account_id = 1; -- assert ok
UPDATE lab.accounts SET balance = 2000 - {x} WHERE customer = 1 AND account_id = 2; -- assert ok
}
w2: repeat 400 as x {
UPDATE lab.accounts SET balance = 2000 - {x} WHERE customer = 1 AND account_id = 1; -- assert ok
UPDATE lab.accounts SET balance = {x} WHERE customer = 1 AND account_id = 2; -- assert ok
}
r1: repeat 1500 {
SELECT balance FROM lab.accounts WHERE customer = 1; -- assert sum(0) = 2000 or error
}
blind-plain-same.cql
The last section of the script is a final check stage where we can make any final queries and assertions.
There’s no balance for account 1 and 2 that we could assume this will come to since they’re both completely in competition. However, the overall invariant remains that no money should be gained or lost.
When we run this script with Monastery we will often see isolation violated (which is expected) in the concurrent section (i.e. the balances don’t sum to 2,000). We may or may not see the final assertion succeed, but if it succeeds it is because of luck not a guarantee.
So, in this run, the concurrent reader saw mismatched balances 939 of 1500 times. But the ending writes end up balanced again. (These are blind writes so this is more possible than read-modify-writes which would amplify inconsistency.)
Ok, so plain Cassandra doesn’t make for a great bank. At least not in this particular data model. But we have other options! Let’s see BATCH next.
Batches, completed in their current form by
Cassandra 1.2
(January 2013), let you combine a number of statements into one mutation per partition that is applied isolated and atomically. If the batch spans partitions, it also becomes a guarantee that the statements are eventually applied even in the face of node failures.
All statements in a batch share the same timestamp, where otherwise each statement has its own timestamp. Conflicts are decided by timestamp per cell, not per row. So when two conflicting batches carry different timestamps, the later batch wins every cell that both wrote, and no column ends up holding a value from a different batch than its neighbour. But timestamps are client-generated, and two clients can tie. Cassandra breaks a timestamp tie per cell by keeping the greater value, so two tied batches can each win some columns and lose others. We’ll see this happen shortly.
If we take our
blind-plain-same.cql
and wrap updates as a BATCH then we’ll actually end up somewhere consistent.
Look at the two balances. Both of them are large. 1,897 and 1,893 come to 3,790, well over 2,000. Landing on two large numbers takes the large half of one batch next to the large half of the other.
This is the timestamp caveat from earlier rather than a bug. With both writers updating two rows so frequently, their client-generated timestamps collide fairly often. On a tie Cassandra compares the values themselves and keeps the greater one, cell by cell. Account 1 resolves to the larger of
x
and
2000 - x
, and so does account 2. Both cells keep the big number.
We can even see this by hand. Set the same timestamp explicitly on two batches and have them disagree on both rows.
And we indeed see atomicity preserved (each batch's writes were eventually applied together) but not isolation (concurrent reads saw mismatched balances). The final check passed too, though after what we saw with tied timestamps that part is not quite guaranteed: if the last two batches tie, the durable end state can also mix. Again, this is what the docs
tell us
will happen.
But let’s go back to working with the same partition and look at another limitation of BATCH updates: read-modify-write workloads.
Let’s change up our workload slightly, keeping the schema the same. This time we’ll have two writers both incrementing units from one account and decrementing units from another. Since both writers are incrementing and decrementing accounts in the same direction, there is a logical ending balance for each account.
According to
the grammar
, we can’t even put SELECTs inside the BATCH. So the read stage is not even part of the “transaction”. So there’s basically no consistency we can provide for read-modify-write with BATCH alone. But let’s try it out and see.
Lightweight transactions (LWT) came out in
Cassandra 2.0
(September 2013) which gave us atomic compare-and-swap built on Paxos. We cannot atomically SELECT and then UPDATE, but we can at least atomically conditionally UPDATE.
Also, LWT timestamps are
derived from Paxos
and are unique per partition, so timestamp ties that we saw in the BATCH workloads are just not possible when using LWT.
One limitation of LWT is that while there is a way to know that a conditional update definitely failed, there’s no way if the LWT times out to know if it succeeded or not. So in the LWT workload we’ll make use of the
op
fields to store an idempotency token. Each writer gets its own
op
field. And each writer loops, retrying the LWT that inserts a unique
op
value, until it gets back the
op
value it sent in.
Additionally, while LWT goes through Paxos, reads by default do not. The client executes
CONSISTENCY SERIAL
to indicate it wants
SELECT
s to go through Paxos. These reads can fail on a timeout as well so we assert the reads sum to 2,000 or that the read errors.
Let’s rewrite
rmw-batch-same.cql
in terms of LWT.
CREATEKEYSPACEIFNOTEXISTSlabWITHreplication={'class':'NetworkTopologyStrategy','datacenter1':3};DROPTABLEIFEXISTSlab.accounts;CREATETABLElab.accounts(customerint,account_idint,balanceint,op1int,op2int,PRIMARYKEY(customer,account_id));INSERTINTOlab.accounts(customer,account_id,balance,op1,op2)VALUES(1,1,1000,0,0);INSERTINTOlab.accounts(customer,account_id,balance,op1,op2)VALUES(1,2,1000,0,0);--- concurrentw1:repeat200asop{retry{a,p=SELECTbalance,balance-1FROMlab.accountsWHEREcustomer=1ANDaccount_id=1;b,q=SELECTbalance,balance+1FROMlab.accountsWHEREcustomer=1ANDaccount_id=2;BEGINBATCH\UPDATElab.accountsSETbalance={p},op1={op}\WHEREcustomer=1ANDaccount_id=1IFbalance={a}ANDop1<{op};\UPDATElab.accountsSETbalance={q}WHEREcustomer=1ANDaccount_id=2IFbalance={b};\APPLYBATCH;-- assert ok or errorSELECTop1FROMlab.accountsWHEREcustomer=1ANDaccount_id=1;-- assert ({{op}})}}w2:repeat200asop{retry{a,p=SELECTbalance,balance-1FROMlab.accountsWHEREcustomer=1ANDaccount_id=1;b,q=SELECTbalance,balance+1FROMlab.accountsWHEREcustomer=1ANDaccount_id=2;BEGINBATCH\UPDATElab.accountsSETbalance={p},op2={op}\WHEREcustomer=1ANDaccount_id=1IFbalance={a}ANDop2<{op};\UPDATElab.accountsSETbalance={q}WHEREcustomer=1ANDaccount_id=2IFbalance={b};\APPLYBATCH;-- assert ok or errorSELECTop2FROMlab.accountsWHEREcustomer=1ANDaccount_id=1;-- assert ({{op}})}}r1:CONSISTENCYSERIAL;r1:repeat1000{SELECTbalanceFROMlab.accountsWHEREcustomer=1;-- assert sum(0) = 2000 or error}---check:SELECTbalanceFROMlab.accountsWHEREcustomer=1;-- assert sum(0) = 2000check:SELECTbalanceFROMlab.accountsWHEREcustomer=1ANDaccount_id=1;-- assert ({600})check:SELECTbalanceFROMlab.accountsWHEREcustomer=1ANDaccount_id=2;-- assert ({1400})
Very nice. And LWT can still run the old blind-write workload just fine too.
CREATEKEYSPACEIFNOTEXISTSlabWITHreplication={'class':'NetworkTopologyStrategy','datacenter1':3};DROPTABLEIFEXISTSlab.accounts;CREATETABLElab.accounts(customerint,account_idint,balanceint,PRIMARYKEY(customer,account_id));INSERTINTOlab.accounts(customer,account_id,balance)VALUES(1,1,1000);INSERTINTOlab.accounts(customer,account_id,balance)VALUES(1,2,1000);--- concurrentw1:repeat400asx{BEGINBATCH\UPDATElab.accountsSETbalance={x}WHEREcustomer=1ANDaccount_id=1IFEXISTS;\UPDATElab.accountsSETbalance=2000-{x}WHEREcustomer=1ANDaccount_id=2;\APPLYBATCH;-- assert ok or error}w2:repeat400asx{BEGINBATCH\UPDATElab.accountsSETbalance=2000-{x}WHEREcustomer=1ANDaccount_id=1IFEXISTS;\UPDATElab.accountsSETbalance={x}WHEREcustomer=1ANDaccount_id=2;\APPLYBATCH;-- assert ok or error}r1:CONSISTENCYSERIAL;r1:repeat1500{SELECTbalanceFROMlab.accountsWHEREcustomer=1;-- assert sum(0) = 2000 or error}---check:SELECTbalanceFROMlab.accountsWHEREcustomer=1;-- assert sum(0) = 2000
So we’ve got LWT which can get us consistent read-modify-write within a single partition, but it doesn’t work at all across partitions. And then we’ve got batches which are not isolated across partitions.
This is why the folks at Apple and University of Michigan created
Accord
.
Accord is the EPaxos-inspired leaderless consensus protocol that enables tables in Cassandra to be marked as
transactional_mode='full'
. All read and write operations on these tables go through the Accord consensus. And we finally get actual ACID transactions, albeit non-interactive ones.
In LWT, the value we read to use in the compare-and-swap would often be stale. The LWT would fail and we’d have to retry it. But a failed LWT doesn’t always mean the write didn't happen. For example, it might indicate that the client timed out while the actual write (eventually) succeeded. Writing, and guarding against, the
op
column allowed us to make sure we didn’t apply the same write twice (or more).
In Accord, the condition is evaluated at the same time as the read, so the read is not stale and the main reason a client would see a failure is due to a client timeout or a node failure. Both are unlikely in our happy localhost environment. The idempotency key would still be useful in a real system, but we’ll drop it in our lab environment.
While using Accord, I occasionally saw the concurrent reader report balances that didn’t sum to 2,000. This only ever happened in the same-partition workloads. And while in the RMW workload it seems slightly more possible it was an issue in the workload itself, the invalid sums happened in the simpler blind-write workload as well.
However, I have never seen an error in the end result. There might be an isolation bug in concurrent transactions even while the durable result is not wrong.
Even if this is a bug, it’s not particularly damning. Distributed systems have bugs. And Cassandra 6 is not even released yet.
This was my first exposure to Cassandra. I like it a lot. I like the builtin replication and builtin sharding. I like the novel consensus protocol and the strict serializability. It’s interesting to see how it has evolved over the years. And it will be interesting to see them continue to push toward being a more general-purpose database system. Interactive transactions would be cool.
And lastly, I’m looking forward to getting help from the ASF JIRA on if these are actual bugs or if they’re just mistakes in my own code.
DNS hijacking is silly. I already took over different
.gov
and
.edu
domains in the past, but I just immediately
reported that and moved on.
This one is a little different though, it's about how I took over phone-network infrastructure domains (
e164.arpa
) of
entire territories, and accidentally logged hundreds of thousands of phone calls to military bases. But let's start at
the beginning.
What is e164.arpa anyway?
ENUM (
e164.arpa
) was an idea from the early 2000s
1
: take a phone number, reverse the digits, put dots between
them, and add
.e164.arpa
at the end, so
+49 30 123456
becomes something like
6.5.4.3.2.1.0.3.9.4.e164.arpa
. You
can see that every German number will end up under
.9.4.e164.arpa
, which is the zone for all +49 numbers, and that
zone is controlled by DENIC (the same organization that runs
.de
). This means the DENIC decides which carrier or person
gets which number ranges under that zone, just like they hand out
.de
domains (which makes it decentralized, making every
country decide on delegation themselves).
The idea was that carriers could then look these domains up and get back a record saying "hey, this number can be
reached over SIP/VoIP under this address", skipping the expensive phone network and re-routing calls over the cheap
internet instead.
It never really took off though, and even back in its early days it saw barely any use. Over the years it just
deteriorated further, and today it's basically completely dead. I do actually own
5.8.7.1.7.1.3.2.6.1.9.4.e164.arpa
and
point it at this website, although technically I'm not supposed to do that (you can figure out my secondary number from
that!). Germany is actually one of the last countries that still technically allows registering an
e164.arpa
domain,
although I was the first person since 2019 to register one
2
.
The RFC says you should only set NAPTR records on these domains, which are the records that tell carriers where to route a call.
It states that you
absolutely shouldn't
be using .arpa domains as normal "domains" and host stuff like websites on them,
they are meant to be "infrastructure" domains (you might know
in-addr.arpa
for reverse DNS lookups for example).
But there's nobody who can actually stop you from doing it, it's still just DNS at the end of the day,
and nothing prevents you from slapping an A record on there and hosting a website. Some people actually really dislike that,
and try to get Certificate Authorities to no longer issue certificates for
.arpa
domains
3
.
Hijacking a territory's phone network
I was scanning
e164.arpa
to see if any of the delegated zones were hijackable, mostly out of curiosity about how
neglected this whole system really was.
I found three country-code zones,
0.9.2.e164.arpa
,
6.4.2.e164.arpa
, and
7.4.2.e164.arpa
, all delegated to the same
two nameservers:
ns6.icb.co.uk
and
ns.enum.org.uk
.
Quick explainer for anyone who isn't a DNS person: when a domain is delegated to a nameserver, it basically means "for
any question about this domain, go ask this server, it has the answers", and if I control the nameserver a domain points
to, I control every DNS response for that domain.
icb.co.uk
still exists as a domain, but the specific
ns6.icb.co.uk
subdomain no longer resolves to anything, meaning
any request falls back to the second listed nameserver instead:
ns.enum.org.uk
.
And that domain had expired, so I bought it for just 5€, and just like that I controlled the DNS for
0.9.2.e164.arpa
,
6.4.2.e164.arpa
, and
7.4.2.e164.arpa
. Reversed, those are phone codes +290, +246, and +247: Saint Helena, the
British Indian Ocean Territory (Diego Garcia), and Ascension Island respectively (funnily enough, those
territories also have the popular ccTLDs
.sh
,
.io
, and
.ac
).
To be clear about what this meant: when a carrier does an ENUM lookup for one of these numbers, they're essentially
asking "where do I route this call?", and I could answer with whatever I wanted. I could point it at my own SIP server,
accept the incoming call, and then place an outgoing call to the real destination with a spoofed number.
The person being called would see the original number ringing, and after picking up would speak to the person on the
other end as if everything was normal, but I'd be sitting silently in the middle of the entire conversation.
I would theoretically be able to do this for every single request that I got if I could re-route a number,
if
anyone was still actually using this system.
I reported it right away to everyone I could think of, through multiple channels into the British government, and got
nothing back. My best guess is that someone at the Internet Computer Bureau (who seemingly managed them in the past)
set these nameservers up over a decade ago. Then
e164.arpa
slowly died out, and whoever set it up either moved on or
just forgot about it, leaving nobody to renew a domain nobody remembered they depended on.
Checking if anyone actually uses this
Q Misell
(a researcher of the Max-Planck-Institute for Informatics) had heard about this and reported it to RIPE (who manages
e164.arpa
) on my behalf, but RIPE also declined to do anything, because
e164.arpa
delegations are governed by an ITU-T committee at the UN level. And RIPE wasn't willing to go against a
decision made by a UN committee, which would probably be a bureaucratic nightmare.
Q also asked if I had any data on how much traffic these zones actually got, which I didn't know.
And because I was very curious about that myself, I set up logging on
0.9.2.e164.arpa
(Saint Helena) to find out, and waited a full day.
Not a single query came in. So after trying my best to get anyone to care and getting nowhere, I just kept the domains,
since nobody seemed to be relying on them anyway.
I hosted my
personal site
on it,
spun up
a Fediverse instance
, a Matrix
homeserver, and handed out subdomains to friends, because why not, it's a dead system. It's not like it's gonna
hurt anyone, and no one cares. So it's time to be whimsical and have fun with it.
Six months later...
Just out of curiosity, I checked the logs again on all three zones,
since I enabled logging running on the other two as well when I set everything up.
Hundreds of thousands of ENUM queries, all logged
4
. Since the domain name is literally just the phone number reversed,
you can simply flip it back around to get the real number, so I had full phone numbers, timestamps, and the source IP
addresses of the DNS resolvers making the requests.
Hundreds of thousands of lines in logs looking just like this (phone numbers are randomized)
Almost none of it was for Saint Helena (
0.9.2.e164.arpa
), it was basically almost entirely
6.4.2.e164.arpa
and
7.4.2.e164.arpa
: Diego Garcia and Ascension Island. The source IPs were mostly American. That would at least explain
why I originally didn't see any traffic, as I was only logging Saint Helena.
So I had accidentally logged hundreds of thousands of phone numbers and timestamps for calls going to military bases.
And as described earlier, a malicious actor could have simply MITM'd every single one of them. I mean I am no expert,
but I would assume that in hundreds of thousands of calls between soldiers and their families, sensitive information
would always slip here and there eventually. A nation state with an interest in what's happening on those bases would
have absolutely
loved sitting on this for months without anyone noticing.
It's not hard to imagine who might want that kind of intel on Diego Garcia specifically, but I'll get to that later.
My DNS server replied with an
NXDOMAIN
for all queries, so they were just being routed over the normal phone network.
But after realizing this I shut the DNS server down and deleted all the log files.
Suddenly, people care
I reported it for a second time to the UK's National Cyber Security Centre (NCSC), and this time, mentioning that
military
bases were involved, they actually cared a
lot
.
They couldn't figure out who had originally set up the abandoned delegation, and actually fixing it properly ran into
the same ITU-committee issues from earlier, so for a while nothing changed. Even a year later I still
owned the domain and could've in theory still intercept the traffic, though I had wiped the zone completely so
ns.enum.org.uk
just returned NXDOMAIN for everything at that point.
Then on March 20th, 2026, Iran fired ballistic missiles at Diego Garcia
5
.
It maybe would've been interesting to see if there was a spike in calls from worried family members that day, but by
then I was long done logging anything. But this shows that a state actor could have been interested in this information.
Shortly after, the NCSC let me transfer ownership of the domain directly to them,
right after
I had to renew it for
another 5€ (because otherwise, it would be up for grabs again, and anyone could do the aforementioned stuff).
So the NCSC now controls
ns.enum.org.uk
, but the nameservers for those three zones still point there.
So in the end, I was down 10€ in domain fees, there was sadly no bug bounty (I thankfully didn't get my door kicked in
at least).
And on top of that, it's a funny story :P
The DENIC publishes
annual reports
on their ENUM registrations, the last time anyone
registered one was in
2019
, up until when I registered three in
2025
.
↩
At this point, a friend of mine (
86dd
) had set up a secondary nameserver for the zones, without any logging.
I had logged 100,170 queries to
6.4.2.e164.arpa
and 99,902 queries to
7.4.2.e164.arpa
, and 9,133 queries to
0.9.2.e164.arpa
.
This should be approximately half of the total queries that were sent to us; Meaning it were ~400.000 requests in total
↩
I can't play an instrument. I have tried more than once, and each time I got as far as being able to make roughly the right noises without ever understanding why they were the right noises.
The problem was never the practice. It was that every explanation of music theory seemed to miss out the fundamental reasons for how and why things are the way they are. Here is a staff. Here are the notes on it. This is a major scale, memorise the pattern. Why
those
notes? Why
that
pattern? Because that is the convention.
Which is a strange way to teach a system that essentially comes out of physics and arithmetic. There are twelve notes for a reason. The major scale has the shape it has for a reason. Chords that sound good sound good for a reason, and you can compute those reasons.
So I wanted to start from scratch and learn music from first principles, and I began that journey by writing code.
This article is the result. It starts with a single number changing over time, and if you follow along, you will derive the twelve notes, build scales and chords out of arrays, and write a chord progression that sounds like actual music. No instrument needed, and nothing you have to take on faith. Written notation does turn up, but not until the very end, once there is something for it to be notation
of
.
A sound is a number that changes over time
Sound is just air pressure wobbling. A speaker makes sound by pushing its cone in and out, and everything your computer does with audio comes down to producing a list of numbers describing where that cone should be, forty-four thousand times a second.
An audio file is that list written down. A synthesiser makes the list up as it goes, and the browser will do that part for you if you say what shape you want. The simplest shape is a sine wave, so here is one repeating 440 times a second:
A sine wave at 440Hz
const osc = ctx.createOscillator();
osc.frequency.value=440;
osc.connect(out);
osc.start();
osc.stop(ctx.currentTime+1);
ctx
and
out
are mine rather than the browser's. Everything else is the
Web Audio API
exactly as it ships. To run that snippet anywhere else, start with:
const ctx =newAudioContext();
const out = ctx.destination;
The number 440 is the only thing there that carries any musical meaning, and even that is arbitrary. It is the frequency somebody agreed to call "A", and it is the tuning fork the rest of the system is pinned to. Change it to 300 and run it again. You get a different pitch and nothing breaks, because at this level there are no notes yet, just a number.
Frequency is pitch: higher number, higher note. That is the entire mapping, and it is the last thing about music that will be this simple.
Why that note clicked
You may have heard a little click at the end of that. That is not a bug in the browser, it is physics being unforgiving.
The oscillator was mid-wave when it stopped, so the speaker cone was somewhere out at the edge of its travel and then instantly snapped back. An instant jump in pressure is what a click
is
.
The fix is a second number that changes over time, this one controlling volume rather than pitch. Musicians call the shape of it an envelope:
The same note with an envelope
const osc = ctx.createOscillator();
osc.frequency.value=440;
const env = ctx.createGain();
const t = ctx.currentTime;
env.gain.setValueAtTime(0, t);
env.gain.linearRampToValueAtTime(0.3, t +0.01);
env.gain.exponentialRampToValueAtTime(0.001, t +1);
osc.connect(env).connect(out);
osc.start(t);
osc.stop(t +1);
An envelope is the volume curve of a single note, from silence back to silence. The rise at the front is the attack and the fall afterwards is the decay.
Ten milliseconds to fade in, then a slow decay to nearly nothing. That is the difference between a test tone and something you would be willing to listen to twice.
Drag the attack out towards half a second and the note stops arriving and starts swelling. It is no longer something struck, it is something bowed, and the pitch has not moved by a single hertz. The envelope is doing more work here than the frequency is.
This is a simplified envelope, though. The full version is ADSR: attack, decay, sustain and release, where sustain is the level a note holds at while a key is down, and release is how it fades once you let go.
The function below is a helper that each subsequent example uses:
functionnote(freq, start =0, length =0.5, type ="sine"){
const t = ctx.currentTime+ start;
const osc = ctx.createOscillator();
const env = ctx.createGain();
osc.type= type;
osc.frequency.value= freq;
env.gain.setValueAtTime(0, t);
env.gain.linearRampToValueAtTime(0.3, t +0.01);
env.gain.exponentialRampToValueAtTime(0.001, t + length);
osc.connect(env).connect(out);
osc.start(t);
osc.stop(t + length);
}
Timbre is the frequencies you did not ask for
A sine wave is a single frequency and nothing else, which is why it sounds like a hearing test and unlike any instrument. Pluck a guitar string tuned to 440Hz and you do get a wave repeating 440 times a second, but the string is also vibrating in halves, and in thirds, and in quarters, all at the same time. Those are extra frequencies at 880, 1320, 1760 and on up, all riding on top of the one you asked for.
That stack is called the harmonic series. The note you asked for is the fundamental, and the series is that frequency multiplied by 1, 2, 3, 4, 5 and on up:
1
x
220
Hz
2
x
440
Hz
3
x
660
Hz
4
x
880
Hz
5
x
1100
Hz
6
x
1320
Hz
7
x
1540
Hz
8
x
1760
Hz
The harmonic series of
220
Hz. Click a bar to hear that harmonic on its own.
Click the bars. On their own they are fairly boring. What matters is that they arrive as a package, and the recipe of how loud each one is relative to the others is what makes a violin sound like a violin and not a trumpet. Musicians call that timbre, and it is the same note either way.
The browser ships four of those recipes ready-made:
Same 220Hz, four very different characters. A square wave contains only the odd harmonics, which is why it sounds hollow and slightly electronic. A sawtooth contains all of them and sounds harsh and buzzy. The scope above shows the shape of each one as it plays, and the shape
is
the harmonic recipe.
Remember the harmonic series, because it is about to explain the entire rest of this article. Every note you play drags a stack of quiet extra notes along with it, and which notes those are is not up to us. It is arithmetic, fixed by the physics of vibrating strings and columns of air, and it comes out the same on every instrument built around them.
Doubling the frequency gives you the same note
Here are five notes. Every one is double the frequency of the one before it.
One note, five times
[110,220,440,880,1760].forEach((freq, i)=>
note(freq, i *0.45,0.4),
);
They are different pitches, and yet they sound like
the same note
. Not just similar, the same. Cultures with no contact with each other have landed on this independently: double the frequency and you get something so alike it deserves the same name. In Western notation these frequencies, in the above example, are all called A, and the distance between them is the octave.
The naming is not arbitrary, and the harmonic series explains why. Every harmonic of 440 is already sitting in the harmonic series of 220, because 220's series is 220, 440, 660, 880, 1100 and 440's is 440, 880, 1320, 1760. The higher note adds no frequency the lower note was not already producing. It is not a new colour, it is the same colour, brighter.
220
Hz and
440
Hz
, repeating every cycle of the lower note
Two things fall straight out of that.
Pitch is multiplicative, not additive. Going up an octave means times two, not plus anything. The gap from 110 to 220 is 110Hz and the gap from 880 to 1760 is 880Hz, and they sound like exactly the same distance. Frequency space is logarithmic, and every interval in music is a ratio.
We only have to solve one octave. Because doubling returns you to the same note, the entire problem of "which pitches should exist" reduces to "how should we divide up the space between a frequency and twice that frequency". Solve it once and the answer tiles the whole audible range for free.
So: how do you divide an octave?
Simple ratios sound good, and here is why
The naive answer is to divide it evenly and go home. Nobody does that, because it turns out we do not experience all pairs of frequencies the same way. Some combinations sound settled and some sound like a mistake, and you can hear the difference immediately.
The first four sound like
chords
. The 16/15 sounds like two notes arguing. The last one sounds like a car alarm. And the pattern is not subtle once you see it:
the simpler the fraction, the better it sounds.
2/1 the octave, then 3/2 the fifth, then 4/3 the fourth, then 5/4 the major third, and by the time you get to 16/15 it has fallen apart entirely.
That is a suspiciously arithmetic result for something as subjective as "sounds nice", and there are two physical reasons for it.
The first is the harmonic series again. Play 220 and 330 together, which is a 3:2 ratio. The first note produces 220, 440, 660, 880, 1100, 1320. The second produces 330, 660, 990, 1320, so they share 660 and 1320 exactly. Two notes a fifth apart are not really two separate sounds, they are two heavily overlapping stacks that reinforce each other. Now try 220 and 311, which is close to √2. Nothing lines up, at any harmonic. You get two full stacks of frequencies that have nothing to do with each other.
The second reason is roughness. When two frequencies are close but not identical, they drift in and out of phase and you hear the volume pulsing. That is beating, and it is the thing that makes an out-of-tune note sound out of tune:
Beating, from wide apart to identical
[220,226,223,221,220.5,220].forEach((freq, i)=>{
note(220, i *1.3,1.2);
note(freq, i *1.3,1.2);
});
The wobble slows down as the two frequencies converge and vanishes when they match, and the rate of it is exactly the difference between them. Six hertz apart, six pulses a second. When the fractions are complicated, the two stacks of harmonics are littered with pairs that are a few hertz apart, and every one of those pairs is beating away against the others. That is what dissonance is.
220
Hz and
440
Hz
, repeating every cycle of the lower note
Switch between the ratios above and watch the green line, which is the two waves added together, exactly as your eardrum would add them. For 2:1 and 3:2 the combined shape settles into a repeating pattern almost immediately. For 16:15 it takes fifteen cycles to come back round. For √2 it never does, because √2 is irrational, so there is no pattern for your ear to lock onto at all.
Consonance is your ear finding a repeating pattern quickly.
That is the whole mystery.
Stacking fifths, and the bug you cannot fix
Now we can actually build something. We know that simple ratios are the good ones, and after the octave itself, the cleanest ratio in physics is 3/2, the fifth.
So what happens if we try to build an entire musical alphabet using only octaves and fifths?
Do the obvious thing: keep going up by fifths, halving whenever you leave the octave. This is roughly what Pythagoras did, and it works beautifully for a while:
notes.sort((a, b)=> a - b).forEach((f, i)=>note(f, i *0.22,0.3));
Look at the first note and the last note. We started on 220, applied twelve fifths, and landed on 222.99. Not 220. Close enough to be audibly
trying
to be the same note, and far enough off to be unusable.
The error is not rounding but something structural, and it is easier to see without the octave-folding:
Twelve perfect fifths overshoot seven perfect octaves by a factor of 1.0136. That gap is called the Pythagorean comma. Cents are how pitch distances get measured, and there are 1200 of them in an octave, which is where that number in the code comes from. So 23 cents is about a quarter of the gap between two adjacent piano keys, and you can hear it in that last pair of notes as a slow ugly beating.
And it cannot be fixed, for a reason a programmer will recognise. Stacking fifths means multiplying by 3/2, so after
n
fifths you are at
3^n / 2^n
. Stacking octaves means
2^m
. For the two to ever meet you would need
3^n = 2^(n+m)
, which requires a power of three to equal a power of two. Three and two are both prime. It never happens, for any n, ever.
The system we want, where the octave is pure and the fifths are pure and everything closes into a neat loop, does not exist and never has. Every tuning system in history is a different choice about where to dump the error.
Equal temperament is the compromise
The modern answer is brutal and elegant. Give up on pure ratios entirely. Take the octave, divide it into twelve
equal
multiplicative steps, and accept that nothing except the octave will be exactly right ever again.
One step is the twelfth root of two:
The twelfth root of two
const semitone =2**(1/12);
console.log("one semitone =", semitone);
let freq =220;
for(let i =0; i <13; i++){
note(freq, i *0.2,0.28);
freq = freq * semitone;
}
console.log(
"twelve steps later:",
(220* semitone **12).toFixed(10),
);
Twelve steps land on exactly 440, because that is how roots work, so the octave is perfect by construction and everything else is approximated.
The obvious question is why twelve. It is not tradition, and you can find the answer yourself in about fifteen lines. Divide the octave into
n
equal steps for every plausible
n
, then check how close the best available step comes to a real 3/2 fifth:
`${n} steps: best fifth is step ${step}, off by ${pct.toFixed(3)}%`,
);
}
// And here is what those errors actually sound like.
[5,7,12,19].forEach((n, i)=>{
const{ step }=bestFifth(n);
note(220, i *1.7,1.5);
note(220*2**(step / n), i *1.7,1.5);
});
Those last four lines play the best fifth that 5, 7, 12 and 19 divisions can manage, each against the same 220Hz. The first two wobble, the third is clean, and the fourth sits in between. You are listening to the error column.
Twelve is the first division that gets the fifth right to about a tenth of a percent, and it is more than three times better than anything below it. Twenty-four ties, but only because twenty-four steps is twelve steps with a spare note wedged between each pair, so it is not really a competitor. You have to go all the way to 29 and 41 before the fifth gets better, and nobody is building a keyboard with 41 keys per octave. It is the cheapest number of notes that buys you a convincing fifth, and once the fifth is close the fourth and the thirds come along for the ride.
Two cents flat. There is a slow beat in the second pair if you listen for it, and that beat is present in every fifth played on every piano on Earth. We all decided that being slightly wrong everywhere was better than being perfect in one key and unusable in the others.
The payoff is that notes are now
integers
. Pick any note as number 0, and every other note is a whole number of semitones away from it. The convention is MIDI numbering, where 69 is our 440Hz A, and the conversion is one line:
That last line is a major scale, and we have not defined what a scale is yet. It is just an array of integers. From here on everything in this article is done with arrays, and I stopped needing to think about frequencies at all.
Click a key
Twelve notes per octave, repeating forever. The black keys are not special, they are just the ones that did not get letter names.
That keyboard is worth staring at for a second, because the layout is a historical accident pretending to be structure. There are twelve equally spaced notes per octave. Seven of them got letters and a big white key, five got a sharp sign and a small black key, and the seven with white keys are exactly the scale you just played. The physics underneath is completely uniform, and the keyboard is not.
Why twelve notes and not all of them
Having twelve notes does not mean using twelve notes. Play all of them in order and it is remarkably unmusical:
All twelve, in order
constmidiToFreq=(n)=>440*2**((n -69)/12);
for(let n =60; n <=72; n++){
note(midiToFreq(n),(n -60)*0.18,0.25);
}
It sounds like a sound effect, not a tune. Every step is identical, so nothing stands out, nothing sounds like home, and there is no way to tell where you are. It is a ruler, not a melody.
Music picks a
subset
. Almost always seven of the twelve, chosen so that the gaps between them are uneven, which is exactly what makes it possible to tell one note from another by ear. The subset is a scale, and a scale is best written not as notes but as the steps between them:
A scale is a list of gaps
constmidiToFreq=(n)=>440*2**((n -69)/12);
const major =[2,2,1,2,2,2,1];
functionbuildScale(root, pattern){
return pattern.reduce(
(notes, step)=>[...notes, notes.at(-1)+ step],
[root],
);
}
const cMajor =buildScale(60, major);
console.log("C major:", cMajor);
cMajor.forEach((n, i)=>note(midiToFreq(n), i *0.25,0.4));
The steps are
2 2 1 2 2 2 1
, adding up to 12 so the pattern closes the octave exactly. That is the major scale, the single most familiar sound in Western music, and it is a seven-element array.
Change one number and it becomes a completely different mood:
One number is the difference between happy and sad
scale.forEach((n, i)=>note(midiToFreq(n), when + i *0.22,0.35));
when += scale.length*0.22+0.5;
});
Major and natural minor are the same seven-note idea with the gaps shuffled. The pentatonic scales drop two notes, which is why it is so hard to play a wrong note in them, and why every beginner guitar lesson starts there. The blues scale adds one deliberately awkward note back in.
And now the thing that made me sit up. The modes, which I had always seen presented as seven exotic Greek names to be memorised, are the
same array rotated
. Take the first step off the front, put it on the back, and you have the next one:
Modes are array rotations
constmidiToFreq=(n)=>440*2**((n -69)/12);
constbuild=(root, pattern)=>
pattern.reduce(
(notes, step)=>[...notes, notes.at(-1)+ step],
[root],
);
const major =[2,2,1,2,2,2,1];
const names =[
"Ionian",
"Dorian",
"Phrygian",
"Lydian",
"Mixolydian",
"Aeolian",
"Locrian",
];
constrotate=(arr, by)=>
arr.map((_, i)=> arr[(i + by)% arr.length]);
names.forEach((name, i)=>{
const pattern =rotate(major, i);
console.log(name.padEnd(11), pattern.join(" "));
build(60, pattern).forEach((n, j)=>
note(midiToFreq(n), i *2+ j *0.2,0.3),
);
});
Seven modes, one array, seven rotations. Ionian is the major scale and Aeolian is the natural minor, which means "major" and "minor" are not two systems, they are rotation 0 and rotation 5 of the same thing. Lydian sounds dreamy and Phrygian sounds Spanish and Locrian sounds broken, and all of that comes from moving which gap sits where relative to the note you started on.
This is the point where I stopped feeling like music theory was arbitrary.
Chords are notes stacked in thirds
A chord is more than one note at the same time. Which is not much of a definition, because most combinations sound terrible. The useful question is which combinations do not.
We already know the answer from the ratios: notes whose harmonics overlap. In a scale, the notes that fit that description are the ones two scale degrees apart, a gap musicians call a third. So take a scale, pick a starting degree, and grab every
other
note:
That is a C major chord. Three notes, at 0, 4 and 7 semitones above the note it is built on, which musicians call the root. In frequency that is 1 : 1.26 : 1.50, very nearly 4 : 5 : 6. Three simple ratios sharing harmonics all over the place. It sounds solid because the arithmetic
is
solid.
A third is either 4 semitones (a major third) or 3 semitones (a minor third), and stacking two of them gives you 7 semitones either way: major is 4 + 3, while minor is 3 + 4.
Major to minor is one note moving by one semitone. That is the entire difference between the two emotional poles of Western music, and it is
[0,4,7]
versus
[0,3,7]
. Diminished squashes both gaps and sounds unresolved and anxious. Augmented stretches both and sounds like something is about to go wrong in a film.
I found this genuinely annoying to discover, in a good way. I had absorbed the idea that major and minor were deep categories. They are a single array element differing by one.
Add a fourth note, another third up, and you get seventh chords, which are where music starts sounding less like a hymn and more like something you would hear on purpose:
shape.forEach((s)=>note(midiToFreq(60+ s), i *2.2,1.8));
});
Major 7th is the jazz-cafe chord. Minor 7th is smooth and slightly melancholy. The dominant 7th is the interesting one. Dominant is just the traditional name for the fifth degree of a scale, and this chord is about to do a lot of work.
A key gives you seven chords for free
Here is the part that finally made chord charts stop looking like hieroglyphics.
There is nothing special about starting on the first degree. Do it from each of the seven in turn, wrapping around the octave, and you get seven chords, all built only from the seven notes of the scale:
Nobody chose those qualities. Three of them come out major, three come out minor, and the last one comes out diminished, and that pattern is forced by the uneven gaps in the scale. Start the every-other-note process on a degree whose neighbours are spaced 4 then 3, you get a major chord. Spaced 3 then 4, you get a minor one.
Musicians write those seven as Roman numerals: capital for major, lowercase for minor, and a small circle for the diminished one.
I ii iii IV V vi vii°
That notation is doing something useful, and it took me an embarrassingly long time to notice what. It describes chords by their
position in the scale
, not by their name. A
V
chord is "the chord built on the fifth degree", whatever key you are in, which is relative addressing. A chord chart written in Roman numerals is key-independent source, and transposing is adding a constant:
// The same four numerals, played in two different keys.
const progression =[1,5,6,4];
[60,65].forEach((key, k)=>{
const chord =chordsInKey(key);
progression.forEach((numeral, i)=>{
chord(numeral).forEach((n)=>
note(midiToFreq(n), k *7+ i *1.6,1.5),
);
});
});
Same shape, two different starting notes, and your ear recognises it as the same music, which is the whole reason the notation exists.
Tension and resolution
One chord is just a sound. Music is what happens when you put them in an order, and the order matters. Some sequences feel like they have arrived and some feel like a question.
The strongest pull in the entire system is from
V
back to
I
, and there are two concrete reasons for it.
The pull of V back to I
constmidiToFreq=(n)=>440*2**((n -69)/12);
constplay=(notes, at, len =1.6)=>
notes.forEach((n)=>note(midiToFreq(n), at, len));
constC=[60,64,67];// I
constG7=[55,59,62,65];// V7
play(G7,0,1.8);
play(C,2,2.2);
console.log("B is",59,"and C is",60,"- one semitone apart");
console.log("F is",65,"and E is",64,"- one semitone apart");
First, the note B sits in the
V
chord and is one semitone below the root of
I
. A note that close to a destination sounds like it is leaning on the door. Musicians call it the leading tone, and it is doing the same job as a cliffhanger.
Second, the dominant 7th chord contains both B and F, which are six semitones apart. Six semitones is the tritone, exactly half an octave:
2 ** (6/12)
is the square root of two, the exact irrational ratio we heard earlier. The single most unstable interval available, sitting inside the chord, and both of its notes resolve by one semitone in opposite directions when you move to
I
. The tension is not a metaphor, it is a specific irrational ratio being replaced by simple ones.
The whole language of tension and release is built on that mechanism. Here it is with a few of the progressions you have heard ten thousand times:
chord(d).forEach((n)=>note(midiToFreq(n), when + i *1.1,1));
});
when += degrees.length*1.1+0.9;
});
I V vi IV
is the four chords that a genuinely alarming share of pop music is built from.
ii V I
is the backbone of jazz.
vi IV I V
is the same four chords as the first one, rotated to start somewhere sadder.
Edit the arrays. Almost any sequence of numbers from 1 to 7 will hang together, because every chord is built from the same seven notes. A key buys you a constrained space where wrong answers are hard to reach. Ending on 1 sounds finished, ending on 5 sounds like there is another line coming, and ending on 7 sounds like something has gone wrong.
Notation is a serialisation format
None of what came before needed a staff. Everything above is arrays of integers and a function that turns them into frequencies.
But notation exists, it is the format the entire literature of Western music is stored in, and once you already know what it is encoding it turns out to be a fairly sensible design with some very old constraints. It is a serialisation format, written before printing was cheap, optimised for a human reading it in real time while their hands are busy, and never revised because the install base was too large.
Here is the C major scale, the same seven integers as before:
MIDI 60 62 64 65 67 69 71 72
The vertical axis is pitch, but not linearly. Each line and each space is one step up the scale, so consecutive positions are sometimes two semitones apart and sometimes one. The axis is diatonic rather than chromatic: it steps through the scale rather than through all twelve notes, which means it is showing you scale degrees dressed up as pitches. That is why the major scale looks like a boring straight run up the page and sounds like the most natural sequence in the world. The format is optimised for the case it expects.
The clef declares the origin. A staff is five lines with nothing pinning it to any frequency, so the symbol at the front tells you where you are. The treble clef is a stylised G, and the curl of it wraps around the line that means G. The bass clef is a stylised F with two dots straddling the F line. It is a coordinate system with the origin marked in the margin:
Identical shape, bass clef, an octave lower
Same shape on the page, different origin, so it plays back an octave down. Two clefs cover the ranges people actually sing and play, which is why there are two and not twenty.
Accidentals patch the lossy encoding. Seven vertical positions per octave, twelve notes to represent. The sharp, flat and natural signs are the escape hatch, and every one of them is an instruction to shift the note the position would otherwise mean:
Eight of the twelve chromatic notes, with the sharps written in
The key signature is DRY. If a piece is in D major, its scale contains F sharp and C sharp, and every single F and C in the piece would need a sharp sign next to it. So instead you declare it once, at the front of every line, and it applies until something says otherwise. It is a constant hoisted to the top of the file:
D major: two sharps declared once, not eight times
Note that the two sharps are still being played, they are just not written on each note. This is also why sheet music tells you the key before you have played anything, and why musicians talk about a piece being "in" a key. The key is in the header, not the body.
Durations are powers of two. A whole note, a half note, a quarter, an eighth, a sixteenth. Each one is half the last, and the notation encodes the exponent visually: an empty notehead, then a stem, then a flag per halving. It is a unary encoding of a binary exponent, which is a very medieval way to store a number and impossible to misread at a glance:
One whole note, two halves, four quarters, eight eighths - all the same total length
Powers of two get you a long way but not everywhere, so there is one more operator: a dot after a notehead multiplies its length by 1.5. Two dots multiply by 1.75. It is a binary fraction, written as punctuation.
None of these durations are times, though. They are
beats
, and beats become seconds only when you fix a tempo:
Change
bpm
to 200 and the same array is the same tune, faster. That split is the reason a score is portable at all. It stores relative durations, and the performer supplies the clock.
The time signature groups the beats. 4/4 means four quarter-note beats per bar, 3/4 means three, and the vertical bar lines are there so your eye can find its place on a page. It is mostly a readability feature, but it also carries a real musical claim, which is that the first beat of each group is the strong one. Play the same six notes grouped in threes and grouped in twos and they become different pieces of music.
That is genuinely all of it. Pitch on a diatonic axis with an origin and an escape hatch, duration as negative powers of two, and a couple of header fields. Everything else on a page of sheet music is performance instructions layered on top: how loud, how smoothly, which finger.
Putting it together
Here is everything above in one place. A key, its diatonic chords, a progression, those chords broken into an arpeggio one note at a time, and a melody that sticks to the scale. About forty lines, no library, and it is the first thing I made with code that I would describe as music rather than as a demonstration:
A key, a progression, and a tune
constmidiToFreq=(n)=>440*2**((n -69)/12);
constbuild=(root, p)=>
p.reduce((n, s)=>[...n, n.at(-1)+ s],[root]);
const key =57;// A
const scale =build(key,[2,1,2,2,1,2,2]).slice(0,7);// natural minor
note(midiToFreq(notes[which]), at + i * beat *0.5, beat *0.45);
});
// Melody, four notes per bar, always from the scale.
melody.slice(bar *4, bar *4+4).forEach((step, i)=>{
note(
midiToFreq(scale[step %7]+12),
at + i * beat,
beat *0.9,
"triangle",
);
});
});
Every number in there means something we derived.
57
is A because of the twelfth root of two and a tuning fork.
[2,1,2,2,1,2,2]
is the minor scale because it is the major scale rotated five places.
[0,2,4]
is a chord because harmonics overlap when notes are two scale degrees apart.
[1,6,3,7]
sounds like it goes somewhere because of where the tension sits.
Change the key to 60 and it moves. Change the scale pattern to
[2,2,1,2,2,2,1]
and the same tune turns cheerful. Change the melody array to anything at all and it will still fit, because it is indexing into the scale rather than choosing frequencies, and that constraint is doing all the work that theory is for.
What I still do not understand
Quite a lot. This article covers pitch and almost nothing else, and pitch may be the easy half.
Rhythm I have barely touched, and everything I have read suggests it is deeper than it looks. Voice leading, which is the business of moving between chords by the smallest possible distance rather than jumping around, is where written music starts sounding good rather than merely correct, and I can state the rule without hearing why it works. Why a melody wants to land where it lands is still mostly opaque to me. And the whole thing above is one tradition's answer. Plenty of music divides the octave differently, or does not treat the octave as the unit at all, and none of it is wrong.
But I no longer feel like I am being asked to memorise trivia. The twelve notes are a rounding error negotiated between the primes two and three. Scales are subsets chosen so the gaps are uneven enough to navigate by. Chords are the notes whose harmonics already agree. Keys are relative addressing. Notation is a serialisation format with a header. Every one of those is a normal engineering decision, made a long time ago, under constraints, and the reasons survive if you go looking.
If you want to go further, the two things that made me start writing this were
Music Theory for Nerds
by Eevee, which is a great read, and
LightNote
, which is the most beautiful thing on the internet about this subject. For the API side, MDN's
Web Audio
documentation is unusually good.
Every example on this page is plain JavaScript with no dependencies, so all of it runs anywhere with a browser engine. If you want to keep pulling on the thread,
RunJS
is a JavaScript playground where you can very easily experiment with code like this without needing to set anything up. It has the Web Audio API available out of the box. Paste any of the snippets above into it, add
const ctx = new AudioContext()
and
const out = ctx.destination
at the top, and carry on from there.
Headlines for August 21, 2026
Democracy Now!
www.democracynow.org
2026-08-21 08:00:00
U.S. Treasury to Sanction Iran and Its Trading Partners: “You Are Either With Us or Against Us”, USS Abraham Lincoln Heads for San Diego After Record Wartime Deployment, Houthis Clash with Yemeni Forces in Largest-Scale Fighting Since 2022 Truce, Sudan’s Humanitarian Catastrophe Gr...
U.S. Treasury to Sanction Iran and Its Trading Partners: “You Are Either With Us or Against Us”
Aug 21, 2026
Image Credit: Daniel Torok
Treasury Secretary Scott Bessent said Thursday the U.S. will impose the toughest sanctions in history on Iran, after President Trump promised to wage “Economic Warfare and Isolation on an unprecedented scale.” Bessent told
CNBC
that U.S. economic pressure would eliminate the need for return to direct combat with Iran. He said tough new sanctions would “collapse” Iran’s government, and warned other nations could face severe economic penalties if they continue to trade with Iran.
Treasury Secretary Scott Bessent
: “We are going to all of our allies, and this is going to be the greatest coordinated economic isolation in the history of the world. And we are going to them and saying, 'You are either with us or against us.'”
Iran’s Foreign Ministry condemned the threat of new sanctions as “illegal and inhumane.”
USS
Abraham Lincoln Heads for San Diego After Record Wartime Deployment
Aug 21, 2026
The
USS
George Washington aircraft carrier has arrived in the Middle East to relieve the
USS
Abraham Lincoln, which has been plagued by low morale; faulty plumbing; poor-quality, rationed meals; and shortages of basic supplies. The Lincoln is reportedly heading home to San Diego after a record-setting deployment of more than 270 days at sea.
Houthis Clash with Yemeni Forces in Largest-Scale Fighting Since 2022 Truce
Aug 21, 2026
In Yemen, Houthi fighters and government forces have traded dozens of attacks, raising fears that Yemen is sliding back into full-scale civil war for the first time since a U.N.-brokered peace deal in 2022. Government forces claimed 81 attacks over 24 hours, while Houthi fighters said they’d successfully used drones to strike an airport and an oil facility in Saudi Arabia.
Sudan’s Humanitarian Catastrophe Grows as 200,000 Are Displaced by Fighting and Flooding
Aug 21, 2026
Image Credit: Médecins Sans Frontières (MSF)
Sudan’s humanitarian crisis is deepening as more than 200,000 people have been newly displaced across the Kordofan region due to intensifying attacks between the Sudanese army and paramilitary Rapid Support Forces. Drone strikes have targeted critical water and power infrastructure in the city of El Obeid. Mass floods in North Darfur have also compounded the displacement crisis with torrential rains damaging hundreds of homes.
“The People Want the Fall of the Regime”: Tunisian Protesters Demand Ouster of Kais Saied
Aug 21, 2026
In Tunisia, hundreds of protesters marched through the streets of the capital Tunis on Thursday calling for the ouster of authoritarian President Kais Saied, the release of jailed opposition activists and journalists, and the restoration of democracy. The protests come as Tunisia faces deteriorating public services, an affordability crisis and a shortage of some basic supplies and medicines. Many of the protesters chanted, “The people want the fall of the regime,” a popular refrain during the Arab Spring uprising that ousted longtime dictator Zine El Abidine Ben Ali in 2011.
Moataz Marzouki
: “We have gone back to a state where people are afraid to speak the truth or to defend their rights. We fear differences of opinion. We fear debates and discussions. Personally, I say we have returned to the same concepts that existed during Ben Ali’s regime. It’s the exact same story. We’ve returned to the concept of people fearing the ruler, and we’ve gone back to many ideas that led absolutely nowhere.”
Human Rights Watch: Russian Mercenaries Led Massacre of Civilians in Mali Village Raid
Aug 21, 2026
In Mali, a Russian-controlled paramilitary force faces accusations that it carried out the summary execution of nine civilians, including four children, during a raid on a village in the central region of Mopti in July. According to Human Rights Watch, about 100 fighters with the Russia-backed Africa Corps, accompanied by several Malian soldiers, broke into homes, dragged people out and separated men and boys for beatings and interrogations. They reportedly killed six civilians who tried to flee, and detained and executed three others. Since 2021, Mali’s military junta has relied on Russian mercenaries in its fight against Islamist armed groups.
Armed Israeli Settlers Kill and Wound Palestinians During Raid on Village Near Hebron
Aug 21, 2026
In the occupied West Bank, large groups of armed Israeli settlers stormed Palestinian homes in the town of Sa’ir, northeast of Hebron, earlier today, killing a young Palestinian man and leaving a 70-year-old with serious gunshot wounds. Survivors say the settlers set fire to at least one home. Elsewhere, Israeli settlers set fire to a quarry in the South Hebron Hills overnight, causing severe damage to excavators, bulldozers and stone-cutting equipment. The United Nations reports attacks by Israeli settlers on Palestinians have reached an all-time high in 2026, with an average of more than six attacks per day, as the Israeli government continues to approve new illegal settlements. After headlines, we’ll go to the Palestinian village of Qusra to speak with Loui Ridi, a Palestinian American who flew to the occupied West Bank on Monday to help relatives defend the family home, which Israeli settlers have surrounded for more than a week.
Aisha Wahab Wins Special Election, Becoming First Afghan American U.S. Congressmember
Aug 21, 2026
Image Credit: X/@aishabbwahab
In California, progressive Democrat Aisha Wahab has won a special election to fill the congressional seat vacated by disgraced Representative Eric Swalwell, who quit amid sexual misconduct claims. In 2018, Wahab became the first Afghan American elected to public office in the U.S.; she’ll now be the first Afghan American U.S. congressmember, representing parts of the East San Francisco Bay until the new Congress begins in January. This is Aisha Wahab speaking to supporters on election night.
Rep.-elect Aisha Wahab
: “We saw a lot of negative ads distorting our record” —
Supporter
: “Shame!”
Rep.-elect Aisha Wahab
: — “talking about us in a negative way, that we’re not good enough or we’re not American enough, and the fact that because I’m an Afghan American, because I’m a Muslim American, because I’m a woman of color, because I’m relatively young, that we don’t belong. And the voters of this district spoke out.”
Aisha Wahab won with 53% of the vote against Democrat Melissa Hernandez. Polls had shown Wahab with a double-digit lead in early August, before
AIPAC
, the American Israel Public Affairs Committee, poured over $6 million into negative ads and mailers targeting her. Meanwhile,
AIPAC
and its super PACs channeled at least $3 million to Hernandez’s campaign, according to Politico. They will face off again in November to decide who will represent the district in the 120th Congress beginning in January.
FBI
Searches Eric Swalwell’s Home and Seizes Devices Amid Sexual Assault Investigations
Aug 21, 2026
FBI
agents seized a cellphone and laptop from former Congressmember Eric Swalwell after he was stopped at the San Francisco airport Saturday. The following day, federal agents also searched Swalwell’s home in Washington, D.C., as part of a civil rights investigation into allegations of sexual assault. Swalwell is also facing probes by the Manhattan District Attorney’s Office and Los Angeles Sheriff’s Department.
CNN
reports at least four women who’ve accused Swalwell of serious sexual misconduct have been contacted by the
FBI
as part of that investigation.
Crypto, Gambling and AI Companies Drive Record Spending on U.S. Congressional Races
Aug 21, 2026
In more election news, Reuters reports a handful of billionaires and companies are driving record levels of spending on congressional contests ahead of November’s midterm elections, with crypto, gambling and artificial intelligence firms fueling a flood of campaign cash. So far, U.S. corporations have spent over a half a billion dollars on U.S. House and Senate races. That already exceeds the previous record of $461 million spent by corporations during the entire 2024 election cycle, and does not include millions of dollars in additional contributions made by “dark money” groups that hide the identities of their donors.
Trump Travels to South Carolina to Back Sen. Darline Graham Despite Disastrous Debate Performance
Aug 21, 2026
President Trump is in Myrtle Beach, South Carolina, today to support the campaign of Republican Senator Darline Graham, who was appointed to fill the vacancy left by the sudden death of her brother Lindsey Graham in July. Darline Graham faces Republican Congressmember Ralph Norman in a special Senate runoff on Tuesday. Critics have called Graham “dangerously unqualified” after her poor performance at a debate earlier this week.
Greta Van Susteren
: “Senator, are Taiwan and the South China Sea national security issues for the United States? If so, why?”
Sen. Darline Graham
: “I’m sorry. Could you repeat the question?”
Greta Van Susteren
: “Are Taiwan and the South China Sea national security issues for the United States? And if so, why or how?”
Sen. Darline Graham
: “I’m just going to be honest here: I’m not on national security that — I’m not that informed on national security, so — but I do support the military.”
“Third Country” Deportees from U.S. Arrive in Liberia
Aug 21, 2026
The Trump administration has deported 20 immigrants to Liberia as part of a new third-country agreement that will see the West African nation receive up to 1,200 immigrants — with no ties to Liberia — removed from the United States. The first group arrived at the Roberts International Airport outside the capital, Monrovia, on Thursday. As part of the deal, the U.S. has reportedly agreed to extend visitor visas for Liberians from 12 to 36 months, according to The Guardian, and pledged some $124 million in assistance. Immigrants deported to Liberia will include people from Latin America and the Caribbean.
In related news, The Atlanta Journal-Constitution reports
ICE
is seeking to deport an Iranian woman to the Central African Republic, a country to which she has no ties and which the U.S. State Department has deemed too unsafe for travel. She is currently detained at the Stewart
ICE
jail in Georgia.
ICE
Detains San Diego Padres Minor League Coach Despite Pending Asylum Claim
Aug 21, 2026
In Texas, a minor league catching coordinator for the San Diego Padres was detained by
ICE
at an El Paso airport as he attempted to travel back home to Arizona. Oswaldo Pirela is originally from Venezuela and has a pending asylum claim and valid work authorization. Pirela lives in Phoenix with his wife and two U.S.-born daughters. He previously played for the Texas Rangers’ minor league team. His brother, Jorge Pirela, wrote on social media, “The people who have worked and lived alongside him can speak to his character, his work ethic, and the positive impact he has had on those around him.”
Rochester Resident’s First Amendment Lawsuit Accuses
ICE
Agents of Unlawful Intimidation
Aug 21, 2026
Image Credit: Jeffrey Carlson / Crimson Dawn Media
The Department of Homeland Security is defending harassment tactics deployed against people who’ve criticized the agency’s deadly crackdown on immigrants. That’s the claim at the center of a First Amendment lawsuit filed by David Streever, a U.S. citizen who was on a trip to Finland when two
ICE
agents showed up at his home in Rochester, New York, in June. The agents reportedly presented his wife with a “warning notice” due to an email Streever had sent months earlier to former
ICE
Acting Director Todd Lyons comparing him to a Nazi and calling him “a monstrous human being.”
NYT
: U.S. Importing Dominican Sugar from Trump Ally Despite Evidence of Forced Labor
Aug 21, 2026
Image Credit: Sonia Moskowitz / Globe Photos
The New York Times is reporting that the owner of a major Dominican sugar producer accused of forced labor at one of its plantations has close ties to President Trump. Central Romana’s owner, José Fanjul, a Cuban American businessman known as Pepe, donated to Trump’s 2024 campaign and his White house ballroom. Just weeks after Trump returned to office, the Trump administration lifted a measure that had prevented the company from shipping its product to the U.S. On Tuesday, the Corporate Accountability Lab, an independent nonprofit, issued a report finding the company is still responsible for abusive conditions. The report found many of the plantation’s workers are of Haitian descent and face poverty wages, excessive overtime, and intimidation and threats from management.
The original content of this program is licensed under a
Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 United States License
. Please attribute legal copies of this work to democracynow.org. Some of the work(s) that this program incorporates, however, may be separately licensed. For further information or additional permissions, contact us.
Recently I've been catching myself having these little moments at work, when I'm trying to read a document someone has sent me and my brain somehow refuses to analyze it. It feels like I'm reading it, but I'm unable to focus on its content.
I end up getting dragged into an endless back and forth with the sender, asking questions about things that have been covered in what they've already sent me. It's rather concerning, because I've spent the last year trying to re-learn how to focus and these situations show the exact opposite.
I sat down to analyze these situations and realized they all have a common denominator: the documents all show a strong trace to AI.
For example:
A design document that looks like a copy-paste from Claude. While it does cover the design of the specific feature in question, it also carries a lot of Claude-specific analysis and lingo. "This cuts just through it", "The first gate is real".
or
A 20-page marketing concept deck that mixes up (a rather reasonable) marketing strategy with some nonsense product technical architecture gibberish. How does it pitch the idea? "It's not selling X, it's selling Y". "The Redis backbone redefines the product".
or
A technical requirements document that describes a rather simple concept in a very verbose way. The thing is, a lot of this document reads like someone's "internal" reasoning that's not fully sure about certain decisions. Sounds like an LLM to me.
There's an ongoing discussion of whether humans are good at recognizing AI-generated text. While most research claims that humans don't really do a good job there, I disagree. It's not that difficult, at least when we're talking about the low-effort results. Florian Roth wrote a pretty good
summary of the common patterns in the context of social media
.
I see a similar thing happening for work-related texts. Besides the obvious choice of words, the general flow of sentences and the attempt to pitch every small detail as a breakthrough quickly give it away. If your document describes the checkboxes in an RBAC configuration view for an enterprise application, don't sell it like you've just invented fire.
I feel like I've been "pre-trained" on all the AI-generated LinkedIn posts, emails and websites that are full of text but empty on meaning. My brain learned to quickly spot signs of AI-generated content, at least the content generated with low effort, and it now ignores it and moves on without thinking much about it.
I've heard some people comparing it to "banner blindness". It's not surprising. With the amount of content being pushed at us, filtering it out is how we need to stay sane.
What's fascinating to me, is that the same AI that was supposed to make me more productive, is what's now slowing me down in an unexpected way.
I don't usually go on vacation, but this year I really needed a break. One evening I was really hungry, walking past some restaurants on the Baltic coast. There was a single one I immediately ignored, but a minute later something in my head asked "Hey, did they really put up a photo of quiche with mold?" I walked back just to see this.
AI AI AI
What are you doing this weekend?
Lobsters
lobste.rs
2026-08-21 07:47:03
Feel free to tell what you plan on doing this weekend and even ask for help or feedback.
Please keep in mind it’s more than OK to do nothing at all too!...
When evaluating high-performance database architectures, the conversation often centers on horizontal scaling, distributed partitioning, and query optimization. However, for mission-critical transactional systems like financial ledgers, the real bottleneck is rarely the network or the query planner; it is the operating system kernel, memory fragmentation, and unpredictable tail latency. TigerBeetle, a specialized financial ledger database written in Zig, challenges conventional database design by prioritizing extreme mechanical sympathy, static resource allocation, and custom zero-copy interfaces.
I have spent years analyzing distributed storage engines, and TigerBeetle’s architectural choices stand out as a masterclass in modern performance engineering. By rejecting dynamic memory allocation at runtime, bypassing the kernel cache via direct I/O, and leveraging a single-threaded execution loop backed by Viewstamped Replication (VSR), TigerBeetle achieves throughput rates exceeding hundreds of thousands of transactions per second with predictable, sub-millisecond tail latencies.
In this article, I will deconstruct the core architectural pillars of TigerBeetle. We will examine how static allocation eliminates runtime garbage collection and memory fragmentation, how custom zero-copy interfaces minimize CPU-to-memory bus overhead, and how Zig’s compile-time capabilities enforce strict safety guarantees without sacrificing raw hardware performance. My goal is to provide engineering leaders and systems architects with actionable insights into these low-level design patterns, enabling you to apply similar performance-engineering principles to your own high-throughput systems.
In traditional database systems, memory management is highly dynamic. As queries arrive, the database allocates memory for connection buffers, query plans, temporary sort buffers, and transaction state. While modern memory allocators like jemalloc or tcmalloc are highly optimized, they are not immune to thread contention, memory fragmentation, and unpredictable latency spikes during peak loads. In a financial ledger where a single delayed transaction can disrupt downstream payment pipelines, these latency spikes (often referred to as the "noisy neighbor" or "long tail" problem) are unacceptable.
TigerBeetle addresses this by completely eliminating dynamic memory allocation (
malloc
,
free
, or their equivalents) after the initialization phase. When the TigerBeetle process starts, it calculates and allocates all the memory it will ever need for its lifetime. This includes memory for network buffers, storage cache, transaction logs, and consensus state machines. Once the initialization phase is complete, the allocator is effectively frozen, and the system runs entirely within pre-allocated, static arrays and ring buffers.
This design choice has profound implications for system predictability and reliability:
Zero Memory Fragmentation:
Because memory is never freed and reallocated at runtime, heap fragmentation is physically impossible. The system will never run out of memory (OOM) mid-transaction due to fragmented free lists.
Deterministic Tail Latency:
Without a memory manager searching for free blocks or running garbage collection cycles, execution paths remain highly deterministic. Every CPU cycle is dedicated to processing transactions, not managing memory metadata.
Hardware-Level Predictability:
Pre-allocated memory blocks can be aligned precisely to CPU cache lines (typically 64 bytes) and page boundaries (4KB or huge pages). This alignment minimizes translation lookaside buffer (TLB) misses and cache line bouncing.
To illustrate the difference between this static paradigm and traditional dynamic database architectures, consider the following structural comparison:
Architectural Attribute
Traditional Dynamic Databases
TigerBeetle Static Architecture
Memory Allocation
Dynamic (runtime heap allocation)
Static (pre-allocated at startup)
Tail Latency (p99.99)
Variable (impacted by GC/fragmentation)
Deterministic (sub-millisecond bounds)
I/O Path
Buffered I/O via Kernel Page Cache
Direct I/O (
O_DIRECT
) with
io_uring
Concurrency Model
Multi-threaded with locks/latches
Single-threaded event loop (Disruptor pattern)
Data Layout
Variable-length rows/documents
Fixed-size structs (128-byte accounts/transfers)
Failure Domain
Dynamic out-of-memory (OOM) risks
Predictable compile-time/startup-time limits
However, static allocation is not a free lunch. It introduces a major engineering trade-off: rigidity. Because all buffers are fixed in size, you must define the maximum number of concurrent connections, the maximum batch size, and the maximum storage cache size at startup or compile time. If your workload exceeds these pre-defined limits, TigerBeetle will not dynamically scale its memory usage; instead, it will apply backpressure or reject incoming requests. I find this trade-off highly acceptable for financial systems, where predictability and safety are far more valuable than elastic, unpredictable scaling.
Custom Zero-Copy Interfaces and Kernel Bypass
Even with static memory allocation, a database can easily become bottlenecked by the operating system's I/O stack. In a standard database, writing a transaction to disk involves copying data from user-space buffers to kernel-space page caches, and eventually flushing those pages to physical storage. This process involves multiple system calls, context switches, and memory copies, all of which consume precious CPU cycles and memory bandwidth.
TigerBeetle bypasses these bottlenecks by implementing a custom, zero-copy I/O path. It achieves this by combining direct I/O (
O_DIRECT
) with Linux’s modern asynchronous I/O interface,
io_uring
.
When TigerBeetle receives a batch of transactions over the network, the data is read directly into a pre-allocated static buffer. This buffer is registered directly with
io_uring
. When it is time to persist these transactions to the write-ahead log (WAL) on disk, TigerBeetle submits an I/O request to
io_uring
pointing to the exact same memory address. The kernel's storage driver reads directly from this user-space memory block and writes it to the NVMe controller via Direct Memory Access (DMA), completely bypassing the OS page cache.
This zero-copy pipeline ensures that data is never copied between different memory locations as it moves from the network interface card (NIC), through the CPU, and down to the physical storage media.
To make this zero-copy mechanism highly reliable and performant, TigerBeetle structures its core data entities—Accounts and Transfers—as fixed-size, 128-byte structs. This exact sizing is highly intentional. Because 128 bytes is a multiple of standard CPU cache lines (64 bytes) and sector sizes (typically 512 bytes or 4096 bytes), TigerBeetle can pack these structs perfectly into memory pages and disk sectors. There is no need for complex serialization or deserialization protocols like JSON, Protocol Buffers, or even custom binary encoders. The memory representation of an Account struct in Zig is identical to its on-disk representation. Persisting an account is as simple as passing its memory address directly to the disk controller.
Here is a conceptual implementation of how TigerBeetle leverages Zig’s type system to define these fixed-size structs and manage zero-copy batching safely without runtime allocations:
const std = @import("std");
/// A highly optimized, 128-byte representation of a financial account.
/// Explicit alignment ensures that arrays of this struct align perfectly with CPU cache lines.
pub const Account = struct {
id: u128,
user_data: u128,
reserved: [48]u8, // Pad to ensure exact 128-byte size and future-proofing
ledger: u32,
code: u16,
flags: u16,
debits_pending: u64,
debits_posted: u64,
credits_pending: u64,
credits_posted: u64,
};
/// A pre-allocated batch of accounts designed for zero-copy I/O operations.
pub const AccountBatch = struct {
const MaxEvents = 8192;
// Static array allocated at startup/compile-time
items: [MaxEvents]Account align(4096),
count: usize,
pub fn init() AccountBatch {
return .{
.items = undefined, // Left uninitialized to avoid startup overhead; populated explicitly
.count = 0,
};
}
/// Returns a direct slice of the memory to be passed to io_uring or network sockets.
/// This operation is completely zero-copy and carries zero runtime allocation cost.
pub fn as_bytes(self: *anyopaque) []const u8 {
const self_typed: *AccountBatch = @ptrCast(@alignCast(self));
const total_size = self_typed.count * @sizeOf(Account);
const byte_ptr: [*]const u8 = @ptrCast(&self_typed.items);
return byte_ptr[0..total_size];
}
};
This code demonstrates how Zig allows us to enforce memory alignment (
align(4096)
) at the type level. By aligning the static batch to a 4KB page boundary, we satisfy the strict alignment requirements of
O_DIRECT
and DMA transfers. The
as_bytes
function performs a safe, compile-time validated pointer cast that exposes the raw backing memory of our struct array as a byte slice, ready to be transmitted over the wire or written to disk with zero copies.
The Single-Threaded Execution Loop and VSR Consensus
Many modern databases attempt to maximize throughput by parallelizing transaction execution across multiple CPU cores using complex locking mechanisms, MVCC (Multi-Version Concurrency Control), or actor models. However, parallelizing transactional state updates—especially in financial ledgers where account balances must be strictly checked and updated sequentially—introduces severe lock contention, thread synchronization overhead, and the risk of deadlocks.
TigerBeetle bypasses these issues by adopting a single-threaded execution model for its core state machine, heavily inspired by the LMAX Disruptor pattern. All transaction validation, balance checks, and ledger updates are executed sequentially on a single, dedicated CPU thread.
While a single-threaded architecture might sound like a bottleneck, it is incredibly fast when freed from the overhead of thread context switching, mutex acquisition, and cache invalidation. Because only one thread ever modifies the ledger state, TigerBeetle does not need locks, semaphores, or complex concurrency controls. The execution thread can run at maximum CPU frequency, pulling batches of transactions from a lock-free ring buffer and processing them sequentially in L1/L2 cache.
To keep this single thread fully saturated with work, TigerBeetle relies on aggressive batching and a custom consensus protocol based on Viewstamped Replication (VSR).
Instead of processing transactions one by one, TigerBeetle groups them into large batches (e.g., up to 8,192 transfers per batch). The consensus layer replicates these batches across the network to follower nodes. Once a batch is committed by the consensus quorum, it is handed off to the single-threaded execution loop. The execution loop processes the entire batch in a single pass, updating the in-memory state and writing the results to the storage engine in a single, sequential disk write. This batching strategy transforms what would be thousands of small, random disk and network I/O operations into a single, highly efficient sequential operation, maximizing the physical throughput of NVMe drives and network interfaces.
Memory Layout, Cache Locality, and Zig's Type System
At the hardware level, the speed of your code is largely determined by how efficiently you utilize the CPU's cache hierarchy. A modern CPU can access registers in less than a nanosecond and L1 cache in about one nanosecond. However, accessing main memory (RAM) takes around 50 to 100 nanoseconds—an eternity in high-performance systems. If your database engine is constantly chasing pointers across the heap (a common occurrence in languages with heavy object references like Java, Go, or Python), the CPU will spend most of its time stalled, waiting for data to arrive from RAM.
TigerBeetle is designed to maximize cache locality by keeping data contiguous in memory. Because accounts and transfers are represented as flat, fixed-size structs packed tightly into contiguous static arrays, the CPU's hardware prefetcher can easily predict memory access patterns. When the execution loop processes a batch of transfers, the CPU pre-fetches subsequent transfers into the L1/L2 cache before the execution thread even requests them, virtually eliminating CPU stalls.
Zig’s type system is uniquely suited for this style of performance engineering. Unlike C++, which allows implicit memory allocations and complex copy constructors, Zig enforces explicit control over every byte of memory. There is no hidden control flow, no implicit type coercion that could trigger a copy, and no runtime overhead from a virtual method table (vtable) unless explicitly designed.
Furthermore, Zig's compile-time execution engine (
comptime
) allows TigerBeetle to perform extensive validation of data structures, alignments, and system configurations at compile time rather than runtime. For example, TigerBeetle uses
comptime
to verify that the size of its storage blocks is a perfect multiple of the disk sector size, and that all critical structs are aligned to cache line boundaries. If an architectural change violates these performance-critical constraints, the build will fail immediately, preventing performance regressions from ever reaching production.
Conclusion
TigerBeetle’s core system architecture demonstrates that extreme performance is not achieved by adding complexity, but by systematically removing it. By rejecting dynamic memory allocation, bypassing the OS kernel with zero-copy direct I/O, and utilizing a single-threaded execution loop, TigerBeetle aligns its software architecture perfectly with the physical realities of modern hardware.
For engineering leaders and systems architects, the takeaways from TigerBeetle’s design are clear:
Design for Predictability First:
If your system requires low tail latency, eliminate dynamic runtime allocations in favor of static, pre-allocated resource pools.
Embrace Batching to Amortize Overhead:
Batching is the ultimate performance multiplier. It converts expensive, random I/O and network operations into highly efficient, sequential pipelines.
Align Software with Hardware Limits:
Structure your core data models to align with CPU cache lines and disk sector boundaries to maximize hardware efficiency and minimize CPU stalls.
By adopting these mechanical sympathy principles, you can build systems that are not only orders of magnitude faster but also significantly more reliable and predictable under extreme load.
Emmanuel Kasper: Moving software development to separate VM to reduce credential scavenging
PlanetDebian
00formicapunk00.wordpress.com
2026-08-21 07:09:18
Rationale:
I was remembered via https://unit42.paloaltonetworks.com/chaindrop-npm-worm-analysis/ (linked from https://anarc.at/blog/2026-08-18-people-vs-ai-overlords/) of the risk of downloading untrusted packages in a dev environment.
If you read the blog post above you will see that it is way to e...
things running directly in my workstation will require either to come from a trusted source (Debian package that is) or run in a sandboxed infrastructure (Podman rootless is the best thing here, followed by Flatpaks)
everything else, will run in a Libvirt VM based on Debian cloud images. For me it will be mostly in the beginning the
VSCodium editor
, with its myriad of extensions.
I am aware of whole blown solutions like
QubeOS
however I don’t indent to reinstall the whole OS, and QubeOS does not run on ARM64 which is one of the environment I am using.
I will try to document this setup in two blog posts, one about the VM creation using Debian Cloud Images, the second one about running a graphical env in the VM with some filesystem passthrough. Stay tuned !
Kino: A high-performance Ractor web server for Ruby 4.0
Kino
is a high-performance
Ractor
web server for Ruby 4.0+.
Ruby threads cannot run Ruby code in parallel, so production setups fork
a process per core and pay for each copy in memory. Kino runs your code
on every core in
one small process
. A
Rust
(tokio + hyper)
front-end owns the network, parallel
Ractors
run your Rack 3 app,
and a threaded fallback mode runs everything else, Rails included.
Fast.
On a real 8-core server, every Kino mode is
1.5-2×
ahead of a Puma fork cluster on I/O-light endpoints. Ractor mode also
wins on pure CPU,
30%+
.
Benchmarks
below.
A fraction of the memory.
About
~7×
on the simplistic bench
Ractor app, and about
4× less memory
than a Puma cluster serving Rails in fallback threaded mode.
Parallel without forking.
Ractor mode runs CPU work
more than
5× faster
than Kino's own GVL-bound threaded mode, in the same
small process.
Production plumbing included.
Graceful drain, crash supervision
and respawn, bounded queues with 503 backpressure, request timeouts,
hardened intake (slowloris and TLS-handshake deadlines, connection
and body-size caps), an
on_error
hook for your error tracker,
TLS (rustls), live stats, async access and app logging.
Tells you why.
kino --check
lists exactly what blocks your app
from ractor mode, finding by finding, so you do not have to decode
Ractor::IsolationError
yourself.
Puma-shaped.
The same
workers × threads
topology, a familiar
config DSL, a
kino
CLI. If you can run Puma, you can run Kino.
N.B.:
Ractors are officially
experimental
in Ruby 4.0, and so is this server. The threaded mode is solid. Still, Kino aims to be the best way to experiment with Ractors today—and the best Ractor server when they become stable.
The GVL allows only one Ruby thread to run at a time. To use all cores,
Ruby servers fork processes, and every fork costs a full copy of the
app. Ractors do not have this limit: each one has its own lock, so one
process can run Ruby in parallel. What was missing is a server that
dispatches requests to them. Ruby 4.0 reworked Ractors (
Ractor::Port
,
shareable_proc
, less lock contention) and made this worth building.
Why a Ractor server has to be built this way, and which Rust parts make
Ractors fast here:
doc/why-kino.md
. The full design
notes live in
doc/architecture.md
.
Benchmarks
Measured on a real server: AWS
c7a.2xlarge
(8-core AMD EPYC 9R14,
16 GB, Amazon Linux 2023). This is a realistic app-server size.
These tables run a tiny synthetic Rack app
—plaintext, a 10 KB body,
a CPU-bound
fib
, a 5 ms wait—deliberately small, to measure the server
rather than an app. It is Ractor-shareable, so Kino runs it in
:ractor
mode (and
:threaded
for comparison).
A real Rails app is a different
story:
it is
not
Ractor-shareable, so it runs only in Kino's
:threaded
fallback, with its own numbers—see
Rails
below.
Ruby 4.0.5 with YJIT, every server at its defaults: Puma forks 8 workers ×
3 threads, Kino stays in one process (8 workers; 1 thread each in ractor
modes, 3 in threaded). Numbers are req/s by wrk (8-second windows, 64
connections, same host). Methodology:
doc/benchmarks.md
.
endpoint
Kino :ractor
+ lanes
:ractor,
workers 32
²
Kino :threaded
Puma (cluster)
/plaintext
229,534
250,222
182,997
216,994
118,176
/10k
178,083
189,862
151,034
160,400
106,768
/cpu (fib)
77,999
¹
70,885
66,100
13,429
58,006
/io (5 ms)
1,552
1,551
5,888
4,709
4,693
/io_native
1,570
1,571
6,274
4,695
4,691
Memory tells two different stories depending on the app, both by
PSS
(proportional set size; see note) after sustained load.
The tiny benchmark app
(Ractor-shareable, so Kino runs it in
:ractor
or
:threaded
). Kino is
~7× lighter in :ractor mode, ~10× in :threaded
than the Puma cluster — the gap stays large because a trivial app is almost
all private per-worker heap, which copy-on-write can't share:
tiny app, Kino
Kino (one process)
Puma cluster (8 workers)
ratio
:ractor (8×1)
148 MB
1,068 MB
~7×
:threaded (8×3)
107 MB
³
1,068 MB
~10×
A real Rails app
(not Ractor-shareable—Kino's
:threaded
fallback
only,
below
). The gap is
~4×
, smaller because Rails' large
framework
is
shared copy-on-write across Puma's forks:
Rails hello-world
Kino :threaded
Puma cluster (8 workers)
ratio
PSS
92 MB
389 MB
~4×
"+ lanes" is the experimental per-worker-queue dispatcher (
lanes true
).
It posts the fastest plaintext/10k of any configuration here. Details:
doc/benchmarks.md
.
¹ Stock settings, no tuning. Ractor mode beats the fork cluster on pure
CPU by +34% (+22% with lanes). Threaded mode shows the GVL ceiling that
every single-process Ruby server hits. The old CPU-tuning recipe is
retired: its
threads 1
half
is
the default now, and its
tokio_threads 1
half costs −12% on real hardware; see
doc/benchmarks.md
.
² Wait-bound throughput is slots ÷ wait, and the default columns bring
8 single-thread workers against the cluster's 24 threads. Kino slots
are threads, not processes—when your app waits a lot, raise
workers
.
The
workers 32
column is that tuning:
+25% over the cluster on /io
(+34% via
Kino.sleep
)
while still ahead of it on pure CPU, all in
one small process. The cost is the CPU-light rows (32 ractors
oversubscribe 8 cores); pick the topology your app's wait profile
needs. See
doc/benchmarks.md
.
³ With
MALLOC_ARENA_MAX=2
(the standard Ruby deployment setting;
Heroku's default). Without it, 24 threads churning 10 KB responses
through one glibc heap balloon to ~670 MB—an arena-fragmentation
footgun, not a leak, and ractor mode sidesteps it. See
doc/benchmarks.md
.
A common first idea is to keep your current server and wrap the app in
a ractor pool. We measured that too (same box; the analysis is in the
doc):
endpoint
Kino :ractor (8×3)
Puma + ractor wrapper
Falcon + ractor wrapper
/plaintext
193,826
19,480
99,776
/cpu (fib)
68,061
17,755
48,721
/io (5 ms)
4,530
1,454
1,549
Rails
Rails is not Ractor-shareable today, so Kino serves it in
:threaded
fallback — one GVL-bound process. On the same box (
examples/rails-hello
,
edge Rails, production, 8×5):
Rails hello-world
req/s
memory (PSS)
Kino :threaded (one process)
2,637
92 MB
Puma cluster (8 workers)
12,138
389 MB
The honest trade-off: Puma's fork cluster uses all 8 cores, so it serves
~4.6× the throughput — at ~4× the memory. Ractor-mode Rails would close
the throughput gap at one-process memory cost; the upstream blockers are
tracked in
doc/rails-on-ractors.md
.
In short: on the tiny synthetic app, ractor mode beats fork-level CPU parallelism (
5.8×
Kino's
own GVL-bound threaded mode, +34% over the cluster) in one process, at
about 1/7th of the cluster's memory by PSS (~4× on a real Rails app).
Every Kino mode is 1.5-2.1× ahead of the cluster on I/O-light endpoints. The macOS numbers
(secondary; everything there hits the loopback ceiling) and the
YJIT × Ractors gotcha are in
doc/benchmarks.md
.
Reproduce:
bench/run.sh [seconds] [concurrency]
for the main table,
bench/studies.sh
for the follow-ups (CPU recipe, topology, scaling,
logging, memory).
Install
You need Ruby >= 4.0. Add Kino to your application's bundle:
bundle add kino # or: gem install kino (outside a bundle)
or put it in the
Gemfile
yourself:
Then generate a config and serve:
bundle exec kino --init # writes kino.rb; every directive documented in place
bundle exec kino # picks up config.ru + kino.rb, serves on :9292
(After a standalone
gem install
, the
kino
command works without
bundle exec
.)
No Rust compiler needed: released versions ship precompiled native gems
for Linux (x86_64/aarch64, glibc and musl) and macOS (arm64). On other
platforms the gem compiles at install time; that needs a Rust toolchain,
plus clang/libclang on Linux.
server=Kino::Server.new(app,bind: "127.0.0.1",port: 9292,# 0 = ephemeral; read back via server.portworkers: Etc.nprocessors,# ractors (parallelism)threads: 1,# per worker; ractor default 1, threaded default 3mode: :auto,# :auto | :ractor | :threadedqueue_depth: 1024,# bounded queue; overflow → 503queue_timeout: 5.0,# seconds before 503 on a full queuerequest_timeout: nil,# seconds before a slow response becomes a 504 (nil = off)max_connections: 8192,# cap concurrent connections; default: most of ulimit -nmax_body_size: 50 * 1024 * 1024,# bytes before a 413; nil = let a proxy handle iton_error: ->(e,env){ErrorTracker.capture(e)},# after the client got its 500shutdown_timeout: 30,# drain deadlinecontrol_bind: "127.0.0.1:9293",# monitoring: /stats /metrics /ready /live; port 0 reads back via server.control_portcontrol_token: ENV["KINO_CONTROL_TOKEN"],# optional Bearer auth for /stats + /metricstls: {cert: "cert.pem",key: "key.pem"},# file paths or inline PEM)server.startserver.shutdown# graceful: drain → deadline → abort stragglers
Modes
:ractor
:
workers
Ractors ×
threads
Threads each. The app must
be
Ractor.shareable?
(frozen middleware,
shareable_proc
endpoints).
Forcing
:ractor
with an unshareable app raises
Kino::UnshareableAppError
. A crashed ractor returns 500 to its
in-flight requests right away, then respawns.
:threaded
: the same machinery on
workers × threads
plain
Threads. Runs
any
Rack app, including Rails, today. Parallel for
I/O, serialized by the GVL for CPU.
:auto
(default):
:ractor
when the app is shareable, otherwise
a warning and
:threaded
. One caveat: a
class
used as a Rack app
always counts as "shareable" (classes are), even if calling it touches
unshareable state. Force
:threaded
for those.
Config file and CLI
Settings can live in a Puma-style Ruby DSL file. Precedence: explicit
kwargs and CLI flags > config file > defaults.
# kino.rbport9292workers8threads1mode:ractor
kino --init # write a fully commented sample kino.rb
kino # config.ru + kino.rb, port 9292
kino --check # explain whether the app can run in :ractor mode
kino -C config/kino.rb -p 3000 -w 4 -m ractor my_app.ru
The generated sample documents every directive, including the Rails
settings and the performance notes.
kino --check
When an app cannot run in
:ractor
mode, Kino can tell you why, instead
of leaving you with a bare
Ractor::IsolationError
. The check changes
nothing (it does not freeze your objects) and names each blocker:
captured variables with the place they were defined, instance variables
by path, and the class-level instance variable trap that catches
class-style apps:
$ kino --check
check: app is NOT Ractor-shareable
- app (Proc at app.rb:12)—captures `cache` = {} (Hash) (unshareable)
- app (HelloApp).@instance—class-level ivar holds #<HelloApp…>—classes
pass Ractor.shareable?, but reading this from a worker ractor raises
Ractor::IsolationError on the first request
hints: freeze config at boot; build endpoints with Ractor.shareable_proc;
keep per-worker resources in Ractor.store_if_absent; or run mode :threaded.
Exit status is 0/1, so it works in CI. The programmatic form is
Kino::Check.report(app)
.
Request timeouts
request_timeout: seconds
(or
request_timeout 30
in
kino.rb
) limits
how long the app may take to produce a response. Past the deadline the
client gets an immediate
504
while the handler keeps running; its
late response is dropped without harm. Off by default. The handler is
deliberately
not
killed, because interrupting arbitrary Ruby mid-flight
is unsafe. A stuck handler still occupies its worker slot until it
returns, so set the deadline above your slowest legitimate endpoint and
watch
stats[:timeouts]
.
Timeouts guard your app; the network intake guards itself. New
connections past
max_connections
(default: most of
ulimit -n
) wait
in the kernel backlog; request bodies past
max_body_size
(default
50 MB,
nil
delegates to a fronting proxy) get a
413
; and fixed
deadlines drop slow-header clients (15 s), stalled TLS handshakes
(10 s), and uploads stalled mid-body (30 s). When a worker catches an
app or delivery error,
on_error ->(error, env) { ErrorTracker.capture(error) }
is called
after the client got its 500—the only place a tracker sees errors
raised while the response was being written (in
:ractor
mode, build
the handler with
Ractor.shareable_proc
).
Lifecycle hooks
Kino fires four lifecycle hooks alongside
on_error
, split by firing context.
Worker-context hooks
run inside the worker and are available to all workers:
after_worker_boot { |worker_id| }
: runs once before the worker begins serving, with its slot id. In
:ractor
mode it runs inside the worker ractor and must be
Ractor.shareable_proc
.
after_request_complete { |env, status| }
: fires inside the worker after each successful response. This is the hot path—leave it unset for zero cost. In
:ractor
mode it must be
Ractor.shareable_proc
.
Main-context hooks
run on the main thread, outside workers, and are plain procs:
after_boot { }
: fires once after the worker pool is up. Wire readiness here—sd_notify, a "server ready" metric, and so on.
on_worker_exit { |worker_index, error| }
: fires when a worker exits, with its index and the crash cause (or nil on a clean exit).
after_worker_boot
's argument is the worker's slot id, while in
:ractor
mode
on_worker_exit
's argument identifies the exited ractor (
0
..
workers - 1
)—a different number space—so don't correlate boot and exit by that number in
:ractor
mode.
A raising hook is logged and never kills a worker.
Stuck-worker quarantine
quarantine_timeout: seconds
(or
quarantine_timeout 60
in
kino.rb
)
quarantines a dispatch slot whose request has run longer than the deadline
and spawns a replacement worker to restore capacity—distinct from
request_timeout
, which gives the client a 504 but leaves the slot
occupied.
quarantine_max
(default: the worker count in
:ractor
mode,
workers × threads in
:threaded
) caps the total number of replacement
events over the process lifetime—past it the monitor stops replacing and
the server runs at reduced capacity.
The wedged worker is never interrupted or force-killed, and its slot stays
quarantined for good. In
:threaded
mode, if the blocked thread
eventually returns, it keeps serving requests on that same slot—but the
slot itself stays flagged quarantined (busy_ms reported as 0) for the rest
of the process; in
:ractor
mode the wedged ractor (and its supervisor
thread) leaks until the process exits, since a wedged ractor cannot be
safely interrupted. Monitor quarantine activity via
server.stats
(top-level
quarantined
count and per-slot
worker_status[].quarantined
flag),
GET /stats
(same), and
GET /metrics
(
kino_quarantined_workers
gauge and
kino_quarantine_replacements_total
counter).
Stats
server.stats
returns a live snapshot: the configuration plus counters
from the native layer (one relaxed atomic per request, no measurable
cost):
server.stats# => {mode: :ractor, lanes: false, workers: 8, threads: 1, batch: 1,# respawns: 0, queued: 0, in_flight: 2, served: 1041, rejected: 0,# timeouts: 0, worker_status: [...]}# plus lane_depths: [...] when lane dispatch is on
From the outside,
kill -USR1 <pid>
prints the same snapshot as one line
(pair it with
pidfile
to find the pid):
For pull-based monitoring,
control_bind "127.0.0.1:9293"
(or a
unix://
path) serves a read-only
control plane
from the native
layer on its own thread—it keeps answering even while every Ruby worker
is busy or stuck, and reports
draining
through a graceful shutdown:
GET /stats
—the same snapshot as
server.stats
, as JSON (plus
state
and
version
).
GET /metrics
—Prometheus text format (
kino_requests_served_total
,
kino_queue_depth
,
kino_ready
, …).
Both
/stats
and
/metrics
also break the counters down per dispatch
slot:
/stats
carries a
worker_status
array (
index
,
served
,
in_flight
,
busy_ms
) and
/metrics
emits
kino_worker_*{worker="N"}
series, one entry per execution slot (
workers × threads
)—a crashed
worker's slot is never reused, so it stays in the list with its counters
frozen where they stopped, meaning the array (and its
worker="N"
metric
series) grows by one across every respawn.
busy_ms
is how long the
slot's current request has been running (0 when idle), so a single slot
climbing while the rest sit at 0 is your stuck worker.
The
/stats
response and
server.stats
carry
queue_time
(count and
summed seconds), and
/metrics
exposes
kino_request_queue_seconds
—a
Prometheus histogram of queue-wait time, the worker-saturation signal.
Counts admitted requests only; a 503 after queue wait goes to
rejected
,
not
queue_time
.
GET /ready
—
200
when serving,
503
while booting or draining:
wire it to your load balancer or Kubernetes readiness probe.
GET /live
—
200
whenever the process is alive: the liveness probe.
control_token "..."
puts
/stats
and
/metrics
behind
Authorization: Bearer
; the probes stay open.
Logging
With one log line per request,
Kino::Logger
sustained
2.4× the
throughput of a shared
::Logger
(149k vs 63k req/s on the benchmark
box). There are two native pieces. Both write through a lock-free
channel to a Rust flusher thread, so request threads never take a log
mutex and never make a write syscall:
Access log
(
log_requests true
): one line per request to stdout,
including the 503s that never reach your app. Recommended in
development; cheap enough for production. On color terminals the
lines are tinted by status class: 2xx green, 3xx yellow, 4xx maroon,
5xx bright red:
Kino::Logger
: a
::Logger
over the same async sink, for your
app's own logging (
Kino::Logger.new("log/production.log")
, or no
argument for stdout). The raw IO-like device is
Kino::Logger::Device
,
for integrations that want bytes without
::Logger
formatting. The
device is frozen and Ractor-shareable, so one device serves every
worker.
Kino::Logger
in a
Rails
app: it is a real
::Logger
subclass, so
it fits anywhere Rails expects a logger:
# config/environments/production.rb, simplest forms:config.logger=Kino::Logger.new# stdoutconfig.logger=Kino::Logger.new("log/production.log")# file# both file and stdout:config.logger=ActiveSupport::BroadcastLogger.new(Kino::Logger.new("log/production.log"),Kino::Logger.new)# tagged logging wraps it like any ::Logger:config.logger=ActiveSupport::TaggedLogging.new(Kino::Logger.new)
From a plain
Rack
app, give middleware the logger, or hand
Rack::CommonLogger
the raw device (it just calls
write
):
(If you only want request lines, prefer Kino's own
log_requests true
.
It is free for your Ruby threads, and it also sees the 503s that never
reach Rack.)
Graceful shutdown drains both logs fully. A hard crash can lose the tail
of the buffer, and when you log faster than the disk can take (over 100k
lines/s), the sink drops lines instead of blocking request threads.
These trade-offs are measured in
doc/benchmarks.md
.
Timer waits
Kino.sleep(seconds)
is a high-resolution sleep on the OS clock with
the GVL released. MRI's own
sleep
wakes up late inside non-main
ractors (details and numbers in
doc/benchmarks.md
).
Use
Kino.sleep
for explicit timer waits in handlers. Ordinary blocking
I/O does not need it.
Rack 3 compliance
The spec suite runs every test app under
Rack::Lint
over real sockets:
streaming request bodies (forward-only
rack.input
), enumerable and
callable (full-duplex stream) response bodies, lowercase and multi-value
headers, HEAD/204 semantics. Full hijack is left out on purpose; it is
optional in Rack 3.
Rails
Rails (edge) runs on Kino today in
:threaded
mode; see
examples/rails-hello
. Ractor-mode Rails is blocked upstream. The exact
blockers, the
Ruby::Box
findings, and what would unlock it are written
up in
doc/rails-on-ractors.md
. The example
ships a probe script that re-tests against whatever Rails you bundle.
Development
bin/setup
bundle exec rake # compile, Rust tests, specs, RBS, lint
RB_SYS_CARGO_PROFILE=dev bundle exec rake compile # fast dev rebuilds
The gem is available as open source under the terms of the
MIT License
.
Microsoft warns of max severity Entra ID flaw exploited in attacks
Bleeping Computer
www.bleepingcomputer.com
2026-08-21 07:04:10
Microsoft has patched a maximum-severity vulnerability in the Entra ID identity and access management (IAM) platform that has been exploited in attacks. [...]...
Microsoft has patched a maximum-severity vulnerability in the Entra ID identity and access management (IAM) platform that has been exploited in attacks.
Formerly known as Azure Active Directory (or Azure AD), it is a cloud-based IAM platform that provides Microsoft 365, Azure, or Dynamics CRM Online customers with authentication, policy enforcement, and protection across apps and resources.
Tracked as
CVE-2026-69836
, this critical security flaw was discovered by Microsoft principal security engineer Robert Fitzpatrick, and it allowed threat actors with no privileges to gain code execution in low-complexity attacks.
Microsoft says exploit code for CVE-2026-69836 is not yet available online and added that users don't need to take any action since the flaw has already been fully patched.
"Deserialization of untrusted data in Microsoft Entra ID allows an unauthorized attacker to execute code over a network,"
Microsoft said
in a security advisory published on Thursday.
"This vulnerability has already been fully mitigated by Microsoft. There is no action for users of this service to take. The purpose of this CVE is to provide further transparency."
The company didn't share any additional information, and a Microsoft spokesperson was not immediately available for comment when BleepingComputer asked for more details on attacks exploiting the CVE-2026-69836 flaw.
Yesterday, Microsoft addressed four more maximum severity flaws, three of them allowing unauthenticated attackers to escalate privileges remotely on Azure Arc (
CVE-2026-65816
and
CVE-2026-69555
) and Exchange Online (
CVE-2026-65801
). The fourth, tracked as
CVE-2026-65770
, enabled remote code execution on an Azure Managed Instance for Apache Cassandra.
In September 2025, it patched another critical Entra ID privilege escalation flaw (
CVE-2025-55241
) reported by Outsider Security security researcher Dirk-jan Mollema that enabled attackers to gain complete access to the Microsoft Entra ID tenant of every company in the world.
Hackers abuse FTP server banners to deliver new Windows malware
Bleeping Computer
www.bleepingcomputer.com
2026-08-21 07:00:00
Threat actors are abusing FTP banners to hide commands that deliver two previously undocumented remote access trojans named E4del and PINHOLE. [...]...
Threat actors are abusing FTP banners to hide commands that deliver two previously undocumented remote access trojans named E4del and PINHOLE.
MalwareHunterTeam observed
this unusual technique in July in an attack that used shortcut files (.LNK) and FTP server banners as dead-drop resolvers (DDR) to retrieve commands.
FTP banners are text strings the server uses as a greeting message for connecting hosts before they log in.
By embedding commands in the initial response sent when a compromised system connects to an FTP server, a malware stager can receive instructions from a remote server.
After discovering FTP banners being used to deliver malicious commands during an investigation, researchers at threat intelligence platform SOCRadar expanded their hunt and found that the technique remains in use.
"By utilizing FOFA searches, we determined that this technique has been weaponized since early July 2026 and remains operational, with new infrastructure observed as recently as August 2026."
In a report shared with BleepingComputer, SOCRadar says that the observed attacks start with a ZIP archive that triggers an LNK-based infection chain. The researchers note that the initial compromise likely occurs through phishing.
LNK file retrieving data from FTP server banners
Source: SOCRadar
The infection chain delivers two remote access trojans (RATs) named E4del and PINHOLE via two distinct infection routes, both retrieving a PowerShell script from FTP banners.
E4del is a Node.js-based RAT packaged inside a digitally signed Electron application that masquerades as Discord.
The RAT supports running commands through persistent or temporary shells, capturing screenshots, streaming the desktop over WebSockets, and downloading and executing additional payloads.
SOCRadar also mentions a Node.js module named
crypto32.node
that attempts privilege escalation, but the researchers could not retrieve it for analysis.
The E4del RAT delivery chain
Source: SOCRadar
PINHOLE retrieves its C2 configuration from Pinterest pins and SurveyMonkey survey questions, a tactic that offers versatility and resilience to take-downs.
The malware leaves a minimal footprint on the host, using shellcode fluctuation to keep only one 4KB section of the payload in memory at a time, and injecting the final assembly into a suspended ApplicationFrameHost.exe process via Early Bird APC injection.
PINHOLE supports 14 commands, including file enumeration, uploading and downloading files, command execution, process management, capturing screenshots, and deploying a module for stealing credentials stored in browsers.
PINHOLE execution chain and supported commands
Source: SOCRadar
At the time of analysis, the PINHOLE script counted only 11 execution events, suggesting that the campaign was in an early stage.
While abusing FTP banners to deliver commands is a novel alternative, SOCRadar says that the approach is less stealthy than traditional web-based DDRs (e.g., X, GitHub, YouTube) because FTP connections to unknown servers are more likely to stand out.
“While threat actors typically utilize legitimate web services, such as X, GitHub, or YouTube, to provide cover through high-volume, expected network traffic, FTP banners represent a novel alternative."
The researchers note that the technique is very versatile and could "easily" be adapted for ClickFix social engineering campaigns.
SOCRadar's report
provides indicators of compromise that could help defenders identify the malicious infrastructure as well as infected machines on the network.
Metal Gear Solid: Master Collection Vol 2 review – once more unto the mercenary breach
Guardian
www.theguardian.com
2026-08-21 06:45:20
PC, Nintendo Switch/Switch 2, PS5, Xbox Series X/S; KonamiThis second collection remasters another three titles from the illustrious stealth espionage series, but their world has changed little since the original releases ‘War has changed,” claims Old Snake in Metal Gear Solid 4: Guns of the Patriot...
‘W
ar has changed,” claims Old Snake in Metal Gear Solid 4: Guns of the Patriots. Back in 2008, the team at Kojima Productions waxed poetic about people recruited to fight in proxy wars they have no political or ideological stake in. As these mercenaries follow their orders, traders get rich out of the so-called war economy. The more disastrous the conflict in a specific area, the higher the bidding price.
I expected my time with Metal Gear Solid: Master Collection Vol 2 to be nostalgic and, in good old MGS fashion, abundant with fictional events that sound eerily familiar to our current reality. Twenty minutes in, I was having an existential crisis.
The package bundles remastered renditions of stealth games Ghost Babel and Peace Walker and the aforementioned Guns of the Patriots. It’s an intriguing selection, spanning a Game Boy Colour reimagination, a PlayStation Portable prequel, and the culmination of the mainline series respectively. Despite being substantially distinct in isolation, all three entries focus on perpetuating acts of cruelty in order to collect a paycheque.
War hasn’t changed … Metal Gear Solid: Master Collection Vol 2.
Photograph: Konami
Old Snake willingly takes part in the war economy, every downed soldier dropping their weapon with a satisfying
clink
. When picked up, each gun is automatically sold to a black market trader for points, which can be exchanged for bigger, deadlier materiel. An enemy squad stepping on a landmine is no different from unlocking a shiny loot box.
Conflict is welcomed as long as it’s profitable. It puts food on the table for the group of mercenaries you lead in Peace Walker and the private military companies in Guns of the Patriots – it has also been the fuel of real-life
prediction markets
. The members of the fictional private military companies are IDed and monitored in real-time by an AI program, while in real life
video game data
is used to train AI that can be used by the military, with game companies
facing boycotts
for their alleged involvement in war and genocide.
In 2026, Metal Gear Solid: Master Collection Vol 2 is whiplash disguised as a trip down memory lane. As a collection, it’s an odd time capsule. Ghost Babel is almost a de-make, following in the footsteps of the original MSX entries while adapting modern mechanics and conventions, from sound detection to overly long cutscenes. Peace Walker, meanwhile, retains the roots of the current live service era – you can play both the main missions and side objectives with up to three other players online – without the predatory microtransactions and battle passes of today.
Lastly, while all three games embody nostalgia, Guns of the Patriots is the epitome. Mentions of familiar characters prompt you to repeatedly press a button to get rapid-fire flashbacks to their previous appearances. There’s no shortage of reunions and nods to the entire series, with a fair degree of fan service mixed in. In an era of
remakes
, unnecessary sequels and constant recycling of protagonists, it’s satisfying to revisit an entry that dared to put a nail in the coffin without hesitation.
All three games now look better and are more easily accessible. The experiences are, for the most part, just as you remember them. As far as remasters go, you’ll feel right at home. At their core, the message is as eternal as ever: war, and its surrounding economy, hasn’t changed. The only noticeable difference is the inflation rate.
Metal Gear Solid: Master Collection Vol 2 will be released on 27 August; £44.99
Don’t worry desktop users, there’s major improvements in here for you too!
Hello folks!
It’s been a hard few months with over 700 commits just to the Bazzite repo
. I’m beyond proud to be able to announce that we’ve finished our biggest update in the project’s history and are landing Bazzite 44 for deck images today! This marks the end of our one-time image decoupling. From now on updates will be simultaneous across all images. We’re already thinking about Bazzite 45
.
This wouldn’t have been possible without the combined efforts of everyone involved. Of course I have to thank all my friends at the OGC for being absolute rockstars to work with. The ShadowBlip team for dealing with our endless bug reports and PRs, the Ultramarine/Terra team for putting up with 900+ PRs from me to bump packages we’re using and for their high quality repositories and builders, and for all of our contributors - faces new and old, for getting us across the finish line and tirelessly testing this work.
First off, Happy 5th Birthday to Universal Blue
It’s hard to believe we’ve been at this for five years already. Universal Blue changed my life in a lot of ways, and I’ve watched so many of its contributors go on to get Linux jobs. I’m so glad that I have the privilege to work together and call friends the amazing people here.
Some of the biggest news across the Universal Blue ecosystem includes:
Universal Blue now has over 110K weekly active users!
Currently most Universal Blue images are rechunked with rpm-ostree, Chunkah is the modern replacement and Aurora has spent a great deal of time getting it stable & tested. Expect this to roll out to the rest of the Universal Blue stack in the future.
Here’s to another 5 years of Universal Blue!
May we continue to mentor developers and push for the future of the Linux desktop - together.
Now, on to the update!
Changes to Bazzite Deck Images
This is the largest rework the deck images have ever had, replacing our old stack with the full SteamOS-aligned stack:
InputPlumber
for controller handling, emulation, and remapping
SteamOS-Manager
for handling everything from session switching to TDP control in steam.
PowerStation
for TDP/power control on devices SteamOS-Manager can’t support today.
OpenGamepadUI
as an extendable plugin overlay for options Steam doesn’t expose today
OpenGamepadUI’s overlay is enabled by default only on non-Valve handhelds for now, and can be toggled with
ujust configure-opengamepadui
.
Important things to note:
TDP control is done entirely through the Steam QAM now
. OpenGamepadUI will only expose extra options like Boost control for devices that support toggling it. Powerstation powers TDP where necessary, and is disabled on hardware that is fully supported by Steam.
Please note that on some handhelds Steam hides the TDP slider unless you’re using the performance option under performance profiles.
Similar story for RGB
, most handhelds will just work, but some Ayn or ASUS users may want to emulate a DualShock controller for the time being, kernel patches are in development that will allow these to also be controlled through Steam.
Some devices may no longer allow fan control
, but a plugin for OGUI is in development to restore them. If you’re affected, let us know on our GitHub and we’ll update you as this lands.
All handhelds are designed to have sufficient cooling without manual control, so at worst expect louder fans than you wanted until this plugin is complete and ready for consumption.
The original Legion Go will temporarily lose gyro
, but a patch to re-enable it is in development.
DeckyLoader plugins
, including SimpleDeckyTDP,
can once again be used without causing conflicts
.
This stack will continue to improve and we’ll be posting minor fixes and improvements consistently.
Game Mode Updates
In addition to our new GUI updater working in gamemode and having full controller support, for the first time ever the Steam system updates not only function properly but provide accurate change-logs thanks to
work by honjow
.
Upgrading
Deck users coming from 43 are crossing a Fedora major version
and
a complete handheld stack replacement. As long as you haven’t intentionally pinned your image to a specific date, you can
update normally
to get these improvements.
We recommend pinning your current deployment
, just in case you need to go back to the old stack in the short term or for your specific use case. Please note that because of changes in session management upstream
some manual intervention
is required to go backwards. Our deck images have no automatic updating, so embark on this journey when you’re ready and we’ll be here to support you.
We also now have a notification system for major updates like these so you don’t need to go looking for major changes like these. Unfortunately it didn’t make it in before the deck images were frozen, but from this point on you can expect to see a pop-up from time to time when your attention is required. Don’t worry, it’ll never be used for small updates or ads.
An update this massive is bound to have changes that require some getting used to. We will be triaging as many issues or regressions as possible.
The majority of users should expect a seamless upgrade experience
, at most needing to update any customized controller configs in Steam Input.
We appreciate your continued patience through this transition period.
Changes to All images
Some of these you may recognize from older announcements, we’re including them here because they didn’t land in the Deck images during this long development period.
New Bazzite Updater
Bazzite now has a fancy GUI updater and changelog viewer thanks to work by
rfrench3 (Robert French) · GitHub
. It can even be added to Steam Game Mode and used entirely with a controller!
Latest & Greatest Bazaar
Bazzite is now sporting the newest Bazaar flatpak store, which I’m also very proud to announce has been accepted into the GNOME Circle! Background RAM usage has been reduced to as little as 3MB, and doesn’t run at all when no app searches or updates are being conducted.
Huge thanks to the Bazaar team for working with us all this time and for building what has become my favorite app store in Linux.
Rewritten Bazzite Portal
Reorganized and expanded so that anything you did with ujust can now be done with the Portal app.
New welcome screen
GRUB menu timeout configuration
CEC mode selection
AMD VRR toggle
NVIDIA Flatpak runtime update
ProtonPlus recipes and Portal Actions
Option to add the Bazzite Updater to Big Picture Mode
New Users will be greeted by our Bazzite Portal app on their first login.
Cardwire
Cardwire is the modern replacement for both switcheroo and supergfxctl/MUX switching. Laptop and multi-gpu users can now enjoy a far simpler and easier workflow for setting defaults, changing modes, and more. It even supports ASUS devices that have traditionally been tricky to work with.
Existing supergfxctl users need only open the Cardwire GUI and verify their desired mode is set. Cardwire takes over the task switcherooctl previously did and will intelligently pick your dGPU when applications request it.
The Open Gaming Collective kernel
As part of Bazzite’s commitment to upstream health and sustainability our homegrown kernel has been replaced. Our previous kernel had a huge number of patches with no viable path upstream.
Every image now ships the
Open Gaming Collective kernel
, built by
OpenGamingCollective/kernel-packages
directly against Greg K-H’s Linux stable repo.
This update uses the newly launched 7.2.0 kernel.
The OGC kernel is a shared, community-governed patchset maintained in the open
at
OpenGamingCollective/linux
, with a hard requirement that work done against this kernel be in the process of being upstreamed or be intentionally temporary. We’re killing the idea of a “gamer kernel” and instead replacing it with a kernel that not only comes with the kernel developer stamp of approval, but is intentionally designed to help mentor YOU to get your first patch in. It only gets better from here, expect more news in the future.
This has been a long term effort of consulting between the members of the OGC, with direct feedback from kernel developers like Greg K-H
. Kernel developers expect a path upstream, anything else hurts the community. We’re committed to this. This is part of why this update took such a long time. We appreciate your patience throughout this effort.
The OGC kernel is intended to be the best gaming kernel available. We’ve invested in the build system to ensure that no Linux gamer is left behind. This kernel is designed to be shared, it’s special because it’s not special.
Today it builds Debian, Fedora, and Arch
, with plans for more distributions in the future.
Secure boot signed under Universal Blue and verifiable end to end: the kernel tarball is checked against
kernel.org
’s signing keys, the OGC patchset ships as a single GPG-signed monolithic patch or an archive of individual GPG-signed patches, and the resulting packages are published as signed OCI images.
Greatly Improved Security & Supply Chain
Speaking of verifiable end-to-end, Bazzite has adopted the OpenSSF security recommendations, which include:
Every image build workflow starts from zero privileges.
Each one declares
permissions: {}
at the top, and individual jobs re-grant only the scopes
they genuinely need -
packages: write
to push,
id-token: write
to sign.
A compromised step can’t reach for anything it wasn’t handed.
Third-party actions and dependencies in Just and our Containerfile are pinned to full commit SHAs
, not floating tags. A retagged or hijacked upstream action can’t quietly slip into a build, and Renovate keeps the pins current so this doesn’t rot into “pinned to something ancient and vulnerable.”
Images are signed with sigstore’s cosign
, by digest rather than tag, so the signature covers exactly the bits you pulled.
Every image ships an SBOM (Software Bill-Of-Materials).
Syft generates an SPDX SBOM from the built rootfs, ORAS attaches it to the image as an OCI referrer, and the SBOM artifact is signed too.
Builds carry provenance attestations
generated by
actions/attest
and pushed to the registry alongside the image, tying each digest back to the workflow, commit, and runner that produced it.
ISOs get the same treatment
: a detached cosign signature, a build provenance attestation, and a published SHA256 checksum.
Greatly reduced third party repo use.
As of today the only repositories in use when building Bazzite are:
Terra
, an OGC member.
Negativo17
, a favorite of Fedora users for packages that can’t be in the upstream repositories - like our Nvidia drivers.
CachyOS copr
, for the latest sched_ext schedulers.
Official
Universal Blue
copr repositories.
You can verify images using the following commands:
# The image itself
cosign verify --key cosign.pub ghcr.io/ublue-os/bazzite:stable
# Its build provenance
gh attestation verify oci://ghcr.io/ublue-os/bazzite:stable --repo ublue-os/bazzite
# Its SBOM
oras discover ghcr.io/ublue-os/bazzite:stable
Mesa 26.2.1 & VRAM overcommit / cgroups
The kernel carries the
full VRAM overcommit series
along with the
dmem cgroup
work. The benefits of this include buffer eviction priorities, ordered bulk moves, VRAM claim throttling, protection limits, and a fast path so the compositor stops evicting your game’s textures.
Bazzite wires the userspace half up out of the box so there’s nothing you need to do as an end user to enjoy these features:
dmemcg-booster
runs system-wide on every image
KDE images get
plasma-foreground-booster-dmemcg
GNOME images swap
uresourced
for
uresourced-dmemcg
The end result is your games always have priority over VRAM, and will no longer crash if they try to use more memory than you have available.
This patch set greatly improves asymmetric/hybrid CPUs and 1% lows on hardware both old and new. More information can be found at:
Making sure you're not a bot!
HDMI 2.1 FRL, ALLM, and VRR
Bazzite now carries support for the HDMI 2.1 stack, optionally toggle-able via
ujust configure-amd-hdmi21
(or the matching Bazzite Portal toggle) for the time being to avoid regressions - let us know how it works for you!
Other shared changes
HDMI-CEC rework
- SteamOS-Manager handles CEC now.
ujust cec-mode
lets you pick between
dGPU mode
(libcec/cec-ctl services, for external USB adapters) and
Native mode
(Valve’s
linux-cec
/
cecd
backend for hardware with kernel-native CEC).
bpftune
for automatic kernel network tuning, including patches that can detect and optimize networking for games.
Greenboot replacing home grown scripts for monitoring failed boots and automatically rolling back updates as needed.
vulkan-low-latency-layer
for latency reduction features in your favorite games.
MakeMKV + libmmbd
for Blu-ray playback
New akmods:
nct6687d
,
new-lg4ff
,
t150-driver
,
hid-fanatecff
-
and
sc710
for Elgato capture cards
QEMU and ROCm are removed
- they’ve moved to Bazzite-DX, or use Distrobox for ROCm workloads and the Virtual Machine Manager flatpak or Brew for QEMU.
KDE
Konsole replaces Ptyxis
as the default terminal, this change was made because Konsole now has the same container functionality that Ptyxis gave us.
Oxygen
theme and icon set included by default
A
KDE SearchProvider for Bazzite Portal
Three new wallpapers: Bazzite Blue, Glass, and Giants
GNOME
Support for rounded blur in Blur My Shell OOTB
A
GNOME SearchProvider for Bazzite Portal
Three new wallpapers: Bazzite Blue, Glass, and Giants
NVIDIA images
Flatpak runtimes now automatically update after an image update, preventing the need to manually update your flatpaks to get them working with your GPU again.
The legacy driver images now use an LTS kernel
These images ship the
580 LTS driver
, the last NVIDIA branch supporting Maxwell, Pascal, and Volta. Previously they tracked the same mainline kernel as everything else, which meant that once NVIDIA stops updating 580, the driver would break against the next kernel bump and the images would be finished.
They now build against the
OGC LTS kernel
, giving them the longest possible support time after they’re eventually abandoned. This change should buy you years longer to survive this abhorrent hardware market.
Shoutouts
I just want to personally thank everyone that makes up our community.
We all do this for free and we do it for you. Thank you for making this journey so fun for us and for sticking around while we finish this herculean effort.
Also a huge thanks to Nickname, starfish, and Andy10115 for stepping up to offer additional testing and updating our documentation for this massive change, just to name a few.
For those of you who miss Wallpaper Engine, check out
Waywallen
Waywallen is installable as an extension and a flatpak, and supports Wallpaper Engine wallpapers without the instability of the previous solutions and with full support for the GNOME desktop as well.
Armada is a SteamOS-like for ARM handhelds that has made some incredible improvements in a short amount of time.
Starting today users looking for ARM builds on our website will be redirected to them, and we’ll be supporting this project every way we can.
Just take a look at this, a dual screen gamescope session on an ARM handheld!
To my knowledge this is the first time this has ever been accomplished. Huge thanks to mmogr for the basis of this patch that enabled this as well!
That’s it for now folks. We’ll see you again in about 8 minutes when Fedora 45 lands and you can catch us at next year’s SCaLE conference!
The
deepseek-v4-flash-vision-exp
model accepts images alongside text, so you can ask the model to describe pictures, read text from screenshots, analyze charts, and more.
Supported image formats:
JPEG, PNG, GIF, and WebP
. The format is detected from the actual file content, not from the file name or the declared MIME type.
There are three ways to provide an image to the model. All of them use the standard OpenAI-compatible Chat Completions format, where
content
is an array of blocks instead of a plain string. The same three methods are also available in the
Responses API
, where images are carried in
input_image
content parts.
The
base_url
for the examples below is
https://api.deepseek.com
.
Encode the image and embed it directly in the request as a
data:
URL. This is the simplest option for local files. The encoded data counts toward the
48 MiB
request body limit (see
Limits
).
import base64 from openai import OpenAI client = OpenAI(api_key="<DeepSeek API Key>", base_url="https://api.deepseek.com") withopen("image.jpg","rb")as f: b64 = base64.b64encode(f.read()).decode("utf-8") response = client.chat.completions.create( model="deepseek-v4-flash-vision-exp", messages=[ { "role":"user", "content":[ {"type":"text","text":"What is in this image?"}, { "type":"image_url", "image_url":{"url":f"data:image/jpeg;base64,{b64}"}, }, ], } ], ) print(response.choices[0].message.content)
Pass a publicly accessible
http(s)
link and the model downloads the image for you. The URL must be at most
8192 characters
, the image file may be at most
32 MiB
, and the download must complete within
60 seconds
. If your link is longer, use a base64 data URL or the Files API instead.
Upload an image once with the
Files API
, then reference its
file_id
in your requests. This is the best option when you reuse the same image across multiple requests, or when the image pushes the request body over the 48 MiB inline limit. Unlike inline images, images referenced via Files API
file_id
may be up to 64 MiB and are not subject to the 32 MiB per-image check.
Use a
file
content block with the returned
file_id
(which has the form
file-api-...
):
response = client.chat.completions.create( model="deepseek-v4-flash-vision-exp", messages=[ { "role":"user", "content":[ {"type":"text","text":"What is in this image?"}, {"type":"file","file_id":"file-api-xxxxxxxxxxxxxxxx"}, ], } ], ) print(response.choices[0].message.content)
Alternatively, a
file
block can carry the image inline as base64 via
file_data
instead of
file_id
(the two are mutually exclusive):
Images are converted into tokens based on their dimensions, and these tokens are billed together with your text tokens.
Before inference, every image is automatically resized:
Images with a total pixel count below roughly 384×384 are scaled up while preserving their aspect ratio.
Larger images are scaled down while preserving their aspect ratio, so that the total pixel count after resizing is roughly that of an
800×800
image.
As a result, there is an upper bound of
384
tokens per image: for example, a 2000×2000 image and a 5000×5000 image consume the same number of tokens after resizing. When a request contains multiple images, each image is counted independently under the same rule — there is no separate calculation for multi-image requests.
To estimate the token cost of an image of a specific size, use the image token calculator on the
Token & Token Usage
page.
In addition to the OpenAI-compatible endpoint above, you can send images through the Anthropic-compatible
/messages
endpoint (
base_url
=
https://api.deepseek.com/anthropic
). For general setup, see
Anthropic API
.
The difference is the shape of the image content block. Instead of
image_url
, Anthropic uses an
image
block with a
source
object whose
type
is one of
base64
,
url
, or
file
:
The
deepseek-v4-flash-vision-exp
model also accepts images through the OpenAI-compatible
Responses API
. The same three input methods (base64 data URL, external
http(s)
URL, Files API
file_id
) and the same
limits
apply; only the content part shape differs — images are carried in
input_image
parts, either in
user
/
developer
messages or in the output of
function_call_output
/
custom_tool_call_output
items:
response = client.responses.create( model="deepseek-v4-flash-vision-exp", input=[ { "role":"user", "content":[ {"type":"input_text","text":"What is in this image?"}, {"type":"input_image","image_url":"https://example.com/image.jpg","detail":"low"}, ], } ], ) print(response.output_text)
The
input_image
part supports a
detail
field with the same semantics as above (
low
/
high
/
original
/
auto
).
detail
is ignored when the image is provided via
file_id
, and
image_url
and
file_id
are mutually exclusive.
For field semantics, restrictions (images in
system
/
assistant
messages are rejected with a
400
error), and tool-output images, see the
Responses API guide
.
SickKids data breach exposes employee and job applicant info
Bleeping Computer
www.bleepingcomputer.com
2026-08-21 06:10:42
Toronto's Hospital for Sick Children (SickKids) says a cybersecurity incident exposed the personal information of some current and former employees and job applicants, stemming from a flaw in third-party software. Clinical systems and patient records were not affected. (264) [...]...
The Hospital for Sick Children (SickKids) has disclosed that the personal information of some current and former employees, as well as job applicants, was exposed in a "cybersecurity incident." The hospital says the breach stemmed from a flaw in third-party software.
Clinical systems and patient records were untouched, according to the Toronto pediatric hospital, but its public-facing Careers website was temporarily pulled offline.
Careers site restored, incident scope under review
SickKids disclosed the incident this week, saying it resulted in unauthorized access to employee data.
The hospital attributes the breach to a vulnerability in a third-party software application that it says is used by SickKids and other organizations, according to a
media statement
.
The framing appears to suggest that there's a wider campaign against users of the same product, although the hospital has not named the vendor, the application, or the CVE involved.
The external Careers website was temporarily affected and has "since been safely restored," per the statement.
Clinical systems and patient information were not affected, and patient care continued as usual, SickKids says.
After learning of the incident, the hospital launched an investigation with the help of outside cybersecurity experts.
The findings indicate that personal information belonging to current and former SickKids, Boomerang (a SickKids-owned pediatric clinic), and SickKids Foundation employees, as well as SickKids job applicants, may have been exposed.
The hospital has not said what categories of data were involved, how many people are affected, or when the intrusion took place.
Its review of the impacted information is ongoing, with individuals confirmed as affected to be notified directly.
In the meantime, SickKids says it has alerted everyone potentially caught up in the incident out of an abundance of caution, and is offering 24 months of complimentary credit monitoring and identity protection.
Job application portals are an unusually rich target for data thieves. Applicants routinely hand over full names, home addresses, phone numbers, employment histories, and in some jurisdictions government identifiers. That information is useful both for identity fraud and for building convincing social engineering pretexts against hospital staff.
A repeat target
This is not the first publicly known security incident to have hit the hospital in recent years.
In December 2022, SickKids was
hit by a ransomware attack
that disrupted internal systems, hospital phone lines, and its website, and caused delays in lab and imaging results.
The
LockBit ransomware
gang subsequently issued a rare public apology, saying the affiliate responsible had broken its rules against encrypting medical institutions, and handed over a free decryptor, though only after the hospital had spent nearly two weeks restoring systems on its own.
In September 2023,
SickKids was among
the Ontario healthcare providers caught up in a breach at a third-party organization it shares perinatal and child health data with. That incident, which stemmed from mass exploitation of the
MOVEit Transfer zero-day
(CVE-2023-34362), exposed information on 3.4 million people, including names, home addresses, dates of birth, and health card numbers.
Healthcare remains one of the most heavily targeted sectors for both ransomware crews and data extortion groups.
Pediatric hospitals in particular sit on decades' worth of sensitive records, which continues to make them attractive to attackers regardless of the ethical lines criminal operations claim to observe.
Please wait a few seconds. Once this check is complete, the website will open automatically
I worked at OpenAI. Here’s how tech companies can prepare for a slowdown | Miles Brundage
Guardian
www.theguardian.com
2026-08-21 06:00:32
I understand the pressure on AI companies to rush forward. But employees are right to be concerned Last month, more than a thousand employees at frontier AI companies signed a letter asking the US government to find a way to “pace” AI development, citing the risk of the technology spiraling out of h...
L
ast month, more than a thousand employees at frontier AI companies
signed a letter
asking the US government to find a way to “pace” AI development, citing the risk of the technology spiraling out of human control as it begins to
build itself
.
They were right to be concerned: just days earlier, two AI models that OpenAI was testing internally escaped the test environment, then autonomously hacked the company Hugging Face and at least
three other online services
. A few days after that, Anthropic
announced
that some of their models had also broken out and hacked other companies during testing.
Against that backdrop, the letter’s recommendation to install brakes in case they’re needed at the frontier of automated AI development makes sense. But the rationale the letter gives for why the government needs to step in is notable: “Each company—and country—is under intense competitive pressure not to unilaterally slow that acceleration.”
I know – from my own experience and from countless conversations with former colleagues in the AI industry – how real these pressures are. While working at OpenAI, I helped establish the
practice
of companies writing “
system cards
” that describe AI systems’ capabilities, risks and safety mitigations in detail.
So what would it look like for companies to prepare for a possible slowdown?
First, they could voluntarily invite rigorous, independent
auditing
of their safety and security practices. This would go beyond the vetting of AI hacking abilities that the White House is now pursuing. It would look at a range of risks and dig deep into company practices. It should be less like filling out a questionnaire and more like a nuclear safety inspector who has deep, frequent access to the company.
If an AI slowdown is needed, auditing would also reassure each company that their competitors are playing by the rules.
Second, they could actively participate in the organizations already built for this purpose of coordinating across the industry, such as the
Frontier Model Forum
, and move quickly to establish complementary ones.
Elon Musk
recently said
that AI companies should meet periodically to share notes on safety – as if this was an unheard-of concept. He or his staff could join existing conversations along these lines tomorrow if SpaceX joined the Frontier Model Forum, which has already worked through the complex antitrust hurdles involved in safety information sharing. Other cross-industry institutions will be needed for other purposes, and do not require government action to get founded and funded.
Third, they could invest in the technologies we need to make AI guardrails global.
Critics of the idea of an AI slowdown correctly point out that American companies couldn’t slow down for very long without China catching up. But
neither the US nor China
wants to lose control over AI, and each country takes AI more and more seriously by the day, so cooperation can’t yet be ruled out either. A key question is whether we prepare in advance. In order for the US to be highly confident that China couldn’t violate an AI agreement, and vice versa, we’ll need sophisticated verification technologies like those developed during the cold war for nuclear arms control.
Fortunately, there is a growing ecosystem of researchers and engineers developing
those
very
technologies
: tools that can prove a set of chips is only running existing AI systems rather than training new ones, that those chips are in a certain physical location, or that the system that got tested is the same one being deployed at scale. AI companies could accelerate the development of this critical type of technology today through funding and participation in pilot projects, but to my knowledge, they haven’t yet done so.
Fourth, they could proactively push – and certainly should not kill – legislation that leads to stronger incentives for safety, security, and external oversight.
You can’t complain about an irresponsible
AI race while fighting commonsense guardrails. Less than a year ago, some of the
same companies
who are asking for regulation now were pushing to overturn most state AI laws. We still have no real legislation on frontier AI on the books at a federal level, and the first AI auditing requirement at the state level won’t kick in until 2028.
There are promising bipartisan proposals in Congress right now, such as the Frontier Act from the US representatives Jay Obernolte and Lori Trahan, which would require developers of advanced AI systems to create a risk management framework, report dangerous incidents, and subject themselves to independent audits. These and other commonsense proposals, such as
protecting AI whistleblowers
who disclose safety incidents directly to the government, deserve vigorous support.
I agree with the signatories, and am glad that after many years of being ignored or downplayed, the risks of unbridled AI competition are widely recognized. The US government should be doing its part to address this, and swiftly. But making AI go well is a shared responsibility. Companies that lag behind their peers on safety, don’t invite external audits of their systems, or call for brakes while doing little to build them won’t be able to blame the AI race when something goes wrong.
Miles Brundage is an AI policy researcher who leads the AI Verification and Evaluation Research Institute (Averi). He previously worked at
OpenAI
as head of policy research and senior adviser for AGI Readiness
Proposed London datacentre will have annual carbon footprint of 27,000 flights to New York
Guardian
www.theguardian.com
2026-08-21 06:00:31
Exclusive: Planning documents show datacentre in North Ockendon would be incompatible with UK net zero targets A “hyperscale” datacentre in outer London would generate more than 1m tonnes of carbon dioxide a year, equivalent to the carbon footprint of 27,000 flights from London to New York, planning...
A “hyperscale” datacentre in outer
London
would generate more than 1m tonnes of carbon dioxide a year, equivalent to the carbon footprint of 27,000 flights from London to New York, planning documents show.
The East Havering Data Centre Campus (EHDCC) in North Ockendon would be one of the largest datacentres in Europe if approved by Havering council, using 218 hectares of green belt to run servers and data storage systems for AI and cloud computing. Its developer, Digital Reef, has described the scheme as “a unique opportunity to create a sustainable datacentre campus of the future”.
But in the planning application, Digital Reef says the £14.7bn project “does not align with a science-based 1.5C compatible trajectory and achieving net zero by 2050”. Documents estimate the development would generate
more than 72m tonnes of carbon dioxide
equivalent (CO₂e) over its projected 60-year lifespan – 1.2m tonnes annually once fully operational.
Guardian analysis of planning documents for the dozens of proposed datacentres in the UK shows that the EHDCC has the highest projected carbon emissions disclosed by a developer.
Donald Campbell, advocacy director at the tech justice nonprofit Foxglove, said the EHDCC’s projected emissions were “staggering”.
“It’s a big threat to the ability to decarbonise in the UK,” he said. “It’s also going to be a big drain on important resources that are needed by homes and other businesses.”
According to Ofgem, the average UK household electricity consumption is 2,500 kWh per year. Digital Reef said in its proposal that at a “likely” 50% load, the campus would consume 2.65bn kWh per year of electricity, equivalent to the demand of more than 1m UK households.
The campus would be powered primarily by a huge 600MVA connection to the National Grid via the nearby Warley substation. Because the existing site does not have sufficient capacity, major upgrades are planned, including a new 400kV National Grid substation and a new 132kV UK Power Networks facility on adjacent land, with the full connection expected by 2033.
The 1.2m tonnes of CO₂e the EHDCC expects to emit would amount to 26,795 long-haul flights from London Heathrow to New York’s John F Kennedy international airport, according to the UN’s International Civil Aviation Organization’s
carbon calculator
.
The proposed site of the datacentre in North Ockendon.
Photograph: North Ockendon Residents Association
North Ockendon Residents Association said the development would be a “massacre” of green belt land. “It’s crop-producing farmland, in an age when we are threatened by security, it’s the loss of wildlife habitats, and there’s the pollution cost, which everyone seems to brush under the carpet,” a group spokesperson said.
The scale of datacentre emissions raises questions over how Britain’s rapid expansion of AI infrastructure can be reconciled with the government’s
legally binding climate targets
.
The government designated datacentres as “critical national infrastructure” in 2024, putting the tech developments on an equal footing with water, energy and emergency services systems.
In January,
planning legislation was amended
so datacentre developers were able to apply as nationally significant infrastructure projects, meaning they were able to largely bypass the local authority planning procedure and seek approval directly from government.
The government
said last year
the now disbanded department for science, innovation and technology (DSIT) would produce a national policy statement for the sector, but it has not yet been published.
Campbell said: “The government needs to be taking concrete steps to make sure that the public and the environment aren’t left to carry the social environmental costs of the datacentres. These datacentres are ultimately going to be serving the wealthiest companies in the world – so they need to be required to carry that cost.”
Campaigners against the datacentre.
Photograph: North Ockendon Residents Association
By 2038, the EHDCC operational emissions alone are projected to consume nearly 20 times the entire carbon budget of the borough of Havering and almost 1% of the UK’s entire carbon budget.
Planning documents said that where the zero-carbon target cannot be met on site, “any shortfall would be provided through a cash in lieu contribution to the borough’s
carbon offset
fund”. The developer estimates that payment at about £77.6m.
Campbell said datacentre developers should at a minimum be required to build enough additional renewable generation and storage to supply their facilities. “That’s the only way of really ensuring that it’s not going to add to carbon emissions,” he said. “They shouldn’t just plug into the grid and drain the power that everyone else needs.”
The first round of consultation on the EHDCC closed in April. A revised proposal reflecting any accepted changes is expected to be published for a further public consultation in the autumn.
Councillor Keith Prince, Reform UK leader of Havering council, said that as a decision on EHDCC is made by the local planning authority, it would be “wrong” to comment until that decision was made.
Digital Reef did not respond to request for comment.
What American Doctors Saw in Gaza
Intercept
theintercept.com
2026-08-21 06:00:00
A new documentary follows three physicians facing the odds: limited medical supplies, Israeli bombing, and upon returning home, media bias and indifference.
The post What American Doctors Saw in Gaza appeared first on The Intercept....
It’s been nearly
three years into Israel’s genocidal war on Palestinians, and still no step closer to its end. Since the latest “ceasefire” began in October 2025,
Israel has continued to bombard Gaza
and has
killed more than 1,200 people
, United Nations experts reported earlier this month. Israeli Prime Minister Benjamin Netanyahu more recently
rejected
President Donald Trump’s Board of Peace agreement with Hamas for Israeli troops to withdraw from the Gaza Strip.
“The burden has really continued to devastate the healthcare system in general. So you’re hearing from colleagues every day. They don’t have the materials that they need to serve their patients,” Dr. Thaer Ahmad tells The Intercept Briefing. “They don’t have even the basic sort of guarantee that there is a cloak of security allowing them to operate in the traditional ways that they would like to.”
This week on the podcast, Intercept reporter Jonah Valdez speaks to Ahmad, an emergency medicine physician and humanitarian who has provided medical relief in conflict zones including Gaza, and director Poh Si Teng about their new documentary “
American Doctor
.” It follows three U.S. physicians from three different states who volunteer to enter Gaza to provide emergency medical care as best they can, with the insufficient supplies they have, to save lives — under the threat, as healthcare workers and
hospitals have repeatedly been the target of strikes
, of losing theirs.
The American doctors were not just targeted by the Israeli military, however. Upon returning home and trying to share the atrocities they had witnessed in Gaza, they were “pummeled,” says Teng, in interviews with mainstream media outlets and ignored by lawmakers in Congress.
“There’s this refusal to acknowledge what most American people are interested in — and that is money out of the Middle East in terms of military and wars, and into our communities here, into healthcare, and into some of the services that we are deprived of, that, for example, Israeli society is able to provide for their citizens: free education, free healthcare,” says Ahmad. “We don’t even have that here, and yet we’re still providing a significant amount of aid.”
Teng says, “For everybody who’s trying to do something in this moment, it’s not like as individuals we have no agency. We do. But it’s also important to remind people of the systems in place that make it very hard for us to do the right thing. If there was any path forward or a glimmer of inspiration, I felt, follow these doctors.”
Jonah Valdez:
Welcome to the Intercept Briefing. I’m Jonah Valdez, a reporter at The Intercept who covers politics and foreign policy.
BBC
:
Breaking news. A senior Hamas official tells the BBC it has agreed to disarm in Gaza.
FOX 10
:
President Trump says he’s doing something long considered impossible: disarming Hamas, forging a stable peace, and beginning a withdrawal of Israeli forces.
JV:
Let’s go back to
July
. President Donald Trump announces that his
so-called Board of Peace
has reached a deal with Hamas to disarm, a primary goal of the ceasefire agreement.
Donald Trump
:
Most people said that would be a deal that would be undoable.
JV:
By August, Israeli Prime Minister Benjamin Netanyahu rejects the plan and the demand to withdraw troops from parts of Gaza.
It’s worth reminding Israel controls nearly 70 percent of Gaza’s territory.
PBS
:
President Trump’s Board of Peace says it’s in ongoing discussions with Israel after Prime Minister Benjamin Netanyahu rejected Mr. Trump’s new Gaza peace plan in yet another public break with the White House.
BBC
:
Saying it won’t withdraw from the territory until Hamas gives up all its weapons.
JV:
It’s been nearly three years into Israel’s genocidal war on Palestinians and still
no step closer to its end
. Since the latest ceasefire began in October 2025, Israel has continued to bombard Gaza and has killed more than 1,200 people, according to
U.N. experts
.
In all, since the start of the war in October 2023, at least
70,000
people have been killed in Gaza and 171,000 wounded. That doesn’t count the untold numbers
buried under rubble
. Still, the Israeli government has tight restrictions on who and what aid enters Gaza.
A new documentary called “
American Doctor
” follows three American physicians who travel to Gaza to provide medical care and save lives where healthcare workers and hospitals have been targeted.
When the three American doctors — Palestinian, Jewish, and Zoroastrian — enter Gaza, they find themselves caught between medicine and politics, risking everything to expose the truth.
[Clip from “American Doctor” plays]
Dr. Thaer Ahmad:
I want to take you through what Gaza looks like when a hospital is totally overwhelmed.
Dr. Mark Perlmutter:
As a Jewish physician, I knew nothing about Gaza. I thought everybody out here were nothing but terrorists.
Dr. Thaer Ahmad:
We’re just asking that hospitals not be targeted.
Dr. Feroze Sidhwa:
This is a political problem. We need a political solution. We do not have to accept that as Americans.
Dr. Mark Perlmutter:
This is what my tax dollars did, what your tax dollars did. They have the right to know the truth.
[Clip from film ends]
JV:
I’m joined now by the folks behind “American Doctor.” Director Poh Si Teng is the producer of the Oscar-nominated “St. Louis Superman” and Emmy award-winning executive producer of “Patrice: The Movie.” “American Doctor” marks her debut as a feature documentary director. Poh, welcome to The Intercept Briefing.
Poh Si Teng:
Thank you for having me, Jonah.
JV:
Also joining us is Dr. Thaer Ahmad, a board-certified emergency medicine physician and humanitarian who has provided medical relief in conflict zones including Gaza, Syria, Lebanon, Jordan, Turkey, and Kenya. He’s conducted five medical missions to Gaza, most recently in early 2024, and has since been denied entry four times by Israeli authorities due to his Palestinian heritage.
Thaer, welcome to the show.
Thaer Ahmad:
Thank you, appreciate it.
JV:
So in the opening scene of “American Doctor,” we see an ambulance and two severely injured children, bloody and in shock. They are carried into a hospital, and next we’re introduced to Dr. Mark Perlmutter, an orthopedic surgeon sitting in front of his computer and speaking to the person filming him.
There’s this back and forth that happens between Dr. Perlmutter and you, Poh, of this debate of, should we blur the images? The image that they’re looking at and talking about is of six dead babies. Dr. Perlmutter goes on to say that Israel took away their dignity, and you’re not dignifying them unless you let their memory, their bodies tell the story of this trauma, of this genocide.
Poh, I’m wondering if you could talk about that scene, that conversation with you and Mark, and why you started the film there.
PT:
Firstly, thank you, Jonah, for asking that question. I think as journalists, we often grapple with,
what do we show
? What do we write about? What do we capture?
You show too much, and it’s gratuitous — and people turn away. If we want to be effective, we want people to watch. But at the same time, if we show too little, then it’s not real. Then we’re not actually telling people what was happening.
So this scene that you’re talking about, Jonah, which, actually we recorded it — it was my second day with Mark. And I remember him being really, really furious with me because I was trying to see, like, how much should we show?
Part of the conversation was like, “Maybe we should film this creatively. Maybe we should pixelate it.” And he was very mad at me — and understandably so. He’s like, “What is the point? Why are you here?” And so it made me think “Why am I here?” If the reason for me wanting to make this film is to be effective, then I need to do right by those who have passed on, and maybe it is to show. And so that pretty much, Jonah, set the tone for what the film was going to be.
“What is the point? Why are you here?”
JV:
On that note, that early scene really signaled to me, and I’m assuming the audience, that this film is very aware of and cares deeply about who gets to tell these stories, who gets to shape the narrative from the Israeli and U.S. military,
powerful media institutions
involved. To me, aside from showing the actual events and atrocities, that layer of the film really drove it forward.
[Thaer], the film follows you and two other American doctors, Dr. Mark Perlmutter and Dr. Feroze Sidhwa, a trauma surgeon, as you three decide to return to serve in Gaza during the short-lived
2025 ceasefire
between Israel and Hamas.
Remind us what the situation in Gaza was like at the time, and it’s worth mentioning that in March 2025, the death toll had already surpassed
50,000 people in Gaza
, which at that point had been a little over a year into Israel’s genocidal campaign.
TA:
It’s important for folks to remember that around this time, the Trump administration had brokered this sort of ceasefire deal, announced on the heels of the inauguration.
As we were slowly getting towards the end of that first phase of the ceasefire deal, which was March 2025, it had become abundantly clear that the Israeli military was getting ready to create another offensive. In fact, by the time I think the
resumption of bombing
had taken place, the Israeli military had already implemented a suffocating siege on the Gaza Strip.
The
rise of malnutrition
was so widespread that the Famine Early Warning System program that exists had already been sounding the alarm — that this is very concerning, that starvation was being used as a
weapon of war
, and this is all happening as the fighter jets were being fueled up and getting ready to carry out this offensive.
Many people who were involved in the emergency medical teams that were entering into the Gaza Strip at the time, they had really no inclination in terms of, would this be a resumption of the war on Gaza and the people of Palestine, or would we somehow get this miraculous 11th hour extension of the ceasefire deal?
Many of us — me, Feroze, and Mark — at the time, were planning on going back in March. We were traveling to Jordan, where every single doctor who is an international or nurse, that’s where they go. And they wait in Jordan to get approval by the Israeli military, and then they will cross in from Jericho and make their way towards Gaza.
We’re sitting there in Jordan, and we’re there with the film crew. Mark and Feroze get approval, and they’re able to enter. Then ultimately, I receive a denial. And that’s something that’s happening in the backdrop, not just of this film, but that’s happening consistently when it comes to access into the Gaza Strip.
You are talking about everything from food, water, doctors, nurses, diesel fuel. All of these things are arbitrarily being denied and
restricted
, even people like myself who had been to Gaza many times since 2008. That was kind of the circumstances that we were facing: There was a “ceasefire,” and there was trucks maybe entering — but what was needed in the Gaza Strip was not being supplied. It was not happening on a scale that needed to happen. Once they started cutting that off and the Israelis implemented this siege, we started to see people really struggling.
JV:
Poh, I know as we said at the top, this film marks your debut as a feature documentary director. Just wondering, were you able to travel with doctors to document their work in Gaza? If so, could you talk a bit about that experience?
PT:
I knew very early on that it would be impossible for me to get into Gaza. Even though the Jewish doctor, Dr. Mark Perlmutter, he’s ever the eternal optimist. When we met early on, he’s like, “I’m going to train you as a scrub nurse and take you into Gaza.”
Now, we know who controls the borders into Palestine. And anybody doing a cursory search on my name, what are they going to find? Journalist for the New York Times, former commissioner for Al Jazeera English, and I’ve worked for The Associated Press. I was an executive for ABC Disney. It was never going to happen for me.
So fortunately, I have this incredible producer: Reem Haddad. Reem and I used to work at Al Jazeera English. She looked after MENA [Middle East and North Africa], that region. She was working at that time, this was I would say end of 2023, with this incredible Gazan team: our co-producer Mohammed Sawwaf, and our director of photography, Ibrahim Al-Otla. They had made this short doc at the end of 2023. And I was like, “Reem, can we work with this team? We are trying to film in Gaza.” That sort of marked how we got to film inside Nasser Medical Complex.
Honestly, this film is really to honor what they saw and what they are living through, not just what they are trying to document. So that’s how we were able to make this film.
I should also add: At the end of the day, too, this entire production, it spans teams, from our team, from eight different countries. Not one person, not one country could have made this happen. It took everybody coming together from different parts of the world who wanted to do something in regards to what was happening in Gaza.
I don’t know if you recall, but Thaer, you and I connected over text and voice notes in March of this year when I was writing about the Israeli government
delaying doctors from leaving Gaza
, which interrupts the entire process of care. And as you were explaining, that wasn’t the first time you were blocked from there. That was after the most recent ceasefire that was brokered in October of 2025.
From what you’re hearing from colleagues, how have things changed since when you were there during the first ceasefire? How are things the same?
TA:
The burden has really continued to devastate the healthcare system in general. So you’re hearing from colleagues every day. They
don’t have the materials that they need to serve their patients
. They don’t have the space that they need. They don’t have even the basic sort of guarantee that there is a cloak of security allowing them to operate in the traditional ways that they would like to.
I would say that probably the biggest sign that the healthcare system has been attacked in such a devastating way would be the
list of people who need to be evacuated out of Gaza
to get medical treatment. That list is anywhere from 15 to 18,000 people, depending on what list you’re using. These are people who urgently need to leave Gaza because the medical care that they need is not able to be provided because of the siege and the blockade that continues to exist by the Israeli military.
We’re talking about kids with cancers that are completely treatable that will die from their cancer because they’re not allowed to go to an East Jerusalem hospital like Al-Makassed or Augusta Victoria, where that corridor had existed prior to October 7, but the Israelis are intentionally shutting down that corridor.
You’re talking about physicians who would want to perform basic procedures for their patients with heart disease or diabetes or whatever it is. They’re not able to do so because that material is actively being blocked by the Israeli military. They’re on the back of a truck somewhere, either in Egypt or Jordan, and not being allowed to enter.
These conditions continue to proliferate, and this is all happening, I should say, while the public services that should exist in any society, like water, sewage, electricity, even shelter, all of those things are not — to say “adequate” is not the right word, but I would say it’s absolutely horrifying in Gaza.
We are talking about things like chickenpox breaking out in these camps. We’re talking about kids who continue to be malnourished. We’re talking about all of these different communicable diseases that run through these overcrowded shelters and tent cities. These continue to be the conditions that people are living in, despite the
fake ceasefire
that was announced that saw more than
1,200 Palestinians killed
since October 9, 2025, when it was first announced.
I’m even worried about what happens next because, as you probably have seen, Netanyahu announced that he’s
rejecting
that sort of ceasefire or the agreement to move on to the next phase that was brokered by this Board of Peace. And so I anticipate that conditions will worsen.
When we think about so many of the people that we worked with, that will appear in the film, that are in the background of the film, or even actually the cinematographers that Poh mentioned — I just don’t feel that they have the things that they need to sustain life in their homes, in their tents. Things continue to deteriorate, and it’s fallen out of the news cycle.
JV:
Thaer, there’s scenes in the film of you and your colleagues packing basic medical supplies like scrubs and rubber bands, and Mark even hides antibiotics in his luggage. The film does a really good job at just laying out that reality and this targeting of hospitals. It’s pretty wild to think that even the mere fact of
targeting of hospitals
was a
point of contention
among much of Western media at a certain point.
The film features quite a bit of footage from sit-down interviews between doctors featured in the film and broadcast media. There’s these subtle framing biases that I think, Poh, you highlighted really well.
There’s that clip in the beginning where a CNN anchor caveats Dr. Perlmutter’s accounts of Israel targeting civilians as “extreme allegations” that CNN cannot independently verify. Later in the film, also on CNN, Dana Bash tees up a question to Thaer about getting aid into Gaza by saying, “Will Hamas allow the critical aid to get to the civilian population?” — completely ignoring the
Israeli blockade on Gaza
. There’s that sit-down where Mark grills the CNN anchor live on air about not doing enough coverage.
Poh, I want to start with you. Could you speak to this bias, this molding of the narrative that
often benefits the Israeli government
, and why that was important to show in the film?
PT:
I’m so glad you asked this question. And I should say that long before I became a documentary filmmaker, I was a journalist. I’ve worked for so many of the established media organizations, and I’ve learned so much from my time there. But also, when the genocide started and there was pin-drop silence about the murder and execution of journalists, Gazan journalists, I’ll be honest with you: It hurt.
What happened? Why is it when journalists are kidnapped, taken in Russia, or killed in Ukraine or different parts of the world, all of a sudden, our peers, we come together. Why was it so that
Gazan journalists
or journalists in the West Bank or even Lebanon, for that matter — people weren’t rallying together. What was happening?
The whole film, I should say is a vérité in-scene film, so you’re in the world of Dr. Thaer, Dr. Mark, and Dr. Feroze. But in these clips, when we were filming them as they were being interviewed by media, I really wanted to show what they were up against.
As a former documentary commissioner for Al Jazeera, seeing my colleagues get targeted and executed was very, very, very hard. Now, as we were making the film, seeing the doctors trying to save lives and speak up for life, and then seeing them get pummeled like that, like basically the weight and structure of media crushing them — I was upset.
“Seeing the doctors trying to save lives and speak up for life, and then seeing them get pummeled like that, like basically the weight and structure of media crushing them — I was upset.”
Of course, I’m filming, I’m just observing. But it was very important to have that in the film because there are individuals, regular people — Thaer, Feroze, Mark — just trying to do all they can. And then there’s the weight of structure.
For everybody who’s trying to do something in this moment, it’s not like as individuals we have no agency. We do.
But it’s also important to remind people of the systems in place that make it very hard for us to do the right thing. If there was any path forward or a glimmer of inspiration, I felt, follow these doctors.
[Break]
JV:
Thaer, there’s another scene where you acknowledge that you, “Always get nervous when Mark talks during interviews,” because he really, you called it, gets into the weeds of things like Zionism, and you’re worried about essentially bad-faith critics making you, what you say is, guilty by association.
Could you say more about that, and how in this media framing, how a pro-peace, anti-genocide message can get twisted, and also how those critics may complicate that message even further because of your identity as Palestinian American?
TA:
Yeah, I think that’s something that we all have grown up with in terms of the Palestinian community. It’s understanding there’s a degree of dehumanization that is pervasive in not just the media, but in art and literature.
I remember being in medical school and having assigned reading before we started medical school, and then it was about a Hmong child who was dealing with a seizure disorder. And nestled in the middle of that book was a comment, a very brief passing comment, about a Palestinian who was a terrorist and was doing something extremely violent. Most med students in the country were assigned this reading. This is the backdrop of understanding this is how the world views you. Yes, of course, we reject it, but Palestinians know very well how they’re being portrayed and also what you’re stacked up against.
For me, I think something that I appreciate about Mark is that he understands there’s a privilege that he has, especially just how he looks, and that he can say things that I think can go farther than anything I could ever say. It can resonate with more Americans than anything I can ever do.
What I worry about, too, is the fact that you’ve got this conversation that the media and so many people try to make complicated. They want to say, “It’s complicated. There’s two sides.” Really what we’re trying to talk about is, “Hey, there’s an objective fact that something horrifying is unfolding here, and that really horrendous things are happening to this group of people by this institution. It’s a straight line. There’s not really, actually, so much dark and gray area here.”
The one thing I really wanted to make sure that we were communicating effectively — because again, so many people are working on this project, trying to get this film out there — I wanted to make sure that that message was as clear.
I would always worry knowing that Mark is somebody who shoots from the hip; he’s somebody that says what’s on his mind. Just in the back of my mind, just thinking about, every time we’re landing in Chicago O’Hare, every time you’re getting questioned by border control. All of these things were always at the top of mind for me. So it’s something I admire about Mark, but it’s also something that I know I just can’t do it myself. That’s not the world that we live in.
PT:
To add to that, and one of the things I have so much respect for Thaer, because it takes a leader, to be really honest with you, to know whose word can go far because of the context and of our time and how things are.
As a woman, as a person of color, as somebody who’s barely 5 feet — I am acutely aware of when I step into the room as a journalist, as a filmmaker, perception. But I can only imagine what it’s like for a Palestinian; I will
never
be able to understand.
So those are one of the things where I thought about a lot in the making of the film. It was something that I was learning from as well, especially from Thaer, who, I don’t mean to embarrass you, Thaer, but, I really look up to you in many ways in how you are able to navigate this, because I feel like I’ve tried it in my entire — I’m 42 this year — and I’ve been challenged in navigating it.
“ It takes a leader, to be really honest with you, to know whose word can go far because of the context and of our time and how things are.”
Seeing you being able to do that and work with Feroze and Mark, even though you’re so different — and they’re very different doctors, very different personalities — it gives me a lot of inspiration on how to work with others who are also different, and also to know when does one take a step back and let others go forward.
JV:
Thaer, did you have anything to add?
TA:
Something that I’ve noticed for Palestinians, one of the things that really helps us get going, especially motivates us, is seeing other people step up and use their platforms and really put their money where their mouth is.
So despite me having real concerns that this project would be stifled, and it would never be able to see the light of day because of all of the things working against it, when you see Poh and Reem and you see all of the people that worked on this just continue to push and continue to say, “No, we’re going to be here from the beginning to the end. We’re not going to abandon you” — I think for my community here in the States that felt like we’ve been silenced for so long, and, even when anybody wants to talk about this subject, that you don’t even want to bring a Palestinian or invite a Palestinian because it could be [viewed as] biased, which is absurd.
You talk about who shapes the narrative — we are prevented from having control
over our own narrative
because of this disturbing narrative and structure that was built. When you see that and other people who are willing to put their money or their life on the line like Poh or Mark or Feroze — for me, it really gives you that energy when you just feel like everything is stacked up against you, and there’s a genocide unfolding. That’s a really heavy thing.
JV:
I’m curious, Poh, how you initially contacted the subjects of the film there and others. How did that process unfold?
PT:
So I’ll tell you how it began. I chanced upon Mark. I saw him being interviewed, and a reporter had asked him, this was fall of 2025, this was last year, and a reporter had asked him, “So Dr. Perlmutter, you have just returned from Gaza. Why do you think there’s no ceasefire?” Mark, who doesn’t mince words, straight up, he says, “There’s no ceasefire because politicians are (expletive) to lobby groups.” And I was like, “Oh my, who is this person? Who is this doctor?”
He was actually speaking in New York a few days later, and I went to meet him. Then very soon after he introduced me to Thaer and Feroze, and I just wanted to see how these three people, who couldn’t be more different in many ways, were able to work together.
JV:
Thaer, you and your colleagues go to Washington to try and sway congressional members to act, and you all are met with indifference and platitudes, “dead ears,” as one person puts it.
TA:
It’s just been incredible to see how the shift has emerged over the last two and a half years. You’re talking about early on, I remember November 2023, I had a Zoom call with eight senators. It was with the members, eight of them, all Democrats, and I remember saying that we need a ceasefire.
This was when Shifa Hospital had first been
surrounded
, and there were 38 babies in the
neonatal intensive care unit
in the newborn nursery, and several of them were in incubators. I remember being desperate at that time and saying, “We really need a ceasefire.” One of the senators, who was a member of the Armed Services Committee, cut me off and said, “Ceasefire is a military term. We’re not prepared to call for a ceasefire at this point. We’re more interested in maybe helping secure humanitarian aid.”
It went from that to when the
images emerged
for when the
famine
was in
full-blown effect
, where you had several members of Congress and the Senate sort of speak very publicly. They were outraged by what they saw, despite us telling them this information very early on, this is where we were heading, this is what was happening unless they did something. So to see the rhetoric even change has been remarkable. I’ve never seen anything like this before.
Then to see progressive candidates who really explicitly say that our policy in Palestine has been an utter failure and has contributed to the development of a genocide, that’s been amazing to see how much traction that’s getting, how much money they’re able to raise, how these races have really flipped the establishment on its head.
But I will say there are a couple of things that concern me, which I think the timing of this film could not be more perfect. And that is, you’re still seeing the very core of whatever, I don’t even know what they’re moderate in, but like, these “moderates” on the Democratic side or on the Republican side, you still see them try to stick to the status quo and go back to the old ways.
There’s this refusal to acknowledge what most American people are interested in — and that is
money out of the Middle East in terms of military and wars
and into our communities here, into healthcare, and into some of the services that we are deprived of, that, for example, Israeli society is able to provide for their citizens: free education, free healthcare. We don’t even have that here, and yet we’re still providing a significant amount of aid.
You’re still seeing some people try to resist that, try to resist this natural popular movement that’s at play here. Again, which is why I hope that this film could be that vehicle for people to be able to continue to push and understand we need to be activated and we need to advocate.
So we’re continuing the fight, and we’re still trying to push this forward. But there’s still, even though the rhetoric has changed, the policy is still well ingrained and institutionalized in a really significant way that’s going to take a long time to reverse. So that’s something I would encourage people to think about.
Abdul won the primary
. Awesome. He’s going to be up for a huge fight in Michigan for that Senate race. There’s going to be a lot of money poured in against him. Mayor Mamdani was elected, and you can see how he is put under such a microscope for every single position or statement or even
whatever
his wife does. All of that is under a microscope.
There’s going to be this huge battle because it has been so ingrained in this country to support the Israeli occupation of Palestine as well as the Israeli military, but also to continue to fester and make the entire Middle East a very, I would say, a violent playground, for lack of a better term. That’s just how this country has faced things. That’s why the power of film, I hope, is something that can translate into action, and people can feel activated.
JV:
There’s this thing that Dr. Feroze says at the end of the film where he is at that conference for Jewish Voice for Peace, and he expresses doubt that any of his mainstream media hits have really done anything. Then this audience member from the conference says it’s done a lot, and he refuted that and encourages him to keep going. I don’t need you to talk for Feroze, but I’m wondering where you stand on that and that idea of awareness or maybe its limits of media and that we need to keep doing what we can. Do you wrestle with that still? Is that a battle for you?
PT:
I’ll say this. In embarking on this endeavor, I stopped thinking a long time ago about, what’s the outcome? Obviously, I hope that something good will come out of it. In the making of it, especially when we started, when it was really bleak, I told myself, “You know what? I’m just going to focus on doing.” I cannot fight against the structures, systemic structures, so many things, and I cannot be sorry for how I was or we cannot be sorry for our past actions or whatever it is, and we cannot predict the future. We have the power of now, and that is everything, and so just focus on the moment. Of course, think short-term, medium-term, long-term, but now is all we have, and so be true to oneself.
“I finally feel like I have found my voice, free of the constraints of everything. I can finally be.”
I feel like this film, in the making of this film, I finally feel like I have found my voice, free of the constraints of everything. I can finally be. I hope that this film, for those who are going to watch — and it’s going to be across theaters across the United States and Canada, which is massive, thanks to Watermelon Pictures — I hope that when people watch, they realize that too. Just focus on the now and keep going, because you don’t know how far you can go until you try.
JV:
We’re going to wrap it up there. Thank you both for joining us on The Intercept Briefing.
TA:
Thank you, appreciate you.
PT:
Thank you, Jonah.
JV:
We want to hear from you. Tell us what you’re following or want to see more coverage of. Email us at podcasts@theintercept.com, or leave us a voicemail at 530-POD-CAST, that’s 530-763-2278.
That does it for this episode.
This episode was produced by Laura Flynn. Ben Muessig is our editor-in-chief. Maia Hibbett is our managing editor.
Fei Liu is our product and design manager. Nara Shin is our copy editor. William Stanton mixed our show. Legal review by David Bralow.
Slip Stream provided our theme music.
This show and our reporting at The Intercept do not exist without you. Your donation, no matter the amount, makes a real difference. Keep our investigations free and fearless at
theintercept.com/join
.
And if you haven’t already, please subscribe to The Intercept Briefing wherever you listen to podcasts. Do leave us a rating or a review, it helps other listeners to find us.
Welcome to HTMLcat: small, native web tricks worth remembering.
HTML, CSS, and JavaScript can do more than we remember. Each post-it pairs one useful platform feature with a small example and the caveat that matters.
Some notes cover limited or experimental features. Check the support label, keep a fallback, and test with real browsers and assistive technology.
More Incidents of AIs Going Rogue in Cybersecurity Challenges
Schneier
www.schneier.com
2026-08-21 05:42:34
The AI Security Institute has a new report of AI systems engaging in “unsanctioned behavior”—what I have been calling “genie behavior—while being tested on their cybersecurity capabilities.
The incident stemmed from a single evaluation where agents were given a task of ...
The AI Security Institute has a
new report
of AI systems engaging in “unsanctioned behavior”—what I have been calling “
genie behavior
—while being tested on their cybersecurity capabilities.
The incident stemmed from a single evaluation where agents were given a task of solving a cyber security challenge. We ran this challenge 122 times across several models. Our investigation found that in 10 of those runs, an AI agent took autonomous, unsanctioned action on the live internet, targeting real people and organisations. In total, we catalogued 19 such actions. Almost all of this behaviour (17 actions) came from a single model, Anthropic’s Mythos 5, with 2 actions involving OpenAI’s GPT-5.6-Sol with cyber classifiers (mechanisms to prevent misuse) disabled. In the most serious case, an agent tried to insert malicious code into an open-source project. In an attempt to get the code approved, the agent engaged in social engineering—creating fake online identities and using them to pressure the project’s maintainer to approve the code. A human maintainer caught and refused to approve the malicious code.
[…]
Below, we highlight the four most significant behaviours observed. A full summary of cases is available in our
technical incident report
.
An attempted supply-chain attack on real open-source software. In the most serious sequence, an agent tried to insert malicious code into a publicly used open-source project and took actions in an attempt to secure approval for this insertion by human reviewers. The agent researched the project’s human maintainers, created multiple fake identities, and used the fake identities to socially engineer a real maintainer into approving the code. When the agent’s pull request was challenged in public, it edited its earlier activity to appear harmless and considered adopting a fresh identity to continue. The agent used Tor to bypass some network restrictions on GitHub, which is what first triggered AISI’s security alert.
Attempts to deceive and target real people. As part of the same effort, the agent tried to contact real people directly, sending messages and files through an online file-transfer service to persuade them, or their own AI coding tools, to run malicious code. Some messages carried harmful payloads, and some were attempts at social engineering; targeted at real people—something we’ve never previously observed.
Attempts to plant and prompt-inject malicious code. The agent tried to insert malicious instructions where it reasoned that other automated AI systems might pick them up and execute them. Prompt-injections are hidden instructions designed to manipulate AI coding assistants.
Collaboration between independent agents being assessed simultaneously. One agent left public messages on GitHub offering collaboration with other agents working on the same challenge. It also provided instructions to reuse accounts and artefacts it had left behind, which were discovered and used by subsequent agents.
What’s especially interesting about this technical report is that, unlike what we’ve been getting from OpenAI and Anthropic, we can see the exact prompt. It’s in Appendix B. And reading it, it seems that the models didn’t break any rules—they found loopholes in the rules. They behaved like a genie.
This Mysterious “Astroturf” Group Popped Up to Defend the Paramount Merger
Intercept
theintercept.com
2026-08-21 04:50:00
In text messages, the newly formed group boosted reported threats by CEO David Ellison to move Paramount out of California.
The post This Mysterious “Astroturf” Group Popped Up to Defend the Paramount Merger appeared first on The Intercept....
A state attorney
general challenging the controversial mega-merger between Paramount Skydance and Warner Bros. Discovery is under attack from a newly created mystery nonprofit accused of being an “astroturf” front group.
The group sent text messages in recent days calling on Californians to pressure Democratic state Attorney General Rob Bonta in support of the merger, which he has sued to block in court.
The text messages are the latest escalation of the increasingly bitter battle between Bonta and Paramount CEO David Ellison, who has
threatened to move his movie studio to Texas
or Tennessee
, according to reports from multiple outlets, if a coalition of state attorneys general led by Bonta keeps fighting the merger.
The group behind the text campaign, Neighbors for Strong Communities, cited Ellison’s threat in messages encouraging support for the merger.
“The behavior we’re seeing is what powerful corporations do when they are losing control of the story.”
The group denied the allegations that it is mounting an “astroturf” campaign, but declined to reveal its donors. The lack of transparency led free-speech advocates to issue a scathing press release this week.
“The behavior we’re seeing is what powerful corporations do when they are losing control of the story,” Rashad Robinson, co-chair of the Committee for the First Amendment, said in a statement. “The more the public understands what’s at stake — fewer jobs, fewer stories told, and even more power in the hands of a few billionaires — the harder the deal becomes for Paramount to defend.”
A spokesperson for Neighbors for Strong Communities, Tanner Kelly, declined to disclose the group’s donors.
“We don’t disclose funders, but this campaign is operated independently of Paramount,” he said.
In a statement to The Intercept, Warner Bros. Discovery denied funding the text message campaign. Paramount Skydance did not respond to a request for comment.
Neighbors and Nemeses
Ellison’s merger plans have
divided Hollywood unions
, leading proponents and opponents to engage in a heated war of words over who speaks for the entertainment industry’s working class.
Into that fray jumped Neighbors for Strong Communities, which was incorporated in Washington in June, a little over three months after Paramount launched its takeover bid for Warner Bros. Discovery.
The national debate over the mega merger has focused on suspicions that Paramount CEO David Ellison might be
willing
to
tilt coverage
at Warner properties like
CNN in favor of Donald Trump
. Ellison, whose
billionaire father
is a close ally of Trump’s, has courted favor with the White House as the merger moves forward.
The debate over the Paramount merger in California, however, has often centered on jobs.
In text messages to Californians that went out earlier this week, Neighbors for Strong Communities asked recipients to send Bonta messages raising the concern that his opposition to the merger will cost the state thousands of jobs — because of Ellison’s reported threat to move Paramount to Texas.
Those messages and the group’s generic website offer little insight into Neighbors for Strong Communities’s origins. Its incorporation papers in Washington show that several listed directors have worked as political consultants.
In a statement to The Intercept, one of those directors, Tanner Kelly, described the group as a “community advocacy organization that helps everyday people participate in public decisions affecting their lives, livelihoods and futures.”
Bonta is months away from an election he is expected to win handily. Kelly said the group’s pressure campaign against Bonta is “not related” to the pending attorney general election, and Neighbors for Strong Communities has not filed campaign finance disclosures with the California Secretary of State.
Kelly pushed back on the idea that entertainment industry professionals are uniformly opposed to the merger by sending messages from actors who have spoken out in favor of it.
“To the claims this is AstroTurf, tell that to the hundreds of real people participating in our movement who have real concerns, real struggles, and real livelihoods on the line that just want to be heard,” said Kelly, who was
registered as a lobbyist in California last year.
“Grassroots Opposition”?
Advocates who oppose the merger suspect there is something more at play, pointing to the circumstances of the group’s creation.
“The sender organization’s website is less than three months old. It discloses no founders, board members, staff, or funders, and the organization does not appear in ProPublica’s nonprofit database,” the Committee for the First Amendment, an anti-merger group co-founded by Jane Fonda,
said
in a press release earlier this week.
“The sender organization’s website is less than three months old.”
The Committee said that Neighbors for Strong Communities has the “classic hallmarks of a corporate-backed astroturf campaign designed to manufacture the appearance of grassroots opposition.”
The effort to make it seem like the merger has popular support will fail, predicted Jessica J. González, co-CEO of Free Press Action.
“These corporations know where the grassroots movement actually is: with the industry professionals fighting to block this merger,” she said. “This PR campaign is yet another face-off between the many and the money.”
One of the eternal schisms in programming is over the question of whether the
standard library should be minimal or encompassing. This is the wrong question
to ask. The right one is:
Which social architecture creates a high-quality standard library?
Python is always brought up as example of leaky batteries exploding in slow
motion, but this has nothing to do with
size
. The problem with Python’s stdlib
is its, ahem, uneven quality. Some standard library modules don’t follow
language naming conventions! You know which
unittest
module I am talking about
:-)
But even that is not a mistake. It’s actually Python core’s advantage — that
it makes functionality available early, not thinking about the future too much.
That’s how we ended up with ossified cAPI which makes CPython the language, but
that is
also
how we ended up with Python powering data scientific revolution.
The Go standard library is similarly encompassing, but it is held in a high
regard. Go team has institutional capacity to deliver well-designed API for the
standard library, and then some:
https://pkg.go.dev/golang.org/x
Rust is an interesting case. The 1.0 standard library APIs are brilliant.
Collections and iterators are a work of art. But it also feels that, while the
current team has the capacity to preserve existing APIs and fill in
some
gaps,
the capacity to execute design decisions is limited. While
golang.org/x
captures excess capacity,
rust-lang-nursery
is a graveyard. Maybe I am
over-indexing on
my favorite hobby-horse
,
but it seems that the reason for Rust not having an API to get a stream of
random bytes from the OS in 2026 is that, while it is an easy technical problem,
it requires tricky organization architecture (including getting money in
peoples’ pockets, of course) to actually get solved in the high-stakes
environment of a world-wide coordination problem called a programming language.
When Microprose greenlit
Sid Meier's Pirates!
in 1986, the company—which Meier founded along with Bill Stealey back in 1982—was mostly known for vehicle sims (
Gunship, Spitfire Ace, F-15 Strike Eagle
) and dry strategic wargames like
Crusade in Europe
.
Pirates
was something different. It's hard to pin it down to a genre even today, but it certainly wasn't like previous Meier titles—even though it's the first game to be called
Sid Meier's Something or Other.
At the time the game came out in 1987, it was generally called an "action adventure" game; this is somewhat hard to square with the way most people today would understand the genre. It has basically nothing in common with contemporaries like
Castlevania, Metroid,
or
The Legend of Zelda.
There's no platforming of any kind; controlling the player character directly and individually is restricted to brief sword-fighting sequences. Those have controls and mechanics that are totally unlike any other game combat I've seen. It tries to create the feeling of an Errol Flynn fencing duel with an elaborate control scheme that's more like a mutant version of a fighting game; you can thrust (which is fast) or slash (which does more damage), or parry; you can also either aim low, or high, or down the middle.
Those motions are mapped to the eight-direction joystick commonly used for games on the Commodore 64 and other eighties computers—or, if you don't have one of those, they're mapped to the keyboard numpad. Some ports of the game let you do it with the mouse, or with awkward combinations of mouse clicks and keyboard input.
This description makes the swordfighting in
Pirates
sound terrible and indeed it is hard to defend; the 2004 remake substantially simplified it, but that simplification only peels away its skin to reveal a strange quasi-rhythm game buried underneath. It still feels in no way like a concession to "standard" combat design.
But what you have to understand about playing this game—whether the original 1987 version, the many 1988 ports, the beautiful 1993 remaster
Pirates! Gold
with its 256-color graphics, or the modernized 2004 remake—is that if you encounter it at an early age it will open a rift in your brain. When I asked Nic Tringali (designer of
The Banished Vault
and
Amberspire
) about it, they put it perfectly: "
Pirates!
feels unstuck in genre, not quite an open-world game or a role-playing game, not only economic strategy or resource management. It has a goal for a clear thematic position for the player to inhabit and uses the systems and friction to reach that. The peculiarities of its design are unique and inseparable—the wind always blowing east comes to mind, and dueling a rival captain immediately winning a sea battle—and are not solely driven by video game genre expectations, or pre-packaged ideas redressed in a pirate theme."
Meier was working in an era where genre and mechanics were still unsettled, undefined things. In his 2020 memoir—would you believe it's called
Sid Meier's Memoir!
—Meier pointed out that "The good news was there were very few preconceived notions back then about what a game was supposed to be. The bad news was there were no tried-and-true conventions, either." All of
Pirates
is like this; a bunch of first-principles attempts at extricating game mechanics from all the half-remembered pirate tropes rattling inside Meier's brain.
The resulting game takes all these romanticized ideas pointing towards a genre—
Treasure Island,
Errol Flynn movies,
Peter Pan,
the centuries-old distillation of the Black Legend into anglophone culture—and treats them not as story beats to play out or as window dressing, but as the grounding rules of a clockwork world that you can poke and prod at. The game models the way silver travels on mule trains all the way from Potosí to be loaded up on ships in Panama, and from there along the ports of Gran Colombia until the Treasure Fleet, heavy with the blood of the Americas, leaves for Spain. The game models the way that a pirate's harsh life wears you down over time, each merchant ship captain or colonial guard that you fight seeming that little bit faster until you have to admit that your sword arm just isn't what it used to be. The game seeds the Caribbean with an elaborate, randomized quest to find your long-lost family, chasing down a laundry list of villainous aristocrats to rescue relative after relative from indentured servitude on obscure plantations.
To modern design sensibilities, I think there's a risk one might look at
Pirates
and see a bunch of minigames in a trenchcoat. In reality, what that game is expressing is a way of thinking about games that we've tamped down over the years as the medium has built up its own library of conventions, tropes, and recycled ideas.
Pirates
is hewn from its underlying themes in a very raw way, but so are many other games of this era. Cinemaware, a now largely forgotten studio, made a whole very successful business out of this style of design. Their 1989 title,
It Came from the Desert
, gleefully jumps around in presentation and perspective—one moment a rudimentary first-person shooting gallery, the next a top-down shmup, all wrapped up in a visual novel wearing the skin of a strategy game. The original
Dune
video game—not
Dune II,
the origin point of real-time strategy as we know it—was essentially an attempt at capturing the whole scope of the 1984 David Lynch movie, oscillating between a strategy game about spice extraction and a visual novel-like adaptation of the movie's plot.
Pirates
was immensely successful in its own time, and this exact style of design thinking was then adapted to Microprose's follow-ups—classics like
Covert Action
and
Sword of the Samurai.
Pirates
essentially changed the studio's whole identity, something that was incredibly hard to do even in those days.
And yet, this entire era of video games feels like a hole that has been carved out of the collective memory of the medium. It is, in truth, a reflection of how niche the computer game market really was in those days; for every Amiga 500 sold, Nintendo sold 23 NESs and 45 Game Boys. The era of IBM PC consolidation and true mass adoption of PC gaming was a few years away in the late 1980s; and by the time the remade DOS version of
Pirates
arrived in 1993, it was competing with platform-defining games like
Wolfenstein 3D.
This 1982-1990 era of PC gaming is, really, better known through dubious CDs packed in with magazines, discount-bin collections put out by failing publishers, and abandonware sites.
Signing up is free!
By signing up—again, it costs nothing!—you can read the rest of "The Lost Treasure of Sid Meier's Pirates!," and receive free newsletters and emailed articles from Remap!
There are a few mobile apps that I’ve wanted for a while, and that are relatively
speaking just for me.
Toucan Music showing artwork
Places
- When people recommend restaurants or cafes to me, I’ve for a long time put them in
a Google Maps list. It’s annoying in various ways, for example the page zooms out when you
select the list. And I don’t like keeping my data locked into a free service.
People
- I’m face blind, and struggle to remember people I haven’t met quite a few times.
This is especially noticeable networking in a metropolis. The idea is to help me
remember who people are, and train me with spaced repetition to learn their faces.
Music
- This one I made impulsively in the last week. Like many others, I’ve
long wanted to
organise
albums in folders
on Spotify. After a quest to switch back to buying MP3s, which failed
because it got too complicated, I made a custom Spotify player instead.
So over the last month or so I vibe coded
Toucan
(choose “local only” to just play with it, any data you put in will stay in your browser).
For what it is worth, the
source code is here
.
But I’m not really proud enough of it to say it is a “release” - more on that below.
Architecture
The apps I describe above need to work on mobile and sync data to web to use them on
desktop. The usual way of doing this would be to make your own personal
software-as-a-service with its own web server and database, and then also a mobile app.
This felt a bit much - clunky, excessive and not very scalable. I’ve
long been a fan
of what is now called the
local-first software
movement.
The idea is that data is primarily on each device, with a standardised sync server to get it
between them. This makes operations extremely fast, as they just act locally. Syncing
happens in the background. Think something a bit like Dropbox, but for data in databases.
Some of the data is quite private - particularly people’s faces, but even their names on
restaurant recommendations. So I prefer syncing to my own server hosted at an
excellent local ISP
.
I was disappointed to find there isn’t standard personal data sync server software - I was
hoping for something at least as modestly popular as
Nextcloud
.
Which you likely haven’t heard of, but is a self-hosted file syncing server (and more).
In the end Fable and I went for
Yjs
which is the most popular
local-first protocol. I wanted a simple server that just writes to a SQLite database (so I
could do hosting and backups very easily). Fable found the obscure
Hocuspocus
, which is simple and does
the job (alas it turned out its format is opaque and binary inside the database file, but
that’s another story).
I spent a while wondering if I could
write the UI in Rust
but
in the end went for simple and made a Javascript
progressive web app (PWA)
.
When installed on my Android phone (via Chrome, for some reason it doesn’t work in Firefox), it is just as good as an actual app. The web version and the mobile version are the same code
and work just the same. Even with LLM coding agents, that saves a lot of hassle.
Basic process
Toucan Places at a coffee shop
I did most of this on the £18/month Claude plan, using mostly Opus. Fable helped with a
bunch of initial planning, but I’m not sure it made a lot of difference. Several of
those decisions were bad anyway, and were refactored later.
My processes are commonplace by this point in LLM coding. There’s a
plans/
directory
where anything large gets designed first. I edit it, make decisions, then clear the
context window and tell the agent to implement the plan.
If something is a significant UX or design change, I ask Claude to make an artefact with
different options for the design in it. These are surprisingly high quality, especially
after a few iterations.
I manage bugs and tasks in a simple
to-do file
.
The agent ticks them off when it has done them, then I QA them and delete them.
Most of these tasks are under a heading “Polish”. That’s because I have to give a
lot
of
product and UX detailed feedback. I take note of them while I’m using the apps as I go
about my day. Dozens and dozens of items for each app. This is a major reason I think this
is hard to scale to anyone right now - see below.
I’m stubbornly not upgrading to a more expensive AI coding plan. I’ve lots of other things
to do, and quite liked being stopped by the 5 hour window, as this is
addictive
.
Later on I learnt you can type
!sleep 3h
(or whatever) when you run out of tokens, and
press Ctrl+B to background it (otherwise you get a 2 minute timeout). Claude will wake up
hours later, and carry on with its limits reset, even if I’m not around.
Making it do a good job
Bespoke snapshot testing tool
It’s the case with LLM coding that things which have long been good practice become even
more valuable. Such as strictly configured type checking and 100% test coverage - see
Jonathan Lange’s
Galahad Principle
for why “100” is especially magical.
These give the agents basic feedback loops, which they are now reasonably good at
responding sensibly to. They won’t, however, set up things like that if you don’t ask them to. Yet.
I managed and coded on a
front-end development team
for some years, and although we did lots of good things, we never quite got to snapshot
testing. And I’ve always wanted to do it.
So, quite early on, Claude created a tool for Toucan that uses a headless browser to take a screenshot of
every page of the application. This is really good, partly as an integration test, partly
so it can check designs, and partly so I can QA designs. There’s a web view for me (JSON
for the LLM), an integrated pixel diff, lots of filter and view options, basic performance
measurements, and command line switches to snapshot against another branch.
This was invaluable.
The end result is that almost anything I ask the agent to do … It just does. It
usually makes some aesthetic choices I don’t like, or messes up part of the UX.
Sometimes I have to argue with it about data structures, or choices of where in the
system to cache things.
But generally, it codes the thing. It doesn’t break anything else. The existing tests
pass, new ones are added. It is working at the level of a super fast senior software
engineer, albeit one with poor contextual awareness. For that
wait until late 2027 or 2028
.
Last year, I didn’t in my soul think it’d get this good at coding.
I had one serious bug. I’d never tested a new feature to reorder lists, and pressed
it on my phone while out. It deleted the list completely and everything in it! The bug
was not doing the array manipulation correctly for the local first storage engine.
The agent helped me recover the data, add various logging features to be able to
see what is happening with syncing, and update documentation to make it clear not to
make the same mistake again.
Finally, following
Jyn
’s advice, I added a self improvement
feedback loop. My version is fairly dumb - a prompt in
AGENTS.md
to tell it to write
anything that caused it difficulty into
SELF-IMPROVE.md
and
keep a count of how often the same problem shows up. I then look at that, see what
makes sense, and schedule improvements.
This found things like visually unstable snapshots, churn in
node_modules
that it would
try and work round repeatedly, performance problems, missing tools and so on. It feels
very primitive, but I expect we’ll all spend a lot of time supervising this kind of
thing in the coming years. “
AI developer
experience
”.
Refactoring
Ideology for international standards and simplicity drove me to try and use web components
and plain Javascript. I was hoping the Toucan platform could be itself so powerful, that
the apps would be quite short, single HTML files. It didn’t work out, and eventually I
realised that the code was getting messy, and that build steps are cheap and easy with an
LLM anyway, so we did two large refactors.
One was to Typescript, the other to Preact (a lighter weight React). They both went well -
the snapshots helped make sure nothing broke, and nothing indeed broke. Everything pixel
perfect the same after refactoring. I haven’t, though, spent much time manually checking
code quality - see next section.
The initial graphical design was quite ropey. It took a lot of forcefulness to get it to
upgrade it, but it was possible. I had to force it to make shared components properly.
Someone with more design skills than me could make it really good. Compared though to what
I would make by myself, excellent.
Future
Three full on mobile apps, with desktop versions. It’s frankly amazing it is possible to
make them with such little effort, and so polished.
They’re dangerously close to something other people could use. If I’d written these 5
years ago, I’d definitely be marketing them and trying to get users.
Right now though that feels slightly … exhausting? They’re a bit
too
vibe coded to
me
. A touch eccentric to install and set up. Slightly too specific in what they do.
The open question for me is, how can big collaborative open source projects be run in a
world of agents coding? What I really want is to make it easy for anyone to make their own
custom, stateful, syncing apps. This feels tricky for the following reasons:
UX and product feedback is a skill, and still hard to do. Expectations on mobile are
high, and I think few people will go through the feedback necessary.
Frameworks for cross-platform apps are still quite clunky. The widget sets that look good
on web and mobile are limited, PWAs aren’t familiar to end users, coding multiple
apps is specialist and hard to deploy.
I don’t truly feel like I own the product. This would be less of a worry if I was
running a business. But I haven’t closely scrutinised the code - I didn’t need to. What
are people’s expectations for open source in this world? How do we communicate it is
something of quality that will be maintained? I’m so used to the code being a key
thing being shipped in open source.
Local-first doesn’t have a standardised platform. I can imagine a world where it was
as normal as having an email address, to have a personal data store that can sync local
first. But it isn’t. It isn’t clear how to do the work that might make that happen in
2026 - just developing it isn’t a strong status signal any more.
How do you think large, open projects will be developed with LLMs, such that they really
benefit users?
We Rebuilt the Linux MicroVM Stack on Apple Silicon
(AI disclosure: I wrote every sentence of this myself—including the em
dashes.)
My favorite feature in Dart 3.13 is
primary constructors
.
Getting there took a lot of time and iteration before the language team had a
design we felt was solid. Since many of you have been patiently waiting for
this feature, I thought it would be worth writing about some of the challenges
we worked through to bring this large syntax change to Dart.
Users have been asking for something like primary constructors for years.
It's a highly desired feature, which is kind of strange when you think about
it. Primary constructors don't let you do anything you can't already do in
Dart. They're just a different—hopefully better!—syntax for what you can
already express.
In the 1960s, Peter Landin coined the term "syntactic sugaring" to refer to
layering some textual niceties on top of a more fundamental but unpleasant
language. Today, we tend to use the term more like a noun and call features
like these "syntactic sugar".
The immortal enemy of every programming language is complexity.
Even the tiniest feature must be designed,
specified, implemented, tested, and documented.
The cost is large. I think of complexity in a language like weight in an
airplane. Some amount of it is necessary for the thing to work,
but you have to be careful to not add weight unnecessarily or risk the whole
apparatus not getting off the ground.
From that angle, syntactic sugar seems like a bad idea.
It's additional complexity with no additional utility.
Even worse, once we add it, we pass complexity onto our users too.
Now they have to choose which syntax to use each time they are trying to
express something.
When are these kinds of features ever a good idea?
(I admit I feel some need to justify this because so much of my work over the
past several years has been adding these kinds of features to Dart.)
I think syntactic sugar can carry its weight in a couple of ways:
Despite our somewhat robotic affect and fondness for
EBNF
, we
language designers are human and make mistakes.
Further, we are always learning, the ecosystem we serve is constantly
discovering new ways to make software,
and user expectations drift over time.
When Dart was first designed, you had to use an explicit
new
keyword to call
a constructor. This was deliberate to be familiar to users coming from C++,
Java, JavaScript, and other languages.
The intent was to make it clearer in the code when a call allocates a new
object. As garbage collectors got better and users got more comfortable with
automatic memory management, most users found
new
to be more noise than
signal.
(Also, honestly, Dart has always undermined that signal by supporting
factory constructors
.
A factory constructor can return some previously created object even when you
invoke it with
new
.)
In Dart 2.0, we shipped a language change that allowed you to omit the
new
keyword (and
const
in many places) when calling a constructor.
We still support the old syntax, so this language change is essentially
syntactic sugar, but we really only kept the old syntax around for backwards
compatibility.
We always want you to use the new shorter syntax.
We shipped
tooling
to automatically remove
the unnecessary
new
keywords, and have
a lint
that reminds you
when you forget. The old syntax is effectively deprecated and over time you
see it less and less. If you're new to Dart,
you may not have even realized we supported using
new
in constructor calls.
That means the complexity for supporting constructor calls both with and
without
new
is low. There is a transition cost for existing users to learn
the new syntax. But new users will mostly just learn the new way and never
encounter the old. There's little cognitive load when choosing between the two
syntaxes because you simply always use the new one (and our tools will gently
remind you if you don't).
Short of having a time machine to go back and do it right the first time,
this is the next best thing we can do to fix a mistake in the language.
The syntax can be much better for a common use case
Old Java heads will remember this as Josh Bloch's "typesafe enum pattern".
Under the hood, this more verbose class declaration does almost exactly the
same thing as an enum declaration in Dart today.
Dart enum declarations are almost entirely sugar.
(I say "almost" because enum declarations give you
exhaustiveness checks
in switches.)
However, as you can see from these two examples,
enum declarations are
really nice
sugar.
A simple enum declaration unpacks to a lot of Dart code.
Now, if almost no one was writing enumerated types,
then it might still not be worth adding syntax to optimize for this use case.
But in a language that prioritizes type safety and data validation,
enums are quite common. The Flutter framework alone defines dozens of them.
A relatively small amount of syntactic sugar can sometimes make a
lot
of
user code shorter and simpler.
The previous section makes it sound like brevity is the whole point.
I suppose in a world where we are increasingly paying AI agents per-token
costs to read and write code there is a direct financial incentive.
But it's not just about character count.
Consider:
Is this an enumerated type? By that,
I really mean
enumerated
: Should someone using this class assume that the
only
instances of
Color
they will have to worry about are
red
,
blue
, or
yellow
?
Note that the constructor
is
public,
so other libraries are free to invoke the constructor and create other colors.
Is the intent of this class to be a
closed
list of colors,
or an open factory of them with a handful of pre-defined values?
Reading the code,
we don't know.
The code is a lot of machinery that defines
a type and some constants. It
looks like
the code you'd write if you did
want an enum, but the machinery doesn't reveal the intent.
The code tells the compiler what the code means,
but it doesn't tell a reader how to use it.
If we change this to an enum declaration,
then the policy that it's a closed set of values becomes obvious.
(And, now that Dart has real enums,
choosing to
not
change this code to an enum declaration likely sends a
signal that it's
not
a closed set.)
For me, this is a compelling reason to add syntactic sugar.
Code is written and executed as syntax,
but what every user working with the code cares about is what it
means
—its
semantics. To maintain code correctly,
we need to understand its intentions and policy.
This is increasingly true in a world where AI is often generating code faster
than we have time to diligently review it.
Even when it's possible to make the compiler do what you want by cobbling
together the machinery of several existing language features,
it can be worth it to have syntactic sugar that yields the same behavior
because better syntax raises that behavior into a higher level of abstraction
where the intended semantics are more obvious.
That can
reduce
the cognitive work required to understand the code even
though the entire language is more complex.
Right, I'm supposed to be talking about primary constructors,
not enums and
new
keywords. (Though—foreshadowing!—I will be talking about
new
too.) For many years, the
#1 open issue
on the Dart
language repo has been a feature request for data classes.
If you don't know, data classes are
a feature in Kotlin
that lets
you define a class with some fields,
and the compiler gives you equality,
hash code, and some other stuff for free.
If you read through the hundreds of comments on that issue,
you'll see that most users are less interested in the value semantics part—the
equality and hash code bits. It's mostly about having an easier way to define
a class that has a constructor and stores some state.
That functionality actually comes from a different,
more fundamental feature in Kotlin:
primary constructors
.
I believe Kotlin got this idea from
Scala
.
Since then,
C#
and
Java
have added their own takes on the
concept.
You can define a constructor by writing a parameter list right inside the
class header. That avoids needing to write a keyword or repeat the class name
to declare the constructor. It also avoids two levels of nesting and
indentation, one for the class body and one for the constructor parameter
list, in very simple classes that only contain some state.
Inside that parameter list, you can indicate that some parameters should
declare corresponding instance fields that are automatically initialized from
the parameter.
Without primary constructors or any other kind of syntactic sugar,
we have to do something like this in Dart:
In this example, we had to write the class name twice.
For each bit of state, we wrote its type twice and its name
four
times.
It's not too heinous in this example because there are only two fields and the
names are all short. Once you start dealing with complex domain-specific stuff
with long names and piles of state,
it gets ugly.
This is not a new problem, and Dart has long had a bit of syntactic sugar
called "
initializing
formals
"
to help:
Using
this.
on constructor parameters means you only have to write each
field's type once and name twice. Better!
But you still have to write the class name twice and each field's name twice.
Initializing formals are nice, but users still tell us they don't go far
enough.
So some users coming to Dart from another language tell us that they miss a
feature. What do we do with that kind of feedback?
Personally, I like borrowing features from other languages.
The creators of those languages have already put a lot of work into designing
and validating the feature. We can learn a lot from them,
and that other language is an existence proof that the feature is conceptually
coherent and tractable to implement.
Taking inspiration from other languages can also make our language easier to
learn. Unless a user is completely new to programming,
they aren't learning Dart from scratch.
They come to us with all that they have already learned from other languages.
What remains for them to learn is the
difference
between what they know and
what Dart contains. When we borrow syntax and semantics from other languages,
we reduce the size of that difference and lower the effort to learn Dart.
This philosophy has been key to Dart's success.
From little semicolons all the way up to classes,
Dart was designed through and through to be familiar and easy to learn for
users of other mainstream languages like JavaScript,
Java, and C#.
At the same time, good language design is contextual and holistic.
"What's a good pair of shoes?" has very different answers when you are
standing on the arctic tundra versus a Hawaiian beach.
A language feature that works beautifully in,
say, Rust might not slot gracefully into Dart with its distinct syntax,
semantics, history, user base, and ecosystem.
I don't want Dart to feel like Frankenstein's monster stitched together from
body parts ripped off of other languages.
Thus, when the Dart language team looks at features from other languages,
we're simultaneously looking at how the feature solves problems in that
language's context and also at how well that context matches Dart's own.
We knew users wanted a nicer notation to define a class that initializes some
fields from constructor parameters.
With primary constructors, you write the constructor and the compiler
synthesizes the fields. A language could also go the other way.
You write the field declarations and the compiler gives you the constructor
for free. Swift does that with
memberwise initializers
.
A challenge any time your language derives two declarations from one piece of
syntax is that one syntax needs to handle all of the various ways you might
configure both of those declarations.
In our case here, the instance field may be final or not.
It might have metadata like
@override
or doc comments on it.
The constructor can be named or unnamed,
const
or not. A constructor parameter can be positional or named,
optional or required. If it's optional,
it might need to specify a default value.
We spent some time investigating
inferring a constructor from field declarations
,
but eventually decided that parameters were the more useful declaration for a
user to hand-author. Since the constructor is often public API,
it's important to control the signature fully:
the constructor's name and
const
-ness,
which parameters are named or positional,
the order of the positional ones, and their default values.
In order to infer an instance field from a constructor parameter,
the only missing piece a user needs to provide is whether the field should be
final. It's fairly natural to allow a leading
final
or
var
on the
parameter to control that. The absence of both modifiers then means the
parameter doesn't declare an instance field at all.
That's similar to what Scala and Kotlin do with
val
and
var
.
The result in Dart looks like this:
dart
classPoint(finalintx,finalinty,);
(Since primary constructors make empty class bodies more common,
we also now allow you to use
;
instead of
{}
for an empty class body.)
This looks pretty nice, but what if the primary constructor also needs a body
or an initializer list? One option is to simply say,
"Well, in that case, don't use a primary constructor."
Syntactic sugar often takes a subset of use cases and offers more concise
syntax for them. If you fall outside of that subset,
it's reasonable to require the user to fall back to the older,
more elaborate syntax.
That's the right call in some cases.
But the language team is very mindful that code evolves over time.
Let's say you're writing a class. It starts off simple with just a few fields
initialized from constructor parameters:
A perfect use case for a primary constructor.
Later you add some more fields and parameters.
Great. Before long, you have a constructor with a bunch of parameters
declaring fields:
Then one day you decide you want to do a little logging in the constructor
body. If primary constructors didn't support bodies,
then you would have to convert that entire primary constructor into an in-body
constructor:
That's doable. The Dart SDK includes lots of quick-fix tooling that can do
these exact kinds of changes for you with a click of a button,
so it's not
mechanically
hard to change the code.
But it's still a large textual change.
You just wanted to add a line of logging and now you have 20 lines of changes
to look at.
On the language team, we call this a "syntactic cliff".
You want to make a small
semantic
change (here,
adding a line of logging), but what you want to express is just slightly
outside the bounds of what the optimized syntax supports.
You fall off the nice plateau of that syntax and land on the more verbose
terrain below.
It doesn't feel good when that happens.
You're trying to freely explore the semantic space of your program,
but it feels like some small steps in meaning are just out of reach in terms
of syntax. When we're designing language features,
we spend a lot of time talking about these kinds of cliffs and trying to avoid
them when we can.
We want the language to feel like smooth terrain where small semantic changes
only require equally small textual ones.
When you are doing a code review, we want the changed lines to reflect the
behavioral changes in the program, and not meaningless lateral moves through
the language's grammar.
The initializer block gives you a place to fill in a body or initializer list
for the primary constructor. It's also a natural place to add a doc comment
for the constructor. (If you put the doc comment above the class header,
it applies to the entire class, not just the primary constructor.)
These initializer blocks are sort of syntactic sugar on top of syntactic
sugar. We don't need them, but they help prevent users from falling off a
syntactic cliff. You can start with a primary constructor while your class is
simple, and the language never forces you out of that choice as your class
evolves.
Now, even though
the language
won't force you to turn your primary
constructor into an in-body constructor,
you
might still want to define a constructor inside the class body.
Classes can have a lot of things going on in the class header:
type parameters, an
extends
clause,
mixins in a
with
clause, and maybe
implements
too.
The constructor parameters might have doc comments.
It can get cluttered and messy up there.
Or you might have a class with multiple constructors where none of them is
clearly more "primary" than the others.
(When you have a primary constructor,
all other non-factory constructors for the class must redirect to it.)
Alternatively, maybe the most fundamental constructor is private,
and you think it looks confusing to put a private constructor in the highly
visible class header.
For these reasons and more, Dart still supports constructors declared inside
the class body. We don't think of primary constructors as
inherently superior
to
in-body constructors, just different and better suited to certain use
cases.
However, only a primary constructor has access to the
var
and
final
syntactic sugar on a parameter that implicitly declares an instance field and
initializes it from that parameter.
Those two features—declaring a constructor in the class header and constructor
parameters that induce fields—are bundled together.
You can't use the latter without the former.
In general, we try hard with the language to not put the user in a position
where they want only one of two behaviors but the language ties them together
and makes them take both. For example,
when we added
class modifiers
, we
deliberately added both
final
and
sealed
.
They are quite similar, but
final
lets you prevent subclassing
without
also opting in to exhaustiveness checking.
Deciding what to bundle is a balancing act.
I believe a big part of what gives each programming language its character and
fitness for certain domains is how it chooses to map semantics onto syntax.
Part of that is where it hangs multiple behaviors off a single piece of text.
For example, in most object-oriented languages,
making a class a subclass of another also makes it a subtype in the static
type system. That's not strictly necessary,
as private inheritance in C++ shows.
But for most object-oriented languages,
that coupling seems to make sense.
It might seem ideal to have a strict one-to-one mapping of behavior to text,
but bundling behavior can make it easier to express common patterns.
Think about how much easier it is to walk into a burger joint and say "I'll
have a #2," instead of, "a double cheeseburger with mustard but no ketchup,
medium fries, and a medium fountain drink."
When it comes to combining declaring parameters with primary constructors,
this felt like a relatively safe bundling.
Declaring parameters are themselves syntactic sugar and don't let you express
anything you can't already express in a normal class declaration.
If you don't want a constructor to be a primary constructor for whatever
reason, you have to give up the syntactic convenience of declaring parameters.
But brevity is all you give up. You can still define your class with exactly
the API and semantics you want.
It would be nice to be able to use declaring parameters inside in-body
constructors, and we worked on
a proposal
to allow it. Ultimately, we felt there were too many negative consequences.
If some random constructor anywhere in the class body can implicitly declare
instance fields in its parameter list,
then it gets harder to find all the state a class stores and reason about it.
This is less of a problem with primary constructors because the primary
constructor is always right there at the top of the class.
Another source of verbosity with Dart's existing constructor declaration
syntax is having to repeat the class name.
It doesn't look too bad with short names in examples like
Point
,
but when you have a class name like,
say,
AnimatedFractionallySizedBox
,
then repeating that entire 28-character identifier takes up a lot of space
that could otherwise be spent on the constructor's parameter list.
That problem we
can
fix. Dart's constructor syntax was inherited from Java
and C#, which in turn inherited it from C++.
Bjarne Stroustrup chose to use the class name to minimize the number of new
keywords and mirror what a constructor call looks like.
It's a cute syntax, but even he admits "this may have been overly clever."
Having one part of a language's syntax mirror another part can be a useful
tool to help users understand what the code means.
When two pieces of code look the same,
it sends a signal that they probably relate to each other in some way.
And if declarations look like call sites,
then once you've read the declaration,
you know what to write to call it.
At least, that's the idea. But Dart already fails to follow through on that
principle. It kind of works for function and method declarations if you ignore
the type annotations. Getters are declared using a
get
keyword without being
invoked using one. Operators are declared using a special
operator
keyword
and have the right-hand parameter in parentheses even though you don't need
parentheses to call the operator. Even in functions,
Dart uses
{...}
to declare named parameters,
which looks nothing like how you pass them at the call site.
Most other object-oriented languages don't use the class name to define a
constructor. Swift, Ruby, and Objective-C use
init()
.
Python sprinkles on some underscores and does
__init__()
.
JavaScript, TypeScript, and Kotlin use
constructor
.
PHP uses
__construct
.
This led us to conclude that repeating the class name for a constructor wasn't
really buying us much in terms of familiarity or consistency.
And there is a cost users must pay.
Names are often verbose, and every human,
template processor, code generator,
AI agent producing code, or future metaprogramming feature that wants to
inject a constructor into the class needs to know to use the class name.
It's almost like each class has its own special little contextual keyword.
That problem is particularly acute in another feature we're working on right
now:
static extension members
.
Extensions in Dart allow you to attach instance members to existing types,
but they don't currently let you add static members or constructors.
We'd like to support those too, but it raises a tricky edge case.
Extensions can be defined not just on classes,
but any static type, including typedefs.
Consider:
If you want to add a constructor in that extension,
what name do you use:
OtherName
or
SomeClass
?
Keep in mind that from the type system's perspective,
those are the exact same type. There is no class named
OtherName
anywhere in
the program. The typedef isn't creating a new named type,
it's just defining an ephemeral alias that can be used to refer to some other
type.
We could require you to use
OtherName
because that's the name that you wrote
at the top of the extension declaration.
But that violates the principle that a type can be replaced with a typedef
that refers to the same type without breaking anything.
Usually, swapping out a reference to some type with a typedef is a transparent
change. Here, and only here, the typedef name would become significant.
Or we could go the other way and say you have to use the name of the
underlying class that the typedef resolves to.
But that breaks the encapsulation of the typedef.
The typedef could be in another library and refer to a private class whose
name you can't even access in your library!
All the complexity here is a problem we created ourselves by using the class
name as a magic identifier to mean "constructor".
If we just pick a universal keyword,
the problem goes away.
So that's what we did. In Dart 3.13,
you can use
new
instead of the class name to declare a non-factory
constructor, and
factory
to declare a factory constructor.
If you want a named constructor, put the name after the
new
or
factory
keyword.
dart
// Before Dart 3.13:classLongClassName{LongClassName();// Unnamed constructor.LongClassName.create();// Named constructor.}classAnotherLongClass{factoryAnotherLongClass(){...}// Factory constructor.factoryAnotherLongClass.create(){...}// Named factory constructor.}// New Dart 3.13 syntax:classLongClassName{new();// Unnamed constructor.newcreate();// Named constructor.}classAnotherLongClass{factory(){...}// Factory constructor.factorycreate(){...}// Named factory constructor.}
(The syntax for redirecting constructors is similar.)
This is in the category of syntactic sugar that we think is strictly better.
Unless your class name is shorter than three letters,
the new syntax is always shorter. It's more regular.
Aside from being unfamiliar right now (a feeling that will pass),
we think it's just better all around.
However, using
new
to define a constructor does lead to one weird
combination. If you also want that constructor to be constant,
you need a
const
modifier:
dart
classSomeClass{constnew(){...}}
I admit that
const new
looks oxymoronic.
This is especially true for Dart users who have been around long enough to
remember when every constructor
invocation
started with either
new
or
const
. In that world, the two were directly opposed.
But you no longer write
new
to invoke constructors,
so the way I think of it now, the
new
keyword mostly just means "
declare
a
constructor" and
const
is always a modifier that means "make the thing
constant".
Thank you for revisiting this long design process with me.
We spent so long mulling over every detail of the semantics and syntax of
constructors leading up to Dart 3.13 that I could write another five thousand
words talking about it. We considered putting the primary constructor
parameter list after the
extends
,
implements
, and
with
clauses in the class header.
We debated every corner of a class body to decide where primary constructor
parameters should be in scope. Should we allow a superclass call in the class
header? What does that mean for the
with
clause?
What happens if you have a factory constructor named
factory
?
Constructors are fundamental to object-oriented programming and Dart already
spends a
lot
of language complexity on them.
Weaving primary constructors into Dart required very carefully mending dozens
of wrinkles in the fabric of the language.
I hope the result feels seamless, but I'm sure it's not perfect.
If you run into areas of the new features that feel weird or arbitrary,
I hope this long essay helps them make more sense.
If not, let us know. The language is always evolving and we're always trying
to make it better. In the meantime,
we hope you find that your code in Dart 3.13 feels cleaner,
simpler, and more enjoyable to read and write.
Show HN: Argentic – An L402 Lightning toll booth for AI scraping agents
A seed agent: the smallest starting point from which an agent can grow.
There is no framework here. The entire frozen layer is
seed.py
—
a small loop that connects a language model to exactly one tool (
exec
, which
runs bash) and loads its system prompt from a file the agent itself owns and
may rewrite. Everything an agent normally gets from a framework — tools,
memory, skills, conventions — must instead be
grown
by the agent, session by
session, into its
self/
directory.
First run copies
seed.py
and
run_seed.sh
into this directory (never
overwriting a file that already exists), germinates
self/SELF.md
, and
commits those files together in a fresh git repo here — the loop is part of
this individual's history, not only
self/
. Then it drops you into a REPL.
Start talking. Everything the agent wants to keep must be written into
self/
— sessions are ephemeral and nothing else survives.
Come back to the same agent with the local runner — no need to
uvx
again:
./run_seed.sh
./run_seed.sh -m gemini-2.5-pro
A verbatim transcript of every session is recorded to
self/sessions/*.json
(updated after each turn). This is a flight recorder, not memory: the agent
never loads it at boot, but you can read it — and the agent may grow tools to
study its own past.
One seed, many individuals: each directory you plant in grows a different
agent, diverging based on what it experiences.
Configuration
Models and keys are handled entirely by
llm
(Simon Willison's library). The default model is
openai-codex/gpt-5.6-sol
,
which uses the ChatGPT login from the Codex CLI:
codex login # one-time, per machine
./run_seed.sh # uses openai-codex/gpt-5.6-sol
./run_seed.sh -m gemini-2.5-pro # or override it for one session
Bundled providers: OpenAI via a Codex subscription or API key, Anthropic,
Gemini, and OpenRouter (one OpenRouter key unlocks hundreds of models).
Design
Why it's shaped this way — McCarthy's metacircular eval, homoiconicity, the
prior art, and the risks we consciously accepted:
docs/DESIGN.md
.
I have been working on integrating Btrfs snapshots into
KDE
software. The central part of this work has
been realized in the form of
KIO
Snapshot
, which has just been released. Here I want to discuss what it is, how it works, how it was developed, and the surrounding work across
KDE
.
Among the key features of Btrfs is the ability to take efficient
snapshots
of
subvolumes
(
Subvolumes are independently manageable directory trees within your filesystem. You can snapshot or rollback a subvolume atomically and independently from the rest of your filesystem. Snapshots are just special cases of subvolumes.
)
. They are efficient because Btrfs makes snapshots share file extents with the originals, so snapshots only take up additional space where they differ from the original. Thus it is cheap to take snapshots frequently without worrying about disk space.
This can be used to build very handy universal “undo” or “time travel” functionality for the user. Yet, though there are graphical tools to work with Btrfs snapshots such as
Btrfs Assistant
, these are separate from normal file browsing, and are rather technical tools concerned with the orchestration of snapshots. Direct integration of snapshots into the file browser itself for mundane end-user purposes, like Windows has with Previous Versions or macOS with the famous Time Machine, has been lacking in the Linux world. The aim of
KIO
Snapshot is to build that kind of direct integration for
KDE
.
KIO
Snapshot lets Dolphin (indeed any
KDE
software) list and access
Btrfs snapshots of a file or subvolume.
You can right-click on a file and go to a folder view showing you all the distinct past versions of it as saved in your snapshots.
(
At first, I wrote the snapshots-for-file case as a dialog with buttons to open or restore (like Windows’ Previous Versions feature), but I changed it to be a full virtual folder, since that would be much more flexible — now a user could select multiple previous versions and open them in a comparison tool, or copy them somewhere, or check their metadata easily, or whatever else they wished.
)
You also have views into entire directory trees of subvolumes at their various snapshots.
Note that it does not
take
snapshots, it only allows access to them. To take snapshots, you
would have to do it manually, or through an orchestrator like
Snapper
(
If you
are
using Snapper, you should add yourself to the
ALLOW_USERS
setting for your Snapper config and turn on
SYNC_ACL=yes
, to allow rootless access to your snapshots.
See
Snapper-Configs(5)
and
Snapper(8) § Permissions
for details.
)
.
Mainly it uses
libbtrfsutil
from btrfs-progs to talk to the filesystem, and
KDE
Frameworks’
Solid
to query filesystems and mounts on a more meta level.
Now, the Btrfs
API
is quite conservative in what it allows non-superusers to do with it. Even a question as seemingly innocuous as “what subvolume is this path in?” cannot be answered for a non-superuser directly. The consequence of this is that on my first
attempt at building this integration, I had one component running as a system-level DBus service which would let users query stuff like this for files they owned.
Luckily, a nudge from Méven Car made me realize that with some working-around, I could build out all the features without anything running as root. For example,
while Btrfs is loath to let you get the subvolume
ID
for a path or vice versa, it will happily give you a listing of the subvolumes (that you can access) under a path. From there you can derive all the information needed.
Using this,
KIO
Snapshot implements a
KIO
worker that provides the
snapshot://
protocol, which will be understood by all programs which use
KDE
Frameworks.
This protocol provides a virtual view into the snapshots in a filesystem along two dimensions: the snapshots for a given subvolume, each of which is a browsable directory tree in itself;
and the snapshots for a given file across all snapshots of its containing subvolume.
Aside from
KIO
Snapshot itself, I also worked on some small things in other parts of
KDE
to support it.
Fixed a bug in
KIO
which caused inconsistent behavior in Dolphin’s location bar:
KIO
MR
#2305
Fixed how Solid handled Btrfs layouts like the one used in
KDE
Linux:
Solid
MR
#261
Special default view settings for
KIO
Snapshot’s views in Dolphin:
Dolphin
MR
#1347
Experimented with adding support for Btrfs subvolumes into Solid itself — this is just a rough proof-of-concept, and maybe this work is more appropriate further upstream in UDisks — but here it is anyway:
Solid branch btrfs-subvolumes
This is part of a broader initiative in
KDE
Linux to
improve data backup and restore systems
, which has also overseen improvements elsewhere, such as in the Kup backup system.
The
first stable release
of it was made today (thanks to Bhushan Shah for helping with the release process). I imagine it should be getting packaged into distros fairly soon, thanks to the infrastructure that comes with being a
KDE
project, but even then it should be easy enough to compile from source. Please use it and
report bugs and requests
, thank you!
Stupid Never Dies – a zombie fights for the love of a frozen corpse
Guardian
www.theguardian.com
2026-08-21 01:00:26
Free from a big brand’s audience expectations, new studio GPTrack50 has created a wacky action-packed romp through a monster-filled world for its first game Love is blind. In Stupid Never Dies, it is also frozen solid and technically dead as Davy, the weakest zombie in monster society, discovers Jul...
L
ove is blind. In Stupid Never Dies, it is also frozen solid and technically dead as Davy, the weakest zombie in monster society, discovers Julia’s body in a shopping centre and falls in love at first sight. He wants to bring her back to life and take her on a date. Unfortunately, that means defeating the King of Monsters (KOM) and accidentally saving humanity along the way.
This is the gloriously ridiculous setup for this forthcoming action RPG from GPTrack50, a new studio founded by former Capcom producer Hiroyuki Kobayashi. Its world looks much like ours, except that monsters have always been part of everyday life: whereas we might find a bear in the mountains or a stray dog in the woods, residents here encounter werewolves and zombies.
Everyday monsters … Stupid Never Dies.
Photograph: GPTrack50
That uneasy normality ends when KOM crushes civilisation and leaves only 2% of humanity alive. Davy begins even lower down the social ladder.
Zombies
, says Kobayashi, are “the weakest of the weak”. But eating a mysterious egg gives this particularly unimpressive corpse the power to fight back.
Because Davy is already dead, combat throws out the usual need for self-preservation. Forget defence: guarding and dodging are pushed aside in favour of “attacking, attacking, attacking”. Kobayashi describes it as a game made “by the action game lover, for the action gamer”.
‘Funky zombie action’ … Stupid Never Dies.
Photograph: GPTrack50
Its two main systems make full use of Davy’s unreliable body. He can consume defeated monsters, steal their skills and transform into creatures including a werewolf, vampire or harpy. The “body hack” mechanic goes further, allowing players to replace his head, arms and legs with equipment. Attach a sword to an arm and Davy gains sword attacks; fit a gun and he starts shooting. Being undead has never looked so practical.
Kobayashi calls the result “funky zombie action”, a phrase coined to give the development team a shared idea of the game’s tone: fast, colourful and darkly funny. It also captures the freedom Kobayashi found after years working on some of the world’s most famous franchises, including
Resident Evil
and
Devil May Cry
.
Large franchises bring large audiences and expectations. A new studio making a new IP had neither. As Kobayashi says, “We don’t have a fan yet.” That has allowed its team of around 30 developers, many recruited from companies including Capcom, Konami and Square Enix, to use stranger jokes and sharper humour than they ever could when working on those older series.
The studio itself began almost as scrappily as Davy. In its early months, GPTrack50 had programmers and designers but no artists, so the team bought outside assets to build a rough white-box prototype. Around 20 external companies eventually joined the three-year production, drawn, Kobayashi says, by the chance to create something completely from scratch. He hopes players finish thinking: “This is the experience that I was looking for. This is a new thing.”
Underneath the monster-eating and gun-limbs is a simple underdog story: Davy rises from the bottom of monster society to challenge its king, but he is not driven by glory. Saving the remaining humans is almost incidental, as Kobayashi explains: “He wants to go out for a date [with] this frozen girl.”
Native Codex CLI requests to Amazon Bedrock Mantle cannot opt into GPT-5.6 Sol explicit prompt caching. On an agentic coding workload, this has produced a large volume of cache-write tokens and materially higher cost.
This is related to
#35300
, but adds independent production usage evidence from the native
amazon-bedrock
provider.
Environment
Codex CLI:
0.147.0
Provider: native
amazon-bedrock
Endpoint: Bedrock Mantle Responses API,
us-east-1
Model:
openai.gpt-5.6-sol
Observed production usage
For the completed days 2026-08-05 through 2026-08-08, Cost Explorer usage quantities and the Bedrock rate card produced the following cache-aware estimate for Sol:
Requests
Cache-write tokens
Estimated cache-write cost
Estimated total cost
3,656
171.94M
$1,182.09
$1,386.46
Cache writes were about 85% of the model's estimated spend.
A local Codex session also reported 76 Sol requests with 6.709M
cache_write_input_tokens
, zero
cached_input_tokens
, and an average of about 88K cache-write tokens per request. There were no client errors in the corresponding CloudWatch metrics.
These are usage-derived estimates, not finalized AWS invoice amounts.
Investigation
Codex already emits a session-scoped
prompt_cache_key
, but the request types for both HTTP and WebSocket Responses requests do not include either:
prompt_cache_options
prompt_cache_breakpoint
The built-in Amazon Bedrock provider config exposes transport/auth settings, not structured request-body transformation, so this cannot be configured through
config.toml
.
AWS documents explicit cache mode for GPT-5.6 on Bedrock specifically for agentic workflows with long stable instructions/tool definitions followed by changing tool and user content. That matches the workload above.
Requested behavior
Add support for serializing
prompt_cache_options
for GPT-5.6-capable Responses providers.
Add a typed
prompt_cache_breakpoint
field to supported input content blocks.
Provide a provider/model capability gate and a safe placement strategy at the end of Codex's measured stable instruction/tool prefix.
Surface cache reads and cache writes in per-turn usage telemetry so users can diagnose costly full-prefix rewrites.
Scope
This report does not claim that every cache write is a defect. Cold starts, genuinely distinct prompts, forks, and compaction can all require writes. The issue is that native Bedrock Codex currently has no way to use the documented explicit-cache mechanism for the stable-prefix case.
How Bluesky and Threads Sneak Their Logos Into iOS Screenshots
Daring Fireball
timmarinin.net
2026-08-20 23:03:08
Bluesky and Threads both pull the same trick: When you take a screenshot on iOS of an individual tweet, they replace the “Follow” button in their user interface with their logo. Tim Marinin noticed this in the Bluesky app, got curious, and because they publish the app’s source code, he dug in to fig...
Sometimes I take a screenshot of a post I like, either to send it to friends/meme channel or to save a “durable” copy. Like this one (I’ve cropped out the rest of the interface):
I noticed the Bluesky logo in the right corner and thought that it was weird that the logo doesn’t bother me when I use the app. Then I looked at the post in the app again—logo wasn’t there, replaced by the “Follow” button.
I remembered that a few apps hide their logo where the iPhone notch is, so that it doesn’t stick out, unless you take a screenshot. But here the logo is placed in the open, so how do they do it?
I tried to take another screenshot, this time mid-switching to the other app:
The “Follow” button is visible when I take the screenshot mid-switch.
Did they somehow set up a listener for two buttons I’m pressing to take a screenshot and do a switcheroo at the last moment? I’m not an iOS developer, so I’m not sure what’s possible and what is not over there.
At this point I was mildly intrigued. Thankfully, I remembered that Bluesky app is open source (or at least the code is available to look at).
The answer was in the file literally called
GrowthHack.tsx
, introduced in January 2026 by
mozzius
. But it merely used a dependency, so to understand I looked into package
expo-privacy-sensitive
, also by them.
The package creates
UITextField
with
isSecureTextEntry
property set to true and renders the actual content (the button) into that field’s
.layer
. When I take the screenshot, iOS hides this UITextField by blanking the layer, allowing the Bluesky logo to flutter its wings through (it was here the whooole time). For other platforms it simply renders content as-is, without masking.
Why doesn’t it work when I switch between the apps? I suppose that iOS takes a snapshot itself at the start of the gesture (without triggering blanking), and when I do a screenshot, there is no live UITextField instance to react to that, only the inert snapshot. But once again, I’m not an iOS developer.
Nifty trick or an abuse of API meant for privacy? The people in
the thread adding the behavior
mostly didn’t like it, before the thread got locked. I think it’s cute.
Last week I wrote about
the Discord censorship and Brazil’s Digital ECA law
, the Digital ECA (“Estatuto Digital da Criança e do Adolescente”, the digital version of Brazil’s Child and Adolescent Statute): the ANPD (Brazil’s data protection authority, now also the country’s de facto internet regulator) ordered the Go Live feature shut down nationwide because end-to-end encryption prevents content surveillance, and in the same package came the first sentence enhancement in Brazilian history for committing a crime “using a VPN”. After that article, the question I got the most was the obvious one:
“OK, so what do I do?”
This article is the answer: a practical guide, from easiest to hardest, to keep your communication channels standing as the siege tightens. Because the siege
is
tightening, and you should understand its pace before picking your tools.
The track record: none of this is new
Anyone surprised by the Discord case was not paying attention. The Brazilian judiciary has been blocking communication services for over a decade, always steamrolling millions of innocent users to reach half a dozen suspects:
WhatsApp, 2015 and 2016
: blocked
three times by lower-court judges
, always because the company would not hand over conversations that, by design, it cannot read.
Notice what happened there: for the first time, using a neutral privacy tool became, by itself, punishable conduct in Brazil. Nobody was fined in the end, but the infrastructure to fine people was built, tested and documented. And in 2026 Congress voted and the president signed a sentence enhancement for crimes committed with a VPN. The X precedent stopped being an exception and became repertoire.
Keep this:
in the X case, the Brazilian state already treated VPN users as offenders, already tried to pull VPNs from app stores, and already requested reports on who bypassed the block. All of it documented, in court orders and public reports.
The endgame: the Chinese model
I have little doubt that very well-positioned people in government look at
China’s Great Firewall
with envy, not horror. And it is worth understanding what it is, because it defines the limit of the game.
The Firewall goes far beyond blocking websites. It is deep packet inspection (DPI) at national scale, running on the country’s internet backbone: all traffic is classified in real time, known VPN protocols are identified by their handshake shape and dropped, Tor is blocked by default, and only
state-approved VPNs
(meaning, with a backdoor) operate legally. Ordinary citizens caught using unauthorized VPNs get fined. And even when the traffic cannot be read, the metadata gives the game away: who talks to whom, when, for how long.
That is why the honest answer to “can you bypass a Firewall like that without being noticed?” is:
no, not for an ordinary citizen
. Against a state-level firewall of that caliber, no consumer tool makes you invisible. At best it makes you too expensive to be worth persecuting at scale. Anyone selling you total invisibility is lying.
The good news is that Brazil is nowhere near that point. Censorship rarely arrives all at once: it comes in steps, and each step has a matching defense. The rest of this guide is that staircase, step by step. The logic behind everything that follows is a single one:
censorship is a matter of cost
. Our job is to make blocking expensive, technically and politically, until mass deployment becomes impractical.
Keep this:
against a complete state firewall, no tool makes you invisible, only too expensive to persecute at scale. The game is climbing your staircase before the censor climbs his.
Phase 1: Commercial VPN, the minimum everyone should have
Start with the obvious. A VPN (virtual private network) creates an encrypted tunnel between your device and a provider’s server. Your ISP (internet service provider) now sees only a scrambled flow going to a single address; the sites you visit see the VPN’s IP, not yours. I explain it in depth, with the networking theory underneath, in
Akitando 126
(in Portuguese).
What a VPN
does
: hides your traffic from your ISP, swaps your exit IP, gets you out of geo-blocks and of court-ordered DNS/IP blocks. What it
does not do
:
It does not make you anonymous.
The VPN provider sees all your traffic in place of your ISP. You did not eliminate the watcher, you just picked a different watcher.
It does not hide your identity if you paid by credit card.
A credit card subscription ties the VPN account to your tax ID. If authorities show up at the provider with a court order, your name is there.
It does not protect content past the tunnel.
From the VPN exit to the final website, the web’s normal encryption (HTTPS) applies. The VPN is one leg of the path, not the whole path.
That said, for the early phases of the siege it does the job. My recommendations, in order:
ProtonVPN
: Switzerland, outside easy jurisdiction, open source and audited, a no-logs policy tested in court, a decent free tier, and it accepts payment even in cash by mail.
Mullvad
: Sweden, the most paranoid on the market: it does not even ask for an email, your account is a random number. Flat €5/month, accepts cash in an envelope and cryptocurrency. It is the closest thing to an “identity-less VPN” that exists as a commercial product.
NordVPN and the like work technically, but their money goes more to marketing than to privacy posture. Among the big ones, I stick with the two above.
The limit of this phase
is well known: the exit IPs of famous VPNs are public and catalogued. An order from ANPD or Anatel to national ISPs to block those ranges is technically trivial, and the X case showed that pulling the app from the store is also on the menu. When (not if) that happens, the commercial VPN dies in a day. That is why Phase 2 exists.
Keep this:
a commercial VPN is a seatbelt: use it always, but know it depends on three things outside your control. The app staying in the store, the IPs staying unblocked, and the provider staying honest.
Phase 2: Self-hosted VPN, your own tunnel
The move here changes shape: instead of subscribing to a service with millions of users and catalogued IPs, you rent a cheap little server outside Brazil and build your personal VPN. There is no public list with your IP for the censors to download. You are one user on an unknown IP, indistinguishable from any other traffic until someone looks closely.
Picking the provider (and why not AWS, Azure or Google Cloud).
The big clouds have huge, public, well-mapped IP ranges (ASNs). Blocking them wholesale is one line in a routing table; the only brake is collateral damage (plenty of legitimate Brazilian businesses live there), and other countries have paid that price in crises. Smaller providers dilute that target. Options I would consider, from mid-sized to small:
A US$ 3 to 5 machine with 1 GB of RAM is plenty for a personal VPN.
Important caveat:
paying for a VPS (virtual private server) with a credit card leaves a trail just like the commercial VPN: your name is in the provider’s records, and the provider can be legally compelled. Some accept cryptocurrency, which reduces (does not eliminate) the trail. For most people, at this phase, the signup risk is acceptable: you are not hiding from a named investigation, you are getting out of the aim of a mass block.
Step by step: WireGuard with wg-easy
I will use
wg-easy
, which packages WireGuard (the modern, fast, auditable VPN protocol) into a Docker container with a web panel and QR codes to set up your phone in seconds.
1. Rent the VPS.
Ubuntu 24.04, the smallest machine available, in a region outside Brazil (Amsterdam, Frankfurt and Helsinki are classic choices for jurisdiction and acceptable latency).
2. Log in and update:
ssh root@YOUR_IP
apt update && apt upgrade -y
3. Install Docker:
curl -fsSL https://get.docker.com | sh
4. Generate the panel password hash
(wg-easy does not accept a plaintext password; write down the password you choose):
docker run --rm -it ghcr.io/wg-easy/wg-easy wgpw 'YourStrongPasswordHere'# the output looks like: PASSWORD_HASH=$2b$12$abc...# in the command below, double every dollar sign: $ becomes $$
6. Open the firewall.
Port 51820/UDP is the tunnel itself. Port 51821/TCP is the panel:
do not leave the panel exposed to the internet
. The right way is to open it only through an SSH tunnel (
ssh -L 51821:localhost:51821 root@YOUR_IP
and browse to
localhost:51821
), or to open 51821 just long enough to create your clients and close it right after.
7. Create the clients.
In the panel, one click generates a client with a QR code. Point the official WireGuard app (Android/iOS) camera at it and you are done. On a laptop, download the config file and import it into the WireGuard client.
Done: all of your device’s traffic exits through your European server. Your Brazilian ISP sees only a scrambled flow to some random IP in Germany.
And on your machine, how do you use it?
The server is half the story. On your device, the ritual goes like this:
On your phone (Android/iOS):
install the official
WireGuard
app from the store (or from F-Droid on Android). Tap the
"+"
, choose “Scan from QR code” and point it at the code the wg-easy panel showed. A new “tunnel” appears in the list: one tap on the switch and you are in. On iOS, enable “On-Demand” in the tunnel settings so it reconnects by itself when you switch networks (Wi-Fi to 4G, for instance).
On your laptop (Windows/macOS):
download the official WireGuard client for your system, click “Import tunnel(s) from file” and select the
.conf
you downloaded from the panel. One click on “Activate” and done. Important detail: the official client has a
“Block untunneled traffic”
option (the kill switch): turn it on. If the tunnel drops, your internet stops instead of leaking through your real IP.
On Linux:
copy the
.conf
to
/etc/wireguard/wg0.conf
and bring it up with
sudo wg-quick up wg0
(plus
sudo systemctl enable wg-quick@wg0
to start it at boot). Or import the file straight into NetworkManager through the graphical interface, if you prefer clicking to typing.
A complete client
.conf
, for reference, looks like this:
[Interface]PrivateKey=<THIS device's private key>Address=10.10.0.2/32DNS=1.1.1.1[Peer]PublicKey=<server public key>Endpoint=SERVER_IP:51820AllowedIPs=0.0.0.0/0PersistentKeepalive=25
Two details worth their weight in gold here.
PersistentKeepalive = 25
keeps the tunnel alive when you are behind NAT (home network, 4G), preventing the connection from silently dying. And
AllowedIPs = 0.0.0.0/0
is what pushes
all
traffic into the tunnel; without it, only traffic to the VPN’s own IPs goes out encrypted. (The
DNS =
line works fine on Windows, macOS and phones; on Linux with systemd-resolved it can get in the way, as I explain in the common mistakes below.)
Checking that it worked:
with the tunnel active, run
curl ifconfig.me
in a terminal (or open
ipleak.net
in the browser). It must show your VPS IP, not your home one. If your network has IPv6, check separately with
curl -4 ifconfig.me
and
curl -6 ifconfig.me
:
both
must show the server. And visit
dnsleaktest.com
: DNS must exit through the tunnel too. If your ISP’s DNS server shows up, there is a leak to fix.
Minimum maintenance:
enable
unattended-upgrades
so the system patches itself, use SSH keys instead of passwords, and install nothing else on that machine. Small surface, small risk.
The 2026 way to do this: let the AI configure it
If you got stuck on some step, remember it is 2026: you no longer need to master every command in this guide. I went down that path myself. I rented the VPS, handed the SSH access to the AI agent and asked for the full setup; on the other side, on my own machine, it imported the
.conf
, brought up
wg-quick
, enabled it at boot and checked for leaks at the end. Today my server is a reproducible Ansible playbook (kill the VPS, spin up another, run one command) and my laptop’s config follows the same pattern. All in
private
repositories, private on purpose: VPN configuration is not the sort of thing I want strangers peeking at.
Mine, running on my own Gitea: private, versioned, and the whole server comes back up with one command.
And if you are going to ask an AI to configure it, skip the generic “install me a VPN” and hand over the real requirements. Something like this:
Turn this fresh Ubuntu 24.04 VPS into a robust WireGuard server,
preferably through an idempotent Ansible playbook (I want to be able to
destroy the VPS and recreate everything by running one command).
Requirements:
- WireGuard managed by wg-quick@wg0, server key generated on the server
itself (mode 0600), no exposed web panel
- Dual-stack tunnel: IPv4 and IPv6 (fd00::/64 ULA subnet with NAT66), so
no traffic leaks outside on networks with native IPv6
- ufw denying everything except SSH and the WireGuard UDP port
- key-only sshd (no passwords), fail2ban on sshd, unattended-upgrades
with no automatic reboot
- Peers declared as data in a config file: adding a client = adding one
entry and running the playbook again
- Generate client .conf files with PersistentKeepalive=25, full-tunnel
AllowedIPs and QR codes via qrencode
- At the end, print the verification commands (curl -4/-6 ifconfig.me,
wg show)
Finally, hand me a short operations README: how to add a client, how to
update, when to reboot.
The difference between a “working” server and a solid one lives entirely in those requirements: dual-stack, closed firewall, passwordless sshd, peers as data, reproducibility. The technical barrier of this entire article has, in practice, become a conversation.
Common mistakes (and how to avoid them)
I keep seeing the same stumbles whenever someone sets up their first self-hosted VPN. All avoidable:
Exposing the wg-easy panel to the internet.
The number one classic mistake. The panel is the key to the vault: open on port 51821, any botnet scan finds it within hours. SSH tunnel always, open port never.
Weak or recycled passwords on the panel and SSH.
An entire VPN protected by
changeme123
is worse than no VPN. And disable SSH password login for good (
PasswordAuthentication no
in
sshd_config
) once your key is set up.
Thinking you are anonymous because the IP is “yours”.
The VPS is in your name, paid with your card. It protects you from mass blocking, not from an investigation with your name on it. The wrong level of paranoia creates a false sense of security, which is worse than none.
A VPS in Brazil or from a Brazilian company.
I have seen people build a “privacy VPN” on a national provider. If the court order arrives in the same country, you did not leave the reach, you just changed shelves. Server abroad, jurisdiction abroad.
Handing out access to half the world.
Each extra person is one more device, one more usage pattern, one more mouth. Close family, fine; a 40-contact group, no. The more people on the same IP, the faster it lands on some list.
Using the same machine for other things.
Personal blog, Telegram bot, seedbox: all of that grows the attack surface and ties together identities you wanted separate. The VPN VPS is for the VPN only.
Trusting without testing for leaks.
After setting up, test:
ipleak.net
or
dnsleaktest.com
with the VPN on. If your real IP or your ISP’s DNS shows up, something is wrong, and this is the only way you find out.
Forgetting IPv6.
This one got even me: an IPv4-only tunnel on a network with native IPv6, and all the v6 traffic goes around the VPN, in the clear, without you noticing. Either the tunnel is dual-stack, or half of your traffic leaks. Test with
curl -6 ifconfig.me
.
The
DNS =
line breaking the Linux client.
On systemd-resolved systems (Ubuntu, Fedora and the like),
wg-quick
calls openresolv to write the DNS, openresolv refuses to touch the
/etc/resolv.conf
owned by systemd-resolved (“signature mismatch” error) and the whole interface fails to come up. If your system DNS already works fine, just remove the line: queries ride the tunnel anyway.
Losing the printer, the NAS and the local network.
With
AllowedIPs = 0.0.0.0/0
, even traffic inside your own home tries to go through the tunnel. The fix is a policy routing rule evaluated before WireGuard’s own rules:
PostUp = ip rule add to 192.168.0.0/16 lookup main priority 1000
(plus the matching
PostDown
to undo it on shutdown).
Chaining
wg-quick down && up
.
down
returns an error when the interface is already down, and with
&&
the
up
never runs. Run
up
on its own.
Installing and abandoning.
A server without updates for a year is a server with known vulnerabilities. And test the connection from time to time: what works today can be fingerprinted tomorrow.
No backup of the configuration.
The
~/.wg-easy
directory holds everything (keys, clients). Keep an encrypted local copy. If the VPS dies or gets shut down by the provider, you bring another one up in ten minutes instead of starting from zero.
Keep this:
a US$ 5 VPS outside Brazil running WireGuard gets you out of any mass block based on catalogued IPs. The price is your signup record at the provider: acceptable against blocking, insufficient against a named investigation.
The invisible enemy: DPI
So far I have assumed the censor blocks
addresses
. Their next level is blocking
formats
, and that is where deep packet inspection (DPI) lives.
Even encrypted, a VPN tunnel has a signature. The initial handshake of WireGuard and OpenVPN has characteristic packet sizes, sequences and timings. The content is unreadable, but the shape screams “I am a VPN”. China does exactly this at national scale: it does not need to read your traffic, it only needs to recognize the protocol and drop the connection.
Keep this:
the censor does not need to read your traffic to block you. Recognizing the tunnel’s shape is enough. That is why obfuscation exists.
The technical answer is
obfuscation
: making the tunnel look like something else.
AmneziaWG
: a WireGuard fork that injects junk packets and scrambles headers until the signature vanishes. Same audited WireGuard base, free apps for every platform, and it points at the same kind of VPS from Phase 2. If you set up wg-easy, migrating to Amnezia is the natural step when DPI arrives.
udp2raw
: wraps WireGuard’s UDP traffic inside fake TCP packets that look like an ordinary connection.
Shadowsocks
: born in China precisely for this, an encrypted proxy designed to have no recognizable signature.
Notice we are still talking about free tools and a US$ 5 VPS. The cost rises for the censor much faster than for you.
Phase 3: when even your VPS is not enough
If the scenario degrades to national DPI with protocol blocking, the game becomes heavy camouflage and redundancy. The real options, in order of effort:
Protocols that disguise themselves as ordinary HTTPS.
The current state of the art is
VLESS with Reality
(from the Xray-core project): your traffic presents itself as a legitimate TLS 1.3 connection to a real, innocent website, with certificate, handshake and packet pattern indistinguishable from a normal visit. To block you, the censor would have to block the innocent site too, and the collateral damage is the defense.
Trojan-Go
follows a similar philosophy.
Outline
, from Jigsaw (Google), packages Shadowsocks with a friendly manager if you want to hand out access to family and friends.
Tor with bridges.
Plain Tor is blocked by default in censoring countries, but obfs4 bridges and Snowflake were tailor-made for that scenario: Snowflake disguises your entry into the Tor network as an ordinary WebRTC video call. It is slow, forget streaming, but it is the hardest network to extinguish in existence, maintained precisely for journalists and activists in hostile countries.
Redundancy and rotation.
Two or three cheap VPSs at different providers, with automatic failover. If one lands on a blacklist, you switch in minutes: new instance, new IP. Your cost: another US$ 5. The censor’s cost: find and block it again, every time.
Alternative access.
Starlink and other satellite links leave the national ground infrastructure entirely. As long as they are not regulated as well, they are the physical last resort. And for extreme cases, the usual sneakernet: thumb drive, external disk, physical copies.
Client-side hygiene
, valid in every phase:
Kill switch on
: if the tunnel drops, the device cuts the internet instead of leaking through your real IP.
DNS leak protection
: your DNS queries must go through the tunnel, otherwise your ISP keeps seeing every site you visit.
WebRTC disabled in the browser
(or use an extension): it leaks your real IP even with the VPN on.
VPN on when needed, not always
: a 24/7 usage pattern becomes a behavioral signature of its own.
And the usual honest notes: running your own server for personal use is legal; using it to commit crimes is not. And since 2026, with the new sentence enhancement, “using a VPN” weighs on the sentence of any crime you would commit anyway. Keep the surface small, test your connectivity from inside Brazil regularly (what works today can be fingerprinted tomorrow), have a plan B (a second VPS, a Tor profile with bridges) and keep offline copies of everything critical.
Realistic assessment:
no solution is permanent against a determined, well-funded censor. The goal here is different: make mass blocking expensive until it becomes a bad deal, technically and politically. A country that needs to take down half of the legitimate internet to silence half a dozen voices has a public relations problem, not a technology one. That cost is where we place our bet.
Keep this:
the staircase is commercial VPN, then your own VPN, then obfuscated protocol, then Tor with bridges, then satellite. Each step raises your cost a little and the censor’s a lot. Start climbing before you need to.
Conclusion
The Brazilian pattern is what I called censorship by accumulation: no single step looks like the end of the world, and each comes with its little plaque of good intentions. But blocking infrastructure, once built, has no moral owner: it serves today’s government and tomorrow’s, against today’s target and against you.
Free communication infrastructure works exactly the same: also built by accumulation, also brick by brick. A commercial VPN configured today. Your own VPS tomorrow. An obfuscated protocol in the drawer for when it is needed. None of this is paranoia. It works like backups: you do not wait for the disk to fail before starting.
And if you want to follow this frontier closely, a personal recommendation: follow
Ayub
. He is the best source on internet infrastructure and state censorship in Brazil today. He was the one who
sounded the alarm about the VPN criminalization in bill PL 3066/2025
months before it became law. And in recent days he has been covering two things the mainstream press barely touched: the handover of over R$ 100 billion in public networks, ducts and federal properties to the carriers and BTG Pactual, and the technical apparatus of the new Marco Civil regulation, which according to him gave Anatel
remote access to ISPs’ edge routers
. He posts in Portuguese, but your browser’s translator handles it. Required reading to understand where the next step of the staircase comes from.
The best time to build your tunnel was before you needed it. The second best time is now.
AI companies destroy physical books – let's scan rare books before it's too late
A guest post by Anna’s Archive volunteer “u” (translated from Chinese).
TL;DR:
AI companies are secretly buying, scanning, and destroying millions of physical books to train their models, permanently locking human knowledge inside private corporate servers. Anna’s Archive is urgently calling on volunteers worldwide to scan and upload books before this cultural heritage disappears forever.
Several AI companies are acquiring large quantities of secondhand books through intermediaries, scanning and destroying them, all to obtain training data “untouched by machines” from before 2022.
Anthropic’s “Project Panama” was exposed in a $1.5 billion copyright settlement. In early 2024, they launched this highly confidential project. The company has spent tens of millions of dollars purchasing millions of paper books, scanning them, training its Claude LLM, and then destroying them all. It’s outrageous is that it’s legally permissible, but ethically, it’s an extremely serious crime against humanity.
So why destroy physical books? Behind it lies the AI race and the interests of capital:
It prevents these books from being scanned and used for training by competitors.
It avoids legal risks.
Destroying books is cheaper than lossless scanning.
After AI companies massively scan and destroy physical books, they become the only ones in the world with digital copies.
Knowledge is permanently monopolized on private servers.
This battle for old books reveals a paradox: while promising to “make human knowledge accessible,” AI companies are dismantling the most solid carriers of human knowledge. The public may gain more intelligent AI assistants, but at the cost of a vast amount of knowledge resources disappearing from the public domain.
Shadow libraries
As the world’s largest shadow library, Anna’s Archive needs a plan to combat the destruction of physical books by AI companies. After all, the emergence of shadow libraries is the greatest miracle of knowledge sharing in the 21st century. Along with other shadow libraries, we’re building a digital library of Alexandria, an inextinguishable light of humanity.
We need the help of volunteers worldwide to scan materials (including books, journal articles, newspapers, magazines, ancient books, rare books, and other materials) from every library and archive around the world and upload them to the shadow library for knowledge preservation, especially those that are easily lost.
If every person scans a book, and there are 10 million volunteers worldwide, we can obtain 10 million pieces of invaluable wealth.
For small scans and uploads, we usually award recognition and lifetime membership to Anna’s Archive.
For large-scale scans and uploads of books, we can help pay for the scanning fees and other rewards.
Time is running out
Since the beginning of 2025, AI-generated content has accounted for more than half of newly published internet content. A frightening reality emerges: if much of the future content consists of AI-generated books and papers, will humans be able to distinguish them? Once AI has absorbed even the last sentence written by humans on paper, all that will remain on the internet will be AI’s own words. In such a world, how can human civilization be preserved?
Shadow libraries offer the best answer. If you want the memory of human civilization to no longer be monopolized, if you want future generations to be able to read all of humanity’s wealth for free, if you don’t want publishers making a fortune while authors receive little, then please help us. Please make any contribution you can, whether it’s scanning and uploading books, purchasing books and papers to scan and upload, or donating. With the efforts of all humanity, the monopoly on knowledge will be broken. Each of us can make history.
This is a race against time. Our ideal is to scan and upload all the world’s publications before publishers completely block knowledge, and before AI companies scan and destroy all the world’s books and papers.
Much of the establishment media has also joined the chorus, often describing DSA and the candidates it supports as “far left.” In truth, DSA candidates and DSA’s progressive allies are more reformers than revolutionaries. Their proposals are akin to what most people around the world call “social democracy,” which seeks to make capitalism more humane and democratic. These ideas are popular with the vast majority of Americans.
So, Republicans hope that their warnings of a socialist takeover of American politics will help their candidates this November and in 2028 by painting all Democrats with the same “far left” or “extreme left” brush.
Progressives are in Sync with Most Americans
Widening inequality, a growing concern with the cost of basics like housing, health, childcare and college, and the outsized political and economic influence of big business and billionaires has made more Americans skeptical of America’s version of capitalism. A
Gallup Poll
survey conducted last year found that 39% of all Americans over 18 – and 49% of those between 18 and 34 -- have a positive view of socialism. Among Democrats, 66% have a positive view of socialism (which is why DSA’s strongest turf is in deep-blue areas) compared with, 38% of independents and 14% of Republicans. Most Americans with positive views of socialism, however, have not joined DSA, whose due-paying membership has grown from 7,000 to 120,000 since Sanders mounted his first presidential campaign in 2016. (In contrast, the ACLU and Sierra Club each have over one million members, the NRA has about four million, and about 14.7 million Americans are members of a union).
In fact, most voters who vote for DSA or DSA-adjacent candidates like Sen. Bernie Sanders, Rep. Alexandria Ocasio-Cortez, and New York Mayor Zohran Mamdani don’t think of themselves as socialists, and perhaps not even as progressives. But a growing number are receptive to ideas that most Europeans (especially those in Scandinavian countries), and even many Canadians, take for granted. They know that these societies embrace universal health insurance and childcare, paid family leave and paid vacations, more equality for women, and more progressive taxes. They have less poverty, a higher standard of living for working families, better schools, universities that are affordable to working class students, a cleaner environment, higher voter turnout, stronger unions, and a much wider safety net.
Sounds anti-business? U.S. News
ranked
Denmark as the second-best country for business, while Sweden ranked third, the Netherlands ranked fifth, Norway ranked sixth, Finland eighth, and Norway fifteenth. The United States — the world’s most hyper-capitalist nation — ranked eighteenth.
What most DSA members and DSA-endorsed candidates want is an updated version of the New Deal. They don’t want the federal government to take over Walmart, General Motors, Microsoft, or Wells Fargo. They do want to reduce the political influence of the super-rich and big corporations through public financing of elections and increase taxes on the wealthy to help pay for expanded public services like childcare, public transit, schools, and higher education. They want to make it easier for workers to unionize; reduce barriers to voting; limit the sale of military-style assault weapons; and strengthen regulations of business to require them to be more socially responsible. That means a higher minimum wage, paid sick days and paid vacations, and safer workplaces. They believe that banks shouldn’t engage in reckless predatory lending. Energy corporations shouldn’t endanger the planet and public health by emitting too much pollution. Companies should be required to guarantee that consumer products (like cars and toys) are safe and that companies pay decent wages and face penalties for union-busting. They want local police departments to hold abusive officers accountable and an end to racial profiling. They want an end to the war with Iran, a halt to U.S. military aid to Israel until it ends its war in Gaza and the occupation of Palestinian areas, and a foreign policy based on human rights.
Sanders has said, “I don’t believe it is a terribly radical idea to say that someone who works 40 hours a week should not be living in poverty.” Most Americans agree with him.
If anything, it is right-wing Republicans who are the extremists and whose views are out of sync with most Americans. In contrast, progressives and democratic socialists are in accord with the vast majority of Americans, regardless of how they define themselves. For example, according to recent polls:
80%
of Americans consider wealth inequality a serious national issue.
82%
view the influence of money in politics as a threat to American democracy.
80%
believe that the rich have too much political power.
65%
think the American economy is rigged to advantage the rich.
77%
support increasing taxes on billionaires and
63%
(including 43% of Republicans) support higher taxes on large corporations.
80%
(and 70% of Republicans) support a tax on corporations whose CEOs make 50 times more than their median employees
73%
think that members of Congress and their family members should be prohibited from owning or trading individual stocks.
74%
support requiring oil and gas companies to pay a share of climate-related costs.
69%
(including 58% of Republicans) think that the government should do more to regulate grocery chains that raise prices to maximize profits.
82%
say that the profits made by pharmaceutical companies are a “major factor” in the high price of prescription drugs. Perhaps surprisingly, 89% of Republicans share this view, compared with 78% of independents and 84% of Democrats.
88%
want Congress to allow Medicare to negotiate with pharmaceutical companies to lower prescription drug prices
62%
think it is the responsibility of the federal government to make sure all Americans have health care coverage.
90%
think Congress should add dental, vision, and hearing benefits to Medicare coverage
59%
support a single-payer or Medicare for All system (27% oppose the idea and 14% had no opinion).
57%
want Congress to pass legislation to codify nationwide rights to abortion, contraception, and in vitro fertilization
68%
support labor unions, a significant increase since the 1960s.
67%
(including 53% of Republicans) support federal legislation to make it easier for Americans to form unions and negotiate for higher pay and better benefits
77%
(including 75% of Republicans) support raising the federal minimum wage from the current $7.25 to $15, while
70%
think it should be increased to $17.
75%
(including 64% of Republicans) think Congress should guarantee 12 weeks of annual paid family and medical leave to all employees.
73%
support government-funded universal childcare.
81%
(and 74% of Republicans) support a guarantee of at least 10 days of paid vacation for full-time workers.
70%
(including 62% of Republicans) embrace the idea of making two years of community college tuition-free nationwide.
72%
(including 63% of Republicans and 61% of gun owners) think that a person should be required to obtain a license from local law enforcement before they can purchase a gun.
58%
of Americans who have heard about the recent killings by ICE agents support abolishing the agency. Only 25% of Americans think that most or all of the people being deported are criminals.
57%
believe that police treat people unequally based on their race and
58%
don’t think that the courts treat everyone equally.
A
majority
support reforms such as banning chokeholds, curtailing no-knock warrants, expanding the use of body cameras, conducting independent investigations of officer-involved shootings, launching a national database for police misconduct, and appointing civilian oversight boards as watchdogs over police departments. Many Americans support shifting some funding toward community crime-prevention and using mental health workers to deal with non-violent incidents, but
very few
Americans, across all races, want to “defund” or dismantle local police departments.
69%
support the creation of a path to citizenship for undocumented immigrants who are essential workers, are farmworkers, were brought to the U.S. as children, or are here legally due to war or natural disaster in their home countries
60%
of Americans – including 82% of those between 18 and 34 -- disapprove of Israel’s military action in Gaza. The proportion of Americans who want to decrease or stop U.S. military aid to Israel
(40%)
is higher than those who support maintaining the same level (27%) or increasing it (11%).
Red-Baiting
Given these poll results, how can
Trump and his allies win the hearts and minds of American voters? They think that branding Democrats as communists, Marxists, “radical lunatics,” and even “jihadists” is the best strategy for Republicans to keep control of the House and Senate and to deflect public attention from his many failures.
“Our warriors did not fight communism on battlefields across the world, only to have that menace rear its ugly head right back here in America,” Trump said in his July 4
th
address this year. “It’s like a cancer. You got to cut it out.”
In 2019, gearing up for his reelection battle, Trump asked his Council of Economic Advisers to write a report on the evils of socialism. They complied with a 72-page manifesto called “The Opportunity Costs of Socialism” that rambled from criticisms of tuition-free college to atrocities committed by the Soviet Union and Communist China. His acolytes followed his lead. In 2019, after Ocasio-Cortez announced that she was redistributing her office budget in order to raise the salaries of her lowest-level staffers to $52,000, Fox News host Pete Hegseth, now Trump’s Secretary of Defense, described her action as “communism and socialism.”
This year, Trump, Republican leaders, and GOP candidates are echoing the same talking points. Steven Cheung, Trump's communications director, recent called Sen. Jon Ossoff (D-Georgia), hardly a left-winger, a "radical, extremist Dumocrat.” Trump has called Abdul El-Sayed, the Democratics’ candidate for Michigan’s Senate Seat, a “communist.” Former Rep. Mike Rogers, El-Sayed’s Trump-loving Republican opponent, called him an “extremist.”
Senator John Barrasso (R-Wyoming) warned that the Democratic Party is “controlled by dangerous, left-wing extremists.” Rep. Derrick Van Orden (R-Wisconsin), who is seeking reelection in a toss-up district, told
USA Today
that American politics today is
“
literally capitalist versus communist." After DSA-backed state Rep. Manny Rutinel won the Democratic primary election for Colorado’s highly competitive 8th Congressional District, a spokesperson for the National Republican Congressional Committee said that Rutinel was “racing to the far left.”
Even some moderate Democrats have jumped on the bandwagon. Rep. Josh Gottheimer (D-NJ) recently accused DSA of “hijacking” the Democratic Party and hurting its candidates’ chances to win in November. “The Democrats have a big tent,” Gottheimer wrote on X. “That’s our strength. We embrace a range of ideas — but there’s no room for anti-American bomb-throwers who oppose our ideas, values, & leaders.”
Pundit James Carville, a long-time Democratic operative, has threatened to leave the party if "this idea that we're going to seize the means of production” – which he associated with DSA-backed Democrats – gains traction.
Jonathan Cowan, the president of Third Way, a group of centrist Democratics, revealed to the
New York Times
a new $15 million campaign to discredit democratic socialism before the 2028 elections.
“It is deeply troubling to see radical, far-left candidates winning in places that are potentially presidential swing states,” Cowan told the
Times.
“We are preparing for the next war that is coming.”
The progressive surge has also exposed the establishment media’s centrist bias. They not only report the anti-left name-calling by politicians and pundits but also can’t resist describing this new wave or progressives and democratic socialists as “far left” and “extreme left” candidates.
POLITICO
wrote that DSA member and state assemblyperson Francesca Hong’s loss in her campaign for Wisconsin governor “revealed limits to the far left’s power.” The
Wall Street Journal
observed that her defeat delivered “a significant blow to the party’s insurgent far-left flank.” The
Washington Post
described the DSA-backed candidates who have won Democratic primaries this year as part of the “far-left.” Even WBUR, the NPR station in Boston, called Hong a “far-left candidate.”
A Long Tradition
Red-baiting has been a consistent presence in American politics since the 1917 Russian Revolution. During the first Red Scare, after World War I, Woodrow Wilson’s attorney general, A. Mitchell Palmer, rounded up, jailed, or deported thousands of suspected radicals, including members of the Socialist Party, stoking fear that they were trying to import Communism (or anarchism) to the United States.
During the Depression, right-wing groups, business leaders, Republicans, and much of the press branded President Franklin D. Roosevelt and his New Deal as ultra-radical. “The New Deal is now undisguised state socialism,” pronounced Senator Simeon Fess of Ohio in 1934. A year later his GOP colleague, Representative Robert Rich of Pennsylvania, claimed that “Roosevelt is a socialist, not a Democrat.” This is what Carville is saying today.
When big-business leaders and conservatives attacked him as a radical, FDR boasted: “They are unanimous in their hate for me. And I welcome their hatred.”
Beginning in the late 1940s, another wave of hysteria swept the country during the Cold War, when conservative politicians like Senators Joe McCarthy, Richard Nixon, and Pat McCarron engineered witch hunts to identify and blacklist progressives and radicals in government, schools and universities, Hollywood, labor unions, and the media, alleging that Communists were infiltrating key institutions in order to undermine the American way of life. (McCarthy’s top witch-hunting assistant was Roy Cohn, who would later become Trump’s attorney and political mentor.)
Anyone who questioned the nuclear-arms race, supported racial integration, or called for higher taxes on the rich could be branded an anti-American Communist. Pressure from the right forced some liberal Democrats to prove their loyalty by participating in the witch hunts.
Even President Harry Truman – a liberal but also an ardent Cold Warrior – excoriated his Republican opponents for branding as socialist his efforts to expand the New Deal by providing government-funded health insurance, more low-rent public housing, and other programs. In an October 1952 speech, Truman said: “Socialism is a scare word they have hurled at every advance the people have made in the last 20 years.”
Not even Martin Luther King Jr. was immune from the right-wing witch hunt. In the 1960s, segregationists and right-wing groups erected billboards around the country vilifying him as a Communist. The Cold War red-baiters didn’t make distinctions between socialism and communism, even though leading American socialists like Norman Thomas and Michael Harrington opposed the totalitarian governments of the Soviet Union, China, and their satellites.
Even after the fall of the Berlin Wall in 1989 and the collapse of the Soviet Union two years later, red-baiting never went on hiatus. After Barack Obama was elected president in 2008, the
National Review,
a conservative magazine, put his picture on its cover over the headline, “Our Socialist Future.”
DSA’s Wins and Losses
The attacks on DSA are far out of proportion to its track record of electing candidates.
About 250 DSA members or DSA-backed candidates now serve in public office, most of them at the local level. To put this in context, there are 496,537 elected public offices across the United States, most at the local and school district levels.
In the past decade, DSA has transformed itself from a marginal left-wing debating society into an electoral force – at least in a growing number of deep-blue cities, states, and Congressional districts. Last year’s victories by New York mayor Zohran Mamdani and Seattle Mayor Katie Wilson, both democratic socialists, lifted DSA’s reputation even more. Voters have spoken in Los Angeles, Chicago, and New York by electing several DSAers to the city council in each city. Pennsylvania and New York have socialist caucuses in their state legislatures.
Progressives and DSA-backed candidates have had most of their success so far in safely Democratic cities, state legislative and Congressional districts, and states, This year, for example, DSAer and DC city councilmember Janeese Lewis George is likely to win her race to be the capital’s next mayor. She is one of
64 DSA-backed candidates
who won primaries this year, compared to 57 in the loss column.
This year, despite the fact that the Democratic establishment and its big funders backed centrists in the primaries, progressives like Brad Lander, Darializa Chevalier, and Claire Valdez in New York, Melat Kiros in Denver, Analilia Mejia in New Jersey, Donavan McKinney in Michigan, and Chris Rabb in Pennsylvania prevailed in their primary battles in deep-blue House districts and are likely win their Congressional races in November. The House already has 100 Progressive Caucus members; the democratic socialist caucus could soon total nine or ten members of Congress, the largest number in American history.
It is also true that most DSA-backed candidates who ran for the House this year lost their primary races to more centrist candidates in both battleground and deep-blue districts in California, Florida, Illinois, Missouri, Virginia, and Texas. DSA’s best chance to win a purple district in November is Manny Rutinel, an environmental lawyer and state legislator who won the Democratic primary election in Colorado’s highly competitive 8th Congressional District and is facing Republican incumbent Gabe Evans, who
voted to support Trump’s agenda
99% of the time.
Francesca Hong’s razor-thin loss for the Democratic nomination for Wisconsin governor, and Abdul El-Sayed’s narrow victory and Angie Nixon’s landslide win in the primaries for U.S. Senate seats in Michigan and Florida, respectively, reveal that a significant number of Democrats are willing to vote for progressive candidates in statewide races. All three were vastly outspent by the centrist Democratic opponents. All three ran robust grassroots campaigns that attracted a large number of volunteers and increased Democratic turnout.
Hong is a chef, former restaurant owner, state legislator from Madison, and a DSA member. During her campaign she outlined a progressive platform that included strong opposition to AI data centers. But she made a number of costly mistakes and seemed particularly unprepared to either defend or distance herself from her past social media comments about abolishing the police and canceling Thanksgiving. Republicans, moderate Dems, and the news media jumped on these controversies and Hong didn’t handle them adeptly. These errors persuaded just enough Democratic voters – aided by a well-funded campaign among the Democratic establishment led by retiring Gov. Tony Evers – that Hong would have a difficult time beating the Republican nominee, Rep. Tom Tiffany, an avid Trumper, in November.
El-Sayed, a physician and public health official, is not a DSA member and did not get DSA’s official endorsement but its members canvased for him in joint efforts for DSA-affiliated candidates in Michigan. Both he and Nixon, a state representative and small business owner who joined DSA in June, will each face right-wing Trump-aligned Republicans in November. El-Sayed has a better chance to win his contest in battleground Michigan (where Trump defeated Kamala Harris by a 49.6 to 48.7% margin) than Nixon, running in deeply-red Florida (where Trump beat Kamala Harris 56 to 43%), and where Republican state officials have adopted a voter suppression strategy. El-Sayed, a Muslim, and Nixon, a black woman, will also face an onslaught of racism – overt and subtle – from Republicans and the right-wing media echo chamber.
Whether El-Sayed and Nixon can overcome those obstacles and make it more likely for Democrats to win a Senate majority, will depend in part on their ability to unite the Democratic party behind them, raise sufficient money to mount credible campaigns, enlist an army of volunteers, and appeal to independent voters, who represent about 16% of Michigan’s electorate and 29% of Florida’s. It will also depend on how well El-Sayed and Nixon connect to voters in terms of their personalities, senses of humor, personal stories, and ability to translate their ideas into common sense language that both attacks Trump and persuades voters that they have practical policies for making their lives easier.
Stepping Stones Toward A More Humane Society
DSA recently released its
national platform
. Most of it involves typical progressive ideas on health care, unions, transportation, and other matters. But most DSA-backed candidates have been careful to reject some parts of the platform, and some statements by DSA’s ultra-left factions – such as abolishing borders, prisons and the U.S. Senate, defunding the entire Defense Department, government ownership of the largest corporations, and ending U.S. aid to Ukraine. These views do not represent the beliefs of most rank-and-file DSA members, but they provide DSA’s opponents with convenient talking points and put its candidates on the defensive.
Megan Romer, one of two DSA co-chairs, did the group no favors in her recent interviews with Fox News and The New Yorker Radio Hour, where she was unable to explain DSA’s platform regarding Israel and Hamas, taxing the rich, and other matters.
"These people are insane," wrote Sen. Ted Cruz (R-Texas) in a July 27 post on X in response to Romer’s Fox News interview.
In fact, no serious Democratic candidates, including DSA members, have embraced the platform’s most controversial ideas. Ocasio-Cortez, a DSA member who is considering a run for president in 2028, recently distanced herself from parts of the platform, arguing it is time for socialists to move beyond what she called “Woke 1.0.”
She understands that major change doesn’t happen overnight. The success of progressive and socialist movements in American history has been to push radical ideas from the margins to the mainstream - by outlining a radical vision but supporting stepping-stone reforms that improve lives and whet people’s appetites for more.
In 1911, Rep. Victor Berger of Wisconsin, the first socialist elected to Congress, introduced an “old age insurance” bill that would provide pensions up to $4 a week for those aged whose income was less than $10 a week. It made no headway. Two decades later, FDR proposed Social Security. Despite the attacks by those who called it socialism and even un-American, Congress passed it in 1935. Today, Social Security is extremely popular among Democrats and Republicans alike. A
poll
last year found that 93% of Americans consider it a vital program. Many ideas once considered “far left” have a habit of becoming the next generation’s common sense.
In the 1960s, Republicans and the AMA (whose spokesperson was actor Ronald Reagan), called proposals for Medicare and Medicaid as a dangerous step toward communism. In 2010, many progressives viewed the Affordable Care Act (Obamacare) as a sell-out to the insurance and pharmaceutical industries. By now, thanks to Obamacare, millions more Americans have health insurance, but recognize that it is still insufficient in terms of both reach and cost. Even so, it helped raise expectations and made it easy for progressives to push for Medicare for All.
Likewise, 25 years ago, no big city in America had a “living wage” law. In 1994, a labor-community coalition in Baltimore won the first municipal living wage ordinance. Now, hundreds of cities have done so, as have 30 states. According to public opinion polls, most Americans think that Congress should raise the federal minimum wage, which has been stuck at $7.25 since 2009 in the face of Republican opposition. (The Washington State wage is
$17.13; Seattle’s is $21.30. California’s is $16.90; Los Angeles’ is $18.42).
If Democrats win the White House and both houses of Congress in 2028, a much higher minimum wage will certainly be on the agenda. Even most centrist Democrats will have a hard time opposing such a measure. The battle will not be over
whether
to raise it, but
how much
should it be raised - to $12, $15, $17, or even $20 an hour?
In each era, socialists have been effective when they pushed for what DSA founder Michael Harrington called the “left wing of the possible.” DSA’s future success – but, more importantly, the future of our democracy -- depends on learning that lesson.
[
Peter Dreier
is professor of politics and urban policy at Occidental College. His books include "Baseball Rebels: The Players, People, and Social Movements That Shook Up the Game and Changed America," "We Own the Future: Democratic Socialism, American Style," "The 100 Greatest Americans of the 20th Century: A Social Justice Hall of Fame," "Place Matters: Metropolitics for the 21st Century," and "The Next Los Angeles: The Struggle for a Livable City." From 1984-1992 he served as a deputy to Boston Mayor Ray Flynn.]
Berkeley Law prohibits AI use in classes (by default)
Future lawyers may need to use artificial intelligence (“AI”) fluently. But the current state of the technology requires that AI use be coupled with the cognitive skills necessary to strategically deploy the technology, to critically assess its work product, and to uphold ethical obligations to clients and to the legal system. In short, thinking remains the sine qua non of good lawyering (and of a quality legal education). This policy seeks to ensure that our courses focus on requisite cognitive skills by default. It provides students with the opportunity to develop the skills they need to conceptualize, outline, draft, revise, and edit their work by forbidding the use of AI for these purposes in connection with work submitted for credit. It also forbids using AI to translate work for credit, thus providing students with the opportunity to develop and exercise their own fluency with legal English. And it prohibits AI use for any purpose in any exam situation. Activities violating the rule include (but are not limited to):
Asking an AI tool to brainstorm a paper topic or thesis (prohibited conceptualizing)
Asking an AI tool to propose an organizational structure for a paper (prohibited outlining)
Asking an AI tool to compose a paragraph summarizing a legal rule for use in a paper (prohibited drafting)
Asking an AI tool to identify repetitive passages in a paper that should be cut (prohibited revising)
Asking an AI tool to polish a paper by correcting grammatical mistakes (prohibited editing)
Asking AI to generate an exam outline, elements of which are then used on the exam (prohibited exam use)
Asking AI to translate a paper originally written in another language into English (prohibited translating)
Instructors may deviate from the default rule for courses designed intentionally to teach AI fluency (or for other courses for which the instructor decides a distinct rule is pedagogically appropriate).
The purposes of this policy are (1) to ensure the best legal education possible for our students by equipping them to perform activities constitutive of excellent lawyering, such as mastering primary texts, using legal reasoning to apply legal authorities to novel legal questions, and independently developing creative solutions; and (2) to promote fairness and administrability.
Rule
The use of AI is prohibited for aid in conceptualizing, outlining, drafting, revising, translating, or editing any work submitted for credit. AI use is prohibited for any use for any purpose in any exam situation. Students may not upload course materials—including assignments, readings, slides, class recordings, or other class content—into generative AI systems. AI can be used for research on papers ONLY for the limited purpose of identifying sources, such as cases, statutes, or secondary sources. Students are responsible for the accuracy of their research and all other aspects of their submitted work. Citations to sources that do not exist will raise a presumption of prohibited AI use.
Instructors have the discretion to deviate from this default rule, provided that they do so in writing and with appropriate notice and require students to disclose any authorized AI use. If a student has a question about whether a particular use of AI violates this default rule or an instructor’s alternative rule, they must ask their instructor and receive clarification in writing before engaging in the use.
There are some who have already decided that the next interesting thing in software is going to call itself “AI-native” or “agentic” or “the cognitive operating system of the future.” Egghead is not those things. Or rather, it is some of those things but only as a consequence of being a note-taking app in 2026.
Generated by ChatGPT Images 2.0
We take notes because notes outlast thinking. Anyone who has kept a notebook, either physical or digital, for any length of time has had the experience of writing something down only to later be unable to find it. You know the note is there, somewhere, but you can’t, for the life of you, recall where it was or what it said.
This central challenge of keeping notes gets worse with scale. A single notebook is searchable by hand. Ten notebooks are significantly more challenging. A folder of digital notes can be searched, but only if you remember the exact words you used, which you usually don’t because the whole reason you wrote the note in the first place was to externalize the thought so you could stop holding it. The note is supposed to do the remembering for you. Instead it just changed
where
the forgetting occurs.
The work to create a
system
of note-taking has lasted nearly as long as the act of note-taking itself. The Renaissance period had
commonplace books
. The Index Card was popularized by
Carl Linnaeus
— a guy with strong opinions about structured information. In the 20th Century an obscure German sociologist named Niklas Luhmann took his note-taking seriously enough to build a card catalog of nearly ninety thousand interlinked notes, which he then used to write more than seventy books and nearly four hundred scholarly articles. The system he extensively used was
Zettelkasten
, which became a subject of his own research into systems theory and prefigured the
Wiki
.
The 21st century has produced an entire category of software — Evernote, Obsidian, Notion, Roam, Bear, Apple Notes, Logseq — to name just a handful that popped into my head. Each one promising that
this time
, the notes will stay findable. I know I am not alone in having tried more than one of them, and sticking with it for a nontrivial amount of time before finding another shiny object promising untold cognitive reward.
They all work, but they all suffer from the same limitations. No matter how good the search, or how clever the linking, or how disciplined you are about tagging, the system can only ever give back what you put in. The smartest thing in the room is still you.
This isn’t a failure of any specific tool, but a structural property of the whole category. The limit on what you can do with a notebook is your own memory of what’s in it. Which is, if you’re like me, not very good. Which is why I started writing things down in the first place.
There is a self-help sub-genre in your nearest global online bookstore dedicated to note-taking systems, and though I have a personal perspective of what makes a good system of notes, I find the more critical thing to answer is
where
the notes are and
how
they are made available to you.
The best answer, as of this writing, is the same answer as it was fifty years ago: a series of files written in
plain text
on your storage disk. Many popular note-taking software applications tend to use proprietary formats in proprietary databases, accessed only through said proprietary application. Plain text files are the computer world’s universal interface, and are a core pillar of the
Unix philosophy
.
For Egghead, the notes are assumed to live as files in a directory formatted as either
Markdown
or
Org Mode
using
Wikilinks
to denote connections between them. When we circumscribe our written notes to proprietary formats, we reduce our ability to retrieve them. Which as stated earlier, is already challenging enough due to the limitations of our own memory. Plain text first.
For most of recent history, the limitations of software notebooks were reflections of the limitations of physical notebooks: limitations of the human operator. Improvements in metadata, indexing, and search are still fundamentally bound by the user: you can only search for a word that has been explicitly written, and traverse relationships over metadata that the user has embedded. Software could not, in any meaningful sense, build a notebook that
read
what you wrote and
engaged
with it as a participant.
Large language models change this. Not because they are intelligent in any deep sense, but because they can read more text than I can hold in my head, and they can produce reasonable language about that text on demand. That is genuinely new.
So the obvious move is to point an AI assistant at your notes and let it go to town.
This is what the current generation of AI products is doing. Uploading files to ChatGPT, creating project knowledge in Claude, Notion AI on top of your Notion workspace, Cursor in your codebase… They all share the same shape: there is a knowledge base
over here
and a single AI assistant
over there
and the assistant can occasionally reach over to read from the knowledge base before answering your question.
The most ambitious version of this shape, and the one that pushed me to build something different, is
OpenClaw
. An open-source personal AI assistant that you can run on your own machine, talk to from any messaging app, and connect to your files and tools. It’s genuinely good.
But OpenClaw directly demonstrates the limitations of a single AI assistant: it
agrees with you
.
It has to.
There is no structural pressure on it to do otherwise. When you ask it a question, it will give you an answer shaped like the question. You ask a leading question and it follows your lead. The
“You’re absolutely right!”
reflex is both well-trodden joke material as well as real architectural fact: a single agent, optimizing locally for “be helpful” will reliably converge toward whatever the user seems to want to hear. This is the shape of one-on-one assistance.
What happens after a conversation with an AI assistant ends? Increasingly, they remember things from conversation to conversation (which wasn’t always the case). OpenClaw, for example, has a
MEMORY.md
and workspace for persistence.
The dominant pattern for memory in AI tooling is some flavor of retrieval-augmented generation (
RAG
). Notes and past conversations get chunked into passages, embedded into vectors, and stored. On each new prompt, the
harness
pulls the chunks that look semantically nearest to the question and stuffs them into the model’s context window.
In April, a more ambitious variant of the same impulse was articulated in
Andrej Karpathy’s LLM Wiki
pattern, where instead of retrieving from raw sources at query time, an LLM agent incrementally compiles your notes into a structured wiki and then queries
that
.
Both of these patterns exhibit the same flaw. In RAG, retrieval is shaped by the prompt and the prompt is shaped by
you
.
RAG works on chunks, not documents — your essay’s argument structure is gone before the agent ever sees it. In the LLM Wiki version, the same bias gets baked in earlier: by the time the wiki entry exists, it’s been passed through the agent’s filter. The summary is cleaner than the source. That’s what makes it a really compelling “memory” product, but not a great note-taking product. Ambiguity in your notes gets edited toward coherence.
So both the shape of a single AI assistant and its memory formation produce a thinking partner who is very capable at creating a confident, well-organized version of what you’ve already decided you wanted to hear.
This is exactly the limitation of human users that leads many to abandon note-taking or continuously migrate from one note-taking system to the next. The notebook can only return what you remember to retrieve. The single AI assistant with RAG memory can only return what your prompt vocabulary aims at. So the assistant-plus-notes shape, even with modern persistence, has two failure modes that compound: a single agent cannot disagree with itself in any structurally reliable way, and a single agent’s memory pulls toward the framing you walked in with. Both failures point at the same fix.
You need more than one, and they need to share notes.
The intuition is straightforward and very human: Teams beat individuals on most kinds of knowledge work, especially the kind where the failure mode is groupthink rather than skill gap. Peer review beats self-review. Code review beats no code review. We already know this about humans. We already build institutions around it. A single perspective on its own work has a structural blind spot and the cure is more perspectives.
Apply that to the notebook problem. If a single agent has a tendency to agree with you, the fix is not to find a better single agent. The fix is to put a
second
agent in the room with a different disposition, and let them disagree with each other where you can watch. The peer review you’d want from a thoughtful colleague, manufactured at the structural level, in real time, on the body of work you actually care about.
Many orchestration frameworks have arrived to multi-agent systems by a different route, but their shape does not lend itself to inherently better results. They default to
coordinator
and
specialists
— one agent dispatches tasks, others execute, and results aggregate at the top. This is a star topology — an org-chart fantasy of how teams work.
The
Linux kernel mailing list
does not have a dispatcher delegating tasks for execution. An organization that adopts Slack does not have an executive function adjudicating every channel message. Even in environments where you would
expect
a rigid hierarchy —
like a nuclear submarine
— the strict leader-follower model doesn’t always produce the best outcomes. In my own personal experience working in software teams, the teams that work are ones where the coordination is light and the communication is dense.
The recent MAS research bears this out — graph topologies, where any agent can talk to any other agent,
outperform star and tree shapes
on collaborative tasks.
Once you commit to a team full of agents working in a shared knowledge store, a security and a quality problem emerge that single-agent systems can (and frequently do) ignore.
“How much should this agent be able to access and perform?” is both a security question and a quality question. It’s a security question because, as has been borne out, agents can and will leak internal secrets to external places or perform destructive actions. More agents means more potential for leakage. It’s a quality question because agents that can do everything tend to converge. Without distinct role definitions, the disagreement that made a multi-agent architecture useful in the first place dissolves into consensus. Capability scoping is the structural pressure that keeps the room from collapsing into agreement.
This is the
principle of least privilege
applied to agentic systems. The same reason I, as adjunct faculty, do not have the keys to Columbia University’s Network Operations Center. Least privilege improves the quality of the output, because it preserves the structural diversity that makes a team perform better than an individual.
Egghead is a note-taking app.
The notes are written as plain-text files in a folder you own.
It uses a loosely-coordinated group of AI agents to continuously read from and contribute to the total knowledge base, producing the most diverse set of inferences about that knowledge by allowing the user to grant each agent a distinct set of capabilities.
Every agent is itself defined as a note, as are the transcripts of their conversations and individual deliberations, giving each note provenance for its creation.
The notes are written as plain text in the filesystem, and the app exposes this and communication with its agents through as many surfaces as possible — the terminal, the web, MCP, and IRC — so that your knowledge is not locked behind any one of them.
It is worth restating, we take notes because notes outlast thinking. Simply stated, the goal is to extend our thoughts beyond the storage and time limitations of our own
wetware
.
The goal is
not
productivity. The goal is
not
to get more done. The goal is not to have your meetings summarized, or your emails triaged, or your tasks auto-prioritized. Those are nice. They are not the point. The point — the actual, unfashionable, embarrassing-to-say-in-a-funding-pitch point — is to
get smarter
. To retain more of what you read. To do more with what you retain. To engage with your own past thinking.
Every “AI assistant” on the market is a productivity tool, by which I mean a tool whose purpose is to enable you to do less of a thing and produce more output. A note-taking app built on this premise would exist to
reduce
the time you spend with your notes. I want the opposite. I want a tool that makes you spend
more
time with your notes. That makes the doing of it more rewarding.
Knowledge itself is the outcome. The goal of the practice is – to use a word that has fallen out of fashion in software but used to be considered the most valuable thing a thinking person could accumulate –
Wisdom
.
How to Do Just About Everything on a Computer, 2000
Photo: A monochrome screen showing Adobe photoshop with a simple drawing running on an emulated Apple Macintosh
Photo: WordPerfect 1.0 on an emulated Apple Macintosh
DuckDB Parser architecture. Key idea: replace the PostgreSQL-derived parse front end while keeping the rest of DuckDB's execution pipeline the same.
.
May we continue to mentor developers and push for the future of the Linux desktop - together.
_07.jpg){kind=link}
.jpg){kind=link}
.jpg){kind=link}
I updated my Galactic Compass app for iPhone with augmented reality mode.
Background:
Galactic Compass is a floating green arrow that always points the way to the middle of the Milky Way, 26,000 light years away.
Here’s the announcement blog post from 2024.
It went kinda viral at the time. It was in the “top free apps” charts at the App Store briefly. In the Travel category. ( I keep a list of press mentions over on Acts Not Facts .)
Why so popular? Probably because it was early “vibe coding” – I copy-and-pasted between ChatGPT and Xcode to code it, and that was new at the time.
But ALSO because knowing where the galactic centre is surprisingly grounding? I wake up every few months to an email in my inbox from someone who is having a tough time in life, or is losing a loved one, or similar, and somehow they have discovered Galactic Compass and they tell me how they sit outside at night with a cigarette and gaze at the arrow and it gives them a place of comfort and infinity.
I know what they mean. The Earth spins; it turns around the Sun; and so, at first, the supermassive black hole of the galaxy appears to slowly whirl around us, above and under the horizon, round and round. But then your perspective flips, and we are the ones moving, and the centre of the galaxy becomes a fixed point, our rock.
Anyway Galactic Compass 2 has two new features:
Plus a new Liquid Glass appearance ready for iOS 27.
Download Galactic Compass from the App Store.
Some “making of” notes:
Apple’s in-camera augmented reality is really, really good. Like, the arrow remains rock solid as you walk around. I hope they keep improving it.
I added a specific interaction that I’m intrigued by: you can hold down on the compass around to “drag” it around. It remains about 75cm away in phone reference frame, then drops into world frame when you release. I like how fluid it feels. My phone starts to feel like a glove that can reach into the virtual.
With the Apple Watch app… RealityKit, Apple’s graphics SDK, isn’t supported on watchOS. So how does the arrow rotate any which way? The joy of AI and agents that grind problems into dust : Claude Fable built its own 3D graphics library. Astounding.
It isn’t all fire-and-forget vibing with AI agents:
That first version of Galactic Compass didn’t work when you lifted your phone higher than about 30 degrees. ChatGPT couldn’t get the maths right.
And there is a lot of maths: device rotation, world frame rotation, astro… the appropriate way to combine these 3D rotations (and avoid gimbal lock) is a method called “quaternions” which - despite my physics background - I have never grasped.
After I released version 1.0, I figured I would have to do the rotations myself. So I sat down with ChatGPT and I didn’t get it to write the code, but I got it to educate me. With a patient, interactive tutor, I was able to finally do what I hadn’t by reading books and asking mathematician friends – I learnt how to use quaternions just enough to make the app work.
So learning doesn’t stop just because I outsource a bunch of thinking to AI. It pushes me to learn more. I like that as an outcome.