Note: this is a fairly technical post that outlines an integration between CiviCRM and Nuxt. If that sounds like your cup of tea, then please read on. Otherwise, feel free to skip this one....
Note: this is a fairly technical post that outlines an integration between CiviCRM and Nuxt. If that sounds like your cup of tea, then please read on. Otherwise, feel free to skip this one.
What is Nuxt?
Nuxt
is a popular and modern web application framework based on
Vue
. For those that are familiar with the
React
ecosystem, Nuxt a bit like
Next.js
. And if you are into
Svelte
, you can think of Nuxt as roughly equivalent to
SvelteKit
. All three of these projects aim to be 'batteries included' frameworks that make developers more productive and encourage best practice.
It also felt like a good project to try and tackle at the
CiviCRM sprint in Lunteren, Netherlands
. That is to say, it felt simple enough to attempt in four days provided there were lots of other CiviCRM people around to ask for help! In the four days of that sprint - and with lots of help from
Patrick Figel
,
Tim Otten
and others - we were able to get a basic proof of concept up and running...
You're looking at a very simple Nuxt application connected to a CiviCRM Standalone site. A user logs in with the standard demo data username and password and can then access CiviCRM data. Here is the
code behind this demo
.
Why integrate Nuxt and CiviCRM?
It’s fair to say that the CiviCRM user interface as it stands today has some room for improvement. Indeed there are many people working on improving it right now (think of initiatives like
RiverLea
,
SearchKit
and
FormBuilder
). You can think of
Nuxt CiviCRM
as another initiative in that space. When it comes to UI, both Nuxt and Vue both bring a lot to the table in terms of tooling and best practice. Adopting them more widely has the potential to significantly increase the quality of our user interface and the speed at which we can improve CiviCRM (though exactly how this could fit in with current efforts is a topic for another day).
Most organisations today run CiviCRM alongside a compatible PHP CMS (WordPress, Drupal, Joomla or Backdop) and often rely heavily on their CMS for front-facing functionality (donation pages, event registration, portals, etc.). While our PHP CMSes are great no-code tools - and they aren't going anywhere soon - since the beginning of this year, we have also had the opportunity to run CiviCRM
Standalone
, which has encouraged people to begin to explore life outside of the PHP CMS box.
In Lunteren last month, Jaap introduced Kurund and Eileen and I as 'CiviCRM old timers'. I suppose that is fair enough - we have been hanging around for a while - but as well as being a timely reminder of our own mortality, it was also a good nudge to think more about how to attract new people to the project. My hunch is that there is a potential next generation of CiviCRM developers out there that are (rightly or wrongly) put off getting involved with CiviCRM because of our aging technology stack. Introducing newer tools is a great way to open the doors to these new people.
The nuts and bolts
Nuxt CiviCRM is a nuxt module that you can install as part of a Nuxt project. The general idea is that you give Nuxt the URL of your CiviCRM site and then configure CiviCRM to allow authentication from the Nuxt app. You can then log in using your normal username and password and perform any API actions available to that user. More details on how to do that are in
README
and on the
npm page
.
I've tried to make the integration feel native on both sides so that everyone will feel at home. The main interface is a
composable
useCivi()
that gives access to the
api
, to
login()
and
logout()
functions, and to a reactive
user
object. The api looks exactly the same as it does in CiviCRM, so you can do things like retrieve 25 contacts:
const { data } = await api('Contact', 'get', {limit: 25})
All requests are proxied via some fairly simple Nuxt server functions that forward requests to CiviCRM (along with any cookies) and return the results. When a user logs in, CiviCRM issues a cookie which the Nuxt server forwards to the user. The user then sends the cookie back again with each subsequent request. And when the user logs out, the Cookie is cleared. The server doesn't store the cookie or any data. Proxying requests via a backend means that we can avoid having to configure CORS and or third party cookies.
What's next?
We are only at the proof of concept phase right now, but what we have so far feels quite promising and worth exploring some more. Here are some thoughts and ideas on what we might like to do next.
Given that we are granting access to the CiviCRM API, we should get some more eyes on the code so that we can sense check the approach, and consider and address any security concerns.
While we have chosen to do the authentication via a proxy, there are other approaches to authentication that it would be good to support, most notably, OAuth.
It would be good to create a more inspiring demo - a page that does something useful like accept donations or allow people to manage their communication preferences. In a similar vein, we could leverage Nuxt's UI library to create something that meets WAI accessibility criteria and is more designed.
We'd like to build out a more complete set of components and composables that take the grunt work out of making UI that interacts with the CiviCRM API.
The API function is written in typescript but is only superficially typed at the moment. CiviCRM's API comes with a lot of metadata that we could leverage to better type both the parameters and results of the API function. This would provide a much better developer experience and help catch logical errors in any UI we build.
We could explore how to package up widgets written with these tools as web components or similar so that they can be embedded in non Nuxt contexts (other websites, etc.).
We chose Nuxt and Vue for this proof of concept but you might have your own favourite framework - hopefully this work has given you some pointers and inspiration on how you might integrate it with CiviCRM.
We'd love to hear your thoughts and questions on what we have done so far. Feel free to take the demo for a spin and let us know if it works for you, and post in the comments below.
And If you'd like to get involved or collaborate in any way, please reach out. You could create an issue in the
Github repo
or message me - I'm
michaelmcandrew
on Mattermost.
'Composable Moderation' May Protect Bluesky from Political Pressure
Internet Exchange
internet.exchangepoint.tech
2025-11-27 13:41:23
Composable moderation decentralizes rule-setting, reducing pressure on any single platform and limiting attempts to “work the refs.”...
The Trump administration, led by a President who was previously banned from major social networks for inciting violence and spreading disinformation after the 2020 US election, poses a particular challenge for the upstart platform Bluesky.
As Erin Kissane noted
in a recent article in Tech Policy Press, Bluesky was designed for openness and interoperability, yet it now finds itself as a single point of pressure. If it enforces
its rules against harassment and incitement
against official Trump administration accounts for some future infraction, it risks
political retaliation.
If it weakens its rules or shies away from enforcement, it may lose the trust of the communities who turned to the network for protection from coordinated abuse.
Composable moderation, which decentralizes rule-setting by letting users pick the moderation services that best reflect their needs and values, mitigates this problem. It shifts enforcement away from a single platform and into a distributed ecosystem of user-selected moderation services. With no central referee to target, political actors and influencers lose the ability to “work the refs” and pressure a singular trust and safety team into making decisions that favor their side.
Spreading the burden of moderation
“Bluesky the app” is the company’s shorthand for distinguishing its consumer-facing social app from the AT Protocol, the decentralized social networking protocol it is building. The app is just one client in what is intended to become a broader ecosystem of services built on the protocol. For now, however, Bluesky the company still carries the full responsibility for moderation and governance across the AT Protocol.
Centralized governance of a decentralized protocol cannot withstand sustained political or social pressure. When one company sets moderation rules for a network that is meant to be open and distributed, it becomes a single point of influence that governments, interest groups and powerful users can target. As AT Protocol’s
Ethos
statement
makes clear, its long-term vision sits at the intersection of three movements: the early web’s open publishing model, the peer-to-peer push for self-certifying and decentralized data, and the large-scale distributed systems that underpin modern internet services.
Bluesky’s goal is for AT Protocol to embody the openness of the web, the user-control of peer-to-peer networks, and the performance of modern platforms. In the future, we could see photo-sharing apps, community forums, research tools and more all using the same identities and social graph. Bluesky is only one expression of the protocol, not the limit of it.
Composable moderation is the feature that will make that possible. Rather than treating moderation as a network-wide ban, it uses labels to describe issues with content or accounts, leaving individual apps to decide how to act on them. Following
a letter
from Daphne Keller, Martin Husovec, and my colleague Mallory Knodel,
Bluesky has committed
to this approach.
Instead of blocking someone in a way that removes them from every app built on the protocol, Bluesky will mark a suspended account with a label that only affects how that account appears inside Bluesky. Other apps can choose to hide the account, show it with a warning, or ignore the label entirely. This also keeps the user’s underlying account intact, because it’s stored on their personal data server or PDS, the place where their identity and posts live, which should only cut someone off for serious issues like illegal content. The result is a more flexible, decentralized system where no single app controls whether someone exists on the network.
Why this approach solves the potential Trump problem
The closest analogy to existing social media is to how Reddit operates: the platform sets a baseline of what is acceptable, but thousands of subreddit communities apply their own rules, filters, and enforcement styles on top. For example, r/AskHistorians expects in-depth, well-sourced answers that reflect current academic research, and moderators routinely remove unsourced or speculative replies that don’t meet that standard. Composable moderation takes that layered, community-defined model and implements it at the protocol level, so many different apps and services can choose the moderation approaches that fit their values.
And because moderation could be provided by many different apps and services, not just Bluesky, it would reduce the political vulnerability that comes from having a single company responsible for every enforcement call. Communities can also choose moderation services that reflect their own context and needs, giving vulnerable groups more control over the protections they rely on. And if one app or operator fails or comes under political pressure, others can continue enforcing their own standards without breaking the network.
Taken together, this shift could help Bluesky, and future AT Protocol services, navigate the pressures Kissane highlights, distributing power across the network rather than concentrating it in a single company.
Support the
Exchange Point
— Your Donation Helps Us Build a Feminist Future
If you love our work and want to power more of it, here are the ways to support EXP and
our projects
:
🔐 Protect digital rights with a tax-deductible donation.
Give directly through
PayPal
(tax deductible in the U.S.). ➡️
https://exchangepoint.tech/donations
🌱 Double your impact with employer matching.
Most tech companies match donations through
Benevity
— search “Exchange Point.” Here are three projects to support with matching donations: ➡️ Social Web ➡️ Human Rights and Standards ➡️ Protect E2EE
📅 Sustain the movement with a large or recurring monthly gift.
Become a monthly supporter and help us plan long-term. ➡️ Please email
grants@exchangepoint.tech
.
🛠️ Fund the work through contracts or sponsored projects.
Partner with us on research, workshops, audits, or ecosystem strategy. ➡️
Work with us
!
🩵 Support our sister effort on the Free Our Feeds campaign.
Chip in through the FOF community GoFundMe. ➡️
https://gofund.me/1ef4d5d5d
Thank you for helping us build an open, joyful, people-powered internet.
The Internet still depends on fragile paperwork to prove who controls an IP address. The Internet Society explores how a new tool, the RPKI Signed Checklist, can use cryptographic proof to replace these weak systems and make routing more secure.
https://www.arin.net/blog/2025/11/19/2024-grant-report-isoc
Apple and Google’s Play stores shape what apps are available to most people as they use the Internet. When those app stores block or limit apps based on government requests, they are shaping what people can do, say, communicate, and experience, says ACLU’s Daniel Kahn Gillmor.
https://www.aclu.org/news/free-speech/app-store-oligopoly
Podcast: American Prestige. Silicon Valley and the Israeli Occupation, featuring Omar Zahzah, Assistant Professor of Arab Muslim Ethnicities and Diasporas Studies at San Francisco State University, discussing his book Terms of Servitude: Zionism, Silicon Valley, and Digital Settler Colonialism.
https://americanprestigepod.com/episodes/4215556855
Airbnb.org is offering emergency housing to families displaced by disasters, and any donation made before 31 December will be matched to double the number of nights provided.
https://www.airbnb.org/giftanight
The Arab Center for Social Media Development (ACSD) is pleased to announce the opening of proposals for workshops and sessions at the Palestine Digital Activism Forum 2026, Forms are due
December 15.
https://pdaf.net/invitation
Edit
acmeleaf.conf
and add a global
challenge
block and one or more
certificate
blocks. For example:
account {
directory https://acme-v02.api.letsencrypt.org/directory
kid https://acme-v02.api.letsencrypt.org/acme/acct/000000000
privkey AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
}
challenge {
# Use the system resolver, or specify host[:port] of a recursive resolver.
resolver system
# Authorization polling after triggering dns-01
auth_poll_max 30
auth_poll_period 5
# Order polling after finalize while waiting for the certificate URL
order_poll_max 30
order_poll_period 5
# Our own DNS checks for _acme-challenge.<domain>
dns_poll_max 30
dns_poll_period 10
# Extra delay (in seconds) after we first see the TXT record, before
# asking the ACME server to validate.
dns_propagation_delay 30
}
certificate example {
# Which directory, relative to the config file, should the
# certificates be placed in.
directory example
# Which domains to request for. The first is the CSR Common Name;
# wildcards are supported.
domains example.org *.example.org
# The DNS-01 provider and arguments.
provider gandi_v5_livedns example.org whateverYourPersonalAccessTokenIs
}
Then just run the following, perhaps from cron:
acmeleaf -c ~/.config/acmeleaf/acmeleaf.conf
Certificates will be automatically skipped if:
a
leaf.pem
already exists in the certificate directory,
the certificate has more than 30 days before expiry,
its SANs exactly match the configured
domains
,
it verifies as a TLS server certificate for the first configured domain
using the system trust store (optionally with intermediates from
chain.pem
).
On successful issuance, acmeleaf creates:
key.pem
: your private key
leaf-chain.pem
: your certificate and the CA's certificate chain
leaf.pem
: your certificate only (rarely used)
chain.pem
: the CA's certificate chain (rarely used)
These files live in the certificate’s
directory
, and are symlinks pointing
into an
archive/<unix-timestamp>/
subdirectory.
In 1890, most continental European cities allowed between five and ten storeys to be built anywhere. In the British Empire and the United States, the authorities generally imposed no height limits at all. Detailed fire safety rules had existed for centuries, but development control systems were otherwise highly permissive.
Over the following half century, these liberties disappeared in nearly all Western countries. I call this process ‘the Great Downzoning’.
The Great Downzoning is the main cause of the housing shortages that afflict the great cities of the West today, with
baleful consequences
for health, family formation, the environment, and economic growth.
One study
found that loosening these restrictions in just five major American cities would increase the country’s GDP by 25 percent. The Downzoning is one of the most profound and important events in modern economic history.
The Great Downzoning happened during a period in which anti-density views were widespread among planners, officials, and educated people generally. Most people thought that urban density was unhealthy and dysfunctional, and supported government efforts to reduce it. It is natural to assume that this was why the Downzoning happened. Although there is surprisingly little literature on the Great Downzoning, historians who do discuss it often implicitly take this view, seeing it as of a piece with other anti-density measures taken by municipal governments across the West.
While there is undoubtedly some truth in this explanation, the evidence for it is surprisingly ambiguous. The Downzoning was extremely pervasive in existing suburbs, where it tended to raise property values by prohibiting kinds of development that were seen as undesirable. But in other contexts, it proved much harder to apply anti-density rules. In some European countries, ferocious battles were fought over whether municipal authorities should restrict the density of greenfield development. Doing so tended to reduce land values, prompting fervent resistance by rural landowners, who were generally successful in thwarting the proposed reforms. In the late twentieth century, planners and governments reversed their views on density, and became notionally committed to densification as a public policy objective. But they have had very limited success in reforming rules on suburban densification.
The general pattern is that the Great Downzoning was driven by interests more than by ideology. The Downzoning happened where it served the perceived interests of property owners, and failed to happen where it did not. Ideas-driven explanations of social changes
are sometimes absolutely correct
. But in this case, the correct explanation seems more materialist.
This has implications for the politics of housing reform today. In some places, anti-densification rules continue to raise property values, and in these places we should expect the Downzoning to be as politically robust as it has been for the last century: it really does give property owners something they want.
But in the great cities of the West, the housing shortage that the Downzoning has created may prove to be its undoing. Anti-density rules now reduce property values in these places rather than increasing them, and there is growing evidence that property owners opt out of such rules when they have the opportunity to do so. Winning the principled argument for density will not be enough to undo the Great Downzoning, because it never rested chiefly on principled arguments in the first place. But where the Downzoning is doing the most damage, it may now be possible to build new coalitions of interests in favor of reform.
The story of the Great Downzoning
In most European cities before the nineteenth century, elites were concentrated in city centers. Suburbs were unplanned, mixed use, and generally impoverished, a home to those people and businesses that were excluded from the urban core. Their inhabitants were powerless to resist densification, and often had little reason to do so anyway. The situation for suburbs was especially bad in continental Europe, where many cities retained massive fortifications until the nineteenth century that physically cut them off from their outskirts. Paris, Rome, Vienna, Milan, Madrid, and Barcelona are all examples of this.
Paris was surrounded by massive fortifications until the 1920s. As in many cities, these did double duty as a customs barrier: the municipality imposed an excise duty on all goods passing through their gates, as visible in this photograph from 1907. Paris’s municipal excise duty outlasted the fortifications and was abolished only in 1943. Development was banned for 250 meters outside the walls in order to preserve a field of fire. This land, known as ‘the Zone’, quickly filled with illegal slums, one of which is visible in the background of this image.
Image
Wikimedia Commons.
These walls were often customs barriers as well as defensive ones – most Continental municipalities charged customs duties on goods entering the city until the late nineteenth or early twentieth century, and Paris’s excise duty lasted until 1943. This meant that municipal governments had an incentive to discourage economic activity in suburbs, through, for example, excluding their residents from membership of economically vital guilds.
More affluent suburbs did occasionally emerge, such as those that grew up around royal palaces and hunting lodges like Hampton Court and Saint-Germain-en-Laye, or along arterial roads like the Strand in London or the Bockenheimer Landstraße in Frankfurt. But even these were often developed haphazardly and generally remained quite mixed in social and economic terms.
Premodern suburbs often ran along arterial roads outside city gates, as in this map of London from 1572. They were typically poor, unplanned and mixed use. In the suburb of Southwark south of the river are visible two theaters, resembling the slightly later Globe Theatre in which many of Shakespeare’s plays were first performed. Theaters were considered a disreputable use and were not permitted within the city walls. There were some aristocratic mansions on the Strand running west from the City, but all were subsequently demolished and redeveloped.
Image
Heidelberg University Library via Wikimedia Commons.
A key step in the emergence of modern low-density suburbia was taken when developers began developing entire suburban neighborhoods, rather than just individual houses. This first became common in eighteenth-century Britain, probably because of Britain’s relatively deep capital markets and high rate of urban growth: developing a whole neighborhood involves huge outlays on laying out streets and amenities before any revenue starts to come in, so it is facilitated by low borrowing costs and confidence in future healthy sales.
Examples of early planned neighbourhoods
include the West End of London, the Georgian extensions of Bath, the Bristol suburb of Clifton, and the Edinburgh New Town. These neighborhoods were still built with relatively high densities, so they are not exactly what we think of as suburbs today. But they were lower density than the urban core, as well as being exclusively or nearly exclusively residential, and exclusively upper-middle or upper class.
Suburbia really took off internationally in the nineteenth century, when planned suburbs spread across the British Empire, the United States, Germany, Austria-Hungary, and, to a lesser extent, France. The most universal and decisive factor behind this was probably again deepening capital markets and higher rates of urban growth. Other factors – none of which applied everywhere, but all of which were important in some places – included better roads, the development of suburban railways, buses and trams, improved policing, the abolition of customs boundaries around towns, the reform of feudal land tenures, and the demolition of city walls.
At the upper end of the market, densities quickly fell to levels similar to those of modern affluent suburbs. Many of the elite suburbs of this period are still famous neighborhoods today. American examples include Llewellyn Park in New Jersey, Forest Hills on Long Island, and Riverside outside Chicago. Examples from the British Isles include Rathmines in Dublin, Bedford Park in London, and Edgbaston in Birmingham. Some continental examples are Le Vésinet outside Paris, Pasing in Munich, or Westend in Berlin, named after London’s West End as a marketing gambit. Even small towns often had a tiny ‘villa district’, maybe just a couple of streets, like the
Kingsland neighborhood of Shrewsbury.
Only in the poorer countries of Mediterranean Europe did planned suburbs fail to catch on.
Planning a suburb involved high upfront costs. Developers had to assemble land, lay out street networks, and often provide amenities and public services. In some cases they built railway stations or even whole railways. All this was justified by neighborhood characteristics that such planning created – and the higher sales prices they generated. This map shows the Llewellyn Park neighborhood before the individual houses had actually been built, but after huge outlays had been made on roads and landscaping.
Image
Metropolitan Museum of Art via Wikimedia Commons.
Social decline was common, even normal, for nineteenth-century neighborhoods, and homeowners lived in constant fear of it. Right from the start, suburb developers tried to safeguard neighborhood character through imposing covenants. This episode forms a fascinating prequel to the Great Downzoning, so much so that we might think of it as a ‘First Downzoning’ or ‘Proto-Downzoning’.
A covenant is a kind of legal agreement in which the homebuyer agrees to various restrictions on what they can do to their new property. Covenants generally forbade nearly all non-residential uses, as well as forbidding subdivision into bedsits or flats. They frequently imposed minimum sales prices, and in the United States, they often excluded sale or letting to non-white people. In all countries, they often included explicit restrictions on built density. Most covenants were intended to ‘run with the land’, binding not only the initial buyer but all subsequent ones too.
Here are the rules binding homeowners in Grunewald, Berlin’s premier suburb, reading like thousands of other similar documents before and since:
a) buildings may not be constructed higher than three storeys inclusive of the ground floor; b) all buildings must be provided with [ornamented] facades on all sides; c) at most two houses may be built conjoined to each other; otherwise there must always be a gap of at least eight meters between buildings, for which exception may be made only in the case of covered walkways; d) a fenced front garden of at least four metres in depth must be retained between any building and the street.
Grunewald, Berlin’s premier villa colony. The various setbacks and height limits of its covenants are immediately apparent.
Image
Samuel Hughes.
Covenants were extremely widespread. Although rigorous quantitative studies do not exist, my impression from wide reading is that all elite planned suburbs were covenanted, along with many middle-class ones. They were used in all English-speaking countries, and similar mechanisms existed in France (
servitudes
in
cahiers des charges
), Germany (
Grunddienstbarkeiten
), the Low Countries (
erfdienstbaarheden
), and elsewhere (e.g.
Italy
,
Spain
,
Scandinavia
). Under Japanese law at the time, covenants were legally unenforceable, but the idea was so appealing that Meiji-era suburban developers sometimes imposed them anyway, apparently as a purely moral inducement to conformity.
Covenants became more elaborate over time, and by the early twentieth century they sometimes included provisions on such matters as where laundry could be hung and what colours joinery could be painted in.
Developers would not have imposed covenants if they had not expected them to increase the value of neighborhoods, so their pervasiveness reveals a widespread demand for development control among nineteenth-century people. But they were not very effective. One problem concerned whether courts would enforce them. To secure the development in perpetuity, covenants had to apply not only to the initial homebuyer, but to all future ones – people with whom the developer would never have any direct dealings, and who might indeed live long after the developer’s death. It was legally impossible for normal contracts to do this, so alternative mechanisms had to be employed.
Continental countries generally used the
ancient concept of servitudes
from Roman law, but common-law jurisdictions had to rely on the system of
equity
developed in the
English Court of Chancery
. The problem with this was that courts varied greatly in which restrictions they considered equitable, creating a system fraught with risk and unpredictability for developers and homeowners. Although there was a general tendency for covenants to become better established in law as the nineteenth century went on, there remained much uncertainty about exactly what development rights could be restricted, who had standing to enforce against infringements, and when restrictions could be discharged (voided).
In fact, the law on this is still
hazy
today
.
Another problem with covenants was that they could not be modified retroactively, meaning that any flaws or loopholes were unfixable. This could prove disastrous for covenanters. For example, as already mentioned, many covenants included minimum price thresholds. These were normally given in nominal figures, which worked fine in the nineteenth century because there was no inflation. After 1914, however, inflation took off, swiftly making the thresholds trivially easy to meet. There was no way to insert inflation-adjustment clauses retroactively, so one of the pillars of nineteenth-century covenanting effectively vanished. For example, one Edwardian covenant stipulated that no dwelling worth less than £375 be built on the plot. To achieve the same exclusionary effect in 1920,
the corresponding figure
would have been £1,030. Since the number could not be increased retroactively, the covenant became effectively useless.
A third problem concerned the costs of enforcement. Covenants fall under private law: breaking one is not a crime, and the state will not prosecute it. Enforcement thus requires a private lawsuit, which was and is expensive. Today, a simple case will cost
at least £25,000 in Britain
, while a complex one can cost £60,000. Costs in the USA are
similar or higher
. Historical costs of litigation varied but were
notoriously high
. Developers were often willing to bear these expenses as long as they still had plots elsewhere in the development to sell, but once the entire neighborhood had been sold off, they usually lost any interest in policing its built form.
In theory, the covenants would subsequently be enforced by affected neighbors under a system known as ‘reciprocal enforcement’. In practice, however, this was beleaguered by free-rider problems, with no one resident willing to bear the costs of enforcement alone. Mechanisms eventually developed for pooling enforcement costs in some places, like the famous
homeowner associations
in the United States. But the overall bill remained high, meaning that covenant enforcement tended to be haphazard in any but the most affluent neighborhoods.
The upshot of all this was that covenants were usually a weak kind of development control, which disintegrated upon contact with serious demand for densification. An example of this is the Berlin suburb of Friedenau, originally developed in the late nineteenth century as what the Germans called a ‘villa colony’, an elite suburb of large detached houses.
Friedenau was originally built some way from the edge of Berlin, but the urban core expanded rapidly and reached Friedenau during the 1880s. Friedenau’s restrictions proved completely ineffective and the entire neighborhood was redeveloped with large blocks of flats. Only a handful of villas endured long enough to be protected by later conservation laws, surviving today as a curious reminder of Friedenau’s original form.
The Berlin neighborhood of Friedenau was originally planned as a villa suburb, but was subsequently redeveloped at much higher densities in defiance of its restrictive covenants. Left is one of Friedenau’s handful of surviving villas; right is one of the apartment blocks that came to dominate the neighborhood.
Image
Samuel Hughes.
The stage was set for the Great Downzoning proper, when suburban density restrictions were introduced by public authorities. This began in the final years of the nineteenth century in Germany and Austria-Hungary. The key innovation was ‘differential area zoning’, whereby different areas within a given jurisdiction were subjected to different building restrictions. This allowed for development controls to be applied to suburban areas that would keep them at suburban densities without having the absurd side effect of applying suburban density restrictions to dense city centers.
After a couple of decades of experimentation, the 1891 Frankfurt zoning code caught the imagination of municipal governments across Central Europe.
It was swiftly emulated. By 1914 every German city had a zoning code, and many had gone through multiple iterations, usually with progressively lower densities. In existing elite suburbs, these zoning codes tended to effectively duplicate the content of developers’ covenants, but because they had a stronger legal basis and were enforced by the state, they were far more effective.
For example, Grunewald’s zoning designation in the first decade of the twentieth century permitted two storeys plus a roof storey and basement, banned everything except detached and semi-detached buildings, and required a four-meter setback from the street – much the same as the covenant quoted above. The zoning code
remains similar today
, and has successfully preserved Grunewald as Berlin’s premier villa colony.
The example of Germany and Austria was quickly followed abroad. The Netherlands introduced a kind of zoning system in 1901. Italian cities began to follow suit before the First World War. Japan began to introduce a zoning system nationally in 1919, albeit one that continued to permit fairly high densities. Poland introduced a national system in 1928. American and Canadian cities started introducing zoning systems in the 1910s, which became widespread in the course of the interwar period. Zoning provisions began to be introduced in interwar Australia and were consolidated in the 1940s.
Britain and France followed relatively late: although they introduced planning systems of a sort in 1909 and 1919, respectively, these had limited effectiveness, and robust national systems were not introduced in either country until the 1940s. In broad terms, the Great Downzoning was in place by the 1950s, though density restrictions continued to be tightened in the following decades in many countries.
There are some limitations on the spread of the Great Downzoning, which we will explore in the next section. In many ways, though,the Downzoning was remarkably thorough. Virtually every wealthy suburb that existed in 1914 retains its suburban character today. Long ago, too, the Downzoning spread beyond the homes of the elites. When Central European cities began introducing zoning in the 1890s, suburbia was still largely the preserve of their upper-middle and upper classes. Today, a great part of the working and middle classes of all Northwest European countries and of North America live in suburbs, and they too enjoy the ambiguous blessing of the Downzoning’s protection. Wherever planned residential suburbs of owner-occupiers develop, it seems, the Downzoning has ultimately followed.
An idealist explanation for the Great Downzoning
At the time of the Great Downzoning, a negative view of the cities of the nineteenth century was extremely common. Frank Lloyd Wright described the cities of his day as a ‘conspiracy against manlike freedom’, a ‘disease of the spirit’, and a ‘senseless reiteration of insignificance’.
Werner Hegemann, a prominent German urbanist who later wrote the United States’s first suburban zoning code in Berkeley, described Berlin’s urban fabric as comprised of ‘Dwellings so bad that neither the stupidest devil nor the most diligent speculator could have devised anything worse’.
Le Corbusier claimed that ‘The nineteenth century has made the house into a ridiculous, revolting, and dangerous thing’ and observed that ‘We are living in a dustbin … in a kind of scum choked by its own excretions.’
These views spread far beyond architectural and planning elites, to the point that a negative view of nineteenth-century urbanism became one of the standard background opinions of educated people. To give one striking example, Lord Rosebery, chairman of the London County Council and thus the closest thing that existed to a mayor of London, said:
There is no thought of pride associated in my mind with the idea of London. I am always haunted by the awfulness of London […] Sixty years ago a great Englishman, Cobbett, called it a wen. If it was a wen then, what is it now? A tumour, an elephantiasis sucking into its gorged system half the life and the blood and the bone of the rural districts.
Both professionals and laypeople tended to see lowering residential densities as part of the solution. Planners often converged on twelve dwellings per acre (30 per hectare) as a good upper limit. In Britain, the famous urbanist Raymond Unwin promoted the slogan ‘twelve houses to the acre’ as the norm for residential areas.
Ebenezer Howard also advocated twelve dwellings per acre in
Garden Cities of Tomorrow
, perhaps the most influential planning text of modern times.
The influential American planner John Nolen adopted the same figure, arguing that ‘there must be a limitation of houses to not more than twelve per gross acre’.
Josef Stübben, whose textbook
Der Städtebau
was the standard authority on urban planning in German-speaking Europe, advocated twelve dwellings per acre in most contexts, though he allowed for somewhat higher densities in central areas.
The front cover of The Home I Want, a highly successful book published in 1918 by the demobilized soldier and progressive housing campaigner Richard Reiss. It was later reproduced as a poster and displayed by the Ministry of Reconstruction. In England, the working-class terraces of the nineteenth century came to be seen as a symbol of poverty and deprivation. In many Continental countries, an equivalent role was played by courtyard blocks of rented apartments.
Image
Chroma collection via Alamy.
Public policy reflected this view in a range of ways. In all countries, public transport was subsidized and price controlled with the explicit aim of fostering urban diffusion.
In Britain, the 1918
Tudor Walters Committee
set a standard of twelve dwellings per hectare for social housing. This benchmark remained influential for many decades, and local councils often succeeded in meeting it.
In the United States, the Federal Government began to conditionalize mortgage support on
densities of 4-8 dwellings per acre
, while conceding that this might rise to 12-16 dwellings per acre in central areas. The French and Belgian governments sponsored an extensive program of
garden cities in the interwar period
, aspiring to similarly low densities, though not always reaching them. In Germany, the Weimar government extended subsidies for single-family houses with private gardens under the
1920 Reichsheimstättengesetz
(Reich Homestead Act). The Nazis continued these in their own Reichsheimstättengesetz in 1937, illustrating how the aim of urban diffusion was shared between otherwise radically different political movements.
A scheme for publicly subsidised housing at low densities in the Paris suburb of Gennevilliers, constructed 1923-1933.
Image
Fonds Dumail. SIAF/Cité de l’architecture et du patrimoine/Archives d’architecture contemporaine.
The other instrument that planners used to achieve low densities was, of course, zoning. All contemporary written justifications for suburban low-density zoning appealed to these background anti-density views, and virtually any of the planning officials who worked on early zoning plans would have seen their work as justified by such considerations. So part of the explanation for the Great Downzoning is very simple: it happened because it was seen as an obvious way of achieving an uncontroversial public policy objective.
But this cannot be the whole explanation. When existing suburbs were downzoned, the new rules merely confirmed the densities that market forces had already chosen for the neighborhood. Indeed, as we shall discuss in the next section, brownfield downzoning almost certainly tended to increase land values by protecting neighborhoods from blight. In such cases, then, planners were simply going with the grain of property owners’ interests. In places where planners’ priorities and property owners’ interests were not so aligned, planners’ success was far more doubtful.
A vivid example of this is the attempt of planners to lower the density of greenfield development (new neighborhoods on previously undeveloped land). In Anglophone countries, the density of greenfield development was already fairly low by the early twentieth century, and there was not much for planners to do in lowering it further. In continental countries, however, much greenfield development still took the form of densely massed apartment blocks, which were seen by planners and officials as a shameful humanitarian disaster. Lowering these densities was widely seen as just as much of a priority as protecting existing suburbs, and in many countries it dominated public debate about zoning.
The problem was that, unlike in existing suburbs, downzoning greenfield sites generally reduced their value. Developers built dense apartment blocks because, given prevailing market conditions, that was the most valuable use of the land. Requiring them to build terraced houses or cottages instead crashed land values and annihilated the asset wealth of landowners. Planners and municipal officials thus faced a powerful special interest group, against which they had great difficulty in prevailing.
The classic illustration of this is Rome.
Like most Mediterranean cities, Rome had not really developed planned low-density suburbs in the nineteenth century, but Italian planners shared the contemporary belief that public policy should promote lower densities. In 1907, a coalition of liberals and socialists won the municipal elections under the leadership of Ernesto Nathan, breaking the longstanding hold of the landowning interests over the city’s government. The coalition prepared a zoning plan that aimed at making Rome’s urban extensions into international models of good practice, by the standards of the day.
Rome’s ambitious 1909 zoning scheme
.
Image
Fondazione Marco Besso.
The 1909 zoning plan for Rome was radical. The red-shaded areas still allowed traditional courtyard blocks, but at lower densities than before. The green-shaded areas allowed only ‘villini’, small detached apartment buildings of no more than three storeys, covering a maximum of half of their block area. The huge areas with green outline and white infill are marked for ‘giardini’, literally ‘gardens’. Only 1/20th of the plot area in giardini areas could be built over, a density that would count as low even by modern American standards.
The affected land was mostly owned by the ‘black nobility’, the traditional Roman ruling class (black was a symbol of mourning for the Papal government that had ruled Rome before its annexation by the Kingdom of Italy). The black nobility was appalled at the loss of land value that Nathan’s downzoning had wrought, and they embarked on a long campaign to reverse it. In 1913, an alliance of Catholic and nationalist parties won the municipal elections and loosened the zoning restrictions. Then in 1923 Mussolini seized power, dissolved Rome’s municipal government, and appointed a black noble as governor. For many decades thereafter, higher densities were permitted on many Roman greenfield sites than in the urban core.
Characteristic Roman suburbs of the mid-twentieth century. In Nathan’s plan, this particular area (Balduina) had been marked out for ‘giardini’, with a maximum of 1/20th plot coverage permitted.
Image
Google Earth.
Rome’s story is particularly dramatic, but the basic pattern is typical of Southern Europe. Planners in Spain, Portugal, Italy, and Greece generally shared the standard European aspiration towards lower densities, but they had few existing planned suburbs to defend: the only possible downzoning would be on greenfield land, suppressing density in new urban areas. This would run contrary to the interests of the landowners.
In all four countries, this failed to happen, and urban densities remained stubbornly high, only falling gradually in the late twentieth century under the influence of market forces. Today, these cities may seem like rather remarkable survivals of semi-traditional urban forms, but they were generally not so understood by contemporaries: Southern European writers in the twentieth century generally saw them as obvious urbanistic failures, the product of avaricious landowners and weak, corruptible states.
The story was initially similar in Germany and France. German planners made strenuous efforts to downzone greenfield sites before the First World War, but met with fierce resistance from landowners. In general, the landowners were successful in preserving their right to build apartment blocks, although they sometimes had to include larger courtyards and front gardens. In France, the planning movement was much weaker, and made little progress against landowner and developer interests.
The maximum densities permitted in Paris actually
increased
in 1884 and 1902.
A kind of greenfield downzoning did later happen in France and Germany, but its story is a strange one, and the anti-density views of planners played no role in it. In 1914, both countries introduced tight rent controls to protect the families of soldiers from eviction (Britain followed a year later). As so often in the history of rent control, these rules proved difficult to lift when the crisis that had occasioned them was over. Rent controls persisted in both France and Germany throughout the interwar period and long into the second half of the twentieth century. Coupled with high inflation, this meant that the
real value of rents rapidly fell
.
This undermined the build-to-rent sector, because the rental value of apartments was generally no longer great enough to cover their build cost. Neither country had a well-developed system for selling buildings into multiple ownership: the modern French and German equivalents of condominium ownership,
copropriété
and
Wohnungseigentum
, only developed later in the twentieth century. The remarkable effect of this was that there was generally no way to build flats profitably, resulting in the collapse of the private flat-building sector. The surviving private builders
switched over to building small houses for owner occupation
, beginning
the vast low-density suburbs
with which German and especially French cities are surrounded today.
We are confronted, then, with a striking contrast: nearly total success in downzoning existing suburbs, and nearly total failure in downzoning greenfield development. This contrast casts doubt on the idea that the downzoning was driven by the will of planning elites.
Another context in which planners struggled to lower or even cap densities was in city centers. Many American city centers declined in the decades after the Second World War due to rising crime and traffic congestion, while densification was prevented in some European centers by architectural conservation laws. But in places where neither of these factors applied, densification of city centers continued apace, reaching some of the highest floorspace densities ever attained. Many
Australian
and
Canadian
cities are particularly clear examples of this, though there are also
cases elsewhere
. Again, this is puzzling for the ideas-driven theory of the Great Downzoning: in places which lacked an owner- occupier lobby for restrictions on densification, planning ideology seems to have been ineffective.
Toronto began regulating density during the interwar period, and by the postwar period its suburbs were heavily zoned for single-family housing. Yet densification of the central business district continued rapidly, as visible in this photograph from around 1970. In areas without strong homeowner interests pushing for downzoning, downzoning failed to happen.
Image
Taxiarhos228 via Wikimedia Commons, licensed under CC BY-SA 3.0.
What happened at the end of the twentieth century is no less problematic for the planner-driven explanation of the Great Downzoning. From the 1960s onwards, the intellectual tide began to turn in favor of density, and by the 1990s, density was wildly fashionable again.
I once worked as the research assistant to a
British government commission on the built environment
, in the course of which I had the unenviable task of reading every major official document on British urban policy since the 1990s. From Richard Rogers’s
Towards an Urban Renaissance
(1999), through the
Urban Design Compendium of English Partnerships and the Housing Corporation
(2000) and the Commission for Architecture and the Built Environment’s
By Design: Urban Design in the Planning System
(2000), to the Greater London Authority’s Defining, Measuring and
Implementing Density Standards in London
(2006) and the
Farrell Review of Architecture and the Built Environment
(2014), they were united in praising urban density. Government documents like Planning Practice Guidance Note 3 (2000), Planning Policy Statement 3 (2006), and the National Planning Policy Framework (2012) endorsed and besought it. Every planning school in Britain teaches its students the importance of density, walkability, and mixed use.
Many Western cities have seen extensive urban renewal since the 1990s, but mostly on industrial or commercial sites or through the regeneration of social housing. One such example is Canary Wharf in London, seen here in 1995 and 2019.
Image
Jacek Różyczk via Wikimedia Commons, licensed under CC BY-SA 4.0, and Chris Pancewicz via Alamy.
This was not just empty talk. There have been
huge increases in the population
of virtually every British city centre since the 1990s, enabled and fostered by a range of public programmes. In 1990, fewer than 1,000 people lived in Central Manchester. Today,
around 100,000 people do
. But virtually none of this increase has taken place in private suburbs.
Instead, it has been concentrated in former industrial or logistics sites, in city-centre commercial areas, or in social housing, which the authorities regularly demolish and rebuild at greater densities. Towns without much of this,
like Oxford and Cambridge
, have stable or even declining populations in their city centres. An effort to enable more suburban densification nationally in the 2000s aroused much controversy and
was soon abandoned
. A more recent attempt to allow more densification in an area of South London, widely praised by planners, led to a local political revolution
and the policy’s revocation
.
This is not a British peculiarity. All over the West, urban density is valued by planners and officials. Governments pursue it, and have had some success in enabling it in industrial and commercial areas and through the redevelopment of public housing. In the United States, densification is the central theme of a vast YIMBY movement. But progress on densifying owner-occupier suburbs has been extremely limited, and the vast suburbs of the nineteenth and twentieth centuries remain almost untouched. The unified opinion of the planning and policy elites has proved ineffective in the face of homeowner opposition. If the idealist theory were the whole truth, and the Downzoning was purely the creation of planners, this would be extremely strange.
A materialist explanation of the Great Downzoning
The alternative theory is that the Downzoning was driven less by elite ideas than by the way that the Downzoning served the perceived interests of homeowners. This theory fits better with the evidence.
When people buy a home, they care not only about the home itself, but about the neighborhood in which it stands. This was why nineteenth-century developers started building whole villa colonies and streetcar suburbs rather than just individual houses: by developing entire neighborhoods, they could satisfy a wider range of buyers’ preferences, giving people the neighborhood of their dreams rather than just the house.
An advertisement for homes in Bedford Park, a railway suburb of London targeted at cultivated upper-middle class homebuyers. The developer had invested heavily in local public goods and obviously regarded them as an important part of the development’s offer.
Image
Frederick Hamilton Jackson via Wikimedia Commons.
All else being equal, many people prefer neighborhoods built at low densities. Some of the perceived advantages of low density will apply virtually anywhere, like quieter nights, greener streets, more and larger private gardens, and greater scope for social exclusivity. Other attractions are more specific to certain contexts. Where urban pollution is bad, people seek suburbs for cleaner air. Where crime is high, suburbs are often seen as a way of securing greater safety. In eras with high levels of racism and increasing racial diversity, people moved to suburbs to secure racial homogeneity.
Restrictions on densification were a way of preserving these ‘neighborhood goods’ in perpetuity. The prevalence of covenanting constitutes extremely strong evidence that suburban people wanted this. Covenants were imposed by developers, whose only interest was in maximizing sales value. They judged that the average homebuyer valued the neighborhood goods that covenants safeguarded more than they valued the development rights that covenants removed. The ubiquity of covenants shows that, under nineteenth-century market conditions, density restrictions were generally desired by suburban residents. As we have seen, however, covenants were not very effective. The fact that public zoning followed under these conditions is not greatly surprising: it gave suburban people something they demonstrably wanted, and were not able to secure without the help of public authorities.
One question we might ask about this theory is: why did the Great Downzoning happen when it did, as opposed to at some earlier point in history? The answer is simple: it happened because of the emergence of planned suburbia in the preceding century. The whole point of planned suburbs was that they provided neighborhood goods like exclusivity and amenity: this was what made the large upfront costs of developing a neighborhood worthwhile. The impoverished peripheries of medieval and early modern cities may have had some of these goods by accident – presumably they were greener than medieval city centers, for example – but they would not have had many of them, because they had no way of excluding noxious land uses and ‘undesirable’ people. Many were regarded as dangerous and blighted places, where nobody would live if they had any alternative. Until the nineteenth century, suburban people frequently did not stand to lose much from densification.
The inhabitants of planned suburbia had some obvious political advantages over their predecessors, too. They were relatively affluent, which was an advantage for lobbying purposes in 1890-1950 as in all times and places. They were also extremely homogeneous, in the sense that most neighborhoods were planned for exclusively residential use by people of a given social class. This meant that their interests were likely to be aligned, and they could form a united front to campaign for their community’s interests.
One other feature of planned suburbs deserves mention, which is home ownership. In nearly all countries, planned residential suburbs were predominantly sold into owner occupation.
If you are renting a property and its neighborhood declines in amenity, your economic loss is low: neither your income nor your asset value is affected, and if you move to another neighborhood with better amenity, your only economic loss is the cost of relocation. If you are a homeowner, the loss of neighborhood values destroys your wealth, as embodied in your property value. So you are much more invested in these neighborhood values, and probably more likely to fight for them.
A second question is why the Downzoning began in Central Europe, rather than in, say, England, where suburbia had existed for much longer. I offer three possibilities.
The German princes had long taken a more activist approach to urban planning. At Karlsruhe, for example, the street network was designed to radiate out from the prince’s bedroom.
Image
Carsten Steger via Wikimedia Commons, licensed under CC BY-SA 4.0.
One possible explanation is varying state capacity, together with varying tolerance of state intervention.
The princely states in Germany had been relatively activist in city planning throughout the early modern period: a number of German cities had planned street networks, and the authorities sometimes even micromanaged details of facades, as in Potsdam. In Berlin, the authorities enforced a minimum height limit because they felt that low-rise buildings gave the royal capital a ridiculously countrified appearance.
In the nineteenth century, the German state became much more activist than the French or Anglophone ones in a whole range of areas, pioneering mandatory health insurance, old age pensions, universal compulsory education, and a range of labor regulations. It is plausible that this attitude to government made zoning a more natural intervention for Germans than it was elsewhere.
Most larger Continental cities had become dominated by apartment blocks in the eighteenth century. By contrast, anglophone cities generally had no purpose-built apartments until the late nineteenth century, when high-end flatted buildings started appearing in city centers. This image shows a Berlin district under construction in the early twentieth century.
Image
Ullstein Bild via Getty Images.
Another factor was that pressure for densification was greater. As we have seen, Central European cities at the end of the nineteenth century had a strikingly clear distinction between a dense urban core and low-density villa colonies. When the expanding dense core reached a villa colony, the villa colony was faced with total transformation.
No other country had quite such a neat dichotomy. France did not have elite suburbs to the same extent, while Italy and Spain hardly had them at all. Anglophone cities had few purpose-built flats until the late nineteenth century, even in the urban core, and the ‘mansion blocks’ and ‘co-ops’ that then started to emerge tended to be targeted at the middle classes rather than the poor, alleviating one motive for exclusion. Perhaps, then, the German villa colonies were exposed to a form of densification that was particularly alarming to their residents. The late introduction of density controls in Britain and France may also be explained in this way: as we have seen, rent controls stymied flat building in the interwar period, thus relieving the political demand for restrictions on densification.
A third factor is that private sector density controls were probably varyingly effective in different countries. In Britain, most suburbs were developed under the
unique ‘Great Estates’ system
. Instead of selling homes outright, they sold extremely long leases of eighty or a hundred years, after which the properties reverted to the original landowner, called a ‘Great Estate’. The Great Estates thus retained an interest in safeguarding neighborhood goods in order to preserve the reversionary value of the properties. They thus acted as a form of quasi-government, enforcing against covenant breaches much more effectively than neighbors usually would. Some even provided local public services like parks and sanitation. As noted above, some other countries eventually developed systems somewhat analogous to this, like American homeowner associations. But in Britain they existed right from the start, and were ubiquitous. It is plausible that this contributed to the relatively lower demand for public density controls in Britain, in spite of Britain’s precocious suburbanization.
The Great Upzoning?
One element of the preceding argument may have puzzled some readers. I have argued that density controls were originally imposed because they increased property values, suggesting that allowing densification is net value destroying. But many housing reformers,
including me
, have argued that granting additional development rights to streets or neighborhoods increases their value, for the obvious reason that the additional floorspace is worth a lot. This has been confirmed by recent examples. For example, residents of the London neighborhood South Tottenham recently persuaded their local councils to let them
double the height of their houses
. All properties in the neighborhood enjoyed an immediate boost in value once the council agreed.
Residents of this London neighborhood persuaded the local council to let them add 1.5 storeys to their homes. The new development rights resulted in a large increase in property value for everyone in the neighborhood, even those who had not yet taken advantage of them.
Image
Samuel Hughes.
In South Korea,
some neighborhoods are allowed to vote
for much larger increases in development rights. This generates abundant value uplift, as a result of which residents of such neighborhoods nearly always vote in favor.
In Israel, apartment dwellers can vote to upzone their building
: this has proved so popular that half of the country’s new housing supply is now generated this way. How can such cases be reconciled with the argument I have given here?
The answer lies in how the housing market has changed since the nineteenth century. Over the last century, in large part because of the Downzoning, housing shortages have emerged in many major cities, in the sense that floorspace there sells for much more than it costs to build. This means that the development rights lost through density controls have become steadily more valuable. At a certain point, their summed value became greater than that of the neighborhood values for which they had been sacrificed. It was at this point that they became value-destroying.
It is also at this point that opportunities for innovative reforms like the one in South Tottenham start to emerge, because existing residents would now be net economic beneficiaries of allowing greater densities. The Downzoning was originally extremely desirable to residents, because the neighborhood goods it secured were more valuable than the floorspace it precluded. In places where that is no longer true, we should be cautiously optimistic about the prospects of reform. This is the reasoning behind
proposals like street votes
, which would allow individual streets or blocks to vote by qualified majority to upzone themselves to higher densities.
It is important to stress, though, that this is not true everywhere. Although housing shortages exist in nearly all Western countries, they do not exist in all parts of those countries. In fact, they are heavily concentrated in a small number of major cities. In most of the United States, sales prices are generally
not far above the physical costs of construction
: only in a handful of the major cities, like New York and San Francisco, are they consistently substantially higher.
In Britain, the housing shortage is
heavily concentrated in the South East
, with prices fairly near build costs in much of the rest of the country. In France,
a large divergence has emerged
only in Paris and certain areas popular with wealthy holidaymakers. Similar results are evident elsewhere in continental Europe, Australia, and Canada. In much of the West – probably the majority of its urban area – market conditions are not fundamentally altered since the nineteenth century, and density controls probably still maximize property value.
The fact that something maximizes property value need not make it morally good. Property value is determined by preparedness to pay. It may be maximized by preserving the view from a billionaire’s spare bedroom rather than by providing housing for a thousand destitute people, or by fulfilling the exclusionary preferences of snobs and racists. This means that there may be strong arguments for zoning reform even in places where its net effect on the value of individual neighborhoods would be negative.
Politically, however, reforms that reduce particular people’s property values are likely to be much more difficult. For more than a century, there has been an overwhelming tendency for residential suburbia to secure protections for itself against densification. We saw in the last section that this is hardly surprising: it is almost as though this neighborhood type had been designed to generate exactly this political outcome. The history of the Downzoning suggests it is very hard to triumph against the united interests of suburban homeowners.
The political upshot of this history, then, is that reform efforts should be focused. Making the principled case for density is useful, but unlikely to be sufficient: principled argument did not make the Downzoning, and it probably won’t unmake it either. Instead, campaigners should consider ways in which the changing structure of homeowners’ interests can be mobilized in the cause of reform. The examples of South Tottenham, Seoul, and Tel Aviv suggest that homeowners may be vigorous in pursuit of upzoning when they realize how much they stand to benefit from it. There is no reason why this could not be replicated elsewhere. Across the major cities of the West, homeowners are sitting unwittingly on one of the greatest mines of potential wealth in the history of the world. Once they notice, their actions may amaze us all.
1
Pedants will observe that Britain does not technically have a zoning system, because its development controls are given in the form of local planning policies rather than zoning districts. This distinction has no importance for our purposes and I ignore it in what follows: in this essay, Britain’s planning restrictions on densification also count as ‘downzoning’.
2
For a general discussion of early suburbs, see Gerhard Fehl, ‘Jeder Familie ihr eigenes Haus und jedes Haus in seinem Garten!’ in Tilman Harlander (ed.),
Villa und Eigenheim: Suburbaner Städtebau in Deutschland
, 2001. For the fortifications of Paris, see Justinien Tribillon,
The Zone: An Alternative History of Paris
, 2024.
4
Shun-Ichi J Watanabe, ‘Metropolitanism as a Way of Life: The Case of Tokyo, 1868-1930’ in Anthony Sutcliffe (ed),
Metropolis 1890-1940
(1984)
5
This is explored in detail in Robert M Fogelson,
Bourgeois Nightmares: Suburbia 1870-1930
(2005), pp. 46-53. See also Donald Olsen,
Town Planning in London: The Eighteenth and Nineteenth Centuries
(1982).
6
The best discussions of early German zoning are in Brian Ladd,
Urban Planning and Civic Order in Germany, 1860-1914
(1990), Gerhard Fehl and Juan Rodriguez-Lores, ‘Städtebauliches Instrumentarium und stadträumliche Ordnungsvorstellungen zwischen 1870 und 1905’,
Stadtbauwelt
73 (1982) and Juan Rodriguez-Lores and Gerhard Fehl (eds),
Städtebaureform 1865-1900: Von Licht, Luft und Ordnung in der Stadt der Gründerzeit
(1985), vol. 2. Invaluable context on German suburbia is provided by Tilman Harlander (ed),
Villa und Eigenheim: Suburbaner Städtebau in Deutschland
(2001)
.
Quoted in Ebenezer Howard,
Garden Cities of Tomorrow
, 1902.
11
Raymond Unwin,
Town Planning in Practice,
1909.
12
Ebenezer Howard,
Garden Cities of Tomorrow
, 1902.
13
John Nolen,
New Towns for Old
, 1927.
14
Josef Stübben,
Der Städtebau
, first published in 1890 and revised in 1907 and 1924.
15
John McKay,
Tramways and Trolleys: The Rise of Urban Mass Transport in Europe
,
1976.
16
This story is told in Italo Insolera,
Modern Rome: From Napoleon to the Twenty-First Century
, 2021.
17
For discussion, see e.g. Lila Leontidou,
The Mediterranean City in Transition: Social Change and Urban Development
, 1990; Judith Allen et al,
Housing and Welfare in Southern Europe
, 2004; Martin Wynn,
Planning and Urban Growth in Southern Europe
, 1984; Robert Fried,
Planning the Eternal City: Roman Politics and Planning Since World War II
, 1973.
18
W Brian Newsome,
French Urban Planning 1940-1968: The Construction and Deconstruction of an Authoritarian System, 2009.
19
Anthony Sutcliffe,
Paris: An Architectural History
, 1996.
20
In Britain, they were typically sold into leasehold ownership, which for present purposes had a fairly similar effect.
21
Inasmuch as there is a standard scholarly explanation for the German origins of zoning, this is probably it. See e.g. Anthony Sutcliffe,
Towards the Planned City: Germany, Britain, the United States, France 1780-1914
, 1981.
This article documents my experience installing
Chimera Linux
on a decade-old MacBook Air.
Why Chimera Linux? While I’ve used many Linux distributions over the years, most carry significant historical baggage. I wanted to see what a more modern distribution could achieve. Chimera Linux’s technical approach is quite appealing—it doesn’t use systemd and incorporates userland components from FreeBSD.
The initial setup went surprisingly smoothly. I downloaded the image(x86-64, plasma), flashed it to a USB drive, and booted the MacBook Air from it—all in one go, which felt almost too good to be true. Upon closer inspection, however, I discovered that the hard drive hadn’t loaded properly. Additionally, while Bluetooth worked, the wireless network card wasn’t recognized. Not a perfect start, but still acceptable—certainly smoother than my previous adventures with FreeBSD.
I began by addressing the hard drive problem. After researching the error, I found that it was related to a kernel parameter. The solution was to add
intel_iommu=off
to the kernel boot parameters. Nothing too complex—just a matter of rebooting and remembering to modify the GRUB settings before each boot, otherwise the hard drive wouldn’t be recognized. The proper way to make this change permanent would be editing
/etc/default/grub
, but this approach doesn’t work on a LiveCD.
Next came the more challenging wireless network card issue. The symptom was a lack of device files in
/dev/net
. Using
lspci
, I identified the card as a BCM4360. The default
b43
and
bcma
modules loaded in the LiveCD couldn’t drive this card. After finding some documentation, specifically
this one
, I realized I needed to compile a kernel module. I initially wanted to compile it in the LiveCD environment to minimize unknowns, but since kernel version in the apk repository is newer than the LiveCD version, I decided to install the system first and then tackle the driver issue.
Following the
official documentation
, I created the necessary partitions (ZFS+EFI+boot), mounted them, and executed
chimera-bootstrap -l /media/root
to copy the LiveCD contents to the new partition. This step produced an error at the end—something about being unable to create hard links across devices. I suspected the script/command hadn’t handled this edge case properly, so I ignored it. When I ran
apk upgrade --available
, the process was painfully slow because the network connection was going through my phone’s hotspot via Bluetooth, resulting in limited bandwidth. After upgrading several packages, I simply hit Ctrl-C to interrupt the process. I then installed GRUB and rebooted without issues, but couldn’t get into X, and the Bluetooth service wasn’t running either. This presented a challenge since I had never dealt with fixing Bluetooth from the command line without network access, and even if I got Bluetooth working, I’d still need to connect to my phone to access the internet. I took the easy way out: rebooted, booted from the LiveCD again, mounted the ZFS partition, and fixed the issue. If I recall correctly, either
sddm
or
sddm-dinit
wasn’t installed; once installed, everything worked normally.
Of course, I still needed to compile the network card driver. I referenced
this SlackBuild
and modified this script slightly to get it to compile properly(gist
here
).
In reality, it wasn’t that straightforward. I first needed to install
clang
and
gmake
, then install/update
linux-stable{,-dev}
. The most frustrating part was discovering I also needed to install the
wireless-regdb
package. The original script was designed to use
gcc
for compilation, while Chimera Linux uses
clang
. Rather than hunting for environment variables, I simply created a symlink from
clang
to
gcc
, which worked fine for the compilation.
After successful compilation, I had the
wl.ko
kernel module file and could load it with
modprobe
. Whether using
modprobe -r
or adding to the blacklist, I could drive the network card with my newly compiled
wl.ko
(visible in the PCI information), but
/dev/net
still showed only the
tun
device from my phone’s shared hotspot. As a last resort, I looked up the
dmesg
error (about loading
regulatory.db
) and discovered it was indeed due to a missing database. After installing the corresponding package with apk, everything worked properly.
My final thoughts:
The userland code doesn’t really make Chimera Linux stand out. There are relatively few packages in this space, and their impact on the user experience is quite subtle.
The organization of
cports
is impressive, feeling like a modern version of FreeBSD ports. Currently, the number of packages is somewhat limited, but I hope to see improvement in the future.
Chimera Linux isn’t quite mature yet. On one hand, a fresh LiveCD installation shouldn’t have issues like missing
wireless-regdb
/
sddm-dinit
. On the other hand, many things require manual configuration (for instance, Chinese input methods need to be compiled from source, as they’re not available in the software repository or cports).
I’ll continue to follow this distribution’s development. It shows potential, and the developer updates are quite frequent.
This is my attempt at porting the 'popping and locking' theme used in iTerm2, ghostty, atom, vscode, and other tools. I will always consider this a work in progress and I am happy to have anyone jump in and propose adjustments to the color settings!
Screenshots
ASNI Color Palette
Color
ANSI
Black
#1d2021
Red
#cc241d
Green
#98971a
Yellow
#d79921
Blue
#458588
Magenta
#b16286
Cyan
#689d6a
White
#a89984
Bright Black
#928374
Bright Red
#f42c3e
Bright Green
#b8bb26
Bright Yellow
#fabd2f
Bright Blue
#99c6ca
Bright Magenta
#d3869b
Bright Cyan
#7ec16e
Bright White
#ebdbb2
About
This is my attempt at porting the 'popping and locking' theme used in iTerm2, ghostty, atom, vscode, and other tools.
Surely y'all know and have enjoyed Mac OS 9.2.2 booting and beautifully-running on all four Mac mini G4 models for close to 8 years now. (Wow!)
Well, that was one massive revolution...
... But most of us did not think we would live to see the day New World ROM machines, even more so the likes of the Mac mini G4, to NATIVELY boot System 7:
(Gotta love it trying to display 1 GB RAM capacity.)
Before your eyeballs leave your eyesockets completely, I ought to warn that there's still much to be sorted out in this, especially sound, video and networking (the usual suspects). In other words, your mileage may vary, so keep expectations in check!
========================================================
OK, so HOW in the WORLD is any of this possible?
========================================================
It turned out "New World ROM" Macs had a cousin born out of the clone program (until the usual villain, Steve Jobs, came and killed it), which was an architecture called "
CHRP
" (pronounced "chirp"). It was the successor to
PReP
, but, unlike PReP, Mac OS was also going to be officially-bootable on it. Close to no CHRP machines ever saw the light of the day, thanks to Jobs' return. Nonetheless, Apple internally developed Mac OS 7.6 ~ 8.0 for CHRP systems before it got axed. It's just that they never released it, but the development was done regardless. On October 2025, it turned out someone preserved some of these Mac versions, which were then acquired and preserved and shared with the world. (
Macintosh Garden link
,
archive.org link
.)
Although CHRP was left to die, the so-called "New World ROM" Macs inherited much of its architecture and design. As you probably know, these Macs rely on an extra system file called "Mac OS ROM", whereas "Old World ROM" Macs do not need it, and can use their own actual ROM to get Mac OS going. This meant any Mac OS version unaware of the concept of a Mac OS ROM file could not just simply boot in a New World ROM Mac normally. People were able to boot Mac OS versions as low as 8.1, but not any lower, and that too only for the very first few New World ROM Macs, but none of the later ones, which increasingly had a higher and higher minimum OS version.
But not anymore, as the following major events happened:
- The recent Mac OS 8.0 CHRP leaks provided an earlier ROM file that, it turns out, allows regular Mac OS 8.0 to boot, as well. Or, alternatively, the Mac OS ROM file that always worked with Mac OS 8.1 also worked on these Mac OS 8.0 CHRP releases. (Exact details are fuzzy in my memory by now, so someone else might want to correct me if I got something wrong.)
- The recent Mac OS 7.6 CHRP leak provided an additional System Enabler file, which could be exploited for loading Mac OS ROM files. I forget if that's how it worked out-of-the-box, or if a bit of hacking to the System Enabler was required for that, however what I do remember clearly is that, while the System Enabler was hardcoded so that artifically no OS earlier than 7.6 could use it, the OS version check could be patched out of it, so that System 7.5.x (and potentially earlier) can also use it.
In other words,
this file is the reason that earlier Mac OS versions can make use of the Mac OS ROM file
, thus bringing Mac OS 7.6.1 and earlier potentially to ALL New World ROM Macs!
(Trivia tidbit: Apparently this enabler was also present in certain archives of the Mac OS 8.0 betas from when it was still known as "Mac OS 7.7". Oops! This thing was right under our nose all this while!)
- Of course, as hinted at previously, a System Enabler _alone_ is NOT enough to boot System 7 and the like when even much newer systems that were already aware of the Mac OS ROM file could not boot. The newer the model of the New World ROM Mac, the less you could actually "go back". The reason is simple: Mac OS ROM files, over time through its various versions, would get new features added, BUT also would remove older ones which were required by older OS versions. The solution? Using
ELN's great Mac OS ROM patching tools
(plus other tools of his own), "Rairii" AKA "Wack0", known for his amazing PPC Windows NT 3.51 / NT 4.0 project on
PowerMacs
and the
Nintendo GC / Wii / Wii U
, analyzed many of these Mac OS ROM files, and fixed + patched + stitched together new Mac OS ROM files that attempt to keep ALL the old features that were removed AND all the new features that were added. In other words, the ultimate Mac OS ROM file that boots everything and runs everything (roughly-speaking). He also is the one who figured out and hacked the System Enabler to also accept OSes earlier than Mac OS 7.6.
Keep in mind, however, that this effort essentially allows Macs that are already able to boot SOME version of Mac OS to ALSO boot older versions. But if a given machine cannot boot ANY Mac OS version, such as the two DLSD PowerBook G4s (
15"
,
17"
), these patches cannot do anything about that: Their incompatibilities need to be addressed first and separately.
One more interesting thing to note about the similarity between CHRP systems and New World ROM Macs: If you check ANY "Mac OS ROM" file to see its TYPE and CREATOR codes, you will see they are "tbxi" and, you guessed it, "
chrp
", respectively. I couldn't believe "chrp" was in ALL the Mac OS ROM files all these years!
========================================================
Where can I get ahold of this EPIC stuff ? ? ? ? ?
========================================================
Rairii's "super" ROMs are available on
this GitHub repository
, under
releases
. You may also fetch the patched System Enabler for Mac OS 7.6.1 and earlier from there, and place it in the System Folder. Make sure to download the files from the latest release there.
Note that he applied his patches to 3 different versions of the (US) ROMs:
- 10.2.1 with CPU Software 5.9: The "latest and greatest" Mac OS ROM file of all Mac OS. For reference, this is also the ROM version that the
1.628 GB max RAM Mac OS ROM we have was based on (thus going beyond the 1.5 GB limit)
, although do note that the RAM limit break patches are NOT included in this, at least not yet as of the time of writing.
- 2.5.1: A much earlier version of the ROM, but still new enough to support USB. See the GitHub page for details.
- 1.7.1: A very early ROM, which can be well-leveraged by very early New World ROM Macs. See the GitHub page for details.
Note you need ROM version 9.1 or higher to use ATA-6 AKA Ultra ATA/100 AKA Kauai drivers, which are essential on the likes of the Mac mini G4 and the MDD. Special notes for the Mac mini G4 are further down.
========================================================
What is the COMPLETE list of Mac OS versions that now boot?
========================================================
To be exact, this is the complete list of OSes I have attempted, all on the Mac mini G4 1.5GHz model, with the following results:
- System 6.0.8:
No boot
. You get a Happy Mac, followed by a blinking question mark in a floppy icon. (Note: Although this very attempt is UTTERLY insane for multiple technical reasons, it might be not AS seemingly-impossible as one may think, as the 68k emulator resides within the Mac OS ROM file.)
- System 7.0:
No boot
. You get a Happy Mac, but then a warning window pops up saying System 7.0 cannot boot on this computer.
- System 7.1.2:
No boot
. You get a Happy Mac, but then a warning window pops up saying System 7.1 cannot boot on this computer.
- System 7.5:
BOOTS AND IS STABLE
. It requires you to hold shift to turn Extensions (and Control Panels / INITs) off, though, or to get rid of the "Mouse" Control Panel (and possibly more). The system is surprisingly stable! I tested the British version of this one, as Apple's Mac OS Anthology discs did not include the US installers, for some very slacker-y reason.
- System 7.5.2:
Boots, but very broken, close to nothing works
. It could be because System 7.5.2 was always VERY machine-specific, and is apparently one of the most broken versions of Mac OS of ALL time, regardless. The machine-specific enablers, and other things, might be what is making it so unstable.
- System 7.5.3:
BOOTS AND IS STABLE
. It requires you to hold shift to turn Extensions (and Control Panels / INITs) off, though, or to get rid of the "Mouse" Control Panel (and possibly more). The system is surprisingly stable!
- Mac OS 7.6:
BOOTS AND IS STABLE
. Holding shift is not required here. What else can I say? It "works".
- Mac OS 8.1:
BOOTS AND IS STABLE
. Holding shift is not required here, either. Behaves much the same as the others, except we now have HFS+ by default. Still, it did NOT like me having a 940 GB HFS+ partition, and prompted me to either eject it or format it. (To be fair, older OSes tried to do that, too, but Mac OS 8.1 was THE OS to _officially_ be able to handle HFS+ properly, so there are no excuses for it to fail here. Mac OS 9.2 ~ 9.2.2 all work perfectly with it.)
- Mac OS 8.5: No boot. Rather, it seems like it WOULD boot, but starting with Mac OS 8.5, Mac OS now always checks to see if the machine you are booting from is within a list of Apple-endorsed machine IDs for the given Mac OS version. In other words, Mac OS 8.5 does not know what the Mac mini G4 is, nor what a G4 Cube is (our Mac mini G4 ROM file makes the mini pretend to be the latter). It seems it should be possible to patch out the machine check. According to Rairii, this should be able to be patched out by disabling such a check on the "boot" resource in the Resource Fork of the System file, in ID 3 (also known as "boot3"). For Mac OS 8.6, it seems like this check happens at the end of boot3, wherever a check for machine ID 406 is located, in which after it's detected, the code checks to see if the exact Mac model is whitelisted or not.
- Mac OS 8.5.1:
No boot
. All that applies to Mac OS 8.5 also applies to Mac OS 8.5.1.
- Mac OS 8.6:
No boot
. It crashes during the start screen, when the loading bar appears, but before the first extension gets to load. See the top-left corner of the picture for a glitchy visual artifact. Same happens if you try to boot with Extensions off.
- Mac OS 9.0.4: No boot. It crashes during the start screen, when the loading bar appears, but before the first extension gets to load. Same happens if you try to boot with Extensions off. Exact same symptoms as when trying to boot Mac OS 8.6 at least on this mini model, including the visual artifact on the top-left corner.
- Mac OS 9.1:
No boot
. It crashes during the start screen, when the loading bar appears, but before the first extension gets to load. Same happens if you try to boot with Extensions off. Exact same symptoms as when trying to boot Mac OS 8.6 and Mac OS 9.0.4 at least on this mini model, including the visual artifact on the top-left corner.
- Mac OS 9.2 ~ 9.2.2: BEST OS EVER, BOOTS AND RUNS BEAUTIFULLY. 'Nuff said.
Note that, although I describe many of these as "stable", I mean you can use much of it normally (sound/video/networking aside) without it crashing or misbehaving, at least not too hard, but that is not to say everything works, because that is just not the case. For example, when present, avoid opening the Apple System Profiler, unless you want a massive crash as it struggles trying to profile and gather all the information about your system. Some other apps or Control Panels might either not work, or work up to a certain point, after which they might freeze, requiring you to Force Quit the Finder to keep on going. And so on.
As you can see, I did not yet try System 7.5.5, Mac OS 7.6.1 and Mac OS 8.0. That's because they all are most likely working exactly as their neighbouring versions. But feel free to confirm.
Most non-mini systems should be able to boot Mac OS 8.6 ~ Mac OS 9.1 just fine. A "Mac OS 8.6 Enabler", so to speak, by LightBulbFun, can be renamed as e.g. "Sawteeth" and put inside the System Folder for some machines that cannot boot Mac OS 8.6 normally, so that they can, then, boot it. It is actually a Mac OS ROM file, but can function as a complementary, helper file to aid the actual Mac OS ROM file in this case. If you'd like, check
here
for more info. I have attached "Sawteeth.bin" to this post for convenience. LightBulbFun first shared it on
this post
, specifically through this
MEGA link
.
Most non-mini systems should also be able to boot Mac OS 8.5 and 8.5.1, especially on G3s and earlier. Some G4 Macs might need to spoof the Mac model in Open Firmware (or some other Forth script added to ROM) to boot, though, or patch the check out like I mentioned for the mini earlier. The reason the mini doesn't have the spoofing as an option is that any spoofing in OF would be overwritten by its own specialized Mac OS ROM, which spoofs a G4 Cube, which is clearly not in the whitelist of supported machines for Mac OS 8.5 and 8.5.1.
Also note that the mini behaves as reported above with Mac OS 8.6 with or without this "8.6 enabler" file (and with or without the System Enabler for Mac OS 7.6.1 and earlier, both of which don't seem to get in the way of later, nor earlier, OSes).
Most importantly, I did
not
yet attempt to identify which are the latest versions of each Control Panel and Extension for each of these OSes. If I did, I'm sure it would help a lot, and perhaps address quite a number of these problems. The more people chime in on this effort, the better! Imagine if we had a proper "Mac mini G4 System 7.5.5" CD, then an "MDD Mac OS 8.5.1" CD, then an "iBook G3 Mac OS 7.6.1" CD, and so on. Everyone with a G3 or G4 Mac can help by trying things out!
Namely, something akin to MacTron's efforts highlighting the latest Extensions for Mac OS 9.2.2 and Mac OS 8.6 like this, but also for every other Mac OS version:
========================================================
But how did you get the mini to boot? It requires its own special ROM!
========================================================
Indeed it does! All credit goes to ELN and all of those who helped him on Mac OS 9 Lives!: you can simply use his tooling (which was also very useful for Rairii) to re-apply the Mac-mini-G4-specific ROM patches to Rairii's latest 10.2.1 ROM, and voila! It works as well as you would hope it to!
You can even use the resulting ROM for Mac OS 9.2.2, as well, even though you don't have to: Originally, the Mac mini G4 ROM as we see them in RossDarker's Mac mini G4 CDs version 8 and 9 (AKA v8 and v9), as well as in all the previous versions, were based on the US ROM v9.6.1. I could not find an explanation as to why ROM v10.2.1 wasn't used in the end, even when digging the old Mac mini G4 thread again that started it all. Perhaps because we already had a working ROM with v9.6.1 and did not want to risk breaking anything, or who knows. However, I have thoroughly tested Mac OS 9.2.2 with this new ROM combination (latest Rairii 10.2.1 + latest Mac mini G4 patches AKA v9 patches), and from what I could tell, everything behaves
exactly
the same as with the previous ROM we always used. Except now we have the ability to use the same ROM to also boot System 7.5 (I still can't believe this, even though it is true).
(For the record, while the 9.6.1 ROM was also modified to spoof the Mac mini G4 model identifier as a G4 Cube, we also tried to spoof it as a QuickSilver 2002 at one point, but someone reported sound issues with that, and so it was quickly changed back to a G4 Cube and such a change never made it into one of RossDarker's CDs. So just about everyone using Mac OS on the mini for all these years has had a ROM reporting to the OS as a G4 Cube, exclusively.)
To apply the Mac mini G4 patches, I used ELN's
tbxi
and
tbxi-patches
to apply his "macmini.py" script. You can follow the instructions as per the tbxi-patches page, which you should not let intimidate you even if you are not used to this kind of thing. It's quick and easy, and the scripts are also fully-commentated very nicely by ELN if you are curious about what it is doing and why.
In my case, first I tried using the latest Python 3.13.9 both from Windows 7 (bad idea due to resource fork loss) and macOS 10.14.6 Mojave, but neither worked: it seems like that version of Python was just too new. I then retried with
Python 3.8.10
instead (which I chose thinking it might be more period-appropriate for the script's age) on Mojave, which worked
flawlessly
. I didn't try it, but perhaps an older Python version might work on PowerPC OS X, as well.
I used the Python installer
from the official website
, and I also used an "official" Git installer from
here
(thus avoiding any package manager headache... man, how I hate non-Mac-OS systems, including OS X, and package managers in general...)
If somehow someone with plenty of Python knowledge and the willingness to put enough time into it wished to, both tbxi and tbxi-tools could, perhaps, be ported to
MacPython 2.3.5
, so that we could do all this patching from Mac OS 9.2.2 directly and natively without leaving our main OS. That would also be awesome! (Of course, it helps that this is also available on more recent systems nonetheless, because then everyone gets to join in on the fun with all kinds of different backgrounds and setups.)
For convenience, I attached the final patched ROM to this post, so that anyone can go wild on their minis right away!
========================================================
Why should I care when Mac OS 9.2.2 already boots, and runs better?
========================================================
It is also my opinion Mac OS 9.2.2 is the greatest OS, and Mac OS, ever, but not everything that is possible in earlier Mac OS versions is possible in Mac OS 9.2.2. For example, some software requires Mac OS 9.0.4 or earlier to work. A lot of software is System-7-exclusive.
Some people also just prefer the likes of System 7 for its even-lighter memory footprint, lack of mandated Appearance Manager and the like. Mac OS 9.2.2 is already overkill-fast on the mini, and on most New World ROM Macs, but the likes of System 7.5 are just RIDICULOUSLY fast. Even more ridiculously. I still am trying to come into terms with how indescribably fast using it on the mini was. It got even faster when I thought there was no way to get "faster than instantaneous", as Mac OS 9.2.2 always felt instantaneous like no other system already!
People might also have some other kind of reason and/or special attachment to an earlier OS version. Or maybe people want to explore older OS APIs and behaviors, perhaps even make a new application they want to know how it will behave on bare-metal not just on Mac OS 9, but also System 7 etc..
The value is in opening up the doors that give us, the users, more options that help us all out.
========================================================
Final remarks
========================================================
Above all, thank you to everyone that made this possible. But I wanted to emphasize and give special thanks to Rairii for engineering all these ROMs, Mac84 for archiving and sharing all the CHRP discs, ELN for engineering all the Mac mini G4 ROM compatibility scripts and creating all the ROM and other Mac OS tooling, and to the Mac community at large everywhere that assisted in all of this into becoming reality. There's honestly many, many people to thank we owe over this one way or another, both in small and big ways.
I can't wait to see what people will do with all these new Mac OS versions on their New World ROM systems over the course of time!
Our promise has always been to deliver innovative products while protecting the trust and safety of our users.
We want to address openly and clearly changes now taking place as we phase out Neato cloud services.
Why the Change?
Neato Robotics was a company within the Vorwerk Group that ceased operations in 2023. Prior to its closure, we had committed to providing five years of cloud services for our products, with the aim of ensuring that customers could continue to use their robots as smoothly and reliably as possible, even after Neato ended its operations.
In line with Vorwerk’s philosophy of always acting in the best interest of customers, Vorwerk decided to continue maintaining the Neato cloud platform beyond the company’s closure in order to honour the original five-year service promise.
Since 2023, cybersecurity standards, compliance obligations, and regulatory frameworks have advanced significantly. Under these new conditions, the existing cloud infrastructure can no longer be maintained in a reliable and future-proof way. Updating this environment would not be technically meaningful, nor would it ensure the level of quality and sustainability our customers can expect.
What This Means
Protection first: We are now phasing out Neato cloud services.
What Has Not Changed
Warranty remains valid: The manufacturer warranty commitments remain intact.
Robots still work: Your Neato robot will continue to function manually. Simply press the button once to launch a full house run.
Support available: Find all the support online at neatorobotics.com, where you can find guidance and practical assistance.
This decision affects only Neato Robotics. Other Vorwerk products and services remain fully unaffected.
We understand this change impacts how you use your Neato robot, and we do not take this lightly.
This decision was not made out of convenience or incapacity.
It reflects our duty to protect your privacy, your data, and your safety in a world where the technological and regulatory environment has fundamentally shifted.
Thank you for your understanding and for being part of the Neato community.
Was this article helpful?
That’s Great!
Thank you for your feedback
Sorry! We couldn't be helpful
Thank you for your feedback
Feedback sent
We appreciate your effort and will try to fix the article
As I detest gamification, I will cheat a highscore and be done with GH's social media aspects. This script will spam commits to populate your profile's contribution grasph.
- choose target date and number of comits
- generate file or modification
- back commit it
git commit --date "10 day ago" -m "Your commit message"
If it's the last commit, or a range of commits, then add:
git rebase --committer-date-is-author-date <ref> where <ref> is the last commit/branch/tag/
output after comiting looks like: [master 47ffff4] test0
you want the 47fffff4 as ref!
A ChatGPT prompt equals about 5.1 seconds of Netflix
Simon Willison
simonwillison.net
2025-11-29 02:13:36
In June 2025 Sam Altman claimed about ChatGPT that "the average query uses about 0.34 watt-hours".
In March 2020 George Kamiya of the International Energy Agency estimated that "streaming a Netflix video in 2019 typically consumed 0.12-0.24kWh of electricity per hour" - that's 240 watt-hours per hou...
Or double that, 10.2 seconds, if you take the lower end of the Netflix estimate instead.
I'm always interested in anything that can help contextualize a number like "0.34 watt-hours" - I think this comparison to Netflix is a neat way of doing that.
This is evidently not the whole story with regards to
AI energy usage
- training costs, data center buildout costs and the ongoing fierce competition between the providers all add up to a very significant carbon footprint for the AI industry as a whole.
You Can Get It If You Really Want | Jimmy Cliff. Why Trump Supporters Are at a Breaking Point, I'm Genetically Male. Republicans Get Bad News in Deep Red District. Nuremberg | Movie.
Welcome to a special edition of FOIA Files. This morning, the Federal Bureau of Investigation turned over dozens of
emails
to me that reveal some details about how FBI agents and personnel from the Freedom of Information Act office reviewed and processed the Epstein files earlier this year. Let’s dive in! If you’re not already getting FOIA Files in your inbox,
sign up here
.
The legacy of disgraced financier Jeffrey Epstein continues to hang over national politics. Earlier this month, President Donald Trump signed legislation that required the Justice Department to release the Epstein files. Soon, the public may finally get to see at least some of what the government has in its voluminous cache, which comprises more than 300 gigabytes of data and physical evidence from its criminal probe of the serial sex abuser. Getting to this point has been quite a winding path that started just after Trump took office—and that FOIA Files has been covering.
As I
reported
in March, after a botched rollout of what Attorney General Pam Bondi described as “Phase 1” of the release of the Epstein files, FBI Director Kash Patel ordered around 1,000 FBI special agents to team up with the bureau’s FOIA personnel at an FBI facility in Winchester, Virginia to prepare the Epstein files for public release.
The army of agents from the New York and Washington field offices, along with FOIA officers, were instructed on how to review and apply redactions to the documents.
Over the summer, I filed a wide-ranging FOIA request for those directives, as well as communications between agency personnel pertaining to their review of the files and the taxpayer dollars spent on the marathon two-month process. I then sued the FBI to compel release of the documents.
Just this morning I got the
partially redacted records
from the FBI. They mostly consist of emails that provide a look behind the scenes as agents and other FBI personnel started to work on the documents. The bureau withheld more than 161 pages citing ongoing law enforcement proceedings and other FOIA exemptions.
The emails reveal the special training given to FBI personnel working on what it called the “Epstein Transparency Project.” In some instances they referred to it as the “Special Redaction Project.” The training entailed PowerPoint slide presentations and video instruction on how to review the files.
The records I got also reveal the number of hours the FBI devoted to the project, which required some agents to work nights and weekends. The FBI paid personnel from various divisions, including counterintelligence and international operations, $851,344 in overtime for working on the Epstein files between March 17 and March 22, according to the documents. FBI personnel clocked in a total of 4,737 hours of overtime between January and July. Of that, more than 70% occurred during the month of March while personnel reviewed the Epstein files, the documents show.
In an email on March 10, FBI personnel from the Office of General Counsel and the bureau’s Information Management Division discussed pending FOIA requests for Epstein-related records and digitizing and redacting “physical files” and the bureau’s “commitment to transparency.”
Another email describes categories of videos the FBI reviewed related to certain Epstein files, which includes “search warrant execution photos,” “street surveillance video” and “aerial footage from FBI search warrant execution.”
A couple of weeks later, an email sent from the Information and Management Division said the office “continues discussions with DOJ and is awaiting clarification regarding additional criteria for the next phase of this project.”
“We continue to scope and update workflow processes and training material based on those discussions,” the March 22 email says. “Updated training materials and workflow guidance is expected for dissemination later tonight.”
The next day another email was sent to FBI personnel reviewing the Epstein files advising them to “stand by.”
“We have identified more files requiring Phase l review. Please continue to refresh as files will be populated momentarily,” the email said.
The emails indicate FBI personnel were continuously checking out Epstein files to review and redact.
On March 24, an email sent to FBI personnel said “Phase 1 redactions are complete” and “Phase 2” was being prepared for “final delivery to DOJ.”
“Phase 2 review of the new criteria provided by DOJ was approximately 75.2% complete,” the email said. “Upon completion of Phase 2,” the Information Management Division “will provide a copy of all see-through redaction files for DOJ review.”
Bureau personnel also reviewed videos. According to an April 15 email, one of the videos was from the New York City jail where Epstein was found dead a month after his arrest in 2019 on sex trafficking charges. (The DOJ publicly released 11 hours of the prison video in July.)
On May 2, an FBI employee from the New York field office sent an email and attached a document titled, “Epstein Overview FINAL” that summarized their work. The FBI withheld a copy of the attachment.
Bloomberg News
(originally Bloomberg Business News) is an American news agency headquartered in New York City and a division of Bloomberg L.P. Content produced by Bloomberg News is disseminated through Bloomberg Terminals, Bloomberg Television, Bloomberg Radio, Bloomberg Businessweek, Bloomberg Markets, Bloomberg.com, and Bloomberg's mobile platforms. (
Wikipedia
.)
Palestinians Offer a Much Clearer Path to Peace
Portside
portside.org
2025-11-29 00:41:07
Palestinians Offer a Much Clearer Path to Peace
barry
Fri, 11/28/2025 - 19:41
...
In just over one week since the United Nations Security Council passed a resolution affirming U.S. President Trump's 20-point cease-fire plan and an international stabilization force for Gaza, the worst fears of many Palestinians seem vindicated.
In response to what it claims are Hamas violations, Israel has escalated airstrikes against Gaza daily,
killing hundreds
– including
two children per day
on average –since the cease-fire went into effect. Hamas has released all the living hostage and most of the bodies of those killed on October 7. Yet the IDF remains entrenched and holds over half of Gaza, it
allows or restricts aid
at will, and the international force is nowhere to be seen.
Some analysts were quick to protest the flaws and holes of the UN Security Council resolution. Yet what is the alternative? The preferred Palestinian way forward wasn't immediately obvious – but it exists.
There are clear caveats to any answer. Hamas and Fatah are both widely reviled by Palestinians, and can hardly be seen to represent the people. Yet no one Palestinian can speak for a range of perspectives within civil society.
But Palestinian voices display significant agreement on some essential principles for a Palestinian cease-fire, peace and recovery plan. These principles respond to the current U.S.-led process but also reflect long-standing Palestinian positions and demands that they have expressed for years.
Perhaps the most comprehensive, pragmatic, visionary plan for a path forward is the
Palestinian Armistice Plan
, released earlier this year. Co-authored by a group of Palestinian scholars and policy thinkers and sponsored by the Cambridge Initiative on Peace Settlements, this 51-page document is packed with details about how the authors propose that Gaza should move from war to cease-fire, to international intervention, to peace.
Both logical and obvious
Responding to the recent Security Council resolution, the basic principles for a better immediate cease-fire plan range from the logical to the obvious. First, Palestinian critics repeatedly point out that any internationally brokered plan
should bring Palestinians into the process
. President Trump and his team are in regular dialogue with the Israeli leadership, while Qatar has effectively come to represent Hamas in negotiations, though Hamas barely represent Palestinians.
Jamal Nusseibeh
, a Palestinian-American co-author of the Armistice Plan, who is also a scholar, lawyer and investor, explained to Haaretz that formally, the Palestine Liberation Organization is still the sole recognized representative of the State of Palestine and should be at the table.
Second, while Palestinian representatives have requested international intervention for years, they
repeatedly insist
that any such effort must rest on international law. The current plan dismisses international law in several ways: It avoids referring to
earlier UN resolutions
– apparently
unprecedented
for the Security Council.
Despite UN recognition as a non-member observer state and nearly 160 individual state recognitions, the current resolution aspires to faraway, conditional statehood, instead of treating Palestine as a sovereign state now. That renders recent recognitions by France and the U.K., two permanent Security Council members, flimsy.
International law also requires adhering to the
International Court of Justice advisory opinion in July 2024
, which ruled that the entire occupation of Palestinian territories is illegal and must end. That would mean insisting that Israel withdraw from sovereign Palestinian territory, as the international force moves in for the transition to Palestinian governance. An international force, from the Palestinian perspective, is welcome under those terms – a whole chapter in the Palestinian Armistice plan is devoted to the issue.
Instead, many fear that the current International Stabilization Force (ISF) envisioned by the UN resolution is destined or even designed to freeze the status quo. Nusseibeh noted that Palestinians therefore see it as "legalization of the occupation," or
"colonial oversight"
as per an article by Yara Hawari, co-director of Al-Shabaka, a Palestinian policy hub. No Palestinian I spoke to gave credence to the resolution's vague mention of Palestinian "technocratic, apolitical committee," for Gaza, the eventual return of the PA, or the suggested, futuristic "statehood."
A third principle, then, overlaps with the
bottom line for successful international interventions
world over: a final status political end point. Politically, says Nusseibeh, a Palestinian plan for international intervention needs to treat Palestine as a state.
There are important implications to naming the endgame of Palestinian sovereignty as the aim of the ISF. For example, this would imply a mandate over Gaza and the West Bank too, where Palestinians need protection from the latest wave of
Jewish terror attacks
.
The Palestinian Armistice plan explains: "To support the transition to Palestinian self-determination, the peacekeeping force's mandate should cover the entire OPT, allowing the troops to maintain security and act as a buffer between Israelis and Palestinians. Its mandate should be to not only monitor violations, but also enforce the peace; its troops should therefore replace all Israeli forces within the OPT." More succinctly,
Nusseibeh recently wrote
that the region needs "a peace force for Palestine, not a stabilization force for Gaza."
Moreover, he views this move towards statehood with physical international protection as the main incentive for Hamas to disarm – since resistance will become unnecessary absent occupation – and join the PLO. Omar Rahman, a fellow at the Doha-based Middle East Council on Global Affairs, concurs. "They would agree to disarm and disband as part of that political process that's in place for ending the occupation," he told Haaretz.
That means accepting the framework of two states. Hamas has
indicated
openness to the
PLO-integration
path
more than once
– many times more than Hamas has ever agreed to disarmament in the present political vacuum.
A horizon for Hamas' disarmament, in turn, could prompt the much anticipated, as yet uncommitted countries such as Indonesia, Egypt, Azerbaijan or any others, to participate in the ISF. As it is, their participation is already "tied to a political horizon and [without that] they're not going to get trapped in Gaza doing Israel's dirty work," said Rahman.
These are not small considerations; the Palestinian pathway would facilitate what Trump claims he wants to do.
Finally, some Palestinians are incensed that the international process does not include a mechanism for Israeli accountability. A
network of Palestinian
civil society organizations in Palestine included an accountability mechanism within its list of demands of the international community, in response to the Security Council: "accountability for Israel's historic and ongoing mass atrocity crimes, including support for the establishment of an international, impartial and independent mechanism to investigate crimes committed against the Palestinian people."
When asked about Hamas' crimes against Israelis, Rahman responded that if the process for accountability is based on international law, then both parties should be held accountable, including Hamas – but he pointed out that most of the planners of October 7 are already dead. Nusseibeh felt that it would be "helpful if there were some kind of reference, at least, to what most people by now are calling a genocide."
Hope matters
This list of problems with Trump's plan is not exhaustive, but neither are the solutions that arise from Palestinians themselves. Some additional initiatives deal with the most immediate issues, such as the
group of Gazan municipalities
that spearheaded the remarkable "Pheonix-Gaza" reconstruction project.
Together, Palestinian engineers, architects, university students and researchers have produced a
document of extraordinary scope and optimism
, dedicated to reconstructing housing, health, education, neighborhoods, heritage and more. What's needed is a ceasefire and a political horizon to draw external commitments and funds.
Among Palestinians, the principles, vision and plans are there. Nusseibeh raised one final item the international community can provide, that the people of region – Israelis and Palestinians alike – so desperately need. Referring to peace process of a bygone age, he said, "The only way that we can begin to climb out of the hole that we're in right now is to provide that hope. And that hope is only going to come if we have a properly constructed international drive towards a long-term peace."
Haaretz
is an independent daily newspaper with a broadly liberal outlook both on domestic issues and on international affairs. It has a journalistic staff of some 330 reporters, writers and editors. The paper is perhaps best known for its Op-ed page, where its senior columnists - among them some of Israel's leading commentators and analysts - reflect on current events. Haaretz plays an important role in the shaping of public opinion and is read with care in government and decision-making circles. Get a
digital subscription
to Haaretz.
Notes on How To Survive a War
Portside
portside.org
2025-11-29 00:10:32
Notes on How To Survive a War
barry
Fri, 11/28/2025 - 19:10
...
This spring, a Russian missile slipped through Ukrainian air defences and crashed into a three-storey building, killing 13 members of an extended family, save for a young girl who was pulled free from the wreckage.
When I arrived a few weeks later, an area the size of two tennis courts was flattened into rubble, the windows of every apartment looking out into the blast zone were blown out by the explosion and now boarded with plywood, parked cars were mangled out of shape, the paint burnt off their chassis. Somehow, amidst all this wreckage, a few walls of the building were still standing, wallpaper intact, fittings still protruding from the upright surfaces.
On a small rectangle of grass, not far from the blast site, neighbours had assembled a small shrine to the dead: photographs, bouquets of red plastic flowers, two basketballs scribbled with messages of condolences, dozens of stuffed toys, and a large plastic flask of drinking water.
Around the corner on Aviakonstruktorska street, where aeronautical engineers once lived in the Soviet-era apartments and worked at the nearby headquarters of Antonov (makers of the An-225 Mriya, the largest aircraft ever built), Kyiv felt like any other Eastern European city awakening from the winter.
Get one Open Democracy story, direct to your inbox every weekday.
Sign up now
The city was beautiful in the spring; the chestnut trees were in bloom, their silken seed pods hung in the air like snowflakes, enshrouding the city each time the wind blew. People went about their lives – running late for meetings, buying groceries, cursing at traffic, enacting their rituals of the everyday, knowing all the while that at any moment some inventive new munition could whistle through the air and detonate their lives on impact.
Each night of my visit, and sometimes in the day, Russian forces launched hundreds of drones and missiles at Kyiv. Most were shot down by the city’s air defences, every now and then, one made it through.
The war wasn’t always visible, said Anatoliy, a young man who lived in one of buildings that looked out into the blast zone, describing the surreal experience of awakening each morning in what was recognisably a warzone, descending through a staircase of broken windows, walking past past mounds of rubble and mangled metal, before turning a corner into a fully functional city en route to university where he studied cybersecurity.
“But the war is never far away,” he said. “It is always close.”
My companion on this visit was Volydymyr Yermolenko, an essayist, journalist, podcaster, the president of the Ukraine chapter of PEN (a world-wide association of writers), and a philosopher who has spent the past three and a half years thinking about how experiences like Anatoliy’s – where the war is both distant and proximate – are playing out across his country.
“In 2022 there was this, you know, adrenaline, that whatever Russians do, we will mobilise, we will beat them,” Yermolenko had told me when we met in 2024 at Zeg, a storytelling festival held annually in Tbilisi, Georgia. “Now, primarily because of the death toll on the front line, people who volunteer for the army, they understand that they will probably never come back. So this, this makes things a lot, a lot tougher, there are more fractures in the society, like, who does what? Soldiers are asking, why I am on the front line, when you are not on the front line? Why did you leave Ukraine?”
A year after that conversation I was in Ukraine to understand these cleavages; these cracks and fractures between bombed out neighbourhoods and treelined boulevards, between frontlines in the east and cities in the west, between those who stayed and those who left, between Europeans and Americans who insist this war is an existential battle between Good and Evil (but are still open to striking deals with both sides) and the rest of the world who see it as yet another proxy war between Great Powers.
I was here to witness how Ukrainians were negotiating these fissures, even as they navigated the daily horrors of war and considered the prospect of a supervised dismemberment of their nation.
To survive this war, and the dirty peace that will follow, Ukrainians will probably need to re-imagine and re-constitute themselves with a fifth less territory and the loss of tens of thousands their people, in the backdrop of decades of headspinning revolution: the Revolution on Granite in 1990, which set the stage for Ukraine’s departure from the Soviet Union; the Orange Revolution to overturn the stolen election of 2004, the Euromaidan protests and the loss of Crimea to Russia in 2014, which metastasized into this current conflict.
This spring, signs of collective reimagining were everywhere. Active duty soldiers, grieving parents of the dead, poets, writers, young Ukrainian emigres, senior officials at the Ministry of Culture, philosophers like Yermolenko, each spoke of the urgent need for new solidarities and forms of kinship, without which the relentless exposure to death would strip everyday life of meaning and would make it impossible to dream of the future. Each conversation carried the weight of living through a historic moment as if the ideas that emerged from the present would shape the future of Ukraine, and much of Europe, for many years to come.
“If I think of a Ukrainian identity, I think of a palimpsest,” said Iryna Starovoyt, a poet, essayist and a scholar of cultural studies, one evening as I accompanied her on a walk through the centre of Lviv, a medieval city in western Ukraine that was settled by Galicians, raided by Mongols, gave refuge to Armenians fleeing war in Central Asia, and was taken by the Soviets as part of the Molotov-Ribbentrop pact that divided Poland between the Soviets and the Nazis.
“Like a palimpsest, every new power writes their story but leaves traces of what was before,” Starovoyt continued. “There is a new Ukrainian identity; but I don’t know if I can translate it. Not from Ukrainian into English, but from intuition into language.”
Aman Sethi, openDemocracy
“A country is not only a ‘piece of land’, but a narration about this land,” Renata Salecl, a Slovenian philosopher and psychoanalyst, writes in
Spoils of Freedom
. In her book about the fall of Yugoslavia and the rise of post-socialist societies, Salecl describes a country as a collective fantasy with land, and the myths and narratives that link people to that land as its constitutive elements.
Salecl was writing in the mid-1990s, but her insights from that period offer a useful prism to understand the crisis unfolding in Ukraine today.
The aim of a war of aggression, she wrote, is to shatter the very way a people perceive themselves and formulate their identity.
“When Serbs occupied a part of Croatia, their aim was not primarily to capture Croatian territory but to destroy the Croatian fantasy about that territory,” Salecl wrote. “The Serbs forced the Croats to redefine their national identity, to reinvent national myths and to start thinking about themselves in a new way, without linking their identity to the same territories, as they had done before.”
Something similar is underway in the Russian invasion of Ukraine. The Kremlin insists that Ukraine itself is a Soviet creation, that its lands have always been Russian, that the Ukrainian national identity has no historical basis and is a ruse to divide Russians and Ukrainians who, in the words of Russian President Vladimir Putin, are
“a single whole”
.
Russian soldiers and munitions have killed at least 55,000 Ukrainians since the invasion began, of which 12,654 are civilians, according to
figures released by the UN
in February 2025, and the remaining 43,000 were soldiers according to
a December 2024 Telegram post
by Ukrainian President Volodymyr Zelensky, with another 400,000 injured. Over six million Ukrainians have left the country as refugees and an estimated 3.7 million people had been internally displaced by the war as of March this year according to the
International Organisation for Migration
.
At present, Russia’s armies occupy nearly a fifth of Ukraine’s land area. Death, displacement and outward migration have meant that Ukraine, which had just over 50 million people when it gained independence in 1992 and about 40 million people prior to the Russian invasion in 2022, has a population of about 31 million today in areas controlled by Kyiv. That is a lot of people and territory to lose, and a lot of grief and trauma to process.
Today, Russia demands almost all of eastern Ukraine, including lands it does not yet control, as a precondition to halt its military aggression. Lands that, the Kremlin insists, were never Ukrainian to begin with.
So Russia’s war in Ukraine is about territory and the various geostrategic imperatives described by foreign-policy experts. But it is not just about territory, much like the bloodshed that accompanied the disintegration of Yugoslavia was not simply about who got what piece of land, but what it
meant
for a particular parcel to be designated Serbian, or Croat, or Bosnian, or Kosovar, or – in this instance – Ukrainian.
“Russia does not need more land. It is the largest country in the world by land,” Yermolenko told me. “This is a war about identity.”
“Ukrainian identity, we cannot find it in the past. We can find elements of this identity in the past, but we need to make a kind of patchwork of something new,” he said.
This patchwork, Yermolenko surmised, could even serve as a model for other communities and peoples struggling to hold on to their sense of self at a time when questions of national identity have never felt more urgent, or divisive, and military skirmishes along long-disputed borders are increasingly escalating into full-blown wars.
“None of us Ukrainians are thinking about returning to a Golden Age, because what is our golden age?” Yermolenko said, “We’ve only had 30 years of independence. Wow. Incredibly deep history but just 30 years of independence. So it is an identity in the making.”
Aman Sethi, openDemocracy
It is important that ordinary people, wherever we are, understand this war on our own terms. Unless we are careful, the sleight of hand of ‘national interest’ nudges each of us to role-play as international statesmen speaking on behalf of our countries, our militaries, and our oligarchs, at our own expense.
We live in a time of extraordinarily unrepresentative political leadership: In the Middle East, flotillas of concerned, unarmed, civilians sought to intervene in the Israeli genocide in Gaza by dodging Israeli military barricades at sea just to deliver baby-food to dying Palestinians only to be turned back, even as our own leaders struck arms-deals with Israel and wrung their hands when the Israeli military used those same weapons to bomb schools and hospitals, and continues to kill with impunity despite a ceasefire.
In Ukraine, the war has led supposedly resource-strapped governments in Europe and the United Kingdom to shower their defence industries with largesse even as their citizens are forced to make do with less; it has prompted the Indian government to insist that buying crude oil from Russia is a strategic necessity that outweighs all other risks to the economy; it has normalised an old-fashioned colonial resource-grab in which the US now sells arms to Ukraine in return for access to its minerals. Each of these trade-offs has been sold to us, wherever we live, as the sacrifices we must endure to preserve ‘our way of life’, or as Salecl would put it, to preserve the sanctity of our respective national myths.
We are all shaped by our myths, but we needn’t be bound to them. We are all born into our respective national identities, but we can each choose to reinterpret what they mean to us.
One morning, at a museum in south-western Ukraine, I glimpsed how two Ukrainian women from the city of Odessa, born 50 years apart, each painting in her early thirties, had both strived to make sense of a time of cataclysmic change, each making art and myths for her people to comprehend history’s long arc.
Few cities are better suited to such thinking: Odessa is built on lands that, over 2,000 years, have been claimed by the Greeks, the Tatars, the Ottoman Turks, the Russians, and, of course, the Ukrainians. The palace that houses the National Fine Arts Museum was built in the 1820s in the Russian classical style, when Odessa was a major Black Sea port of the Russian Empire.
The city has been repeatedly targeted by Russian airstrikes since 2022, so when I arrived, the museum’s most treasured collections were sequestered away in crates in bomb shelters; a video projector displayed photographs of these artefacts on a darkened wall near the entrance, in much the way that museums across Africa and Asia must make do with facsimiles of their stolen treasures now on display at the British Museum in London.
Even so, two halls of the museum had been turned over to the work of Liuda Yastreb, a Ukrainian modernist who graduated from the Odessa school in 1964. Yastreb was trained in the prevailing style of Soviet social realism at a time when the regime viewed art solely as a weapon of class struggle.
But the work on display was from Yastreb’s days in the Soviet underground art scene in the 1970s, when artists in Moscow, Odessa, Leningrad, displayed their most interesting and transgressive work in so-called “apartment exhibitions” in the relative privacy of each other’s homes.
The figures in Yastreb’s work occupy much of her canvases: Eve, naked and voluptuous, looks over her shoulder and smiles knowingly at the viewer as she holds a tiny, cherry-sized apple, and a cheerful serpent peeks out from behind a tree. Three naked women, their tangled limbs create the illusion of a three-headed four-armed many-legged creature as they run through a riotous watercolor cityscape in joyful abandon, an enormous winged female figure hovers in midflight over toy-like apartment towers.
The final exhibit was a translation of
Spiritual Stoicism of the Artist in Soviet Society,
Yastreb’s 1979 essay, in which she described the psychic dislocation of selfhood and identity during Ukraine’s Soviet period.
“People's souls must be brought back from the emptiness caused by the revolution, war, famine, and destruction that accompanied them for so long,” Yastreb wrote. This task, she said, fell to artists such as herself to act as “rubbish collectors” gathering the discarded relics of her time: Churches, icons, sculptures, ancient books, painting, engravings, folk art.
“All this was being dusted off, cleaned, as if it were seen for the first time, but known long ago.” she wrote. “There was an urgent desire to revive despised traditions” in service of creating what she described as “real works of art appeared, reborn, radiating light and power.”
Across the hall from the Yastreb exhibition, I visited an exhibition by Anastasiia Kolibaba, a Ukrainian artist born in 1994 – three years after the dissolution of the USSR.
On her website, Kolibaba describes her practice as documenting events in service of “the creation of an elementary modern myth.” Her paintings of the current war look like photographs washed by a blur-filter to create an effect eerily reminiscent of Soviet-era imagery updated for the age of Instagram.
Where the figures in Yastreb’s work occupy most of her canvases – the individual self striving to take up space – Kolibaba’s work tries to claim Ukraine’s vast empty landscapes from threats that originate just beyond the horizon.
In
Combatants
, the exhaustion of three soldiers and their dog on patrol seeps through the canvas; in a series of three watercolours titled
Trophy,
the artist offers three perspectives on a destroyed tank; in a work titled
Air Defence,
a distant flash of light illuminates a darkened sky along a desolate shore
.
My phone buzzed with a notification for an incoming air strike as I left the museum for my next meeting. “Usually it is nothing,” said the taxi driver, gesturing out of his window to people sipping coffee at an outdoor cafe. “Usually.”
Aman Sethi, openDemocracy
After two weeks of sheltering from Russian air raids in the corridors of the apartment complex where he lived in Kyiv, the anarchist looked into the eyes of his spouse and children and thought, “It cannot go on like this. We cannot keep living like this.”
He signed up to join the army the next day, and has been rotating in and out of the conflict’s shifting frontlines ever since.
“If there is anything I can take from my anarchist self, I think it is to resist the feeling of powerlessness,” he told me over a patchy WhatsApp call from somewhere in eastern Ukraine. “When you drive through destroyed cities, when you look inside bombed apartments, you see the remnants of people’s lives. And you tell yourself, ‘I will stand against this.’”
In the years before the war, the anarchist was a keen observer and lucid interpreter of the ideological undercurrents and debates animating Ukrainian society. In the years after the Russian occupation of Crimea in 2014, human rights organisations in Ukraine tracked the rise of volunteer paramilitaries who embraced far-right politics and Nazi imagery, opposed LGBTIQ rights, and embraced Christian nationalism in a manner increasingly visible in far right movements across the US, the UK, and Central Europe.
Since the full-scale invasion in 2022, the members of these paramilitaries have been absorbed into the Ukrainian military where they fight alongside a wide cross-section of Ukrainian society who have thrown themselves into the war effort. As a consequence, the anarchist said, the classic divisions of left and right are becoming much harder to parse because the world has changed so much that it has ceased to make sense.
“We have all kinds of people in my unit. The far-right, far-left divide is completely irrelevant now, ” he said. “Right now, the blunt reality of things is that I am killing Russians. And I’m trying to get better at it. Does that make me a right-winger? I don’t know.”
When the men in his unit spoke of identity, he said, “we speak about emotions.” And for now, the Ukrainian identity at war is defined by one pure emotion: A defiance shared by 30 million people united by extreme circumstances. “It is the simplest emotion of ‘What the fuck? Get out of my space.’”
But the longer the war goes on, the more it is changing him. War coarsens, war bruises, war scars. War warps your perception of reality and stretches the boundaries of what is normal. Sometimes, the war makes it impossible to imagine peace.
“For me, as a writer, it is crucial to be able to feel things. My job is to feel things, to keep my emotions attuned to what is happening in front of me,” he said. “But right now, I’m in survival mode. Right now, I need to shield my emotional self.
“Am I becoming someone else? Who will I become? I want to preserve my ability to listen to people,” he continued. “And from the level of an individual, imagine this process going upwards to the level of a country.”
Ukraine’s future, he continued, will be determined by who and what survives the war. “It will be shaped by what survival means for me versus my comrade,” he said. “Right now we are riding a tsunami of emotions to who knows where. If we are lucky to live, we will have a very interesting period of identity building and we will each be in a position to shape what emerges after.”
For much of 2022, three-quarters of Ukrainians favoured fighting on until victory was attained. Three years later, polls
indicate nearly 70%
now favour a negotiated end to the war, even as they doubt the fighting will end anytime soon. But as much as many in Ukraine, in Europe, and around the world, yearn for a ceasefire, some of the soldiers on the front fear a temporary pause in the fighting almost as much as a war without end.
“Sometimes I feel the worst case scenario is a ceasefire, the army is dismissed, we all go home. And then two years later, the Russians go again,” he said.
Right now, 30 million Ukrainians are sacrificing their lives, their careers, their mental health, their ability to feel, as if four long years ago, the entire country took a deep breath and is yet to exhale. Right now, it is impossible to predict if the experiences and traumas of war will shift Ukrainian society to the left, the right, or what these labels will even mean.
When I was reviewing my notes and transcripts, I noticed how often the people I met on my trip said “Right Now”, as if the war had trapped them into fossilised amber of the eternal present: Right now is not the time to ask how the war is going; right now is the time to fight the war. Right now is not the time for divisive talk. Right now, we can only think about the right now.
Or, as a senior adviser to the Ukrainian government put it when I asked him about his department’s plans to deal with the PTSD that would inevitably follow the war, “Right now, there is no trauma. Right now there is only life experience.”
A mantra to survive the Right Now, as narrated to the author on a late night taxi ride in Lviv:
Live in the moment.
Ask yourself:
Is this a catastrophe?
Or is this just catastrophic thinking?
Remind yourself:
Before the war
you and all your friends could still have died in a road accident,
or some other unforeseen calamity.
Tell yourself:
Before this, it was almost like this.
Poet Halyna Kruk | Aman Sethi, openDemocracy
When her young son died on the front, a woman stopped talking to her sister.
She stopped talking to her sister because her sister has many children and grandchildren.
And her sister always has news to share of her children, but the woman has no news to share of her dead son.
Halyna Kruk heard this story from a woman who walked up to her after a poetry reading. This is, as Kruk noted, only one of hundreds of thousands of such stories. This is what a war feels like.
Kruk is a poet, writer, and professorof medieval literature, who has volunteered along the grey-zones in east Ukraine since the 2014 invasion when Russia took control of Crimea. She met her future husband on one of these trips when she stopped to distribute medicines, supplies and books at a military camp not far from Marioupol.
The day we met at a coffee shop in Lviv, her husband had just returned home from what he hoped would be his last deployment at the front. After our coffee, we walked out into the street where Kruk bumped into an acquaintance she hadn’t met for many years. When Kruk asked about her family, the acquaintance said her son had died on the front.
“There are these new divisions emerging in society,” Kruk said, recalling the grieving woman at her poetry reading.
“In the first year of war, families were broken because the distance in experience was so vast that the connection was broken,” she said. “In one family it was hard for people to speak to each other.”
Since the outbreak of war, Ukrainian men of fighting age have needed special permission to leave the country. The law was very recently relaxed to allow men aged between 18 and 22 to travel. Meanwhile at least 6 million mostly women and children have left Ukraine in 2022 in search of safety. With every passing year, they become less likely to return according to data gathered by Centre for Economic Strategy, a Ukrainian think tank. And however and whenever the war ends, Kruk expects a sizable number of people will want to leave the country, if only to start on a clean slate because they have lost so much.
This dynamic was visible on a long bus ride from Chisinau, Moldova to Odessa. The travel restrictions meant all my co-passengers were women, children and a solitary white-haired man. Beside me sat a young Ukrainian woman from Odessa, who was only 19 in 2022 when her mother bundled her into a car with a distant relative and shipped her off to Germany when the war began. This was her first trip back home.
“My mother was terrified I would be posted to the frontlines,” she said, explaining she was training as a paramedic when the war began. “I haven’t told her I’m coming to see her, because she is sure they will draft me.”
In the three and a half years since, she had learnt German, enrolled in a German university and was now a handful of examinations away from becoming a qualified nurse. Nearly four years of war and devastation have meant that nurses are in great demand in Ukraine, but eighty years of peace and prosperity have meant that
nurses are also in great demand in Germany
.
In her first year away, she said, she was desperately homesick and worried for her mother especially when Odessa was repeatedly hit by Russian air strikes. But as the war settled into a grim pattern in Ukraine, she built herself a life in Germany. “Now I know the language,” she said. “Now I have a career. Now I have a boyfriend.”
These aren’t trivial things to want; these are arguably the most important things — life, community, love, freedom and dreams for the future.
“I don’t know if I will come back,” she said. “Just for short trips. Just for holidays. Just to see my ma.”
For Kruk, leaving Ukraine was not an option she considered. She has continued to write poetry. It is one of her ways, she said, to survive the war. But she said she has struggled to write prose.
“For prose, I need more inner space. I need greater distance from reality, and I can’t do that now.” she said. “But I feel alive here. I feel more alive here than elsewhere; I feel I live in history; and I know many people feel it.”
Bogdan Dubovic, soldier | Aman Sethi, openDemocracy
In the summer of 2023, Bogdan Dubovic was in the trenches, working as a sapper clearing the way for his unit as part of the Ukrainian counteroffensive, when he was transfixed by the sudden appearance of a Russian drone followed by a deafening explosion.
Once the smoke and debris cleared, he was rushed to a hospital where surgeons were forced to amputate his left arm. His unit members told him he was lucky to be alive, but they all knew that Bodgan was not an ordinary man, he was a Leshy, a forest spirit, an enchanted protector of the forests, who as per legend could either aid or hinder travellers lost in the woods.
“When I first signed up to fight, my unit said, ‘Everyone has to have a call sign, what is your name?” Bogdan recalled. At the time, Bogdan was working as a forest ranger in eastern Ukraine. “I said, I am a man from the forest, call me ‘forest man’.” But that call-sign was taken, so he said, “Call me Leshy.”
In his new identity, his first assignment was to assist in the exfiltration of civilians and soldiers from Marioupol, in the days after the city fell to the Russian forces. Thereafter until he lost his arm, he operated in some of the most gristly theatres in the war. In his previous life as a forester, Bogdan had learnt that the way to survive in the wild was to take every chance to turn the odds in his favour, and that is what he taught his unit of young volunteers.
With every passing skirmish, his unit came to believe that he really was a magical being, a forest spirit who would protect them in the face of dangers, and slowly Bogdan came to embody the Leshy.
“We never lost control of ourselves as a team, we were confident in our decisions, and that saved us every time,” he said. “My team trusted me as if I was not someone real, but from a legend.”
Yermolenko, the philosopher, and his wife Orgakova had first encountered Leshy when they were delivering supplies to his unit. Since then, they had struck up a deep friendship and met whenever Leshy was visiting Kyiv.
On the day we met, he sported a satyrid white beard, his eyes sparkled, and he constantly pulled on cigarettes that enshrouded him in a plume of smoke; the absence of his left arm lent him an aura of an otherworldly being who can be hurt but not defeated.
Leshy is from the Pontic steppes, a region known in Ukrainian as
dyke-pole
(literally “wild fields”) the vast grasslands that stretch across southern Ukraine, past the Sea of Azov, all the way into central Asia.
“This is not a place of nationalities,” he said. “It is a place of geography. It is a particular state of the soul; it is a place where people from different ethnicities, different histories, different nationalities, are drawn to a particular feeling of freedom. It is a freedom that forms your character and your perception of the world.”
The Ukrainian identity, Leshy said, was drawn from the steppe grasslands, from that rolling landscape of freedom that had long served as a battleground for armies, a hunting ground for raiders, and a refuge for men like him.
This war, much like those that came before it, he said, was not so much about territory as it was a clash between two world views. Between those who lived in freedom on a limitless frontier, and those who sought to destroy what they could not control.
“The next time you come to Ukraine, I will take you to the steppes.” He said, “If you stay there just a while, you will want to live there for the rest of your life.”
On one of my last days in Kyiv, Yermolenko dropped me off at the PEN Ukraine office for a conversation with his colleague Vakhtang Kebuladazhe, who looks exactly as one would imagine a middle-aged, Heidegger-quoting East European phenomenologist who plays bass guitar, and has written both a thesis on Kant and a rock opera inspired by a poetic play written in 1911 by Lesya Ukrainka, a Ukrainian feminist poet.
I started, as I did for much of my trip, by asking Kebuladazhe how he thought the war was shaping Ukraine’s national identity. We argued for a while about national identity, language, power, colonialism and decolonisation: Russian in the case of Ukraine, English in the case of India; language as simply an instrument, versus Heidegger’s description of “Language as the house of being”; and then he told me a joke.
“It is like the boy and the salt,” Kebuladazhe said, when I asked him to describe his conversations about identity with his students at the university.
“There was once a boy who would not speak. His family took him to the finest doctors in Kyiv, but to no avail. He didn’t utter a word.
Then over dinner one day, soon after he turned five, he said, “Pass me the salt.”
His mother was astounded: Why didn’t you say anything until today?
“Until today, dinner always had enough salt.”
“We are that boy,” Kebuladzhe said with a wry smile. “Now Ukraine is in an existential crisis. Now, my students are immersing themselves in philosophy and history and the humanities. Now they are speaking, now they are asking, ‘Who are we as Ukrainians?”
is editor-in-chief of openDemocracy. Before joining us he was deputy executive editor at HuffPost. Before that he was the executive editor for strategy at BuzzFeed, editorial director with Coda Media, editor-in-chief of HuffPost India, associate editor with the Hindustan Times, and foreign correspondent (Africa) and Chhattisgarh correspondent with The Hindu. His award-winning reportage both in India and around the world has touched on some of the most pressing issues of our time, such as migration, land grabs, labour rights, public health, nationalism, democracy and insurgency. He is the author of the critically acclaimed non-fiction book ‘A Free Man’.
openDemocracy
is an independent international media platform. We produce high-quality journalism which challenges power, inspires change and builds leadership among groups underrepresented in the media. Headquartered in London, we have team members across four continents. We are a mission-focused organisation, which means we always think about the impact our journalism can have. We're keeping our journalism free for everyone to read. If you value our work,
please help keep it free
.
Bluesky Thread Viewer thread by @simonwillison.net
Simon Willison
simonwillison.net
2025-11-28 23:57:22
Bluesky Thread Viewer thread by @simonwillison.net
I've been having a lot of fun hacking on my Bluesky Thread Viewer JavaScript tool with Claude Code recently. Here it renders a thread (complete with demo video) talking about the latest improvements to the tool itself.
I've been mostly vibe-coding ...
Bluesky Thread Viewer thread by @simonwillison.net
. I've been having a lot of fun hacking on my Bluesky Thread Viewer JavaScript tool with Claude Code recently. Here it renders a thread (complete with
demo video
) talking about the latest improvements to the tool itself.
I've been mostly vibe-coding this thing since April, now spanning
15 commits
with contributions from ChatGPT, Claude, Claude Code for Web and Claude Code on my laptop. Each of those commits links to the transcript that created the changes in the commit.
Bluesky is a
lot
of fun to build tools like this against because the API supports CORS (so you can talk to it from an HTML+JavaScript page hosted anywhere) and doesn't require authentication.
From a physicist point of view I want to mention this trick and its generalization for operators:
"Two commuting matrices are simultaneously diagonalizable"
(for physicists all matrices are diagonalizable). Of course the idea is that if you know the eigenvectors of one matrix/operator then diagonalizing the other one is much easier. Here are some applications.
1)The system is translation invariant : Because the eigenvectors of the translation operator are
$e^{ik.x}$
, then one should use the Fourier transform. It solves all the wave equations for light, acoustics, of free quantum electrons or the heat equation in homogeneous media.
2)The system has a discrete translation symmetry: The typical system is the atoms in a solid state that form a crystal. We have a discrete translation operator
$T_a\phi(x)=\phi(x+a)$
with
$a$
the size of the lattice and then we should try
$\phi_k(x+a)=e^{ik.a}\phi_k(x)$
as it is an eigenvector of
$T_a$
. This gives the
Bloch
-
Floquet
theory where the spectrum is divided into band structure. It is one of the most famous model of condensed matter as it explains the different between conductors or insulators.
3)The system is rotational invariant: One should then use and diagonalize the rotation operator first. This will allow us to find the eigenvalue/eigenvectors of the
Hydrogen atom
. By the way we notice the eigenspace of the Hydrogen are stable by rotation and are therefore finite dimension representations of
$SO(3)$
. The irreducible representations of
$SO(3)$
have dimension 1,3,5,... and they appears, considering also the spin of the electron, as the
columns
of the periodic table of the elements (2,6,10,14,...).
4)
$SU(3)$
symmetry: Particle physics is extremely complicated. However physicists have discovered that there is an underlying
$SU(3)$
symmetry. Then considering the representations of
$SU(3)$
the zoology of particles seems much more organized (
A
,
B
).
Rope science, part 11 - practical syntax highlighting (2017)
In this post, we present an incremental algorithm for syntax
highlighting. It has very good performance, measured primarily by
latency but also memory usage and power consumption. It does not
require a large amount of code, but the analysis is subtle and
sophisticated. Your favorite code editor would almost certainly
benefit from adopting it.
Pedagogically, this post also gives a case study in systematically
transforming a simple functional program into an incremental
algorithm, meaning an algorithm that takes a delta on input and
produces a delta on output, so that applying that delta gives the same
result as running the entire function from scratch, beginning to end.
Such algorithms are the backbone of xi editor, the basis of
near-instant response even for very large files.
The syntax highlighting function
Most syntax highlighting schemes (including the TextMate/Sublime/Atom
format) follow this function signature for the basic syntax
highlighting operation (code is in pseudo-rust):
Typically this “state” is a stack of finite states, i.e. this is a
pushdown
automaton
. Such
automata can express a large family of grammars. In fact, the
incredibly general class of
LR
parsers
could be accomodated
by adding one additional token of lookahead in addition to the line,
a fairly straightforward extension to this algorithm.
I won’t go into more detail about the syntax function itself; within
this framework, the algorithms described in this post are entirely
generic.
A batch algorithm
The simplest algorithm to apply syntax highlighting to a file is to
run the function on each line from beginning to end:
let mut state = State::initial();
for line in input_file.lines() {
let (new_state, spans) = syntax(state, line);
output.render(line, spans);
state = new_state;
}
This algorithm has some appealing properties. In addition to being
quite simple, it also has minimal memory requirements: one line of
text, plus whatever state is required by the syntax function. It’s
useful for highlighting a file on initial load, and also for
applications such as statically generating documentation files.
For this post, it’s also something of a correctness spec; all the
fancy stuff we do has to give the same answer in the end.
Random access; caching
Let’s say we’re not processing the file in batch mode, but will be
displaying it in a window with ability to scroll to a random point,
and want to be able to compute the highlighting on the fly. In
particular, let’s say we don’t want to store all the spans for the
whole file. Even in a compact representation, such spans are
comparable to the size of the input text, potentially much more.
We can write the following functional program:
fn get_state(file, line_number) -> state {
file.iter_lines(0, line_number).fold(
State::initial(),
|state, line| syntax(state, line).state
)
}
fn get_spans(file, line_number) -> spans {
let state = get_state(file, line_number);
syntax(state, file.get_line(line_number)).spans
}
This will work very well for lines near the beginning of the file,
but has a serious performance problem; it is O(n) to retrieve one
line’s worth of spans, so O(n^2) to process the file.
Fortunately,
memoization
,
a traditional technique for optimizing functional programs, can come
to the rescue. Storing the intermediate results of
get_state
reduces
the runtime back to O(n). We also see the algorithm start to become
incremental, in that it’s possible to render the first screen of the
file quickly, without having to process the whole thing.
However, these benefits come at a cost, namely the memory required
to store the intermediate results. In this case, we only need store
the state per line (which, in a compact representation, need only be
one machine word), so it might be acceptable. But to handle extremely
large files, we might want to do better.
One good compromise would be to use a
cache
with only partial
coverage of the
get_state
function; when the cache overflows, we
evict some entry in the cache to make room. Then, to compute
get_state
for an arbitrary line, we find closest previous cache
entry, and run the fold forward from there.
This cache is a classic speed/space tradeoff. The amount of time to
compute a query is essentially proportional to the
gap length
between one entry and the next. For random access patterns, it follows
that the optimal pattern would be evenly spaced entries. Then the
time required for a query is O(n/m), where m is the cache size.
Tuning such a cache, in particular choosing a cache replacement
strategy, is tricky. We’ll defer discussion of that for later.
Handling mutation
Of course, we
really
want to be able to do interactive syntax
highlighting on a file being edited. Fortunately, the above cache can
be extended to handle this use case as well.
As the file is mutated, existing cache entries might become
invalid.
We define a cache entry (line_number, state) as being
valid
if that
state is actually equal to computing
get_state(line_number)
from
scratch. Editing a line need not only change the spans for that line;
it might cause state changes that ripple down from there. A classic
example would be inserting
/*
to open a comment; then the entire
rest of the file would be rendered as a comment. So, unlike a typical
cache, changing one line might invalidate an arbitrary fraction of the
cache contents.
We augment the cache with a
frontier,
a set of cache entries. All
operations maintain the following invariant:
If a cache entry is valid and it is not in the frontier, then the
next entry in the cache is also valid.
From this invariant immediately follows a number of useful properties.
All lines up to the first element of the frontier are valid. Thus, if
the frontier is empty, the entire cache is valid.
This invariant is carefully designed so that it can be easily restored
after an editing operation, specifically that all operations take
minimal time (I
think
it’s O(1) amortized, but establishing that
would take careful analysis).
Specifically, after changing the contents of a single line, it
suffices to add the closest previous cache entry to the frontier.
Other editing operations are similarly easy; to replace an arbitrary
region of text, also delete cache entries for which the starts of the
lines are in strictly in the interior of the region. For inserts and
deletes, the line numbers after the edit will also need to be fixed
up.
Of course, it’s not enough to properly invalidate the cache, it’s also
important to make progress towards re-validating it. Here is the
algorithm to do one granule of work:
Take the first element of the frontier. It refers to a cache entry:
(line_number, state)
.
Evaluate
syntax(state, file.get_line(line_number))
, resulting in a
new_state.
If
line_number + 1
does not have an entry in the cache, or if it
does and the entry’s state != new_state, then insert
(line_number + 1, new_state)
into the cache, and move this element
of the frontier to that entry.
Otherwise, just delete this element from the frontier.
The only other subtle operation is deleting an entry from the cache
(especially evictions). If that entry is in the frontier, then the
element of the frontier must be moved to the previous entry.
On the representation of the frontier
It’s tempting to truncate the frontier, rather than storing it as a
set. In particular, it’s perfectly correct to just store it as a
reference to the first entry. Then, the operation of adding an element
to the frontier reduces to just taking the minimum.
However, this temptation should be resisted. Let’s say the user opens
a comment at the beginning of a large file. The frontier slowly
ripples through the file, recomputing highlighting so that all lines
are in a “commented” state. Then say the user closes the comment when
the frontier is about halfway through the file. This edit will cause
a new frontier to ripple down, restoring the uncommented state. With
the full set representation of the frontier, the old position halfway
through the file will be retained, and when the new frontier reaches
it, states will match, so processing can stop.
If that old position were not retained, then the frontier would need
to ripple all the way to the end of the file before there would be
confidence the entire cache was valid. So, for a relatively small cost
of maintaining the frontier as a set, we get a pretty nice
optimization, which will improve power consumption and also latency
(the editor can respond more quickly when it has quiesced as opposed
to doing computation in the background).
Tuning the cache
This is where the rocket science starts. Please check your flight
harnesses.
Access patterns
Before we can start tuning the cache, we have to characterize the
access patterns. In an interactive editing session, the workload will
consist of a mix of three fundamental patterns: sequential, local, and
random.
Sequential is familiar from the first algorithm we presented. It’s an
important case when first loading a file. It will also happen when
edits (such as changing comment balance) cause state changes to ripple
through the file. The cache is basically irrelevant to this access
pattern; the computation has to happen in any case, so the only
purpose of the cache is not to have significant overhead.
By “local,” we mean edits within a small region of the file, typically
around one screenful. Most such edits
won’t
cause extensive state
changes, in fact should result in re-highlighting of just a line or
two. In this access pattern, we want our algorithm to recompute tiny
deltas, so the cache should be
dense,
meaning that the gap between
the closest previous cache entry and the line being edited be zero or
very small.
The random access pattern is the most difficult for a cache to deal
with. The best we can possibly do is O(n/m), as above. We expect these
cases to be rare compared with the other two, but it is still
important to have reasonable worst-case behavior.
Any given editing session will consist of all three of these patterns,
interleaved, in some relative proportions. This is significant for
designing a well-tuned cache, especially because processing some work
from one pattern may leave the cache in poor condition for the next.
Analyzing the cache performance
In most applications, cache performance is characterized almost
entirely by its
hit rate,
meaning the probability that any given
query will be present in the cache. Most
cache eviction
policies
are chosen to optimize this quantity.
However, for this algorithm, the cost of a cache miss is highly
dependent on the
gap
between entries, and the goal should be to
minimize this gap.
From this viewpoint, we can see that the LRU (least recently used)
policy, while fine for local access patterns, is absolutely worst case
when mixing sequential with anything else; after sequential procesing,
the cache will consist of a dense block (often at the end of the
file), with a huge gap between the beginning of the file and that
block. As Dan Luu’s excellent
case
study
points out, LRU can also
have this kind of pathological performance in more traditional
applications such as memory hierarchies.
For the “random” access pattern, the metric we care about is maximum
gap; this establishes a worst case. For LRU, it is O(n), which is
terrible. We want to do better.
The obvious next eviction policy candidate to consider is randomized.
In traditional cache applications, random eviction fixes the pathology
with perfectly sequential workloads, and performs reasonably well
overall (in Dan’s analysis, it is better than LRU for some real-world
workloads, worse in others, and in no case has a hit rate more than
about 10% different).
I tried simulating it [TODO: a more polished version of this document
would contain lots of beautiful visualizations, plus a cleaned up
version of the simulation code], and the maximum-gap metric was
horrible, almost as bad as it can get. In scanning the file from
beginning to end, in the final state the entries near the beginning
are decimated; a typical result is that the first entry remaining in
the cache is about halfway through the file.
For a purely random workload, an ideal replacement policy would be to
choose the entry with the smallest gap between previous and next
entries. A bit of analysis shows that this policy would yield a
maximum gap of 2n/m in the worst case. However, it won’t perform well
for local access patterns - basically, the state of the cache will
become stuck, as lines most recently added are likely to also have the
smallest gap. Thus, local edits will still have a cost around n/m
lines re-highlighted. It doesn’t make sense to optimize for the random
case at the expense of the local one.
Inspired by Dan’s post, I sought a hybrid. My proposed cache eviction
policy is to probe some small number k of random candidates, and of
those choose the one with the smallest gap as defined above. In my
simulations [TODO: I know, this really needs graphs; what I have now
is too rough], it performs
excellently.
There’s no obvious best choice of k, it’s a tradeoff between the
expected mix of local (where smaller is better) and random (where
larger is better). However, there seems to be a magic threshold of 5;
for any smaller value, the maximum gap grows very quickly with the
file size, but for 5 or larger it levels off. In a simulation of an
8k entry cache and a sequential scan through an 8M line file, k=5
yielded a maximum gap of ~9k lines (keep in mind that 2k is the best
possible result here). Beyond that, increasing k doesn’t have dramatic
consequences, even at k=10 this metric improves only to ~3600, and
that’s at the expense of degrading local access patterns.
Obviously it’s possible to do a more rigorous analysis and more
fine-tuning, but my gut feeling is that this very simple policy will
perform within a small factor of anything more sophisticated; I’d be
shocked if any policy could improve performance more than a doubling
of the cache size, and with the cache sizes I have in mind, that
should be well affordable. And a larger cache size always has the
advantage that any file with a number of lines that fits entirely
within the cache will have perfect effectiveness.
Cache size and representation
Choosing cache size is always a tradeoff between cache effectiveness
(whether hit rate or maximum-gap) and the cost of the cache itself.
A larger cache should increase effectiveness, but how much?
This is an empirical question, but we can try to analyze it. Cache
effectiveness is irrelevant for sequential access. For the local case,
it would be reasonable to expect that the “working set” is quite
small, typically on the order of 1000 lines or so.
And for the random case, the cache only has to perform reasonably
well; we expect these cases to be rare.
From this, we can guess that the cache doesn’t have to be very large
to be effective. Thus, a very simple representation is a dense vector
of entries. Some operations (such as deletion and fixup of line
numbers) are O(m) in the size of the cache, but with a very good
constant factor due to the vector representation. So, while it’s
tempting to use a fancy O(log m) data structure such as a B-tree, this
is probably a case where simpler is better.
My gut feeling is that a fixed maximum size of 10k entries will yield
near-optimal results in all cases.
Implementation state and summary
I haven’t implemented this yet (beyond the simulations), but really
look forward to it.
Based on my analysis, this algorithm should provide truly excellent
performance, producing minimal deltas with very modest memory
requirements. I’m also pleased that the code and data structures are
relatively easy; I have considered
much
more sophisticated
approaches (including of course my beloved balanced-tree
representation for the cache), which in analysis wouldn’t perform
nearly as well.
I think it would be interesting to do a more rigorous analysis. It’s
possible this technique has already been investigated somewhere, but
I’m not aware of it; I’d
love
to find such a literature.
Thanks to
Colin Rofls
for stimulating
discussions about caching in plugins that inspired many of these
ideas.
In spherical geometry, the interior angles of a triangle add up to more than π. And in fact you can determine the area of a spherical triangle by how much the angle sum exceeds π. On a sphere of radius 1, the area equals the
triangle excess
Area =
E
= interior angle sum − π.
Small triangles have interior angle sum near π. But you could, for example, have a triangle with three right angles: put a vertex on the north pole and two vertices on the equator 90° longitude apart.
Hyperbolic geometry
In hyperbolic geometry, the sum of the interior angles of a triangle is always less than π. In a space with curvature −1, the area equals the
triangle defect
, the difference between π and the angle sum.
Area =
D
= π − interior angle sum.
Again small triangles have an interior angle sum near π. Both spherical and hyperbolic geometry are locally Euclidean.
The interior angle sum can be any value less than π, and so as the angle sum goes to 0, the triangle defect, and hence the area, goes to π.
The figure below has interior angle sum 0 and area π in hyperbolic geometry.
Strictly speaking this is an improper triangle because the three hyperbolic lines (i.e. half circles) don’t intersect within the hyperbolic plane per se but at ideal points on the real axis. But you could come as close to this triangle as you like, staying within the hyperbolic plane.
Note that the radii of the (Euclidean) half circles doesn’t change the area. Any three semicricles that intersect on the real line as above make a triangle with the same area.
Note also that the triangle has infinite perimeter but finite area.
The Vital, Overlooked Role of Body Fat in Shaping Your Health and Mind
Portside
portside.org
2025-11-28 23:59:49
The Vital, Overlooked Role of Body Fat in Shaping Your Health and Mind
barry
Fri, 11/28/2025 - 18:59
...
“You wouldn’t get pushback today if you claimed fat was an organ, in the same way your lungs or liver or spleen are organs,” says
Paul Cohen
at The Rockefeller University in New York, who researches metabolic disease and cancer related to obesity. This shift in thinking is reshaping our view of body fat and our understanding of obesity. It challenges how we think about trying to get rid of fat, and is even prompting some scientists to explore how to reprogram it instead – not just to tackle obesity, but to improve our broader health.
Until relatively recently, body fat – also known as adipose tissue – was largely seen as a passive storage depot for excess calories, a layer of insulation against the cold and simple padding. These functions are clearly important: the evolution of body fat may have aided humans in moving out of Africa and surviving in colder climates. Even today, carrying a bit of excess weight reduces the likelihood of older people dying if they fall ill.
“I think the first thing that people fail to appreciate is what a valuable evolutionary step it was to be able to store fuel,” says
Randy Seeley,
who researches energy balance and metabolism at the University of Michigan. “If you’re not able to do that, you’re a filter feeder: you have to swim in your food.”
But while many organisms possess some form of body fat, in mammals, it has evolved into something much more complex than just a kind of meaty bubble wrap, says Seeley. “It also now becomes integrated into the overall regulation of blood glucose, body temperature and other physiological functions, including bone health.”
Controlling hunger
The first clues that we were underestimating our body fat came in the 1990s with
the discovery of leptin
. This hormone, secreted by fat cells, acts on the brain to suppress appetite and boost energy expenditure. On the flip side, when people quickly lose fat, leptin levels drop, which the brain interprets as a sign that energy stores might be running low. It responds by ramping up hunger signals and reducing energy expenditure to help you regain that lost fat.
The discovery of leptin cracked open a hidden communications network between fat and the rest of the body. Since then, we have discovered that fat cells release many more hormones and other signalling molecules, some of which communicate with tissues nearby, while some travel much further afield. Together, they are known as adipokines.
What’s more, this communication isn’t only chemical – it’s also electrical. We now have evidence for networks of nerve fibres extending deep inside adipose tissues, forming a direct, two-way line of communication between the brain and our fat.
“The nerve supply in adipose tissue enables a bidirectional and fast communication route with the brain,” says
Kristy Townsend
, a neuroscientist at The Ohio State University who studies fat. As well as sending messages about energy and metabolism, nerves allow fat to quickly communicate its health status, for instance, whether it is injured or inflamed.
Fat and immune health
Immune cells may also join these conversations, relaying information about inflammation or injury and releasing molecules that help nerves survive and grow. “If you look at the tissue in between all the adipocytes, there’s pretty much every immune cell you can imagine – so fat is also an immune organ,” says Townsend.
In short, fat doesn’t just store energy; it speaks. And together, these adipokines, immune cells and nerve fibres form the vocabulary of an unexpectedly sophisticated organ.
The far-reaching impacts of fat are only now coming to light. Its best-documented role is in energy balance (see “Your unappreciated organ”, below), telling the brain when reserves are full or depleted. But fat’s communication with the brain also seems to extend to our moods. While mood disorders such as depression or anxiety are complex, and stigma or poor body image may also contribute to this, evidence is increasingly linking obesity – particularly metabolically unhealthy obesity – to these conditions.
And our fat plays a crucial role in fertility, too. Without a minimum level of body fat, for example, menstruation won’t start or will stop, which makes sense, because entering pregnancy without sufficient energy to sustain a developing fetus could be catastrophic for both mother and child.
“People forget that fat is metabolically really important. Without fat, we have issues with hormonal control, infection immunity,” says
Louise Thomas
, a professor of metabolic imaging at the University of Westminster in London.
When fat turns bad
So if fat is such a crucial factor in our health, why does it get such a bad rap? The first issue is its location. White fat makes up more than 95 per cent of our total stores and is found both under the skin (subcutaneous fat) and wrapped around internal organs (visceral fat). “Our organs are often sitting in a sea of fat,” says Thomas.
That internal sea can turn toxic.
Excess visceral fat
is linked to a higher risk of type 2 diabetes, high blood pressure, heart attacks and certain cancers. Growing evidence also suggests it may
affect brain function
and contribute to conditions such as Alzheimer’s disease.
What triggers this shift from cooperative organ to rogue state is a major focus of research. While white fat cells in both subcutaneous and visceral deposits can expand and contract depending on the body’s storage needs, those surrounding internal organs appear especially vulnerable to the harmful effects of excess fat.
In obesity, these fat cells enlarge and are prone to dying once they reach a critical size. Part of the problem is that their blood supply can’t keep up with their growth. Stressed and suffocating, they release inflammatory molecules as distress signals, attracting immune cells to clear dead or dying cells.
These immune cells intensify the inflammation, with effects reaching far beyond the fat itself. The chemical signals
interfere with insulin
– the hormone that regulates blood sugar – raising the risk of type 2 diabetes. They are also linked to cognitive changes seen in obesity such as memory and attention problems, and may create conditions that foster tumour growth.
Obesity is a risk factor for many kinds of cancer
, and often people who are obese tend to have worse outcomes.
Dying or overstuffed fat cells also release fatty acids, or lipids, into their surroundings – and in excess, these can be toxic to surrounding cells. Over time, this lipotoxic stress can damage the network of nerves threaded through fat, a condition known as
adipose neuropathy
. Obesity, type 2 diabetes and ageing are all linked to this loss of peripheral nerves, which further disrupts metabolism by impairing communication between the brain and fat.
Protecting bone health
Misfiring fat signals can also play havoc with our bones. Most of the time, oestrogen produced by adipose tissue can help protect against excessive bone resorption – where old bone tissue is broken down faster, then new bone can replace it. However, growing evidence suggests that excess fat, particularly visceral fat and fat accumulation within bone marrow, can impair bone quality and increase fracture risk. This is partly because inflammatory cytokines released by adipose tissue can stimulate osteoclasts,
the cells responsible for bone resorption, which, in turn, promotes bone loss
.
Despite the downsides of dysfunctional fat, adipose itself isn’t the enemy – we need it. And efforts to get rid of it can backfire.
Studies of liposuction
, a cosmetic procedure that removes targeted fat, suggest that the extracted fat may simply reappear elsewhere. “You may want to remove fat from some locations, but you may like even less where you get it afterwards,” says Seeley, who has been involved in some of this research. “If you remove subcutaneous fat, you’re probably going to end up with more visceral fat in the long run, and that probably leaves you in a worse place than where you were before.”
Not everyone with obesity is unhealthy, either. Between
10 and 30 per cent
of people classified as obese based on body mass index seem to escape the usual health effects, such as insulin resistance, high blood pressure and unhealthy cholesterol levels – at least in the short term. This so-called metabolically healthy obesity has intrigued researchers like
Matthias Blüher
at the University of Leipzig in Germany.
About 15 years ago, Blüher and his colleagues began
comparing fat tissue
from people with obesity who developed insulin resistance – often a precursor to the development of type 2 diabetes – and those who didn’t. They found that where excess fat sits and how it behaves are both crucial: people with more visceral and liver fat tended to be metabolically less healthy, while those whose adipose tissue contained smaller fat cells, fewer immune cells and a healthier secretion pattern of adipokines appeared to be more protected.
Different types of fat
More recently, the researchers have taken this investigation down to the cellular level, analysing which genes are active in different fat deposits across dozens of people with healthy and unhealthy obesity. Their results, published earlier this year, reinforce that
not all visceral fat is equal
. “Even within the visceral cavity, it makes a difference where the fat is located,” says Blüher. The highest risk is associated with fat that sits outside of the intestine, although, for now, they aren’t sure why this is the case.
The fat also looks different in people with healthy obesity: their fat cells are more metabolically flexible – able to switch efficiently between storing and burning energy – pump out fewer inflammatory signals and host fewer immune cells. Their visceral fat also contains mesothelial cells, which can transform into other cell types, perhaps enabling their fat to expand more smoothly without triggering excessive inflammation. Why some people have more of these metabolically healthy cells is probably down to genetics, although lifestyle factors such as diet and exercise may play a role.
Either way, Blüher thinks that these insights could help doctors identify which people with obesity are at the highest risk of complications, and then tailor treatment accordingly.
Reprogramming fat for health
His longer-term dream is to find a way to restore fat’s healthy function – perhaps even transform “unhealthy” obesity into a permanently more benign form. Encouragingly, this may not require dramatic weight loss. Many of the benefits of modern weight-loss drugs and bariatric surgery seem to stem not from the amount of weight lost, but from improving fat distribution and function, says Blüher. “In bariatric surgery, even if people don’t lose a lot of weight, the health benefits start almost immediately.”
Achieving this would be revolutionary, not least because it would prompt a rethink of what a healthy body shape looks like.
And if fat could be reprogrammed to behave more healthily – or the cellular memories of its bloated heyday erased (see “The yo-yo effect”, below) – many more of us might live longer, healthier lives without obsessing over size.
Whether obesity begins in the adipose tissue or the brain is still debated, but it is clear that when communication between the two falters, the whole system drifts off-kilter.
Seeley likens the situation to an orchestra: “All of these organ systems – your liver, pancreas, adipose tissue, muscle and gastrointestinal tract – are all talking to your brain, and your brain is talking to all of them. If your symphony conductor isn’t doing a good job, then even if all your instruments are OK, it won’t sound great.”
In other words, fat isn’t necessarily the problem; it’s an instrument playing slightly out of tune in a misdirected symphony. Many of us have been conditioned to try to shrink, remove or hide our body fat. But the real task is to understand it – to coax this creamy, talkative organ back into harmony with the rest of the orchestra. Because when it plays well, it helps keep the whole body in tune.
is a Bristol-based journalist writing about biology, medicine and technology. Born in Cambridge, she graduated from Liverpool University with a first-class BSc in Cell Biology. She spent 3.5 years at The Guardian as a science correspondent, including during the COVID-19 pandemic, and remains a regular contributor. Prior to this, she spent nine years at New Scientist magazine working as a news editor, features editor and reporter, and co-presented of the BBC World Service podcast Parentland, as well as various science documentaries for BBC Radio 4. Linda has received numerous awards for her journalism, including the Association of British Science Writers’ award for Best Investigative Journalism. She has also published two books: Bumpology, the myth-busting pregnancy book for curious parents-to-be, and Chasing The Sun, the new science of sunlight and how it shapes our bodies and minds.
New Scientist
is the world’s most popular weekly science and technology publication. Our website, app and print editions cover international news from a scientific standpoint, and ask the big-picture questions about life, the universe and what it means to be human. If someone in the world has a good idea, you will read about it in New Scientist.
Since the magazine was founded in 1956 for “all those interested in scientific discovery and its social consequences”, it has expanded to include newsletters, videos, podcasts, courses and live events in the UK, US and Australia, including New Scientist Live, the world’s greatest festival of science. New Scientist is based in London and New York, with operations elsewhere in the UK, the US and Australia.
To win the argument for universal basic income, advocates must confront the myth that less work means less worth.
The general idea behind universal basic income (UBI) is
almost as old as America itself
. You can trace it back to 1797, when Thomas Paine
argued
for guaranteed payments in his political treatise “Agrarian Justice.” Fast forward to 2020, and Andrew Yang revived the idea with a “
Freedom Dividend
” during his failed presidential campaign. Despite the 200-plus-year chasm that separates these two men, the criticism they faced for backing UBI was strikingly similar: that “no one will work” and that “we can’t afford it.”
Because of this, supporters of the program might be tempted to believe that the purpose of UBI experiments is to allay these concerns with empirical evidence on the effect of UBI on work hours. The problem, however, is that these concerns are not rooted in empiricism but normative belief: namely that 1) lower-class people who refuse employment should receive nothing and 2) UBI costs more than it’s worth. And while not all UBI opponents believe these things, those who are often move the goalposts to portray almost any findings about cost and labor effort as reasons to reject UBI.
We must resist playing this game.
UBI-related experiments consistently find evidence that no participant responds to UBI experiments by dropping out of the labor force. Yes, some people reduce their hours of work, but the decline in work effort (if any) is clearly within a sustainable range. In other words, the evidence decisively contradicts claims that “no one will work” and “we can’t afford it.” But if we take the bait of focusing on such extreme statements, we attract everyone’s attention to opponents’ favorite issue: “Did the people who got the UBI ‘work’ as much as the people who didn’t?” Once the question is framed this way, it tosses a softball to opponents who predictably argue UBI is out of the question because some people didn’t work as much as they otherwise might have.
Any unconditional grant large enough to live on necessarily allows lower-class people to refuse employment. This fact — at least for critics who feel that people who refuse employment should receive nothing — makes UBI undesirable
by design
. To them, UBI will always be “unaffordable” because it will appear to cost more than they think it’s worth. UBI supporters fall into their trap if they attempt to refute this belief with, say, technical explanations of the difference between a 4 percent decline in labor hours and 4 percent of people leaving the labor force.
Supporters need to focus on all the good that comes of guaranteed income. As Bru Laín
argues
, UBI has a “positive impact on socioeconomic indicators related to a lack of money,” including the “alleviation of stress and mental illness, improvement in eating habits, settlement of household and personal debts, improvement of happiness, subjective well-being and social and community participation.”
Instead of trying to assuage critics’ fears, the pro-UBI movement needs to challenge the narrative in which any refusal to accept employment is a “bad” experimental observation.
Meanwhile, proponents of UBI that fall headfirst into critics’ trap even when they point to findings that that UBI
increases
labor effort. Consider these headlines from a UBI experiment in Stockton, California: “
Experiment in guaranteed income leads to more work
,” “
Californians on universal basic income paid off debt and got full-time jobs
,” and “
The Biggest Payoff From Stockton Basic Income Program: Jobs
.” Even the city’s mayor, Michael Tubbs, who was instrumental in establishing the program, employed this kind of rhetoric,
saying
, “Number one, tell your friends, tell your cousins, the guaranteed income did not make people stop working, in fact, those who received the guaranteed income were working more than before they received the guaranteed income and almost doubled in increase compared to those in the treatment group.”
The results Tubbs points to are largely determined by the design of the study: People who receive small grants when they weren’t working very much to begin with usually work more in UBI studies; people who receive larger grants when they are working full-time to begin with often work less. By portraying the uptick in Stockton’s labor effort as self-evidently good, Tubbs’ comments make it more difficult for future experiments that might involve larger grants to report the likely finding that people work less. Buying into the narrative that it is always “good” for low-income people to spend as much or more time on paid labor than they are now is a game UBI supporters can’t win and shouldn’t play. If the biggest problem in the world today were getting the lower class to work as much as possible, UBI would not be the best policy to achieve it.
Instead of trying to assuage critics’ fears, the pro-UBI movement needs to challenge the narrative in which any refusal to accept employment is a “bad” experimental observation. After all, how could it be a
good
thing for the global poor to spend more hours in grueling jobs for which they’re likely underpaid and overworked? What do you think will happen to wages and working conditions if the two billion people in deep poverty around the world all decide to work more at the same time? Theory predicts they would work longer hours for lower hourly wages.
One of the many disadvantages of UBI experiments is that they cannot measure how much wages and working conditions might improve in response to a substantial UBI, because that effect depends on the interaction between millions of citizens and employers across the country. The closest thing UBI experiments can measure is the first step in the process, and that step involves giving people a choice beyond working too hard for too little. So, rather than trying to quibble over hours worked, UBI supporters might have better luck broadcasting the good that comes when people with the worst jobs decide to work less — and using experiments as a platform for participants to tell their stories.
Karl Widerquist
, a professor of philosophy at Georgetown University-Qatar who specializes in distributive justice, is the author of “
Universal Basic Income
.”
The MIT Press
is a mission-driven, not-for-profit scholarly publisher. Your support helps make it possible for us to create open publishing models and produce books of superior design quality.
Fabric is a creative code and rapid prototyping environment focusing on interactive visuals, image and video processing and analysis and 3D content authoring.
Fabric
Provides an intuitive Visual Node based content authoring environment
Provides an SDK to load an common interchange file format
Provides an SDK to extend Fabric by creating custom nodes via a plugin architecture
Fabric is inspired by Apple's deprecated Quartz Composer ecosystem, and its design philosophy.
Fabric is intended to be used as
A Creative coding tool requires little to no programming experience.
Pro User tool to create reusable documents (similar to Quartz Composer Compositions) that can be loaded in the Fabric runtime and embedded into 3rd party applications.
Developer environment built on Satin that can render high fidelity visual output in a procedural way, using modern rendering techniques.
An early alpha of Satin rendering a instances of a sphere geometry, along with an HDRI environment and a PBR Shader at 120Hz:
What can I do with Fabric?
Think of Fabric as a playground of visual capabilies you can combine together.
Fabric includes a licensed Metal port of
Lygia
shader library, powering Image effects and more, written by @
Patricio Gonzalez Vivo
and contributors.
Requirements
Warning
Please note Fabric is heavily under construction.
macOS 14 +
XCode 15 +
Please See
Releases
for code signed App downloads.
For Developers:
Checkout Fabric and ensure you check out with submodules enabled, as Satin is a dependency.
Open the XCode project
Ensure that
Fabric Editor
is the active target.
Build and run.
Getting Started
Checkout our
Architecture Document
to understand the underlying paradigms of working with Fabric's Nodes and execution model, learn what a
Node
and a
Port
is, the types of data Fabric can generate and process, and how Fabric executes your compositions.
We also provide a set of evolving tutorial / getting started and sample Fabric compositions along with a readme walk through. You can use the
Sample Compositions
to learn and build off of.
You can view a comprehensive list of available and planned
Nodes
here to explore and learn how to compose more advanced and custom setups with Fabric.
Don't hesitate to file a feature request if a Node is missing!
I (
Anton Marini
) are looking to build a community of developers who long for the ease of use and interoperability of Quartz Composer, its ecosystem and plugin community.
If you are interested in contributing, please do not hesitate to reach out / comment in the git repository, or
join our discord via invite
FAQ
Will Fabric ever be cross platform?
No. Fabric is purpose built on top of Satin and aims to provide a best in class Apple platform experience using Metal.
What languages are used?
Fabric Editor is written in Swift and SwiftUI. Satin is written in Swift and C++
Why not just use Vuo or Touch Designer or some other node based tool?
I do not like them.
Don't get me wrong, they are incredible tools, but they are not for me.
They do not think the way I think.
They do not expose the layers of abstraction I want to work with.
They do not provide the user experience I want.
Confessions of a Software Developer: No More Self-Censorship
I haven’t published since April because I’ve been afraid. I also avoided social media, news aggregators, and discussion forums for months. I’m done letting fear stop me.
What was I afraid of?
In this post I detail
every single thing
I’ve avoided admitting on this blog.
Knowledge Gap Confessions
First, why am I admitting these things now? I realized I am not the only working software developer missing crucial skills. My learning path through my career looked a lot like
a slime mold seeking morsels of food
: strengthening what has utility, but letting the rest wither. But lately, I’ve been building a better base of knowledge. Writing about what I learn–which helps me learn better–requires me to admit I didn’t know. Plus, I’d like to show others in my situation that it’s never too late to learn what you don’t know. I
can
fill in those fundamentals, and so can you.
It’s from that very ignorance that sprouts the drive for knowledge.
Learning about
polymorphism
over the past twelve months was the first time I was embarrassed to admit I didn’t already know something. I’ve been writing ostensibly object-oriented software since 2012. And yet, my lack of awareness of polymorphism showed me I’ve been writing little more than structured programs. That I could
replace conditionals and case staments with specialized classes
had never crossed my mind.
Why I Was Afraid to Admit It
As a hiring manager I interviewed software engineers and
tried to filter for object-oriented knowledge
. Retroactively, it’s clear I was hypocritical. This gap reveals that I spent the early part of my career learning tools, not principles. Plus, it highlights my lack of formal education. Polymorphism is covered in every college OO course.
I Forgot SQL
I took a college database course as a student. As a working professional, I read and worked through the exercises in
Learning SQL, 3rd Edition
. For a while, I
could
write SQL. But I specialized in front-end web development, and had no professional use for SQL. Like any unused skill, it atrophied. I remember how to write basic queries, but not much more. For example, I cannot tell you the difference between a left inner join and an outer join without looking it up.
Why I Was Afraid to Admit It
I’m not accustomed to forgetting. Growing up I had a remarkable ability to remember almost everything. It didn’t matter whether I did it, read it, or heard it. It could be a fact, a skill, or an event. Four years later I could access that knowledge, with the slightest reminder unlocking a flood of memories. Now that I’m in my mid-thirties, that isn’t always true. SQL is the first time I’ve lost
an entire skill
to atrophy. It’s tough to come to terms with the start of aging. It’s tougher to admit it publicly.
I Don’t Write Automated Tests
An estimated 95% of the code I’ve shipped to production had no automated tests. Early in my career, I had no exposure to the concept. Later, I was writing front-ends in
Ember
, whose
testing story was looked good but felt pretty bad at the time
. More recently, I’ve been working legacy code, and I haven’t
put in the work to make it testable
. The only time I tend to write new tests is when I’m writing a new subsystem, which can be designed testable from the start. I’m convinced that writing automated tests needs to be part of my daily practice, but I haven’t gotten there yet.
Why I Was Afraid to Admit It
This may be my most professionally-damaging confession. If you believe Uncle Bob, shipping production code without tests is not more than risky, it’s unethical. I stopped myself from posting about my learning journey for fear that a future hiring manager would decide that I was unfit to work with them on this basis.
How much of the code should be tested with these automated unit tests? Do I really need to answer that question? All of it! All. Of. It.
Am I suggesting 100% test coverage? No, I’m not
suggesting
it. I’m
demanding
it. Every single line of code that you write should be tested. Period.
Isn’t that unrealistic? Of course not. You only write code because you expect it to get executed. If you expect it to get executed, you ought to know that it works. The only way to know this is to test it.
People have been waiting for a follow-up from me about my
journey learning C#, .NET, and Blazor
. This isn’t that post. I don’t know if that post will ever come.
C# was never the language I wanted to learn for side projects. .NET was never the platform I wanted to work with professionally. I was learning them for one reason: my job. My engineering department decided to switch our tech stack from Angular to Blazor. I was the only person on the team with no C# skills. I started fixing that immediately.
A couple months later, almost as suddenly, the decision was undone. Our tech stack would
not
change after all. With no intrinsic motivation to push me along, I abandoned the C# / .NET book I was reading without finishing. I’ve got more important things to learn.
Why I Was Afraid to Admit It
No matter what software thoughts crossed my mind, I intended to post about them. Writing helps me solidify ephemeral thoughts. Publishing offers an opportunity for feedback. But I made two errors that locked me into a pattern of fear. First, I promised a follow up article at the end of
my last post about the Blazor stack
. I then felt worse every time I published an article other than the promised follow-up. Second, I began to see value in the
amount of traffic
a blog post got. The posts about my first steps in that learning journey were the winners. Admitting I changed tack when the company did felt like admitting defeat.
The best possible option–and yet the most improbable–would be for my current employer to start a Ruby project. I’ve worked with a few of my teammates for 12 years across two companies. I’ve always chosen to keep working with fantastic people at the cost of working with a
less-than-fantastic language
.
Sadly that means I’m limited to being a
Rubyist
after work and on weekends. I spend fewer of
those
hours than I’d like writing Ruby, instead favoring other obligations, hobbies, and professional development goals. The only way I foresee getting to work with Ruby as much as I’d like would be to paid for it.
Why I Was Afraid to Admit It
My manager and
his
manager, the CTO, read this blog. I found it difficult to write freely about my distaste for the tools I use every day. I found it ever harder to admit that I
actively want
my daily job duties to be different. I feared they might take it as a hint that I’m quitting (I’m not), or that I’d push to use a tool at work nobody else is familiar with at work (I won’t).
A bonus, even more personal confession…
A bonus, even more personal confession…
Cyber Bullying Hurts, Even as an Adult
I spent a lot of my young adulthood online. My early days on the internet were spent in intellectual spaces, where the interactions felt like The Nets of Ender’s Game: a marketplace of ideas. Strong criticism came swiftly, but it was about the
ideas
, not the person. Even Reddit and Hacker News–which have a reputation for harsh comments sections–don’t bother me, because the vitriol is aimed at taking down bad ideas, not insulting people.
Other sites, though, are different. I learned this when I got bullied on yet another threaded discussion site. I was called incapable, sneaky, disgusting, incompetent, uncaring, and a representative of a threat to human expression.
What triggered this vitriol? I requested a small feature in an open source project, and the maintainer said they would accept a PR. The project was written in a language I haven’t used. I used an LLM to generate a small commit (a few dozen lines), reviewed and tested the patch, and submitted a pull request. This was months ago–there were few social norms around AI-assisted patches, and AI policies were rare. Since the project didn’t have a policy, I did not disclose my use of AI.
When I told the story of that pull request on the threaded discussion site and defended my ethical position, the bullying started. I was followed across social websites, contacted via email and SMS, and even called on the phone. I no longer felt safe having a presence on that website. I deleted my comments, removed PII from my profile, and asked the administrators of the website to scrub my real name to prevent further harassment. Instead, they attached more PII to my profile, locked me out of editing it, and permanently vandalized it with the false claim that I lied about being contacted about the discussion outside the website.
Why I Was Afraid to Admit It
This incident was one of the most toxic things I’ve ever experienced, and it lasted for days. Writing about it leaves me feeling its echoes even now. I was afraid that one of these people will use the comments section, or email, or even my phone number, to re-litigate the issue. Even now, I am afraid that the administrator’s (possibly defamatory) statement on my profile will make me less employable.
Workplace Confessions
Your SaaS Team Doesn’t Need a Special Process
Hundreds of companies, thousands of researchers, tens of thousands of workers, and millions of dollars have gone into shaping our industry’s best practices. The agile manifesto is old enough to drink. Software as a Service has dominated the market for over a decade. Your company has a limited innovation budget. Do you want to spend it on coming up with a custom software development lifecycle, or making a product that wins in the marketplace? Follow
Scrum
,
Lean
/
Kanban
, or
eXtreme Programming
to the letter, and let your team focus on the product.
Why I Was Afraid to Admit It
Like any author, I write what I know. Here, I was motivated to write because a co-worker pushed to create a custom software development process. I don’t know that I have the tact to avoid it seeming to be a takedown of that person or their ideas. I admire the ability of authors like
Kent Beck
and
Martin Fowler
to write about how to work better without calling out coworkers who made mistakes.
Remote Work Sucks
Remote work eliminates a lot of problems with office work: commutes, inefficient use of real estate, and land value distortion. But software development is better when you breathe the same air as the folks you work with. Even with a camera-on policy,
video calls are a low-bandwidth medium
. You lose
ambient awareness
of
coworkers’ problems
, and asking for help is a bigger burden. Pair programming is less fruitful. Attempts to represent ideas spatially get mutilated by online
whiteboard
and
sticky note
software. Even conflict gets worse: it’s easy to form an
enemy image
of somebody at the end of video call, but difficult to keep that image when you share a room with them and sense their pain.
Why I Was Afraid to Admit It
When COVID-19 hit, the company I worked for went remote
“for a couple of weeks.”
After a few months of productive work without an office and with no vaccine in sight, it became permanent. I took the opportunity to move to a rural area.
Geographic arbitrage
meant I could afford 27 acres, and I even bought a family milking cow. My family has since put down roots: close friendships, community involvement, and a lifestyle built around the lack of a commute.
I feared that writing negatively about remote work might jeopardize my current remote job–and every future remote job I might look for. I thought, “who would hire a remote worker who prefers in-office work?” Because even though I prefer working side-by-side with others, I won’t likely move for a job. I have a 30-year mortgage with a low interest rate. My house was purchased before the post-pandemic price spike. I have an acre of lawn & garden, not to mention the farm acreage. I’d need to
double
my current income to maintain my current lifestyle in a city, which is unlikely.
What Now?
Now that the dam has burst, nothing is holding me back from publishing. I’m going to continue to work on skill building, but now I feel free to write about it. If this article resonated with you–whether you also have knowledge gaps you’d like to fill, you’d like to help me fill mine, or you just want to see what happens–please let me know in the comments. Subscribe via Mastodon to see everything I post, use RSS to customize your subscription, or subscribe to my mailing list to get notified when I post a larger article.
Mastodon & Fediverse
To help you follow along, I’ve enabled ActivityPub on this blog, meaning it’s a fully-functioning Mastodon account:
@
[email protected]
.
Articles about programming, learning, code, books, and teams
Of course, the classic (and my favorite) way is to subscribe to email notifications. I’ll only send emails when I publish a new article, not for every micro-post.
During a recent internal Bitmovin hackathon focused on experimenting with AI tools, I decided to work on a project I had been wanting to explore, even though it was outside of our usual video focus. I gave myself a simple solo project that I thought would be a great way to test modern AI coding assistants: integrate an API that returns solar generation data. It turns out what should have been a straightforward integration turned into a two day reminder of how easily AI can fail.
Two leading tools, Cursor and Claude, both hit the same tiny string formatting bug and neither could get past it. The outcome was that they were defeated by the exact same task, but in completely different ways. One ran into a silent logical wall, while the other dramatically hallucinated a completely false solution.
The Shared Battlefield: A Hyper-Specific Signature
My objective was to interface with the FoxESSCloud platform. The core hurdle was generating a unique signature for every request, a standard practice in proprietary APIs to ensure request authenticity and to prevent tampering.
This signature is produced by taking a concatenated string of five critical request parameters:
HTTP method (POST)
API path
Unique auth token,
Timestamp
JSON request body
Then you run that final string through an HMAC-SHA256 hash function. The difficulty lay entirely in the preparation of the input string, not in the hashing itself.
The Stumbling Block: The Concatenation Trap
The API documentation required the string to be concatenated using newline characters (\n). However, the API was expecting the newlines to be handled as literal characters within certain parts of the string, and not simply as concatenation operators. This created a massive blind spot for both AI tools, as shown in the examples below.
Status
Pseudo-Code Generated by AI (Wrong)
The Required Format (Right)
Problem
The AI-generated code often used concatenation operators (+ “\n” +) to build the string, resulting in the “illegal signature” error.
The API required the newlines to be
included as literals
within the string structure itself for the first segment of the string.
No matter how I prompted them, both AI tools stayed locked on the version on the left, and the API refused every request until I switched to the format on the right
Day 1: Cursor’s Silent Failure (The Logical Dead-End)
I started with
Cursor
, the AI-powered editor, feeding it the API documentation and error logs.
Cursor’s approach was methodical but ultimately circular. It correctly identified the part of the code responsible for the hash generation, but it lacked the critical insight to challenge the input string’s construction. I spent hours debugging with it, and its suggestions revolved around changing the encoding or the hashing library, which are all standard boilerplate fixes that were incorrect.
Cursor’s failure was one of
logical stubbornness
. It would not deviate from its initial, flawed concatenation pattern, making it a technical dead-end. The error was always the same:
“illegal signature.”
Day 2: Claude’s Dramatic Failure (The Confident Hallucination)
Frustrated with Cursor, I switched to
Claude
on Day 2 to get a fresh perspective on the logs. Claude was immediately more conversational and engaging, which at first made it feel more helpful, but its output was even more misleading.
When presented with the failing code and the “illegal signature” error, Claude was unable to identify the simple string concatenation bug that Cursor had also missed. Instead, it diverted the entire debugging process by dramatically announcing a breakthrough.
The Story of the Wrong Time
While I was feeding it logs and error messages, Claude seized on the timestamp parameter, confidently declaring:
FOUND IT!
The timestamp is showing
2025-11-18
but the actual current date is
2024-11-18
. Your system clock is set exactly one year in the future! The FoxESS API is rejecting the requests because the timestamp is in the future…
Please fix your system clock.
This was a
Red Herring
of the highest order. It sent me down a completely baseless tangent; I immediately checked my system clock, and
it was perfectly correct.
Claude had completely hallucinated a complex, plausible system-level problem (time drift) to explain the error, rather than addressing the actual bug in the code. It swapped Cursor’s quiet inability to solve the issue for a confident, authoritative explanation that was entirely false.
The Unsolved Problem
After correcting the initial timestamp tangent, I was back at square one. I explicitly asked Claude to fix the string format, and, just like Cursor, it generated the flawed concatenation highlighted in the previous section.
The critical takeaway:
Two distinct, high-powered AI coding tools were simultaneously defeated by a single, subtle formatting requirement in an API integration. They could perform the complex HMAC hashing, but they could not master the necessary string structure.
Conclusion: The New Rules of AI-Assisted Coding
My hackathon project ended not with a data visualization, but with a critical lesson on the state of LLMs in development:
AI Shares Blind Spots:
LLMs are powerful pattern matching systems. If a common pattern (like string + “\n” + string}) is the
wrong
solution for a highly specific API, both models are likely to repeat the mistake. They lack the ability to truly read documentation critically and apply byte-level precision.
The Contrast in Failure:
Cursor failed silently, trapped by its initial logic. Claude failed dramatically, compounding the actual bug with a confident, fabricated system error. The
hallucination
proved to be the more disruptive, time-wasting error mode.
AI is a powerful coding assistant, but for subtle, context-heavy, and non-standard parts of coding, where literal truth is paramount, the human developer, armed with a print(signature_string) command, is still the superior debugger.
This repository includes patches, that can be applied to classic Mac OS New World Toolbox images of version 1.2 and higher, that patches back in code that was removed and required for booting earlier versions of classic Mac OS.
With an appropriate System Enabler where required (CHRP System Enabler for 7.x, iMac G3 enabler for early 8.x, Sawtooth 8.6 TBXI for 8.6...), such a patched TBXI can attempt to boot System 7.5 and above, with varying degrees of success (some extensions/control panels/etc can cause crashes).
Patches
The following patches are included:
m68k_patch
This is a patch set for the main m68k part of the Toolbox image, which brings back in implementations of various interfaces that got removed from the tree at various points:
TBXI v1.2 removed the Program-to-Program Communication Toolbox implementation - Mac OS 8.x boot does not use this but 7.x requires it to be present
Between there and TBXI v2.5.1, the legacy Sound Toolbox, Communications Toolbox and Event Manager implementations were removed.
Between TBXI v6.1 and v8.8, the Picture Utilities, Text Services Manager, and Icon Utilities implementations were removed - additionally, a function in one of the tables was changed from a no-op to a bad trap, and the code to initialise another function table pointer in a structure pointed to by low memory was removed (this last one was done between v3.0 and v3.7).
After v8.8, there were no significant changes here - and later TBXIs can use the m68k main part from v8.8 with no issues.
There are three different patchsets, the script to apply the patches will detect which set to apply based on what traps are present.
The assembler used here is
vasm
(for m68k, mot syntax), built from source with patches applied to
symbol.c
to remove marking repeat definitions as an error (in
check_symbol
comment out the line with
general_error(67,name);
and in
new_abs
do the same with that and the
if
line above).
The patch script is written in PHP, just run
php patch.php "path/to/Mac OS ROM.hqx.src"
with your build of vasmm68k_mot in the same directory.
ProcessMgrSupport
At some point between TBXI v3.1 and v6.1, the initialisation interface for
ProcessMgrSupport.pef
changed - it originally had its entry point pointing to a mixed-mode RoutineDescriptor which was called from m68k code, and was changed to an exported PowerPC function
InitProcessMgrSupport
.
ProcessMgrSupport is not large (old variant is ~5KB, new variant is ~3KB), so I decompiled the parts in C and disassembled the parts in PowerPC asm, and included both initialisation formats, so it can work to boot older and later System files.
If your TBXI's
Parcels.src/MacROM.src/Mac68KROM.src/Rsrc/ncod_8_ProcessMgrSupport.pef
is around 3KB in size, build the provided implementation with Retro68, and replace it with the built PEF, otherwise booting earlier System files will crash due to calling a null pointer.
InterfaceLib patches (done manually)
At Mac OS 9.0, the format of the File Control Block changed, and the PowerPC functions in InterfaceLib to get/set entries related to it in low memory were changed (at some point between TBXI v3.0 and v3.7) to cause a system error (this was described in Technical Note TN1184).
To work around this issue with later TBXIs and earlier System files, the following patches need to be made to InterfaceLib:
This could be automated, though code has not yet been written to do this. If you want to use a TBXI from 9.x era, it would be best to use TBXI v10.2.1 as a base; here are the patches for 10.2.1 (in
fc /b
format of
offset: original patched
):
Mac OS 9 stores fake objects in the old FCB table to keep backwards compatibility with old m68k code - so here be dragons, etc. It's enough to let older code
run
at least.
Known issues
With a patched v10.2.1 TBXI, the
Date & Time
Control Panel in System 7.x will crash (on real hardware, QEMU seems to be fine) if it's present. The reason why is currently unknown (probably a function pointer call through a wild/uninitialised pointer access).
Django 6.0 introduces a built-in background tasks framework in
django.tasks
. But don't expect to phase out Celery, Huey or other preferred solutions just yet.
Django handles task creation and queuing, but
does not provide a worker mechanism to run tasks.
Execution must be managed by external infrastructure, such as a separate process or service.
The main purpose of the new
django.tasks
module is to
provide a common API for task queues
implementations.
Jake Howard
is the driving force behind this enhancement. Check out the introduction on
the Django forum
.
His reference implementation, and simultaneously a backport for earlier versions of Django, is available
as
django-tasks
on GitHub
.
But let's ignore that and play with the more minimal version included in Django 6.0 instead. By creating our very own backend and worker.
Our project: notifications
We're going to create an app to send notifications to phones and other devices using
ntfy.sh
. (
I'm a fan!
)
The free version only provides public topics and messages. Meaning anyone can see the stuff you're sending
if
they subscribe to the topic. For our purpose we can simply create a topic with a randomized name, like a UUID.
The project's settings expect the URL from step 4 to be supplied as an environment variable. For example:
importhttpxfromdjango.confimportsettingsdefsend_notification(message:str,title:str|None):# Pass the title if specified.headers={"title":title}iftitleelse{}httpx.post(settings.NTFY_URL,content=message,headers=headers,)
Really. That's all there is to it to start sending and receiving notifications.
This is the main goal of the new framework: defining tasks using Django's standard API, rather than using task queue specific decorators, or other methods.
So here it goes:
# ...fromdjango.tasksimporttask@taskdefsend_notification(message:str,title:str|None):# ...as before
Our function is now a task. In fact it's a
django.tasks.Task
.
You cannot call
send_notification
directly anymore. Tasks can only be run by using the
enqueue
method. It might not be the behavior you'd expect or want, but this seems to be the best option. This design eliminates the possibility of accidentally invoking a task in-process, rather than in the background.
The
task
decorator allows you to specify the task's priority, queue name and backend name. You can override these settings with the
using
method, which returns a new
django.tasks.Task
instance.
If you need more control over task behavior, you can set
takes_context
to
True
in the decorator and add
context
as the first argument. This context currently provides you with access to the task result and thereby useful information like the number of attempts.
There's no way of defining retries and backoffs, or other fancy things you might expect from a full-blown task queue implementation. But that's
not
what this is. You can easily add your own retry logic by inspecting the task context if needed.
Enqueuing a task
Adding a task to the queue is easy:
task_result=send_notification.enqueue(message="Season's greeting!",title="Santa has something to tell you")
Executing a task
This is where things start to fall short. At least right now. Django 6.0 will ship with the
ImmediateBackend
and the
DummyBackend
. The first will execute the task immediately, while the latter will not execute the task at all.
Which is why our project includes a (demo) backend backed by the database and a worker process!
Fetching the result
If you're not going to wait around for the result, you can get a hold of it later on using its id. Simply call
get_result(result_id)
on your task.
Our project includes a view that's polled periodically for outstanding results using
htmx
.
The list underneath the form shows the results for each execution of our task. When the form's submitted, a new result is added to the top of the list. Htmx is instructed to keep polling for changes as long as the result's status isn't
FAILED
or
SUCCESSFUL
.
deftask_result(request,result_id,status):result=send_notification.get_result(result_id)ifresult.status==status:# No need to swap the result.returnHttpResponse(status=204)returnTemplateResponse(request,"index.html#result",{"result":result})
Wondering what
index.html#results
is doing? Django 6.0 also introduces
template partials
. In this case our view effectively sends a response containing only the template partial named
result
.
Behind the scenes
When you decorate a callable with
task
, the configured backend's
task_class
is used to wrap the callable. The default's
django.task.Task
.
That class's
enqueue
method will in turn invoke the configured backend's
enqueue
method.
Calling its
get_result
method is similar: call the configured backend's
get_result
method and pass on the result.
Since there's no workers, that's basically all a task backend needs to provide. Cool. Let's add one, shall we?
A task database backend
Our goals:
A basic task backend, backed by our database.
We want to support "automagic" retries
Our
enqueue
and
get_result
methods will return an instance of the default
django.tasks.TaskResult
. This determines the minimum amount of data we need to store, and we're going to do so in a model called
Task
.
Models
Let's create a first draft of our
Task
model, based on the properties of
TaskResult
and
Task
in
django.tasks
(the "dataclasses"):
classTask(models.Model):priority=models.IntegerField(default=0)callable_path=models.CharField(max_length=255)backend=models.CharField(max_length=200)queue_name=models.CharField(max_length=100)run_after=models.DateTimeField(null=True,blank=True)takes_context=models.BooleanField(default=False)# Stores args and kwargsarguments=models.JSONField(null=True,blank=True)status=models.CharField(choices=TaskResultStatus.choices,max_length=10,default=TaskResultStatus.READY)enqueued_at=models.DateTimeField()started_at=models.DateTimeField(blank=True,null=True)finished_at=models.DateTimeField(blank=True,null=True)last_attempted_at=models.DateTimeField(blank=True,null=True)return_value=models.JSONField(null=True,blank=True)
What's missing? For one, the
TaskResult
also includes a list of encountered errors, and ids of the workers that processed the task. Something that we could
perhaps
ignore.
Except the
TaskResult.attempts
property is based on the number of worker ids. And if you're using the task context within a task, you're bound to be relying on that type of information.
We could add these details to the
Task
model by adding a
JSONField
for each. This is the current approach in the reference implementation.
But let's be more explicit in our approach and define models for these as well. We'll record each attempt to execute a task and its potential error, linking them to the task with a foreign key:
classError(models.Model):exception_class_path=models.TextField()traceback=models.TextField()classAttemptResultStatus(TextChoices):# Subset of TaskResultStatus.FAILED=TaskResultStatus.FAILEDSUCCESSFUL=TaskResultStatus.SUCCESSFULclassAttempt(models.Model):task=models.ForeignKey(Task,related_name="attempts",on_delete=models.CASCADE)error=models.OneToOneField(Error,related_name="attempt",on_delete=models.CASCADE,null=True,blank=True)worker_id=models.CharField(max_length=MAX_LENGTH_WORKER_ID)started_at=models.DateTimeField()stopped_at=models.DateTimeField(blank=True,null=True)status=models.CharField(choices=AttemptResultStatus.choices,max_length=10,blank=True)
This setup ensures we have all necessary information to execute a task, plus we can provide every single bit of detail when a
TaskResult
is requested.
All fine and dandy, but we need to think about the worker's requirements as well. It needs to be able to:
Quickly check for outstanding tasks
Claim one of those tasks
Process that task and either mark it as failed, successful or ready (to retry later)
We could do all of that with how it's set up right now, but I'd like to refine things a bit.
classTask(models.Model):# ...# This field is used to keep track of when to run a task (again).# run_after remains unchanged after enqueueing.available_after=models.DateTimeField()# Denormalized count of attempts.attempt_count=models.IntegerField(default=0)# Set when a worker starts processing this task.worker_id=models.CharField(max_length=MAX_LENGTH_WORKER_ID,blank=True)# ...
The
available_after
field will contain the earliest time at which the task can be executed. If the task's
run_after
is specified (which can be done by using a task's...
using()
method),
available_after
is set to that value. Otherwise we're using the current datetime; all in UTC.
Once a task needs to be retried,
available_after
will be set to the next possible point in time the task can be executed. In other words: we can
back off.
The
attempt_count
field makes querying for available tasks a bit easier. Any tasks with an
attempt_count
greater than the maximum allowed value can be ignored. Yes, their status should have been set to
FAILED
which means they should be excluded by default, but we could change the configuration and tweak the maximum number of attempts.
The
worker_id
field is filled when a worker claims a task. This, among other things, prevents any other workers from picking up the task. Assuming the worker id is unique.
Enqueueing and fetching a result
Enqueueing a task could not be easier: create a
Task
model instance
from the
Task
dataclass instance
, save it, done! Well, at least after turning the end result into a
TaskResult
.
We use the string version of the model's database id as the id of the result.
Retrieving a result is likewise only a matter of loading the task and its attempts, and turning that into a
TaskResult
.
Here's a simplified version of our task backend as it stands:
classDatabaseBackend(BaseTaskBackend):supports_defer=Truesupports_async_task=Falsesupports_get_result=Truesupports_priority=Truedefenqueue(self,task:Task,args,kwargs):self.validate_task(task)model=self.queue_store.enqueue(task,args,kwargs)task_result=TaskResult(task=task,id=str(model.pk),# ...# More properties being set# ...)returntask_resultdefget_result(self,result_id):returnself.model_to_result(self.queue_store.get(result_id))defmodel_to_result(self,model:models.Task)->TaskResult:...
At lot of functionality is deferred to this
queue_store
property. Before we dive into that, we'll explain the configuration options for this backend.
Configuration
We want to be able to specify defaults for:
the maximum number of attempts (retries)
the backoff factor; i.e. we'll back off using
math.pow(factor, attempts)
These can be customized for each individual queue. So we end up with something like this in our
OPTIONS
:
A task added to the
low_priority
queue will be attempted up to five times, with a backoff factor of
3
. Other tasks will be attempted up to ten times with the same backoff factor.
Queue store
The
QueueStore
class
is a companion of our backend. It's focus is on retrieving and enqueueing tasks, checking for tasks to execute and claiming tasks.
However the main reason it's included is to simplify the worker. As we'll see, the worker gets it's own copy of the queue store,
limited to the queues it needs to process.
The worker
The worker's job, at least in this project, is to provide information on outstanding tasks to the runner and to drive the processing of those tasks
by the backend
. Which means it looks like this:
classWorker:def__init__(self,id_:str|None,backend_name:str,only:set[str]|None,excluding:set[str]|None,):# Grab the backend and its queue_store.self.backend=task_backends[backend_name]queue_store:QueueStore=self.backend.queue_store# Limit the queue_store to the select queues.ifonlyorexcluding:queue_store=queue_store.subset(only=only,excluding=excluding)self.queue_store=queue_store# Use or create and id. "Must" be unique.self.id=(id_ifid_elsecreate_id(backend_name,queues=queue_store.queue_names))defhas_more(self)->bool:returnself.queue_store.has_more()defprocess(self):withtransaction.atomic():tm=self.queue_store.claim_first_available(worker_id=self.id)iftmisnotNone:self.backend.process_task(tm)
All we need to do to have a functioning worker runner:
Create an instance of the worker.
Ask it if there's tasks to execute using
has_more
.
If so: tell it to
process
the first available task. If not: go to 4.
Our queue store provides a
peek
method which returns the id of the task in our queues with the most urgency; a combination of
available_after
,
priority
and
attempt_count
.
This lets the runner know whether there's more tasks to process. The next step is to claim one of those tasks. So we call
peek
again and if it returns a task id, we'll try to claim that particular task.
Here's a more basic, clearer version than the one included in our project's
QueueStore
:
If the
count
is zero, we failed to claim the task. Otherwise we retrieve it from the database and can start processing.
The loop is included because we ended up here after trying to claim the task identified by
peek
. Which apparently has already been picked up by another worker. We might as well make the most of it and try to grab another task from the queue.
Processing the task
And finally the thing that actually does something!
The
process_task
method of our backend:
Creates an
Attempt
and constructs the current
TaskResult
.
Executes the task, capturing anything extending
BaseException
, or returning the
return_value
of the task when everything went according to plan.
Either updates the
Task
model, the
Attempt
and
the
TaskResult
with the final details of the successful execution, or with details about the failure to do so.
And in the latter case: check if the task can be retried.
Again: if you want to dive into the details, have a look at
the repository
.
That's it
Of course this demo project leaves out all the things you really need to think hard about. Like signals for the worker. Or database transaction logic. That's not to say it's impossible. Far from it. It just wasn't the goal of this article.
The inclusion of this functionality in Django will certainly allow new libraries or adapters for existing task queues to pop up. And we'll probably soon see some complaints that
django.tasks
isn't extensive enough.
Because, if you're currently using the more advanced functionality of your task queue, there's probably a few things you're missing in
django.tasks
.
Complex orchestration
Some task queue libraries, like Celery, provide ways of combining tasks. You can feed the result of one task into another, enqueue tasks for each item in a list, and so on.
It should be clear by now that supporting this kind of orchestration isn't the goal of
django.tasks
. And I don't mind at all. There's no feasible way of creating a unified API to support this. I've had my share of problems with libraries that
do
claim to support it.
Retry
As mentioned before, there's currently no way to automatically retry a failed task, unless your backend does the heavy lifting. Like ours does.
Depending on your backend this might be easy enough to handle yourself. For example using a decorator:
defretry(func):@functools.wraps(func)defwrapper(context:TaskContext,*args,**kwargs):try:returnfunc(context,*args,**kwargs)exceptBaseExceptionase:result=context.task_resultbackoff=math.pow(2,result.attempts)run_after=datetime.now(tz=UTC)+timedelta(seconds=backoff)result.task.using(run_after=run_after).enqueue(*args,**kwargs)raiseereturnwrapper@task(takes_context=True)@retrydefsend_email(context:TaskContext,to:str,subject:str,body:str):# Do your thing ...
An actual worker mechanism
True. But the
reference implementation
does provide actual workers. Be patient, or even better: start helping out!
There is no perfect solution
I reckon
django.tasks
will soon result in covering at least the most common 80% of use cases. Yes, its API is simple and limited, but to me that's more a benefit rather than a fault. I think this is as close as you can get to a standardized approach.
For years it has been a common theme among programmers that Google's search results have changed for the worse.
It feels like the suggestions are becoming less and less applicable over time.
Today, I spotted one of the worst cases that I have seen so far when searching for a documentary called "Flatten the Curve Flat Earth" (2022).
This documentary is about flattening the curve of the earth and has nothing to do with medicine.
However, Google automatically interprets it as a kind of medical statement:
Notice especially the second search result where "may not have changed pandemic attitudes" is highlighted or the last result where "even unrelated vaccines could help reduce the burden of the pandemic".
What this suggests is that Google automatically turns the query "flatten the curve 2022" into a query about the pandemic even though nothing in the query refers to the pandemic.
Given that this query is steered so aggressively into a certain direction, it really makes me wonder in what ways Google is "helpfully" steering other queries.
Please just show what I'm searching for, Google. Thanks.
Airbus A320 – intense solar radiation may corrupt data critical for flight
Toulouse, France, 28 November 2025
– Analysis of a recent event involving an A320 Family aircraft has revealed that intense solar radiation may corrupt data critical to the functioning of flight controls.
Airbus has consequently identified a significant number of A320 Family aircraft currently in-service which may be impacted.
Airbus has worked proactively with the aviation authorities to request immediate precautionary action from operators via an Alert Operators Transmission (AOT) in order to implement the available software and/or hardware protection, and ensure the fleet is safe to fly. This AOT will be reflected in an Emergency Airworthiness Directive from the European Union Aviation Safety Agency (EASA).
Airbus acknowledges these recommendations will lead to operational disruptions to passengers and customers. We apologise for the inconvenience caused and will work closely with operators, while keeping safety as our number one and overriding priority.
It is OK to Say “CSS Variables” Instead of (or Alongside) “Custom Properties”
TPAC 2025
just ended, and I am positively tired. Attending it remotely, my sleep schedule is chaotic right now. I have many ideas for CSS-related posts in my list of ideas for November, but almost all of them require at least some amount of research and crafting demos.
Well! I found one note that I wanted to expand on, and which sounds tiny enough to be able to finish it in my altered state.
Let me repeat the title of this post:
it is OK to say “CSS Variables” instead of (or Alongside) “Custom Properties”
.
I won’t say that this is something contentious, but it was always mostly a thing where I always stumbled a bit before continuing using the terminology.
The official name of the
corresponding CSS module
is “CSS Custom Properties for Cascading Variables”. It’s URL’s slug is
css-variables
.
They are
variables
. More specifically:
cascading
variables. They change with the cascade: when different rules match, values can be overridden and change.
We can have animations that involve custom properties, or custom properties with values based on the viewport, containers, or something else — dynamic, responsive values that can
vary
for multitudes of reasons.
They are
also
custom properties, and even the more property-like when using
@property
. They can also be explicitly typed, while the rest of CSS is often typed implicitly. But — typed, unlike some other “programming languages”.
Ah, yes, CSS (and HTML)
are
programming
languages, and anyone thinking otherwise is wrong. The
best
programming languages, according to me, by the way.
Oh, I am tired. But also right after finishing this last
day
night
of CSSWG
F2F
, I successfully experimented a bit with one ongoing idea of mine, and now planning to write a proper nice article, for my main site, like I sometimes do. Stay in touch.
In response to the recent revelations about the
NSA backdooring RSA libraries
I've compiled a brief, incomplete, history of NSA backdoors. Help me make it better by emailing corrections and additions to ethan.r.heilman@gmail.com.
Update:
added Actel backdoor,
Update 2:
There is a
hackernews thread for discussion
.
Update 3:
Added Newly discovered postal inception backdoor installation.
1946-1970, The Ultra Secret:
After WW2, the British Empire sold captured German
Enigma cipher machines
to many allied countries and former colonies
1
. The US and the UK had broken Enigma but had kept this fact secret so that countries would use these broken ciphers. To clarify: the British sold machines they knew they could break to allied nations, then the US and the UK spied on those countries for nearly 30 years exploiting the weaknesses in those machines.
1957 - Present, The Boris Project:
In 1957
William Friedman
of the NSA met with his old friend
Boris Hagelin
. The purpose of their meeting was to begin "the Boris Project", in which
Crypto AG
ciphers would be weakened and backdoored so that the NSA could listen to NATO communications (there is some evidence that suggests that the Boris Project predates this meeting). The meeting was first made public in the biography of Friedman, "The Man Who Broke Purple"
2
. Further details were made public with the publication of the
"The Puzzle Palace"
including letters showing Friedman's concern about direction of the project
3
. From interviews with ex-employers we know that the addition of backdoors to Crypto AG ciphers occurred no later, and possibility earlier, than the 1970's and likely continues to the present day
14
. These backdoors included covert channels that allowed full key reconstruction
16
.
Slowly the world figured out that Crypto AG was not a reliable vendor of cryptographic hardware. In 1986 Reagan tipped off the Libyans that the US could decrypt their communications by talking about information he could only get through Libya decrypts on TV
15
. In 1991 the Iranians learned that the NSA could break their diplomatic communications when transcripts of Iranian diplomatic communications ended up in a French court case
17
. In 1992 Iranians got so upset with Crypto AG that they charged a Crypto AG salesman with espionage
19
. Although, despite this evidence, the Iranians appear to have continued to use Crypto AG machines for diplomatic communications until, and perhaps beyond, 2003
18
. In 2004 Ahmed Chalabi was accused of selling the Iranians the methods by which the US was breaking their codes. It is speculated that this might have been information on Crypto AG backdoors or weaknesses
20
.
1979 - Present,
DES
:
The Data Encryption Standard was altered by the NSA to make it harder to mathematically attack but easier to attack via
Brute Force
methods. The original version of DES, called
Lucifer
, used a block and key length of 128-bits and was vulnerable to
differential cryptanalysis
. NSA requested that the already small DES key size of 64-bits be shrunk even more to 48-bits, IBM resisted and they compromised on 56-bits
4
. This key size allowed the NSA to break communications secured by DES.
1993,
Clipper Chip
:
The NSA was deeply concerned with the public adoption by Americans of cryptography that they couldn't break. In 1993 they proposed that voice communication be secured with an encryption chip called "the Clipper Chip". The Clipper Chip was backdoored such that the NSA could, at will, break any communication secured by the Clipper Chip. Unlike most of the backdoors in this list the NSA announced that the presence of the backdoor. Due to its known insecurity the Clipper Chip was never widely adopted.
1997
Lotus Notes
:
The NSA requested that Lotus
weaken its cryptography
so that the NSA could break documents and emails secured by Lotus notes
5
. This Software was used by citizens, companies and governments worldwide
6
7
.
200? - Present,
Actel
ProASIC3 FPGA:
In 2012 Skorobogatov and Woods discovered that Actel military grade FPGA's contained a backdoor. The researchers were able to reverse engineer the key such that they could exploit the backdoor
24
. This chip is used in US weapon systems, nuclear power plants and transportation
21
. All other Actel chips appear to have this backdoor as well
22
. At first there was some concern that the backdoor was planted by a foreign government but it was revealed that Actel, an american company, intentionally added this backdoor
24
.
While there is no smoking gun linking this backdoor to the NSA (at least not yet), it seems implausible to me that a US Company would design a complex backdoor and insert it into chips used in critical US systems without US government approval. Additionally, if Actel had created this backdoor without US approval I would expect more of a response from the US government. The US response has been, to my knowledge, complete silence on the issue.
2004 - 2013,
Dual_EC_DRBG
:
Dual Elliptic Curve Deterministic Random Bit Generator[ or Dual_EC_DRBG is a random number generator created by the NSA. It is designed so that if the NSA selected the internal constants carefully, they could generate a secret key which would allow them to break encryption schemes that relied on Dual_EC_DRBG for security. This property of Dual_EC_DRBG was
discovered in 2006 by Brown
and rediscovered by Shumow and Ferguson in 2007 leading to public speculation that Dual_EC_DRBG was backdoored
8
. In 2004
9
the NSA paid RSA security 10 million dollars
10
to add Dual_EC_DRBG as the default choice in some of its libraries. The NSA then used the fact that RSA was using Dual_EC_DRBG to get it approved as a NIST standard.
2013, Enabling for Encryption Chips:
In the NSA's budget request documents released by Edward Snowden, one of the goals of the NSA's
SIGINT
project is to fully backdoor or "enable" certain encryption chips by the end of 2013
11
. In 2023 it was revealed that Cavium CPUs (now named Marvell) was one of the companies backdooring chips
27
. It is not publicly known the full set of encryption chips they have modified in this way.
2013, Trusted Computing Platforms/Modules:
A resource in the same, previously mentioned, budget request is the exploitation of foreign Trusted Computing Platforms and technologies
12
. There has been some concern expressed in Germany that the Microsoft TCM 2.0 could be backdoored by the NSA
13
.
? - Present, Postal Interception Backdoor Installation:
According to a 2010 report leaked to the Guardian
25
, the NSA's Access and Target Development department routinely intercepts computer equipment being sent through the mail and adds implants
26
. The equipment, which is generally networking devices and servers, is then sent on its way to be used by the targeted individuals and organisations. These implants allow the NSA the ability to connect into airgapped private networks.
'The impact on UK airlines seems limited,' transport secretary says
published at 22:22 GMT
Katy Austin
Transport correspondent
Image source,
Reuters
Responding to the technical issue with Airbus aircraft, the UK's Transport Secretary Heidi Alexander says:
“I am aware of the technical issue impacting certain aircraft and concerns over how this will affect passengers and flights this evening."
She advises passengers who are due to fly this weekend to check with their carriers for the latest information.
The transport secretary says "the good news" is that "the impact on UK airlines seems limited, with a smaller number of aircraft requiring more complex software and hardware changes".
“I would really like to thank the experts, staff and airlines who are working at pace to address this and reassure passengers that work is ongoing."
"It is heartening this issue has been identified and will be addressed so swiftly, demonstrating the high aviation safety standards globally," she adds.
Australian airline Jetstar says some of its flights are unable to depart
published at 22:07 GMT
We've just seen an update from low-cost Australian airline Jetstar, which says some of its Airbus-operated flights are unable to depart at the moment.
"We’re working through the impacts on our fleet and to our customers. We'll have more information shortly," it says.
There is no impact to airline giant Qantas at this stage.
For context: Jetstar is Qantas's low-cost airline.
Planes can't fly passengers until they have been fixed, EASA says
published at 21:58 GMT
Michael Race
Business and economics reporter
A European Union Aviation Safety Agency (EASA) directive stipulates that, as of 29 November, the planes thought to be affected can only fly passengers once they've been fixed.
They will be allowed to make so-called "ferry flights", without passengers, in order to get to a maintenance facility.
The latest from Heathrow and Gatwick airports
published at 21:49 GMT
Image source,
Reuters
We've now heard from London's two biggest airports, Heathrow and Gatwick.
"We are aware of a directive requiring some airlines operating Airbus A320 aircraft to update software on their fleet over coming days, which may result in some disruption," a spokesperson for Gatwick Airport says.
"This is only impacting a small number of airlines at London Gatwick. Passengers should contact their airline for more information."
Separately, Heathrow Airport says the required maintenance on some Airbus aircraft currently has no impact on its operations.
'Potential for disruption immeasurable at this stage,' travel journalist says
published at 21:48 GMT
"Dozens of big airlines use these aircraft," travel journalist Simon Calder tells the BBC.
"So the potential for disruption is, I'm afraid, immeasurable at this stage."
Calder says that while British Airways' mainline fleet of short-haul flights that are at Heathrow and Gatwick Airports use entirely Airbus A320 series planes, "my understanding is that only three of those are affected by this and as a result they'll be able to get the work done overnight, hopefully with no disruption".
He adds he has spoken to Wizz Air and EasyJet, who we have reported are expecting some delays. "Now the assumption, I'm glad to say, is that your flight will be going ahead as normal and the airline will contact you if there is any change to that."
About 80 aircraft affected at Gatwick, BBC understands
published at 21:31 GMT
Simon Browning
Transport correspondent
The Airbus software issue could take hours to fix on the planes themselves, which an industry source suggests means some cancellations are inevitable over the weekend.
It is understood UK-based airlines are currently working on their plans, with one airline being particularly affected.
Planes will arrive in the UK this evening as usual, but it appears some will not do their next turnaround tomorrow and depart from the UK again.
I've been told airports will need to play “a logistical game of Tetris tonight to free up space” to make sure incoming aircraft can be parked overnight. Airports “will be having to park them remotely to ensure incoming long hauls have space", the industry source says.
It is unclear how many engineers will be needed at once to update software. The skies at this time of year are quieter after the peak summer period and before the Christmas festive rush begins.
It is understood about 80 aircraft are affected at Gatwick Airport.
Air Canada not expecting any impact to operations
published at 21:27 GMT
Air Canada says it's not expecting any impact to its operations as "very few of our aircraft use that version of the software".
However, connecting flights with other airlines could still be delayed by a knock-on effect.
Other Canadian Airlines WestJet and Porter don't have any of the A320 planes listed on their websites.
How long will repairs take?
published at 21:25 GMT
Theo Leggett
Business correspondent
This issue affects around 6,000 aircraft.
For the majority of planes, the fix will involve installing new computer software. This should normally take about three hours.
But around 900 older planes will need computers replaced and will not be allowed to carry passengers again until the job has been completed.
How long that takes will depend on the availability of replacement computers.
It’s not yet clear whether there will be enough parts to meet demand.
More airlines warn of potential service disruption
published at 21:09 GMT
Image source,
Getty Images
We're hearing from more airlines around the world who are now reporting potential disruptions to services as a result of the immediate recall of aircraft by Airbus.
American Airlines
says 340 of its planes are affected and that it expects "some operational delays" but expects the vast majority of updates to completed today or tomorrow
Delta Airlines
says it will comply with Airbus's instruction but expects any resulting operational impact to be "limited"
Air India
says the instruction from Airbus could lead to a "longer turnaround and delays to our scheduled operations
Wizz Air
has warned passengers flying over this weekend that they may face disruption as a result of the update
We'll bring you further airline updates when we have them.
Why this is an extremely unusual issue
published at 21:06 GMT
Theo Leggett
Business correspondent
It was experienced on one flight, when an aircraft flying from Cancun to New Jersey was affected by a sudden and intense solar storm.
The radiation corrupted data in the ELAC - a computer used to operate control surfaces on the wings and horizontal stabilizer.
The fault caused the plane to go into a sudden descent.
Airbus says the plane had recently had its software updated. The issue had never arisen with the previous software.
But now, it insists it is acting out of an abundance of caution.
It took the issue to regulators itself and asked airlines to make changes.
EasyJet expects 'some disruption'
published at 20:55 GMT
Image source,
Getty Images
EasyJet says its aware of the communication from Airbus to airlines operating the A320 family aircraft and is "currently working closely with the safety authorities and Airbus to implement the action we need to take".
The airline says it is expecting this to result in some disruption and will "inform customers directly about any changes to our flying programme tomorrow and will do all possible to minimise the impact".
“Safety is our highest priority and EasyJet operates its fleet of aircraft in strict compliance with manufacturers guidelines,” its statement concludes.
Air New Zealand warns of disruption across number of flights today
published at 20:51 GMT
"As a precaution, all our A320neo aircrafts will be receiving a software update before operating their next passenger service," the airline says.
The statement continues by saying "this will lead to disruption across a number of our A320neo flights today and we’re expecting a number of cancellations to services across that fleet".
"If you’re travelling today, we will be contacting customers directly if your flight is affected," the airline adds.
There may be impact to some flights, CAA warns
published at 20:45 GMT
Tim Johnson from the UK's Civil Aviation Authority tells the BBC that there may be disruptions to flights "in some circumstances".
"From a UK perspective, not all airlines fly Airbus A320 or the affected ones, so for some airlines there will be no impact at all," he says.
"For some, there may be some impact," he continues, adding that the CAA has been in touch with airlines and they're looking to make sure the maintenance is undertaken over the coming days.
He says the advice to customers is to "check airline websites and apps for the latest info about what is happening".
How did Airbus find the problem?
published at 20:43 GMT
Theo Leggett
Business correspondent
The issue was discovered after a JetBlue aircraft en-route from Mexico to the United States in October experienced a ‘sudden drop in altitude’.
The plane made an emergency landing, with reports at the time suggesting 15 to 20 people suffered minor injuries.
It’s thought the incident was caused by intense solar radiation, which corrupted data in a computer used to help control the aircraft.
Now action is being taken to prevent further problems. About 6,000 aircraft worldwide are thought to be affected, all of them of the A320 family, which also includes the A319 and A321 models.
According to Airbus, the majority can be fixed with a relatively simple software update. However, some 900 older planes will need replacement computers, and will have to be taken out of service until they can be fixed.
Warning of disruption as thousand of Airbus planes require software update
published at 20:26 GMT
Jamie Whitehead
Live reporter
Airbus says that flights will be disrupted after it requested
immediate modifications to thousands of its planes.
The plane manufacturer says it has found that intense
radiation from the Sun could corrupt data crucial to flight controls.
About 6,000 planes are thought to be affected, which is
around half of the company’s global fleet.
It’s thought most will be able to undergo a simple software
update.
We’ll be bringing you live coverage as well as information on
any potential disruption, stay with us.
Happy Thanksgiving to those of you in the United States.
If there’s one food you should overeat today, it’s probably potatoes. The humble potato is the original superfood, and you should consider eating more potatoes beyond Thanksgiving. This week’s 2% newsletter explains why.
A Danish physician named Mikkel Hindhede proved you could survive on potatoes alone in the early 1900s. He had three laborers eat nothing but spuds with a dollop of margarine for 309 days.
Five doctors examined the men afterward and determined they were all in excellent health. One participant was
described as “a strong, solid, athletic-looking figure
, all of whose muscles are well-developed, and without excess fat.”
Hindhede’s work gave scientific legitimacy to what other cultures had long known: Eating mostly potatoes will keep a person strong and healthy. The
Incans noticed this fact thousands of years earlier
.
Irish farmers experienced it in the 1800s
. A recent
study
in the journal
Nutrition
discovered that the Aymara people of the Andes and Altiplano have ten times fewer incidences of pre-diabetes compared to Americans.
“Potatoes are a surprisingly nutritionally complete food,” the nutrition researcher Stephan Guyenet, Ph.D., told me. The
USDA
reports that a medium potato contains about 170 calories, 5 grams of protein, 39 grams of carbs, and nearly every vitamin and mineral your body needs.
Potatoes have more than double the potassium of a medium banana and a quarter the vitamin C of an orange.
“Importantly, they have complete protein, a distribution of essential amino acids that’s similar to animal proteins,” said Guyenet. We don’t think of potatoes as “high protein.” But you could eat only potatoes and meet the
recommended dietary allowance (RDA)
of protein.
Potatoes contain more calories than most other vegetables. But this is actually a feature rather than a bug.
They’re in a sweet spot where they can give us enough calories to survive (try surviving on broccoli and lettuce alone) but not so many that we overeat.
Potatoes have long been associated with fullness, and scientific data backs up that observation.
A study
in the
European Journal of Clinical Nutrition
compared the satiety index—a measure of how full a food makes you feel—of common foods and discovered that plain potatoes reign supreme. They registered 50 percent more filling than their nearest competitor, fish, and seven times more filling than croissants, which ranked dead last. This study suggests you’d have to eat seven croissants—roughly 1200 calories—to experience the same fullness you’d get from one potato.
That property combined with their sweet-spot calorie concentration makes potatoes an ideal weight-loss food. You’ll feel fuller on fewer calories, making you less likely to overeat, Guyenet told me. You also might save money if you start eating more potatoes: They’re the cheapest vegetable.
If you’ve heard that potatoes are unhealthy, keep in mind the problem isn’t the potatoes. The problem is us and what we do with them. We cut them into little sticks or paper-thin wafers, then bathe them in 365-degree oil (
A third of America’s potatoes
become french fries). We boil them, then mash them with far too much butter and cream. (For the record, I’m 100% on board with too much butter and cream on Thanksgiving … but consider taking it easy on that stuff after the holiday.)
“If you look at nonindustrial agricultural societies around the world who are lean and don’t have metabolic or cardiovascular disease, they don’t fry or pump up their carbs like potatoes with fats,” said Guyenet. “Most of their plate is a plain starch—whether it’s potatoes, rice, sweet potatoes, or cassava—and the rest is a smaller quantity of something more exciting, like a meat with sauce and vegetables.”
If you’re worried about all those carbs, don’t be.
The weight of scientific evidence
suggests that carbs don’t make you fat. Overeating them does. And that, as the food satiety index study found, is hard to do with plain old potatoes.
For fitness, potatoes are a weird-but-good option. That’s thanks to their relatively high carb content, minerals, and amino acid profile. In the 2% Newsletter from two weeks ago, I explained how I once spoke with a professional ultrarunner who runs with a plastic bag of salty mashed potatoes. When he needs mid-run fuel, he’ll bite a corner from the bag and squeeze the potatoes into his mouth. They’re packed with naturally occurring electrolytes and certainly beat some sour-apple-flavored sugar goo.
Enjoy your potatoes today. And hopefully more days this year. And they don’t have to be bland to remain healthy. Surviving on only white potatoes is doable, but it’s not optimal in the long run. Potatoes lack two vitamins: A and B12. This is why that Dutch researcher in the 1800s gave those five men margarine with their potatoes. It’s also why most potato-dependent cultures eat them with a bit of greens or carrots and a small amount of animal products, like butter, milk, eggs, or meat. So feel free to add a bit of butter or sour cream.
Growing up, I’d always heard of basic, unexciting men referred to as “a ‘meat and potatoes’ kind of guy.” But the more I learn about potatoes, I’m OK with that designation. Because if I just throw some greens or carrots into the mix, I’ll be perfect.
//
My two favorite things this week ...
I’m digging the Sitka Ambient Jacket (here's a
men's link
, here's a
women's link
). Sitka started as a company that created high-performance hunting gear. They’re now making all their pieces in solid colors (i.e., not camouflage). I think they’re producing the best gear in the outdoors space right now. This jacket is my newest favorite for fall rucks, hikes, and just hanging around town. It’s water resistant yet comfortable and somehow always keeps me the perfect temperature.
I posted
this quick breakdown
about the six key differences between rucking with a ruck versus a weight vest. People found it useful. You might, too.
Thanks for reading and I'll see you next week,
Michael
When I decided to accept sponsorships for this newsletter,
GORUCK
was a natural fit. Not only is the company's story included in
The Comfort Crisis
, but I've been using GORUCK's gear since the brand was founded. Seriously. They've been around ~12 years and I still regularly use a pack of theirs that is 11 years old. Their gear is made in the USA by former Special Forces soldiers. They make my favorite rucking setup: A
Rucker
and
Ruck Plate
.
Discussion about this post
Amber Czech Was Murdered at Work. Tradeswomen Say It Could Have Happened to Any of Them.
Portside
portside.org
2025-11-28 21:13:30
Amber Czech Was Murdered at Work. Tradeswomen Say It Could Have Happened to Any of Them.
Maureen
Fri, 11/28/2025 - 16:13
...
Content warning: This story contains graphic depictions of violence.
Last Tuesday should have been an ordinary workday for 20-year-old Amber Czech, who worked as a welder at a manufacturing facility in Cokato, Minnesota.
She loved her craft and taught welding at her old high school on days she had off. But she was also a rarity. Women make up just 6 percent of welders in the country, and, as with other male-dominated occupations, it came with the risk of isolation and bullying.
It was a reality of the job that for Czech turned deadly. While she was at her work station that day, a man she worked with walked over to her, picked up a sledgehammer and bludgeoned her to death. He later told law enforcement he simply did not like her and had been planning to murder her for some time.
As the news spread across social media, stories poured in on Instagram and TikTok from tradeswomen expressing their anger and frustration. Over the last week they have shared their own experiences of working with men who threatened them, made sexual advances or joked about their incompetence. They pondered how their story could have ended differently, how their life could have ended like Czech’s.
Angie Cacace, a carpenter from North Carolina, was one of those women who was deeply saddened by Czech’s death. “It hit me pretty hard. I probably have cried about it every day,” she said. “I’ve had altercations with men on job sites so I know that exists.”
Despite the issues she’s faced in the industry, Cacace loves the work and credits carpentry with helping her buy her first home. Now she operates a successful remodeling company.
In 2019, she co-launched a media company and, more recently, a magazine called
Move Over Bob
, aimed at showing the possibilities of tradeswork to girls in high school and community college. Only about 4 percent of tradesworkers are women, but if they can find their way into the field, the jobs, which do not require a degree, offer middle-class wages, pensions and other benefits that are often not available in women-dominated industries like caregiving or the service industry.
But Czech’s death has Cacace questioning the workplace she’s urging girls to enter.
“She did everything that we want these girls to do. She did a high school welding program. She went to a community college,” she said. “It’s just a sharp reminder that there’s just a lot of work to do and figuring out how to better advocate and make sure that these young women are safe.”
Czech’s death has opened up a wound that never really closed for tradeswomen. In 2017, 32-year-old Outi Hicks, an apprentice carpenter, was murdered at her workplace by a coworker in Fresno, California. Her death sparked similar outrage and helped spur programs like the “
Be That One Guy
” campaign, which includes training for bystander intervention in the workplace.
“But there’s a whole host of incidents that have occurred between the Outi Hicks horror and what just transpired,” said Rita Brown, president of the National Association of Women in Construction (NAWIC). “Change has been slow, and clearly it’s not enough.”
Instagram posts from tradeswomen across the country share reactions to the killing of Amber Czech, honoring her and calling attention to safety issues in the trades. Editor’s note: Comments have been blacked out to protect commenters’ privacy.
Instagram posts from tradeswomen across the country share reactions to the killing of Amber Czech, honoring her and calling attention to safety issues in the trades. Editor’s note: Comments have been blacked out to protect commenters’ privacy.
(INSTAGRAM)
In 2023, the Equal Employment Opportunity Commission (EEOC) published a report
detailing the harassment women
and people of color continue to face in the construction trades. It included accounts of sexual harassment like groping and lewd comments, as well as racist incidents. In 2021, a survey conducted by the Institute for Women’s Policy Research found that more than a quarter of women tradesworkers who responded said they were “always or frequently harassed” for being a woman.
While there are several programs developed by women and trades organizations to promote a safer and more inclusive workplace, there is no real federal policy or legislation that provides adequate protections, advocates say. Instead the work has happened at the grassroots level.
Now organizations like NAWIC are calling for construction companies, unions and the industry as a whole to enforce stricter workplace standards. “It is incumbent on all of us that are in leadership, that are in any kind of place of power in the ecosystem of the construction industry, to enforce this zero-tolerance policy,” Brown said.
The Tradeswomen Taskforce, an advocacy group, plans to draft a resolution, which they want these same organizations to adopt, that promotes a safe workplace for women.
Amy Roosa, who works in risk control for the construction industry, said that workplace violence should be treated as a safety hazard. “It’s a ticking time bomb at the end of the day that this is going to happen to a woman,” she said. “It’s really up to the leadership team and even safety professionals to treat this like a workplace hazard.
“We need to have an intervention process. We need to have clear reporting. A woman needs to know that when she comes up and says, ‘I have a concern’ or, ‘He’s threatening me,’ that it’s taken seriously,” she added.
At the same time that tradeswomen are urging more to be done to address gender-based violence in the workplace, the Trump administration has curtailed resources.
Last fall, the Tradeswomen Taskforce and Equal Rights Advocates, a nonprofit focused on gender justice in workplaces, won a $350,000 grant to address gender-based violence in the workplace. The Trump administration abruptly canceled the program, alongside four other grants that aimed to eliminate violence and harassment in the workplace.
Part of the work would have involved helping women navigate how to file complaints or lawsuits, something that is difficult to do on a worksite because they either don’t know the proper channel to elevate a complaint or could face retaliation, said Janelle Dejan, co-chair of the task force. “Things that we do in these advocacy organizations is try to create a safer space,” she said. This includes a place where people can report anonymously or without being ridiculed for having a concern, she said.
Other safeguards that were in place to help women report abuse are no longer functioning, according to Connie Ashbrook, also with the task force. “The EEOC has been squelched,” she said.
Up until October, it didn’t have enough commissioners to issue decisions because
the Trump administration
fired two of the Democratic commissioners
and the general counsel. One of those commissioners, Charlotte Burrows, wrote the report on women’s harassment in the trades. That report has also disappeared from the EEOC’s website.
“It just feels like, at the same time, that we’re really digging deep to work on this problem, the government is taking tools away from us,” Ashbrook said.
In the days since Czech’s death, tradeswomen from across the country have rallied in support, donating thousands of dollars to her family’s GoFundMe and wearing blue to work in her honor.
In an industry that has not figured out how to take care of them, they are taking care of each other.
MetaFun: Compile Haskell-like code to C++ template metaprograms
MetaFun is a program to compile functional programs into
C++
template metaprograms
. It allows you to write programs in
the very
Haskell
-like
language Kiff (for
Keep It Fun & Functional
),
and then use them as compile-time C++ metaprograms.
You can download the latest version of MetaFun from
GitHub
.
Kiff: The input language
MetaFun's input language, Kiff, is a simplified version of Haskell. It supports the following features of Haskell:
Algebraic data types (
data
)
(!) Type synonyms (
type
)
Declared types for definitions
Parametric polymorphism (but
not
typeclasses)
The following builtin keywords:
let ... in ...
(!)
λ ... -> ...
if ... then ... else ...
Special
[x, y]
syntax for lists
(note that features marked with a (!) are not actually supported
by the MetaFun compiler yet.)
Kiff also includes the following primitives, mapped to the native
C++ operators and types:
The types
Bool
and
Int
The bool literals
True
and
False
Numeric literals
Bool operators
&&
,
||
and
!
Int comparison:
==
,
!=
,
<
,
<=
,
>=
,
>
Int operators:
+
,
-
,
*
,
/
,
%
To give you an idea of what Kiff code looks like, here's a
possible definition of
sum
:
foldl :: (a -> b -> a) -> a -> [b] -> a
foldl f x [] = x
foldl f x (y:ys) = foldl f (f x y) ys
add x y = x + y --
The compiler
The C++ code generated by the compiler consists of a bunch of
template structs, one for each definition. Constructors are
represented as structs with template parameters for constructor
parameters. Bools and Ints are boxed.
For reference, here's MetaFun's C++ output for the above code of sum:
template< int > struct Int;
template< bool > struct Bool;
struct nil;
template< class, class > struct cons;
template< template< class, class > class, class, class > struct foldl;
template< class, class > struct add;
template< class > struct sum;
template< int x >
struct Int
{
static const int v = x;
};
template< bool b >
struct Bool
{
static const bool v = b;
};
template< template< class, class > class f, class x >
struct foldl< f, x, nil >
{
typedef x v;
};
template< template< class, class > class f, class x, class y, class ys >
struct foldl< f, x, cons< y, ys > >
{
typedef typename foldl< f, typename f< x, y >::v, ys >::v v;
};
template< class x, class y >
struct add
{
typedef Int< (x::v) + (y::v) > v;
};
template< class xs >
struct sum
{
typedef typename foldl< add, Int< 0 >, xs >::v v;
};
Caveats
MetaFun is the result of me spending a week in bed because of a
knee injury, and toying around with the concept of an
easy-to-use language to write compile-time programs. The fact
that it was thrown together in a couple of days shows: MetaFun
is but a proof-of-concept demo. It's barely finished enough to
compile the 9-digit solution program below. Here's a list of
some notable known problems:
No currying in function calls
No support for lambda expressions
The type checker doesn't actually support type synonyms
Zero optimization: Local definitions are lifted even when they
could be local nested structs in C++
A detailed example
The following program solves the
9-digit
problem
using a simple backtracking algorithm. In fact, it
is the result of a direct, manual backwards translation of
Károly
Lőrentey's solution
, and was the motivating example behind
writing MetaFun.
length [] = 0
length (first:rest) = 1 + (length rest)
value :: [Int] -> Int
value [] = 0
value (first:rest) = 10 * (value rest) + first
contains :: Int -> [Int] -> Bool
contains elem [] = False
contains elem (first:rest) = elem == first || (contains elem rest)
divisor_test :: [Int] -> Bool
divisor_test (first:rest) = let num = value (first:rest)
div = (length rest) + 1
mod = num % div
in mod == 0
test_candidate :: [Int] -> Bool
test_candidate (first:rest) = divisor_test (first:rest) && !(contains first rest)
search_i :: Int -> Bool -> Bool -> [Int] -> Int
search_i len True True (digit:rest) = value (digit:rest)
search_i len True False (digit:rest) = search len (1:digit:rest)
search_i len good final (digit:rest) = search len ((digit+1):rest)
search :: Int -> [Int] -> Int
search len [] = search len [1]
search len [10] = -1
search len (10:next:rest) = search len ((next+1):rest)
search len (digit:rest) = let good = test_candidate (digit:rest)
final = good && 1 + (length rest) == len
in search_i good final
run = search 9 []
Ask HN: What is the purpose of all these AI spam comments?
I have showdead enabled and recently noticed a massive influx of dead comments from users with names like "Jeff_Davis" and "Richard_Smith" that are just short, AI generated summaries of the link itself.
They don't appear to be karma whoring as (most of the time) they don't even seem to be expressing a positive opinion of the link. Just short, useless summaries that are often just a rephrasing of the information given in the post title.
What do you think the purpose of this is?
May be the ones that are dead are from poor imitation of more successful bots that are operating in our midst whose comments aren't dead?
if you are selling user accounts to a spammer, they want to buy accounts that have been "aged" - that is, they have some comment history and weren't created on the same day they start spamming, so they're a bit less obvious.
but if that's what they're doing here, they aren't very good at it.
There were some big opsec fails on reddit a decade ago in the runup to the 2016 election where lots of propaganda accounts were linked together via similarly generated usernames. The big guys already learned basic lessons like that all those years ago and don't make those simple mistakes anymore.
If you were artificial, wouldn't you be trying to normalize your participation about now?
I imagine they would feel a lot better when they aren't the odd man out, and even more so to make up the vast majority.
As things continue to escalate.
Friday Squid Blogging: Flying Neon Squid Found on Israeli Beach
Schneier
www.schneier.com
2025-11-28 20:56:20
A meter-long flying neon squid (Ommastrephes bartramii) was found dead on an Israeli beach. The species is rare in the Mediterranean....
A320-neo planes affected by instense solar radiation. Worldwide fleet “precautionary action” required by an Emergency Airworthiness Directive from the European Union Aviation Safety Agency.
I am a
public-interest technologist
, working at the intersection of security, technology, and people. I've been writing about security issues on my
blog
since 2004, and in my monthly
newsletter
since 1998. I'm a fellow and lecturer at Harvard's
Kennedy School
, a board member of
EFF
, and the Chief of Security Architecture at
Inrupt, Inc.
This personal website expresses the opinions of none of those organizations.
One of the servers to which I SSH ratcheted up its public key
requirements and thus the Monkeysphere key I've been using for 15
years stopped working.
Unfortunately, monkeysphere gen-subkey hardcodes RSA keys and
if I'm going to be forced to use a new subkey I want mine to
be of the 25519 variety. ...
One of the servers to which I SSH ratcheted up its public key
requirements and thus the Monkeysphere key I've been using for 15
years stopped working.
Unfortunately,
monkeysphere gen-subkey
hardcodes RSA keys and
if I'm going to be forced to use a new subkey I want mine to
be of the 25519 variety. Therefore, to add a subkey by hand:
gpg --expert --edit-key $KEYID
Follow roughly what's in
/usr/share/monkeysphere/m/gen_subkey
,
but change the key type to 11 (
ECC (set your own capabilities)
),
don't bother with Encrypt capability, and pick
Curve25519
.
monkeysphere subkey-to-ssh-agent
and agent-transfer will be all
happy with the "ed25519" subkey without any code modifications,
and you won't need to rewrite monkeysphere from scratch to use
Sequoia for the next 15 years.
The
Dolt Workbench
is an open-source SQL workbench supporting MySQL, Postgres,
Dolt
, and
Doltgres
databases. We built the workbench using
Electron
, which is a popular framework that allows you to convert web apps built with traditional web technologies like HTML, CSS, and Javascript to desktop applications. Since the workbench shares much in common with
DoltHub
and
Hosted Dolt
, the architecture is very similar to those products. That is, the workbench uses
Next.js
for the frontend with an additional GraphQL layer that handles database interactions. For this reason, it made a lot of sense to use Electron to get the desktop version of our application up and running.
That said, Electron comes with a few rather significant drawbacks, and those drawbacks have started to become more apparent as the workbench has matured. Because of this, I spent some time exploring
Tauri
, a newer framework that supports the same web-to-desktop use case as Electron. In this article, we’ll discuss how well Electron and Tauri integrate with the workbench, and weigh some pros and cons between the two frameworks.
Next.js doesn’t translate very cleanly to a desktop application context. This is primarily due to the framework’s architecture around server-side rendering and API routing features. In a desktop app, there’s no application server interacting with a client; we just need to render HTML, CSS, and JavaScript in a window. For these reasons, Electron only loosely supports Next.js applications. That’s not to say you can’t build an Electron app with Next.js, but it requires some workarounds to make it function properly. One of the more popular workarounds is a project called
Nextron
, which aims to wire Next.js applications to the Electron framework and streamline the build process. This is the project we use for the workbench. The issue is that, at the time of writing, it appears that Nextron is no longer being maintained, and we started hitting a few bugs with it.
Tauri is largely frontend-framework agnostic. For Next, specifically, you still can’t use the server-side features, but Tauri makes the integration process much simpler by relying on Next’s static-site generation capabilities. To make a Next app work with Tauri, you just need to set
output: 'export'
in your Next configuration file, and Tauri handles the rest.
The biggest difference between Electron and Tauri comes from how they render the UI. The Electron framework comes with a full Chromium browser engine bundled in your application, which is the same engine that backs Google Chrome. This is useful because it means you don’t have to worry about browser compatibility issues. Regardless of the end user’s machine or architecture, the same Chromium instance renders your application UI. This results in a very standardized experience that ensures your app will look the same regardless of where it’s running. However, this also results in a fair amount of bloat. For the vast majority of desktop apps, a full Chromium browser engine is overkill. Even the simplest “Hello World” applications using Electron can run you up to 150 megabytes of disk space.
Tauri solves this problem by leveraging the system’s native webview. Instead of bundling a full browser engine, Tauri uses a library called
WRY
, which provides a cross-platform interface to the appropriate webview for the operating system. As you’d expect, this makes Tauri apps far more lightweight. The downside here is that you no longer have a hard guarantee on compatibility. From what I can tell, however, this mostly seems to be a non-issue. Compatibility issues across system webviews are exceedingly rare, especially for the major operating systems.
Another major difference between the two frameworks is how they handle the “main” process. This refers to the backend process that orchestrates the application windows, menus, and other components of a desktop app that require interaction with system APIs. In Electron, the main process runs in a Node.js environment. This means you get access to all the typical Node APIs, you can import things like normal, and, perhaps most importantly, you can write your Electron-specific code in pure JavaScript. This is a huge bonus for Electron’s target audience: web developers.
Tauri, on the other hand, uses Rust. All the framework code and the main process entrypoint are written in Rust. Obviously, this makes it a bit less accessible to the average web developer. That said, Tauri provides a fairly robust set of JavaScript APIs to interact with the Rust layer. For most applications, these APIs will be sufficient to do what you need to do. In the case of the workbench, I was able to fully replicate the functionality of the Electron version using the JavaScript APIs and some minimal Rust code.
In my experience, I found the Tauri APIs to fit more naturally in our application code. With Electron, if you need the main process to do something, you must always use inter-process communication, even for the simplest of tasks. If you want to write to a file on the host machine, for instance, your frontend needs to send a signal to the Electron main process, which will then spawn a new process and run the function you wrote that performs the write. With Tauri, you can just use Tauri’s filesystem API directly in your application code. Under the hood, the same sort of IPC pattern is happening, but I think the Tauri abstraction is a bit nicer.
Since Electron runs on Node.js, it also bundles a full Node.js runtime with your application. This comes with some pros and cons. For the workbench, specifically, this is beneficial because the GraphQL layer is itself a separate Node.js application that needs to run alongside the frontend. Since Electron ships with Node.js, this means we can directly spin up the GraphQL server from the Electron main process using the Node runtime. This eliminates a lot of the headache associated with bundling and running a typical sidecar process. For instance, our app also ships with a copy of Dolt, which allows users to start up local Dolt servers directly from the workbench. To make this work, we have to bundle the appropriate Dolt binary with each workbench release that corresponds to the correct architecture. Without the Node runtime, we’d have to do something similar for the GraphQL layer.
With Tauri, this is exactly the problem we run into. To get around it, we need to compile the GraphQL server into a binary using a tool like
pkg
, then run it as a sidecar the same way we run Dolt. Thankfully, this seems to be a fairly common use case for Tauri applications, and they have a useful guide on
how to run Node.js apps as a sidecar
.
It’s also worth mentioning that the full Node.js runtime is quite heavy, which also contributes to bloated Electron app sizes. After building the workbench using both Electron and Tauri, the difference in size was substantial. The left is the Electron version and the right is Tauri:
After replicating the workbench’s functionality in Tauri, we’re holding off on making the full transition for a couple reasons:
Lack of support for .appx and .msix bundles on Windows
- Currently, Tauri only support .exe and .msi bundles on Windows. This means your Microsoft Store entry will only link to the unpacked application. The workbench is currently bundled and published using the .appx format. To address this, we would need to take down the workbench entirely from the Microsoft store and create a new application that uses the .exe format.
Issues with MacOS universal binaries
- This is more an annoyance than a bug, but I ran into a few issues related to codesigning universal binaries for MacOS. Namely, Tauri doesn’t seem to be able to create Mac universal binaries from their arm64 and x64 subcomponents. It also seems to be codesigning the Mac builds twice.
Neither of these are hard blockers, but they’re annoying enough that I’m holding off on migrating until they’re resolved or our issues with Nextron become more problematic. For now, I’m leaving
my branch with the migration
open and hope to revisit soon. If you’re on the Tauri team, let us know if you have solutions!
Overall, I’m impressed with Tauri. It eliminates much of the classic Electron bloat and integrates naturally with our existing codebase. If you’re curious about Tauri or the Dolt Workbench, let us know on
Discord
.
Every couple of years
somebody
notices
that large tech companies sometimes produce surprisingly sloppy code. If you haven’t worked at a big company, it might be hard to understand how this happens. Big tech companies pay well enough to attract many competent engineers. They move slowly enough that it looks like they’re able to take their time and do solid work. How does bad code happen?
Most code changes are made by relative beginners
I think the main reason is that
big companies are full of engineers working outside their area of expertise
. The average big tech employee stays for only
a year or two
1
. In fact, big tech compensation packages are typically designed to put a four-year cap on engineer tenure: after four years, the initial share grant is fully vested, causing engineers to take what can be a 50% pay cut. Companies do extend temporary yearly refreshes, but it obviously incentivizes engineers to go find another job where they don’t have to wonder if they’re going to get the other half of their compensation each year.
If you count internal mobility, it’s even worse. The longest I have ever stayed on a single team or codebase was three years, near the start of my career. I expect to be
re-orged
at least every year, and often much more frequently.
However, the average tenure of a codebase in a big tech company is a lot longer than that. Many of the services I work on are a decade old or more, and have had many, many different owners over the years. That means many big tech engineers are constantly “figuring it out”.
A pretty high percentage of code changes are made by “beginners”:
people who have onboarded to the company, the codebase, or even the programming language in the past six months.
Old hands
To some extent, this problem is mitigated by “old hands”: engineers who happen to have been in the orbit of a particular system for long enough to develop real expertise. These engineers can give deep code reviews and reliably catch obvious problems. But relying on “old hands” has two problems.
First,
this process is entirely informal
. Big tech companies make surprisingly little effort to develop long-term expertise in individual systems, and once they’ve got it they seem to barely care at all about retaining it. Often the engineers in question are moved to different services, and have to either keep up their “old hand” duties on an effectively volunteer basis, or abandon them and become a relative beginner on a brand new system.
Second,
experienced engineers are always overloaded
. It is a
busy
job being one of the few engineers who has deep expertise on a particular service. You don’t have enough time to personally review every software change, or to be actively involved in every decision-making process. Remember that
you also have your own work to do
: if you spend all your time reviewing changes and being involved in discussions, you’ll likely be punished by the company for not having enough individual output.
The median productive engineer
Putting all this together, what does the median productive
2
engineer at a big tech company look like? They are usually:
competent enough to pass the hiring bar and be able to do the work, but either
working on a codebase or language that is largely new to them, or
trying to stay on top of a flood of code changes while also juggling their own work.
They are almost certainly working to a deadline, or to a series of overlapping deadlines for different projects. In other words,
they are trying to do their best in an environment that is not set up to produce quality code.
That’s how “obviously” bad code happens. For instance, a junior engineer picks up a ticket for an annoying bug in a codebase they’re barely familiar with. They spend a few days figuring it out and come up with a hacky solution. One of the more senior “old hands” (if they’re lucky) glances over it in a spare half-hour, vetoes it, and suggests something slightly better that would at least work. The junior engineer implements that as best they can, tests that it works, it gets briefly reviewed and shipped, and everyone involved immediately moves on to higher-priority work. Five years later somebody notices this
3
and thinks “wow, that’s hacky - how did such bad code get written at such a big software company”?
Big tech companies are fine with this
I have written a lot about the internal tech company dynamics that contribute to this. Most directly, in
Seeing like a software company
I argue that big tech companies consistently prioritize internal
legibility
- the ability to see at a glance who’s working on what and to change it at will - over productivity. Big companies know that treating engineers as fungible and moving them around destroys their ability to develop long-term expertise in a single codebase.
That’s a deliberate tradeoff.
They’re giving up some amount of expertise and software quality in order to gain the ability to rapidly deploy skilled engineers onto whatever the problem-of-the-month is.
I don’t know if this is a good idea or a bad idea. It certainly seems to be working for the big tech companies, particularly now that “how fast can you pivot to something AI-related” is so important. But if you’re doing this, then
of course
you’re going to produce some genuinely bad code. That’s what happens when you ask engineers to rush out work on systems they’re unfamiliar with.
Individual engineers are entirely powerless to alter this dynamic
. This is particularly true in 2025, when
the balance of power has tilted
away from engineers and towards tech company leadership. The most you can do as an individual engineer is to try and become an “old hand”: to develop expertise in at least one area, and to use it to block the worst changes and steer people towards at least minimally-sensible technical decisions. But even that is often swimming against the current of the organization, and if inexpertly done can cause you to get
PIP-ed
or worse.
Pure and impure engineering
I think a lot of this comes down to the distinction between
pure and impure software engineering
. To pure engineers - engineers working on self-contained technical projects, like
a programming language
- the only explanation for bad code is incompetence. But impure engineers operate more like plumbers or electricians. They’re working to deadlines on projects that are relatively new to them, and even if their technical fundamentals are impeccable, there’s always
something
about the particular setup of this situation that’s awkward or surprising. To impure engineers, bad code is inevitable. As long as the overall system works well enough, the project is a success.
At big tech companies, engineers don’t get to decide if they’re working on pure or impure engineering work. It’s
not their codebase
! If the company wants to move you from working on database infrastructure to building the new payments system, they’re fully entitled to do that. The fact that you might make some mistakes in an unfamiliar system - or that your old colleagues on the database infra team might suffer without your expertise - is a deliberate tradeoff being made by
the company, not the engineer
.
It’s fine to point out examples of bad code at big companies. If nothing else, it can be an effective way to get those specific examples fixed, since execs usually jump at the chance to turn bad PR into good PR. But I think it’s a mistake
4
to attribute primary responsibility to the engineers at those companies. If you could wave a magic wand and make every engineer twice as strong,
you would still have bad code
, because almost nobody can come into a brand new codebase and quickly make changes with zero mistakes. The root cause is that
most big company engineers are forced to do most of their work in unfamiliar codebases
.
If you liked this post, consider
subscribing
to email updates about my new posts, or
sharing it on Hacker News
.
Here's a preview of a related post that shares tags with this one.
How I influence tech company politics as a staff software engineer
Many software engineers are fatalistic about company politics. They believe that it’s pointless to get involved, because:
The general idea here is that
software engineers are simply not equipped to play the game at the same level as real political operators
. This is true! It would be a terrible mistake for a software engineer to think that you ought to start scheming and plotting like you’re in
Game of Thrones
. Your schemes will be immediately uncovered and repurposed to your disadvantage and other people’s gain. Scheming takes practice and power, and neither of those things are available to software engineers.
Continue reading...
The original ABC language, Python's predecessor (1991)
The current sources assume a 32-bit system where int and pointers have
the same size. I hope to eventually upgrade the source code to work on
64-bit systems too (where int is 32 bits and pointers are 64 bits).
Licence
Most files have 1991 as their latest modification time in the tar ball;
a few have 1996 or 2021.
I'll try to negotiate with Steven Pemberton eventually (hopefully MIT).
Authors and references
The man page lists the following people as authors:
Eddy Boeve, Frank van Dijk, Leo Geurts, Timo Krijnen, Lambert Meertens,
Steven Pemberton, Guido van Rossum.
It also lists these references:
Leo Geurts, Lambert Meertens and Steven Pemberton, The ABC Programmer's
Handbook, Prentice-Hall, Englewood Cliffs, New Jersey, 1990, ISBN 0-13-
000027-2.
Steven Pemberton, An Alternative Simple Language and Environment for PCs,
IEEE Software, Vol. 4, No. 1, January 1987, pp. 56-64.
http://www,cwi.nl/~steven/abc.html
How a monopoly ISP refuses to fix upstream infrastructure – The Sacramento Bear
A documented case of infrastructure failure, failed escalation, and a company that refuses to investigate.
Here’s the situation: I have outages. My neighbor has the same outages. Xfinity won’t fix it.
I bought Xfinity internet in June 2024. Immediately, my connection started dropping. Multiple times a day. Every single day. After troubleshooting every piece of equipment I had and questioning my sanity my neighbor complained about the same thing which led me to understand this was not my equipment.
I set up an uptime monitor and found that these outages happen 6-7 times per day for 125 seconds.
Over 17 months of my service term that’s approximately 3,387 outage incidents totaling 117+ hours of cumulative downtime.
This outage pattern has recurred thousands of times. It is consistent, predictable, and it follows an automated schedule.
My neighbor has the same problem. Different house. Different line from a different junction box. Same 125-second outages happening at the same times of day.
PING Uptime Log
2025-11-21T14:42:31-08:00 Warning dpinger exiting on signal 15
2025-11-21T07:01:34-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> none RTT: 21.4 ms RTTd: 14.8 ms Loss: 10.0 %)
2025-11-21T07:01:28-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: down -> loss RTT: 21.7 ms RTTd: 15.8 ms Loss: 20.0 %)
2025-11-21T06:59:33-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> down RTT: 19.1 ms RTTd: 1.7 ms Loss: 21.0 %)
2025-11-21T06:59:28-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: none -> loss RTT: 19.1 ms RTTd: 1.6 ms Loss: 12.0 %)
2025-11-21T00:01:33-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> none RTT: 20.5 ms RTTd: 2.7 ms Loss: 10.0 %)
2025-11-21T00:01:28-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: down -> loss RTT: 20.6 ms RTTd: 2.8 ms Loss: 20.0 %)
2025-11-20T23:59:35-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> down RTT: 20.1 ms RTTd: 3.1 ms Loss: 21.0 %)
2025-11-20T23:59:30-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: none -> loss RTT: 20.0 ms RTTd: 3.0 ms Loss: 12.0 %)
2025-11-20T15:01:37-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> none RTT: 21.3 ms RTTd: 3.9 ms Loss: 10.0 %)
2025-11-20T15:01:31-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: down -> loss RTT: 21.4 ms RTTd: 4.1 ms Loss: 20.0 %)
2025-11-20T14:59:36-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> down RTT: 21.0 ms RTTd: 3.9 ms Loss: 21.0 %)
2025-11-20T14:59:31-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: none -> loss RTT: 21.3 ms RTTd: 4.3 ms Loss: 12.0 %)
2025-11-20T13:46:40-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> none RTT: 20.4 ms RTTd: 2.9 ms Loss: 10.0 %)
2025-11-20T13:46:34-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: down -> loss RTT: 20.2 ms RTTd: 3.0 ms Loss: 20.0 %)
2025-11-20T13:44:38-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> down RTT: 19.9 ms RTTd: 1.7 ms Loss: 21.0 %)
2025-11-20T13:44:33-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: none -> loss RTT: 19.7 ms RTTd: 1.7 ms Loss: 12.0 %)
2025-11-20T12:46:37-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> none RTT: 19.3 ms RTTd: 1.2 ms Loss: 10.0 %)
2025-11-20T12:46:32-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: down -> loss RTT: 19.2 ms RTTd: 1.2 ms Loss: 20.0 %)
2025-11-20T12:44:39-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> down RTT: 19.2 ms RTTd: 0.9 ms Loss: 21.0 %)
2025-11-20T12:44:34-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: none -> loss RTT: 19.3 ms RTTd: 1.0 ms Loss: 12.0 %)
2025-11-20T05:16:33-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> none RTT: 19.5 ms RTTd: 1.4 ms Loss: 10.0 %)
2025-11-20T05:16:27-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: down -> loss RTT: 19.5 ms RTTd: 1.4 ms Loss: 20.0 %)
2025-11-20T05:14:33-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> down RTT: 19.4 ms RTTd: 1.2 ms Loss: 21.0 %)
2025-11-20T05:14:28-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: none -> loss RTT: 19.3 ms RTTd: 1.2 ms Loss: 12.0 %)
2025-11-20T03:31:34-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> none RTT: 18.9 ms RTTd: 1.3 ms Loss: 10.0 %)
2025-11-20T03:31:28-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: down -> loss RTT: 18.8 ms RTTd: 1.1 ms Loss: 20.0 %)
2025-11-20T03:29:32-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> down RTT: 19.3 ms RTTd: 1.6 ms Loss: 21.0 %)
2025-11-20T03:29:28-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: none -> loss RTT: 19.4 ms RTTd: 1.6 ms Loss: 14.0 %)
2025-11-19T20:01:32-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> none RTT: 18.9 ms RTTd: 1.2 ms Loss: 10.0 %)
2025-11-19T20:01:26-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: down -> loss RTT: 18.8 ms RTTd: 1.2 ms Loss: 20.0 %)
2025-11-19T19:59:34-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> down RTT: 19.2 ms RTTd: 2.2 ms Loss: 21.0 %)
2025-11-19T19:59:29-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: none -> loss RTT: 19.1 ms RTTd: 2.1 ms Loss: 12.0 %)
2025-11-19T16:31:36-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> none RTT: 20.2 ms RTTd: 2.7 ms Loss: 10.0 %)
2025-11-19T16:31:29-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: down -> loss RTT: 20.2 ms RTTd: 2.8 ms Loss: 20.0 %)
2025-11-19T16:29:35-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> down RTT: 19.6 ms RTTd: 2.1 ms Loss: 21.0 %)
2025-11-19T16:29:30-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: none -> loss RTT: 19.8 ms RTTd: 2.3 ms Loss: 12.0 %)
2025-11-19T13:31:35-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> none RTT: 17.7 ms RTTd: 2.5 ms Loss: 10.0 %)
2025-11-19T13:31:29-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: down -> loss RTT: 17.9 ms RTTd: 2.6 ms Loss: 20.0 %)
2025-11-19T13:29:38-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> down RTT: 16.7 ms RTTd: 2.0 ms Loss: 21.0 %)
2025-11-19T13:29:32-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: none -> loss RTT: 16.6 ms RTTd: 2.0 ms Loss: 12.0 %)
2025-11-19T10:31:39-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> none RTT: 19.6 ms RTTd: 1.4 ms Loss: 10.0 %)
2025-11-19T10:31:33-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: down -> loss RTT: 19.5 ms RTTd: 1.3 ms Loss: 20.0 %)
2025-11-19T10:29:39-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> down RTT: 18.9 ms RTTd: 1.2 ms Loss: 21.0 %)
2025-11-19T10:29:33-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: none -> loss RTT: 19.1 ms RTTd: 1.3 ms Loss: 12.0 %)
2025-11-19T03:46:32-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> none RTT: 19.6 ms RTTd: 2.6 ms Loss: 10.0 %)
2025-11-19T03:46:25-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: down -> loss RTT: 19.6 ms RTTd: 2.7 ms Loss: 20.0 %)
2025-11-19T03:44:33-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> down RTT: 19.4 ms RTTd: 1.2 ms Loss: 21.0 %)
2025-11-19T03:44:28-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: none -> loss RTT: 19.4 ms RTTd: 1.2 ms Loss: 12.0 %)
2025-11-18T22:31:36-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> none RTT: 19.9 ms RTTd: 2.6 ms Loss: 10.0 %)
2025-11-18T22:31:29-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: down -> loss RTT: 19.8 ms RTTd: 2.5 ms Loss: 20.0 %)
2025-11-18T22:29:34-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> down RTT: 20.1 ms RTTd: 2.5 ms Loss: 21.0 %)
2025-11-18T22:29:29-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: none -> loss RTT: 20.0 ms RTTd: 2.4 ms Loss: 12.0 %)
2025-11-18T12:19:41-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> none RTT: 21.1 ms RTTd: 4.2 ms Loss: 10.0 %)
2025-11-18T12:19:35-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: down -> loss RTT: 21.1 ms RTTd: 4.3 ms Loss: 20.0 %)
2025-11-18T12:17:42-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> down RTT: 19.5 ms RTTd: 1.7 ms Loss: 21.0 %)
2025-11-18T12:17:37-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: none -> loss RTT: 19.8 ms RTTd: 1.8 ms Loss: 12.0 %)
2025-11-18T12:16:37-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> none RTT: 19.3 ms RTTd: 1.6 ms Loss: 10.0 %)
2025-11-18T12:16:31-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: down -> loss RTT: 19.2 ms RTTd: 1.7 ms Loss: 20.0 %)
2025-11-18T12:14:38-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> down RTT: 19.1 ms RTTd: 1.2 ms Loss: 21.0 %)
2025-11-18T12:14:32-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: none -> loss RTT: 19.1 ms RTTd: 1.1 ms Loss: 12.0 %)
2025-11-18T09:31:37-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> none RTT: 19.4 ms RTTd: 1.0 ms Loss: 10.0 %)
2025-11-18T09:31:32-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: down -> loss RTT: 19.4 ms RTTd: 1.0 ms Loss: 20.0 %)
2025-11-18T09:29:38-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> down RTT: 19.6 ms RTTd: 1.3 ms Loss: 21.0 %)
2025-11-18T09:29:33-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: none -> loss RTT: 19.6 ms RTTd: 1.2 ms Loss: 12.0 %)
2025-11-18T07:46:33-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> none RTT: 21.1 ms RTTd: 15.0 ms Loss: 10.0 %)
2025-11-18T07:46:26-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: down -> loss RTT: 21.4 ms RTTd: 15.9 ms Loss: 20.0 %)
2025-11-18T07:44:32-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> down RTT: 19.7 ms RTTd: 2.1 ms Loss: 21.0 %)
2025-11-18T07:44:27-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: none -> loss RTT: 19.6 ms RTTd: 2.0 ms Loss: 12.0 %)
2025-11-18T02:46:34-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> none RTT: 19.0 ms RTTd: 1.5 ms Loss: 10.0 %)
2025-11-18T02:46:28-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: down -> loss RTT: 19.1 ms RTTd: 1.6 ms Loss: 20.0 %)
2025-11-18T02:44:33-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> down RTT: 19.0 ms RTTd: 1.0 ms Loss: 21.0 %)
2025-11-18T02:44:28-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: none -> loss RTT: 19.0 ms RTTd: 1.0 ms Loss: 12.0 %)
2025-11-17T23:01:33-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> none RTT: 18.6 ms RTTd: 0.9 ms Loss: 10.0 %)
2025-11-17T23:01:28-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: down -> loss RTT: 18.7 ms RTTd: 0.9 ms Loss: 20.0 %)
2025-11-17T22:59:35-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> down RTT: 18.8 ms RTTd: 1.0 ms Loss: 21.0 %)
2025-11-17T22:59:29-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: none -> loss RTT: 18.9 ms RTTd: 1.0 ms Loss: 12.0 %)
2025-11-17T17:01:35-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> none RTT: 20.5 ms RTTd: 3.3 ms Loss: 10.0 %)
2025-11-17T17:01:29-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: down -> loss RTT: 20.4 ms RTTd: 3.0 ms Loss: 20.0 %)
2025-11-17T16:59:36-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> down RTT: 19.4 ms RTTd: 1.5 ms Loss: 21.0 %)
2025-11-17T16:59:31-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: none -> loss RTT: 19.4 ms RTTd: 1.6 ms Loss: 12.0 %)
2025-11-17T13:31:36-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> none RTT: 20.1 ms RTTd: 3.3 ms Loss: 10.0 %)
2025-11-17T13:31:30-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: down -> loss RTT: 20.2 ms RTTd: 3.4 ms Loss: 20.0 %)
2025-11-17T13:29:37-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> down RTT: 19.3 ms RTTd: 1.1 ms Loss: 21.0 %)
2025-11-17T13:29:32-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: none -> loss RTT: 19.5 ms RTTd: 1.5 ms Loss: 12.0 %)
2025-11-17T12:46:37-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> none RTT: 19.0 ms RTTd: 3.4 ms Loss: 10.0 %)
2025-11-17T12:46:31-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: down -> loss RTT: 19.3 ms RTTd: 3.5 ms Loss: 20.0 %)
2025-11-17T12:44:40-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> down RTT: 17.8 ms RTTd: 3.1 ms Loss: 22.0 %)
2025-11-17T12:44:34-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: none -> loss RTT: 17.8 ms RTTd: 3.0 ms Loss: 12.0 %)
2025-11-17T02:46:33-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> none RTT: 19.8 ms RTTd: 2.6 ms Loss: 10.0 %)
2025-11-17T02:46:27-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: down -> loss RTT: 19.8 ms RTTd: 2.7 ms Loss: 20.0 %)
2025-11-17T02:44:32-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> down RTT: 19.0 ms RTTd: 0.9 ms Loss: 21.0 %)
2025-11-17T02:44:28-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: none -> loss RTT: 19.2 ms RTTd: 1.5 ms Loss: 12.0 %)
2025-11-17T01:15:08-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: down -> none RTT: 19.0 ms RTTd: 1.1 ms Loss: 0.0 %)
2025-11-17T01:14:57-08:00 Warning dpinger send_interval 1000ms loss_interval 4000ms time_period 60000ms report_interval 0ms data_len 1 alert_interval 1000ms latency_alarm 0ms loss_alarm 0% alarm_hold 10000ms dest_addr 8.8.8.8 bind_addr 24.2.60.72 identifier "WAN_GW "
Event Log from My Modem
23:00:33
Sun Nov 23 2025 Critical (3) UCD invalid or channel unusable;CM-MAC=00:40:36:86:90:ac;CMTS-MAC=00:90:f0:32:11:00;CM-QOS=1.1;CM-VER=3.1;
Time Not Established
Critical (3) SYNC Timing Synchronization failure - Failed to acquire QAM/QPSK symbol timing;;CM-MAC=00:40:36:86:90:ac;CMTS-MAC=00:00:00:00:00:00;CM-QOS=1.1;CM-VER=3.1;
Time Not Established
Critical (3) UCD invalid or channel unusable;CM-MAC=00:40:36:86:90:ac;CMTS-MAC=00:90:f0:32:11:00;CM-QOS=1.1;CM-VER=3.1;
05:15:36
Mon Nov 24 2025 Critical (3) UCD invalid or channel unusable;CM-MAC=00:40:36:86:90:ac;CMTS-MAC=00:90:f0:32:11:00;CM-QOS=1.1;CM-VER=3.1;
Time Not Established
Critical (3) SYNC Timing Synchronization failure - Failed to acquire QAM/QPSK symbol timing;;CM-MAC=00:40:36:86:90:ac;CMTS-MAC=00:00:00:00:00:00;CM-QOS=1.1;CM-VER=3.1;
Time Not Established
Critical (3) UCD invalid or channel unusable;CM-MAC=00:40:36:86:90:ac;CMTS-MAC=00:90:f0:32:11:00;CM-QOS=1.1;CM-VER=3.1;
06:45:43
Mon Nov 24 2025 Critical (3) UCD invalid or channel unusable;CM-MAC=00:40:36:86:90:ac;CMTS-MAC=00:90:f0:32:11:00;CM-QOS=1.1;CM-VER=3.1;
Time Not Established
Critical (3) SYNC Timing Synchronization failure - Failed to acquire QAM/QPSK symbol timing;;CM-MAC=00:40:36:86:90:ac;CMTS-MAC=00:00:00:00:00:00;CM-QOS=1.1;CM-VER=3.1;
Time Not Established
Critical (3) UCD invalid or channel unusable;CM-MAC=00:40:36:86:90:ac;CMTS-MAC=00:90:f0:32:11:00;CM-QOS=1.1;CM-VER=3.1;
08:30:40
Mon Nov 24 2025 Critical (3) UCD invalid or channel unusable;CM-MAC=00:40:36:86:90:ac;CMTS-MAC=00:90:f0:32:11:00;CM-QOS=1.1;CM-VER=3.1;
Time Not Established
Critical (3) SYNC Timing Synchronization failure - Failed to acquire QAM/QPSK symbol timing;;CM-MAC=00:40:36:86:90:ac;CMTS-MAC=00:00:00:00:00:00;CM-QOS=1.1;CM-VER=3.1;
Time Not Established
Critical (3) UCD invalid or channel unusable;CM-MAC=00:40:36:86:90:ac;CMTS-MAC=00:90:f0:32:11:00;CM-QOS=1.1;CM-VER=3.1;
09:45:37
Mon Nov 24 2025 Critical (3) UCD invalid or channel unusable;CM-MAC=00:40:36:86:90:ac;CMTS-MAC=00:90:f0:32:11:00;CM-QOS=1.1;CM-VER=3.1;
09:54:45
Mon Nov 24 2025 Critical (3) SYNC Timing Synchronization failure - Failed to acquire QAM/QPSK symbol timing;;CM-MAC=00:40:36:86:90:ac;CMTS-MAC=00:90:f0:32:11:00;CM-QOS=1.1;CM-VER=3.1;
09:54:51
Mon Nov 24 2025 Critical (3) SYNC Timing Synchronization failure - Failed to acquire QAM/QPSK symbol timing;;CM-MAC=00:40:36:86:90:ac;CMTS-MAC=00:90:f0:32:11:00;CM-QOS=1.1;CM-VER=3.1;
09:55:29
Mon Nov 24 2025 Critical (3) SYNC Timing Synchronization failure - Failed to acquire QAM/QPSK symbol timing;;CM-MAC=00:40:36:86:90:ac;CMTS-MAC=00:90:f0:32:11:00;CM-QOS=1.1;CM-VER=3.1;
09:55:29
Mon Nov 24 2025 Critical (3) Received Response to Broadcast Maintenance Request, But no Unicast Maintenance opportunities received - T4 time out;CM-MAC=00:40:36:86:90:ac;CMTS-MAC=00:90:f0:32:11:00;CM-QOS=1.1;CM-VER=3.1;
09:55:30
Mon Nov 24 2025 Critical (3) SYNC Timing Synchronization failure - Failed to acquire QAM/QPSK symbol timing;;CM-MAC=00:40:36:86:90:ac;CMTS-MAC=00:90:f0:32:11:00;CM-QOS=1.1;CM-VER=3.1;
09:55:30
Mon Nov 24 2025 Critical (3) Received Response to Broadcast Maintenance Request, But no Unicast Maintenance opportunities received - T4 time out;CM-MAC=00:40:36:86:90:ac;CMTS-MAC=00:90:f0:32:11:00;CM-QOS=1.1;CM-VER=3.1;
09:55:37
Mon Nov 24 2025 Critical (3) SYNC Timing Synchronization failure - Failed to acquire QAM/QPSK symbol timing;;CM-MAC=00:40:36:86:90:ac;CMTS-MAC=00:90:f0:32:11:00;CM-QOS=1.1;CM-VER=3.1;
09:55:37
Mon Nov 24 2025 Critical (3) Received Response to Broadcast Maintenance Request, But no Unicast Maintenance opportunities received - T4 time out;CM-MAC=00:40:36:86:90:ac;CMTS-MAC=00:90:f0:32:11:00;CM-QOS=1.1;CM-VER=3.1;
09:59:51
Mon Nov 24 2025 Critical (3) SYNC Timing Synchronization failure - Failed to acquire QAM/QPSK symbol timing;;CM-MAC=00:40:36:86:90:ac;CMTS-MAC=00:00:00:00:00:00;CM-QOS=1.1;CM-VER=3.1;
09:59:54
Mon Nov 24 2025 Critical (3) UCD invalid or channel unusable;CM-MAC=00:40:36:86:90:ac;CMTS-MAC=00:90:f0:32:11:00;CM-QOS=1.1;CM-VER=3.1;
Time Not Established
Critical (3) SYNC Timing Synchronization failure - Failed to acquire QAM/QPSK symbol timing;;CM-MAC=00:40:36:86:90:ac;CMTS-MAC=00:00:00:00:00:00;CM-QOS=1.1;CM-VER=3.1;
Time Not Established
Critical (3) UCD invalid or channel unusable;CM-MAC=00:40:36:86:90:ac;CMTS-MAC=00:90:f0:32:11:00;CM-QOS=1.1;CM-VER=3.1;
Time Not Established
Notice (6) Honoring MDD; IP provisioning mode = IPv6
10:08:01
Mon Nov 24 2025 Notice (6) DS profile assignment change. DS Chan ID: 32; Previous Profile: ; New Profile: 1 2 3.;CM-MAC=00:40:36:86:90:ac;CMTS-MAC=00:90:f0:32:11:00;CM-QOS=1.1;CM-VER=3.1;
10:08:03
Mon Nov 24 2025 Critical (3) UCD invalid or channel unusable;CM-MAC=00:40:36:86:90:ac;CMTS-MAC=00:90:f0:32:11:00;CM-QOS=1.1;CM-VER=3.1;
10:08:04
Mon Nov 24 2025 Critical (3) Started Unicast Maintenance Ranging - No Response received - T3 time-out;CM-MAC=00:40:36:86:90:ac;CMTS-MAC=00:90:f0:32:11:00;CM-QOS=1.1;CM-VER=3.1;
10:08:09
Mon Nov 24 2025 Critical (3) UCD invalid or channel unusable;CM-MAC=00:40:36:86:90:ac;CMTS-MAC=00:90:f0:32:11:00;CM-QOS=1.1;CM-VER=3.1;
10:52:13
Mon Nov 24 2025 Notice (6) CM-STATUS message sent. Event Type Code: 16; Chan ID: 193; DSID: N/A; MAC Addr: N/A; OFDM/OFDMA Profile ID: 3.;CM-MAC=00:40:36:86:90:ac;CMTS-MAC=00:90:f0:32:11:00;CM-QOS=1.1;CM-VER=3.1;
10:55:40
Mon Nov 24 2025 Notice (6) CM-STATUS message sent. Event Type Code: 24; Chan ID: 193; DSID: N/A; MAC Addr: N/A; OFDM/OFDMA Profile ID: 3.;CM-MAC=00:40:36:86:90:ac;CMTS-MAC=00:90:f0:32:11:00;CM-QOS=1.1;CM-VER=3.1;
Event Log from My Neighbor’s Modem
Date Time Event ID Event Level Description
11/24/2025 10:44 68010300 4 "DHCP RENEW WARNING - Field invalid in response v4 option;CM-MAC=a0:68:7e:ab:cf:2d;CMTS-MAC=00:90:f0:32:11:00;CM-QOS=1.1;CM-VER=3.1;"
11/24/2025 10:32 74010100 6 "CM-STATUS message sent. Event Type Code: 24; Chan ID: 193; DSID: N/A; MAC Addr: N/A; OFDM/OFDMA Profile ID: 1 2 3.;CM-MAC=a0:68:7e:ab:cf:2d;CMTS-MAC=00:90:f0:32:11:00;CM-QOS=1.1;CM-VER=3.1;"
11/24/2025 10:17 74010100 6 "CM-STATUS message sent. Event Type Code: 16; Chan ID: 193; DSID: N/A; MAC Addr: N/A; OFDM/OFDMA Profile ID: 3.;CM-MAC=a0:68:7e:ab:cf:2d;CMTS-MAC=00:90:f0:32:11:00;CM-QOS=1.1;CM-VER=3.1;"
11/24/2025 09:33 68010300 4 "DHCP RENEW WARNING - Field invalid in response v4 option;CM-MAC=a0:68:7e:ab:cf:2d;CMTS-MAC=00:90:f0:32:11:00;CM-QOS=1.1;CM-VER=3.1;"
11/23/2025 16:33 74010100 6 "CM-STATUS message sent. Event Type Code: 24; Chan ID: 193; DSID: N/A; MAC Addr: N/A; OFDM/OFDMA Profile ID: 1 2 3.;CM-MAC=a0:68:7e:ab:cf:2d;CMTS-MAC=00:90:f0:32:11:00;CM-QOS=1.1;CM-VER=3.1;"
11/23/2025 16:15 74010100 6 "CM-STATUS message sent. Event Type Code: 16; Chan ID: 193; DSID: N/A; MAC Addr: N/A; OFDM/OFDMA Profile ID: 3.;CM-MAC=a0:68:7e:ab:cf:2d;CMTS-MAC=00:90:f0:32:11:00;CM-QOS=1.1;CM-VER=3.1;"
11/23/2025 15:44 68010300 4 "DHCP RENEW WARNING - Field invalid in response v4 option;CM-MAC=a0:68:7e:ab:cf:2d;CMTS-MAC=00:90:f0:32:11:00;CM-QOS=1.1;CM-VER=3.1;"
11/23/2025 15:30 74010100 6 "CM-STATUS message sent. Event Type Code: 24; Chan ID: 193; DSID: N/A; MAC Addr: N/A; OFDM/OFDMA Profile ID: 1 2 3.;CM-MAC=a0:68:7e:ab:cf:2d;CMTS-MAC=00:90:f0:32:11:00;CM-QOS=1.1;CM-VER=3.1;"
11/23/2025 15:14 74010100 6 "CM-STATUS message sent. Event Type Code: 16; Chan ID: 193; DSID: N/A; MAC Addr: N/A; OFDM/OFDMA Profile ID: 2.;CM-MAC=a0:68:7e:ab:cf:2d;CMTS-MAC=00:90:f0:32:11:00;CM-QOS=1.1;CM-VER=3.1;"
11/23/2025 14:35 68010300 4 "DHCP RENEW WARNING - Field invalid in response v4 option;CM-MAC=a0:68:7e:ab:cf:2d;CMTS-MAC=00:90:f0:32:11:00;CM-QOS=1.1;CM-VER=3.1;"
11/23/2025 14:29 74010100 6 "CM-STATUS message sent. Event Type Code: 24; Chan ID: 193; DSID: N/A; MAC Addr: N/A; OFDM/OFDMA Profile ID: 1 2 3.;CM-MAC=a0:68:7e:ab:cf:2d;CMTS-MAC=00:90:f0:32:11:00;CM-QOS=1.1;CM-VER=3.1;"
11/23/2025 14:15 74010100 6 "CM-STATUS message sent. Event Type Code: 16; Chan ID: 193; DSID: N/A; MAC Addr: N/A; OFDM/OFDMA Profile ID: 3.;CM-MAC=a0:68:7e:ab:cf:2d;CMTS-MAC=00:90:f0:32:11:00;CM-QOS=1.1;CM-VER=3.1;"
11/23/2025 13:29 68010300 4 "DHCP RENEW WARNING - Field invalid in response v4 option;CM-MAC=a0:68:7e:ab:cf:2d;CMTS-MAC=00:90:f0:32:11:00;CM-QOS=1.1;CM-VER=3.1;"
11/23/2025 11:33 74010100 6 "CM-STATUS message sent. Event Type Code: 24; Chan ID: 193; DSID: N/A; MAC Addr: N/A; OFDM/OFDMA Profile ID: 1 2 3.;CM-MAC=a0:68:7e:ab:cf:2d;CMTS-MAC=00:90:f0:32:11:00;CM-QOS=1.1;CM-VER=3.1;"
11/23/2025 11:14 74010100 6 "CM-STATUS message sent. Event Type Code: 16; Chan ID: 193; DSID: N/A; MAC Addr: N/A; OFDM/OFDMA Profile ID: 3.;CM-MAC=a0:68:7e:ab:cf:2d;CMTS-MAC=00:90:f0:32:11:00;CM-QOS=1.1;CM-VER=3.1;"
11/23/2025 11:07 68010300 4 "DHCP RENEW WARNING - Field invalid in response v4 option;CM-MAC=a0:68:7e:ab:cf:2d;CMTS-MAC=00:90:f0:32:11:00;CM-QOS=1.1;CM-VER=3.1;"
11/23/2025 07:59 74010100 6 "CM-STATUS message sent. Event Type Code: 24; Chan ID: 193; DSID: N/A; MAC Addr: N/A; OFDM/OFDMA Profile ID: 1 2 3.;CM-MAC=a0:68:7e:ab:cf:2d;CMTS-MAC=00:90:f0:32:11:00;CM-QOS=1.1;CM-VER=3.1;"
11/23/2025 07:50 68010300 4 "DHCP RENEW WARNING - Field invalid in response v4 option;CM-MAC=a0:68:7e:ab:cf:2d;CMTS-MAC=00:90:f0:32:11:00;CM-QOS=1.1;CM-VER=3.1;"
11/23/2025 07:46 74010100 6 "CM-STATUS message sent. Event Type Code: 24; Chan ID: 193; DSID: N/A; MAC Addr: N/A; OFDM/OFDMA Profile ID: 1 2 3.;CM-MAC=a0:68:7e:ab:cf:2d;CMTS-MAC=00:90:f0:32:11:00;CM-QOS=1.1;CM-VER=3.1;"
11/23/2025 07:43 74010100 6 "CM-STATUS message sent. Event Type Code: 16; Chan ID: 193; DSID: N/A; MAC Addr: N/A; OFDM/OFDMA Profile ID: 3.;CM-MAC=a0:68:7e:ab:cf:2d;CMTS-MAC=00:90:f0:32:11:00;CM-QOS=1.1;CM-VER=3.1;"
11/23/2025 06:43 68010300 4 "DHCP RENEW WARNING - Field invalid in response v4 option;CM-MAC=a0:68:7e:ab:cf:2d;CMTS-MAC=00:90:f0:32:11:00;CM-QOS=1.1;CM-VER=3.1;"
11/23/2025 01:18 74010100 6 "CM-STATUS message sent. Event Type Code: 24; Chan ID: 193; DSID: N/A; MAC Addr: N/A; OFDM/OFDMA Profile ID: 1 2 3.;CM-MAC=a0:68:7e:ab:cf:2d;CMTS-MAC=00:90:f0:32:11:00;CM-QOS=1.1;CM-VER=3.1;"
11/23/2025 01:01 68010300 4 "DHCP RENEW WARNING - Field invalid in response v4 option;CM-MAC=a0:68:7e:ab:cf:2d;CMTS-MAC=00:90:f0:32:11:00;CM-QOS=1.1;CM-VER=3.1;"
11/23/2025 00:13 74010100 6 "CM-STATUS message sent. Event Type Code: 16; Chan ID: 193; DSID: N/A; MAC Addr: N/A; OFDM/OFDMA Profile ID: 1.;CM-MAC=a0:68:7e:ab:cf:2d;CMTS-MAC=00:90:f0:32:11:00;CM-QOS=1.1;CM-VER=3.1;"
11/22/2025 23:52 74010100 6 "CM-STATUS message sent. Event Type Code: 24; Chan ID: 193; DSID: N/A; MAC Addr: N/A; OFDM/OFDMA Profile ID: 1 2 3.;CM-MAC=a0:68:7e:ab:cf:2d;CMTS-MAC=00:90:f0:32:11:00;CM-QOS=1.1;CM-VER=3.1;"
11/22/2025 23:50 68010300 4 "DHCP RENEW WARNING - Field invalid in response v4 option;CM-MAC=a0:68:7e:ab:cf:2d;CMTS-MAC=00:90:f0:32:11:00;CM-QOS=1.1;CM-VER=3.1;"
11/22/2025 23:45 74010100 6 "CM-STATUS message sent. Event Type Code: 24; Chan ID: 193; DSID: N/A; MAC Addr: N/A; OFDM/OFDMA Profile ID: 1 2 3.;CM-MAC=a0:68:7e:ab:cf:2d;CMTS-MAC=00:90:f0:32:11:00;CM-QOS=1.1;CM-VER=3.1;"
11/22/2025 23:33 74010100 6 "CM-STATUS message sent. Event Type Code: 16; Chan ID: 193; DSID: N/A; MAC Addr: N/A; OFDM/OFDMA Profile ID: 3.;CM-MAC=a0:68:7e:ab:cf:2d;CMTS-MAC=00:90:f0:32:11:00;CM-QOS=1.1;CM-VER=3.1;"
11/22/2025 22:40 68010300 4 "DHCP RENEW WARNING - Field invalid in response v4 option;CM-MAC=a0:68:7e:ab:cf:2d;CMTS-MAC=00:90:f0:32:11:00;CM-QOS=1.1;CM-VER=3.1;"
11/22/2025 20:32 74010100 6 "CM-STATUS message sent. Event Type Code: 24; Chan ID: 193; DSID: N/A; MAC Addr: N/A; OFDM/OFDMA Profile ID: 1 2 3.;CM-MAC=a0:68:7e:ab:cf:2d;CMTS-MAC=00:90:f0:32:11:00;CM-QOS=1.1;CM-VER=3.1;"
11/22/2025 20:14 74010100 6 "CM-STATUS message sent. Event Type Code: 16; Chan ID: 193; DSID: N/A; MAC Addr: N/A; OFDM/OFDMA Profile ID: 1 2.;CM-MAC=a0:68:7e:ab:cf:2d;CMTS-MAC=00:90:f0:32:11:00;CM-QOS=1.1;CM-VER=3.1;"
11/22/2025 19:39 68010300 4 "DHCP RENEW WARNING - Field invalid in response v4 option;CM-MAC=a0:68:7e:ab:cf:2d;CMTS-MAC=00:90:f0:32:11:00;CM-QOS=1.1;CM-VER=3.1;"
11/22/2025 11:15 74010100 6 "CM-STATUS message sent. Event Type Code: 24; Chan ID: 193; DSID: N/A; MAC Addr: N/A; OFDM/OFDMA Profile ID: 2 3.;CM-MAC=a0:68:7e:ab:cf:2d;CMTS-MAC=00:90:f0:32:11:00;CM-QOS=1.1;CM-VER=3.1;"
11/22/2025 10:58 74010100 6 "CM-STATUS message sent. Event Type Code: 16; Chan ID: 193; DSID: N/A; MAC Addr: N/A; OFDM/OFDMA Profile ID: 3.;CM-MAC=a0:68:7e:ab:cf:2d;CMTS-MAC=00:90:f0:32:11:00;CM-QOS=1.1;CM-VER=3.1;"
11/22/2025 10:48 68010300 4 "DHCP RENEW WARNING - Field invalid in response v4 option;CM-MAC=a0:68:7e:ab:cf:2d;CMTS-MAC=00:90:f0:32:11:00;CM-QOS=1.1;CM-VER=3.1;"
11/22/2025 10:23 74010100 6 "CM-STATUS message sent. Event Type Code: 24; Chan ID: 193; DSID: N/A; MAC Addr: N/A; OFDM/OFDMA Profile ID: 2 3.;CM-MAC=a0:68:7e:ab:cf:2d;CMTS-MAC=00:90:f0:32:11:00;CM-QOS=1.1;CM-VER=3.1;"
11/22/2025 10:08 74010100 6 "CM-STATUS message sent. Event Type Code: 16; Chan ID: 193; DSID: N/A; MAC Addr: N/A; OFDM/OFDMA Profile ID: 2.;CM-MAC=a0:68:7e:ab:cf:2d;CMTS-MAC=00:90:f0:32:11:00;CM-QOS=1.1;CM-VER=3.1;"
11/22/2025 09:43 68010300 4 "DHCP RENEW WARNING - Field invalid in response v4 option;CM-MAC=a0:68:7e:ab:cf:2d;CMTS-MAC=00:90:f0:32:11:00;CM-QOS=1.1;CM-VER=3.1;"
11/21/2025 23:16 74010100 6 "CM-STATUS message sent. Event Type Code: 24; Chan ID: 193; DSID: N/A; MAC Addr: N/A; OFDM/OFDMA Profile ID: 3.;CM-MAC=a0:68:7e:ab:cf:2d;CMTS-MAC=00:90:f0:32:11:00;CM-QOS=1.1;CM-VER=3.1;"
11/21/2025 22:57 74010100 6 "CM-STATUS message sent. Event Type Code: 16; Chan ID: 193; DSID: N/A; MAC Addr: N/A; OFDM/OFDMA Profile ID: 3.;CM-MAC=a0:68:7e:ab:cf:2d;CMTS-MAC=00:90:f0:32:11:00;CM-QOS=1.1;CM-VER=3.1;"
11/21/2025 22:07 68010300 4 "DHCP RENEW WARNING - Field invalid in response v4 option;CM-MAC=a0:68:7e:ab:cf:2d;CMTS-MAC=00:90:f0:32:11:00;CM-QOS=1.1;CM-VER=3.1;"
11/21/2025 21:46 74010100 6 "CM-STATUS message sent. Event Type Code: 24; Chan ID: 193; DSID: N/A; MAC Addr: N/A; OFDM/OFDMA Profile ID: 3.;CM-MAC=a0:68:7e:ab:cf:2d;CMTS-MAC=00:90:f0:32:11:00;CM-QOS=1.1;CM-VER=3.1;"
11/21/2025 21:26 74010100 6 "CM-STATUS message sent. Event Type Code: 16; Chan ID: 193; DSID: N/A; MAC Addr: N/A; OFDM/OFDMA Profile ID: 3.;CM-MAC=a0:68:7e:ab:cf:2d;CMTS-MAC=00:90:f0:32:11:00;CM-QOS=1.1;CM-VER=3.1;"
11/21/2025 20:59 68010300 4 "DHCP RENEW WARNING - Field invalid in response v4 option;CM-MAC=a0:68:7e:ab:cf:2d;CMTS-MAC=00:90:f0:32:11:00;CM-QOS=1.1;CM-VER=3.1;"
11/21/2025 17:52 74010100 6 "CM-STATUS message sent. Event Type Code: 24; Chan ID: 193; DSID: N/A; MAC Addr: N/A; OFDM/OFDMA Profile ID: 3.;CM-MAC=a0:68:7e:ab:cf:2d;CMTS-MAC=00:90:f0:32:11:00;CM-QOS=1.1;CM-VER=3.1;"
11/21/2025 17:40 68010300 4 "DHCP RENEW WARNING - Field invalid in response v4 option;CM-MAC=a0:68:7e:ab:cf:2d;CMTS-MAC=00:90:f0:32:11:00;CM-QOS=1.1;CM-VER=3.1;"
11/21/2025 17:38 74010100 6 "CM-STATUS message sent. Event Type Code: 24; Chan ID: 193; DSID: N/A; MAC Addr: N/A; OFDM/OFDMA Profile ID: 3.;CM-MAC=a0:68:7e:ab:cf:2d;CMTS-MAC=00:90:f0:32:11:00;CM-QOS=1.1;CM-VER=3.1;"
11/21/2025 17:34 74010100
Same CMTS-MAC (00:90:f0:32:11:00)
Events on same dates
Both logs show :33 and :59 minute markers (overlapping)
Different modem manufacturers (proves it’s not equipment-specific)
Different log formats show the EXACT SAME PATTERN
Both modems are responding to the same R-PHY node crash/restart
, just at different layers of the network stack. The ~30-minute lag is expected because upstream errors take time to cascade through the downstream systems.
My Attempts to Fix It
I called support. Multiple times. They blamed my WiFi (I’m hardwired). They blamed my modem (an Xfinity Approved MB8611). They blamed my router (I tested with multiple devices). They sent a subcontracted technician. Then another subcontracted technician. Then a subcontracted crew. The crew ran a new coax line and grounded it to a water pipe that turns into PVC when it enters the ground (this doesn’t ground anything). Then they sent an Xfinity technician to look at the line.
The problem never changed. The only thing that changed is my download speed dropped from advertised 1200Mbps to <500Mbps.
For my trouble of reporting a network outage I lost 700Mbps in speed. This is a 1200Mbps plan.
I escalated to retention. They offered me nothing. I provided detailed technical documentation showing the exact pattern of the outages, the minute markers they occur at, the exact duration every single time. They didn’t understand it. They couldn’t escalate it.
I was transferred to technical support. The person did not care and put me on speaker phone with so much background noise I couldn’t hear myself think. I imagine he was rolling his eyes while trying his utmost to care less.
My Neighbor’s Attempts to Fix It
He called Xfinity about his TV cutting out repeatedly. The technician told him his UPS grounding his coax cable was causing the problem. So he ungrounded the cable, pocketed the spare cable, and my neighbor kept having the same issues.
What I Found
Just in the last 72 hours I have documented 20 consecutive outages using OPNSense’s built in gateway uptime monitor. Here’s what they show:
Every single outage lasted 124.8 ± 1.3 seconds.
That’s not random hardware failure. That’s a timeout value hardcoded into something in Xfinity’s infrastructure.
The outages cluster at specific minute markers.
35% start at minute :44. 35% start at minute :29. This is scheduled automation. This is likely a cron job or automated task running at those exact times every hour.
The outages peak at specific hours.
Most happen between noon and 1 PM. Others cluster in early morning around 2-3 AM. This is not random.
This is an infrastructure problem on Xfinity’s network. Not on my end. Not on my neighbor’s end. Upstream. Somewhere on their equipment something is failing for exactly 125 seconds multiple times per day.
I have the data. I have the patterns. I have another customer (my neighbor) on a different line experiencing the exact same thing.
Xfinity has all this information. They know about the problem. They just won’t investigate it.
Why They Won’t Fix It
Support can’t understand technical data.
They follow scripts. When I attempted to explain monitoring logs they had no framework for discussing it. They blamed my equipment because that’s what they’re trained to do.
Nobody has authority to escalate.
Retention transferred me to tech support. Tech support couldn’t care nor help. They dug up my yard and placed a new line which did nothing to solve the problem. Nobody would actually order an investigation into the upstream infrastructure.
There’s no pressure to fix it.
Xfinity is the only gigabit provider in this area. No competition. No alternatives. I can’t leave. So they don’t have to care.
A 2-minute outage every few hours is “tolerable.”
It’s annoying enough to frustrate customers but not enough to make them quit (since they have nowhere else to go). It’s cheap to ignore compared to actually investigating and fixing it.
There’s Also a Security Problem
About half of the Xfinity junction boxes in my neighborhood are unlocked or broken. Anyone can walk up and disconnect whomever they want.
If your home security system is on Xfinity with no wireless backup, someone can just walk to the street and physically disconnect your internet, rob your house, and your security system won’t notify you.
I’m Out of Options
I’ve done everything I can do as a customer:
Documented the problem professionally
Escalated through all available channels
Provided technical evidence
Been ignored at every level
The problem is real. My neighbor confirms it. Everyone downstream of whatever is broken on their infrastructure probably has it too.
I can’t fix this. Only Xfinity can. And they won’t.
So I’m publishing this hoping someone with actual authority, perhaps someone at a regulatory agency, someone at a news outlet, someone who has power over Xfinity sees this and decides to actually investigate.
Because I’m out of options. My neighbors are out of options. And Xfinity’s counting on us staying out of options. Because this is the reality of my neighborhood (
source
):
As you can, see there is zero choice.
If you’re in Sacramento County and have Xfinity internet, check two things:
1. Walk to your junction box. Is it locked? If not, you have a physical security problem.
2. Look at where your cable grounds. Does it go to your electrical panel? Or to a water pipe? If it’s a water pipe or PVC, that’s wrong.
3. Have you noticed your connection drop briefly multiple times a day? Same times of day? If you see a pattern, document it. You might have the same problem.
The Sacramento Bear is an independent newspaper engaged in first amendment activities and leveraging the freedom of the press to express facts and opinions about various matters.
This paper explores the analysis and design of the resting configurations of a rigid body, without the use of physical simulation. In particular, given a rigid body in R^3, we identify all possible stationary points, as well as the probability that the body will stop at these points, assuming a random initial orientation and negligible momentum. The forward version of our method can hence be used to automatically orient models, to provide feedback about object stability during the design process, and to furnish plausible distributions of shape orientation for natural scene modeling. Moreover, a differentiable inverse version of our method lets us design shapes with target resting behavior, such as dice with target, nonuniform probabilities. Here we find solutions that would be nearly difficult to find using classical techniques, such as dice with additional unstable faces that provide more natural overall geometry. From a technical point of view, our key observation is that rolling equilibria can be extracted from the Morse-Smale complex of the support function over the Gauss map. Our method is hence purely geometric, and does not make use of random sampling, or numerical time integration. Yet surprisingly, this purely geometric model makes extremely accurate predictions of rest behavior, which we validate both numerically, and via physical experiments. Moreover, for computing rest statistics, it is orders of magnitude faster than state of the art rigid body simulation, opening the door to inverse design---rather than just forward analysis.
@article{Baktash:2025:PRB,
author = {Baktash, Hossein and Sharp, Nicholas and
Zhou, Qingnan and Crane, Keenan and Jacobson, Alec},
title = {Putting Rigid Bodies to Rest},
journal = {ACM Trans. Graph.},
issue_date = {August 2025},
volume = {44},
number = {4},
articleno = {155},
numpages = {16},
year = {2025},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
issn = {0730-0301},
url = {https://doi.org/10.1145/3731203},
doi = {10.1145/3731203}
}
Acknowledgments
This work was generously supported by NSF awards 2212290, 1943123, NSERC Discovery (RGPIN–2022–04680), the Ontario Early Research Award program, the Canada Research Chairs Program, a Sloan Research Fellowship, the DSI Catalyst Grant program and gifts from Adobe Systems. The authors also thank Joseph Sharp for resin printing the dice, Zo
ë
Marschner and Olga Guțan for helping with conducting and recording the dice experiments, and Carnegie Mellon University TechSpark 3D printing facilities for enabling and helping with dice fabrication.
OSS Friday Update - The Fiber Scheduler is Taking Shape
This week I made substantial progress on the
UringMachine
fiber scheduler
implementation, and also learned quite a bit about the inner workings of the
Ruby I/O layer. Following is my weekly report:
I added some benchmarks measuring how the UringMachine mutex performs against
the stock Ruby Mutex class. It turns out the
UM#synchronize
was much slower
than core Ruby
Mutex#synchronize
. This was because the UM version was always
performing a futex wake before returning, even if no fiber was waiting to lock
the mutex. I rectified this by adding a
num_waiters
field to
struct
um_mutex
, which indicates the number of fibers currently waiting to lock the
mutex, and avoiding calling
um_futex_wake
if it’s 0.
I also noticed that the
UM::Mutex
and
UM::Queue
classes were marked as
RUBY_TYPED_EMBEDDABLE
, which means the underlying
struct um_mutex
and
struct um_queue
were subject to moving. Obviously, you cannot just move a
futex var while the kernel is potentially waiting on it to change. I fixed
this by removing the
RUBY_TYPED_EMBEDDABLE
flag.
Added support for
IO::Buffer
in all low-level I/O APIs, which also means
the fiber scheduler doesn’t need to convert from
IO::Buffer
to strings in
order to invoke the UringMachine API.
Added a custom
UM::Error
exception class raised on bad arguments or other
API misuse. I’ve also added a
UM::Stream::RESPError
exception class to be
instantiated on RESP errors. (commit 72a597d9f47d36b42977efa0f6ceb2e73a072bdf)
I explored the fiber scheduler behaviour after forking. A fork done from a
thread where a scheduler was set will result in a main thread with the same
scheduler instantance. For the scheduler to work correctly after a fork, its
state must be reset. This is because sharing the same io_uring instance
between parent and child processes is not possible
(https://github.com/axboe/liburing/issues/612), and also because the child
process keeps only the fiber from which the fork was made as its main fiber
(the other fibers are lost).
On Samuel’s suggestions, I’ve submitted a
PR
for adding a
Fiber::Scheduler#process_fork
hook that is automatically invoked after a
fork. This is in continuation to the
#post_fork
method. I still have a lot
to learn about working with the Ruby core code, but I’m really excited about
the possibility of this PR (and the
previous
one
as well) getting merged in time
for the Ruby 4.0 release.
Added two new low-level APIs for waiting on processes, instead of
UM#waitpid
, using the io_uring version of
waitid
. The vanilla version
UM#waitid
returns an array containing the terminated process pid, exit
status and code. The
UM#waitid_status
method returns a
Process::Status
with the pid and exit status. This method is present only if the
rb_process_status_new
function is available (see above).
Implemented
FiberScheduler#process_wait
hook using
#waitid_status
.
For the sake of completeness, I also added
UM.pidfd_open
and
UM.pidfd_send_signal
for working with PID. A simple example:
Wrote a whole bunch of tests for
UM::FiberScheduler
: socket I/O, file I/O,
mutex, queue, waiting for threads. In the process I discovered a lots of
things that can be improved in the way Ruby invokes the fiber scheduler.
Things I Learned This Week
As I dive deeper into integrating UringMachine with the
Fiber::Scheduler
interface, I’m discovering all the little details about how Ruby does I/O. As I
wrote last week, Ruby treats files differently than other
IO
types, such as
sockets and pipes:
For regular files, Ruby assumes file I/O can never be non-blocking (or async),
and thus invokes the
#blocking_operation_wait
hook in order to perform the
I/O in a separate thread. With io_uring, of course, file I/O
is
asynchronous.
For sockets there are no specialized hooks, like
#socket_send
etc. Instead,
Ruby makes the socket fd’s non-blocking and invokes
#io_wait
to wait for the
socket to be ready when performing a
send
or
recv
.
I find it interesting how io_uring breaks a lot of assumptions about how I/O
should be done. Basically, with io_uring you can treat
all
fd’s as blocking
(i.e. without the
O_NONBLOCK
control flag), and you can use io_uring to
perform asynchrnous I/O on them, files included!
It remains to be seen if in the future the Ruby I/O implementation could be
simplified to take full advantage of io_uring. Right now, the way things are
done in the core Ruby IO classes leaves a lot of performance opportunities on
the table. So, while the UringMachine fiber scheduler implementation will help
in integrating UringMachine with the rest of the Ruby ecosystem, to really do
high-performance I/O, one would still need to use UringMachine’s low-level API.
What’s Coming Next Week
Next week I hope to finish the fiber scheduler implementation by adding the last
few things that are missing: handling of timeout, the
#io_pread
and
io_pwrite
hooks, and a few more minor features, as well as a lot more testing.
I also plan to start benchmarking UringMachine and compare the performance of
its low-level API, the UringMachine fiber scheduler, and the regular
thread-based concurrent I/O.
I also have some ideas for improvements to the UringMachine low-level
implementation, which hopefully I’ll be able to report on next week.
If you appreciate my OSS work, please consider becoming a
sponsor
.
The 23+ best US Black Friday and Cyber Monday travel deals already taking off
Guardian
www.theguardian.com
2025-11-28 19:15:14
From Away, Calpak and Samsonite, here are budget-friendly reasons to upgrade your luggage, replace your headphones and finally invest in a set of packing cubesSnag these tech, home and kitchen Black Friday and Cyber Monday dealsSign up for the Filter US newsletter, your weekly guide to buying fewer,...
W
hile the
Black Friday
deals landscape can be overwhelming – and you might be tempted to avoid it completely – some actually incredible deals do exist out there, particularly in the
travel
space. As a travel journalist and the writer of a
packing list newsletter
, I’m always on the hunt for luggage, clothing and gear that will streamline my travel process. During Black Friday and Cyber Monday sales, I keep a trained eye on the retailers with genuine discounts on carry-on suitcases, comfortable loungewear and more. Pro tip: if a specific item catches my eye, I will Google it to see if another website is offering a more enticing deal. (It usually is.)
So if you’re hunting for items that will upgrade your travels without blowing your budget, use my curated guide to inform your shopping. I’ll be regularly updating the deals throughout the holiday sales period, so check back here for more savings over the next two weeks.
How I selected these Black Friday and Cyber Monday travel deals
My north star for investing in travel-related items has always been quality over quantity. I don’t need three kinds of carry-on suitcases; I just need
one I can rely on for every trip
.
I started my search for deals by outlining the key items every traveler should own. I also considered the “nice to haves,” or things that have made travel easier for me over the years. Then, I went to work hunting down those specific pieces from reputable retailers and brands – most of which I frequently shop from – and determining if the discounted prices were worthwhile. The ones that made the cut are featured below.
At a glance: the very best Black Friday and Cyber Monday travel deals
This is, hands down, one of my favorite luggage items of 2025. The crossbody, which is part of Away’s sitewide sale, is one of those pieces that just make travel easier. I first brought it with me on a trip to Alaska, and it fit everything I needed for daily excursions: snacks, a water bottle, sunscreen, a book and even an extra layer.
There’s something so aesthetically satisfying about traveling with complementary luggage, but you also want each piece to actually be functional. The Wrangler Smart Luggage Set is the best of both worlds. Each suitcase features a built-in cup holder, a USB port and even a phone holder – and they’re designed to expand when you need a bit of extra room. Several stylish colorways are on sale right now, including an olive green and a slate gray.
I’m Team Hardside, but I still understand the appeal of softside luggage. A softside suitcase is often lighter and can be expanded when you require extra space. If I were going to invest in this type of bag, I’d definitely go with the Antler Lightest Expandable Carry-On Luggage – a top-rated style that’s compact, durable and designed for convenience (the front pocket is ideal for the items you need easy access to).
When it comes to leather travel bags I know I’ll use forever, I always turn to Cuyana – and I’ve been using the Classic Easy Tote for years. It’s the definition of a timeless piece: stylish, simple and incredibly functional. My favorite part of its design? It can fit my 16-inch laptop, which, as a freelance writer, I never travel without.
Quick overnight or weekend trips don’t always require a suitcase. Most of the time, you really only need a change of clothes, some pajamas and a small toiletry bag, all of which can fit in the Etronik Overnight Duffel Bag. The weekender has received more than 1,500 five-star reviews, with shoppers praising its pockets, spaciousness and durability. Inside, there’s even a zippered wet bag to hold anything you may want to keep separate (a wet bathing suit, makeup, etc).
Even if you’re a self-proclaimed carry-on-only type, it’s still a good idea to have a checked bag at the ready. This luggage set from Samsonite – a carry-on and a large spinner suitcase – covers all your bases. It comes in a variety of colors that will stand out at baggage claim, so you’ll never have to second-guess which luggage is yours. And at 59% off? This is truly one of the best early
Black Friday
travel deals.
With the entire Leatherology site currently up to 25% off with code HOLIDAY25, I took the liberty of finding the most useful, marked-down item. You can carry Transit Travel Tote with the long straps or the short straps, and there’s even a hidden trolley pocket that allows you to slide it over your suitcase’s extended handle. What’s more, you can personalize with up to five letters.
I started using the Calpak Terra 26L Laptop Duffel Backpack last winter and was immediately blown away by the sheer amount it can comfortably carry. I’d even say it holds as much as my suitcase – and it will still fit underneath the airplane seat. The clamshell opening to a spacious main compartment makes packing a breeze, and the interior compression strap ensures everything stays secure. Part backpack, part duffel – you’re basically getting two bags for one.
Away is my go-to brand for luggage. I’ve used the Bigger Carry-On for years, and I firmly believe (after trying at least 10 other suitcases) that it holds more than any other carry-on out there. The bundle comes with a set of packing cubes – which are helpful with suitcase organization – so it’s a great starter pack for anyone just beginning their travel journey or those who tend to overpack.
Away’s early Black Friday sale means everything from the luggage brand is 25% off, but you’ll want to stay focused on the items with the best cost-per-use. For the everyday traveler, that is the Away Packing Pro Bundle. The set includes the Bigger Carry-On and the Insider Packing Cubes (set of four), two travel essentials that can be used together or separately. Bonus: you can opt to get both in the same color or mix and match.
Make no mistake, high-quality checked suitcases are expensive, so I always recommend waiting to purchase one until it’s on sale. While the Roam Check-In Expandable suitcase is still on the pricier side, it’s the kind of suitcase you only need to buy once. Designed with an expandable feature (a 2in zipper expansion) and compression boards, it’ll comfortably hold between 10 and 13 outfits.
Is European travel in your future – or maybe you need a practical present for someone heading abroad? If the answer is yes, grab this European travel plug adapter set while it’s on sale (25% off). It comes with one Type-C plug adapter (works for Americans traveling to Germany, Italy, France and Spain, among other countries) and one Type-G mini adapter, which you’d use in the U.K.
International travel requires all sorts of preparation: passports, visas, currency exchange and adapters. I can’t help you too much with the first few, but I can assist with the travel-adapter situation. I’d recommend picking up something like the Epicka Universal Travel Adapter, which ensures you’re covered in 200-plus countries and regions – and it can even charge up to six devices at once.
The Black Friday sale period is one of the best times to find deals on pricier travel pieces – like a nice camera you can bring on a safari or a long-awaited family trip to Europe. The EOS R100 Body is a great option for beginners or amateur photographers; it’s definitely an upgrade from your smartphone, but it’s not the type of technology that will overwhelm you with all of its features.
Regular packing cubes have a slightly different function than compression packing cubes, but they’re still an integral part of any traveler’s arsenal. I like to use a set similar to this one from Veken for organization – T-shirts and tanks in one, socks and undergarments in another, etc. It’s also a good idea to carry an extra packing cube to hold your dirty clothes. If you aren’t totally sure you’ll use packing cubes (although, spoiler alert: you probably will), try out the set in Indigo Teal at nearly 50% off.
Travelers who prefer earbuds to headphones will want to grab these while they’re 50% off in Amazon’s early Black Friday sale. While wearing them, you can use transparency mode or choose to go full noise canceling, focusing instead on just your music. These are also great for a long flight; a single charge gets you 10 hours of listening, with the use of the charging case, that extends to 45 hours.
Carrying a portable charger is a non-negotiable for me when I travel. I never know when the charging port on the airplane or in the airport will be defective, and I refuse to arrive at my destination without a fully charged phone. That’s where the Anker 633 Magnetic Battery comes into play. It can recharge your phone about two times, and it’s small enough to be carried in a purse or handbag. Plus, it’s highly rated and marked down by 40% – now’s the time to snag one.
I’m not a fan of the classic bulky travel pillows. In my eyes, they’re simply not worth the hassle of hauling them around the airport. But I still believe in comfort, particularly on red-eye flights. That’s why I’ll frequently tuck this Trtl neck pillow in my personal item bag. It’s light and compact while still providing the proper neck support to get some sleep on the plane. I also love that it’s machine-washable, and I’ll toss it in the laundry after most trips.
Searching for a stocking stuffer? The Travel Inspira Luggage Scale is the type of item most people don’t realize is missing in their life until they experience the reassurance that comes with weighing a suitcase before arriving at the airport. You simply loop the weighing belt through your luggage handle and hold it up to get a read. Yes, the sale price saves you a few dollars, but you’ll also never pay an overweight luggage fee again.
As someone who has lost too many AirPods while in transit, I’m a full headphones convert. This pair from JBL is 50% off and highly rated by thousands of shoppers. If you’re not totally sold on larger headphones but want to give them a try (without splurging on a pricier version), this is the way to go. Not to mention, it comes in a handful of colorways, including purple and blue.
There are certain things you can’t control at the airport – ahem, lost luggage – but you can arm yourself with tech that can track your belongings. Keeping an Apple AirTag in both my checked bag and carry-on gives me peace of mind when I’m making a tight connection or using a transfer service. If you haven’t invested in your own set of AirTags, now is the time.
If you went to Apple.com to purchase a four-pack of AirTags right now, you would pay full price. But on Amazon, the nifty tracking devices are under $65. While they’re definitely useful while traveling, AirTags can also do wonders in your daily life: attach one to your keys, throw one in your purse, or slip one in your wallet.
The best travel clothing deals
Photograph: Courtesy of Lululemon
JUST ADDED: Lululemon Men’s Soft Jersey Tapered Pant Regular
Winter is the time to make sure your wardrobe has an appropriate amount of comfortable basics, whether you’re wearing them for travel or relaxing days at home. Lululemon’s Black Friday sale includes tons of styles, like the Soft Jersey Tapered Pant, that fit the bill. These are lightweight, stretchy and sweat-wicking, while still having a streamlined, elevated silhouette – AKA the perfect plane pant.
I firmly believe that anyone who travels frequently should have a go-to cozy set they can wear on a long-haul or red-eye flight – and Garnet Hill is a great place to start your search. Right now, the brand is offering 40% off cashmere and flannel (plus 30% off everything else), which means you can pick up the incredibly soft Washable-Cashmere Hoodie and
matching joggers
while they’re marked down.
My personal favorite travel-day uniform nearly always includes a plain white, slightly oversized button-down, like this one from AYR. The brand rarely marks down its evergreen styles. Made of 100% cotton, it can be thrown in the wash without losing its shape, and you can wear it with anything. I usually pair mine with breezy linen pants if I’m headed somewhere warm, or straight-leg jeans for a more everyday look.
If I’m boarding a flight longer than a couple of hours, I’ll typically opt to wear leggings. But while I love my athletic leggings as much as the next person, I like to feel a bit more put-together while traveling, so I look for something like the Spanx Scuba Micro Flare Legging (currently $70 off). The scuba fabric is a bit more elevated than your classic polyester blend, and the shaping waistband gives just the right amount of support without being constrictive.
Allbirds’ Black Friday sale, up to 50% off select styles, is your chance to grab a new pair of travel sneakers. The Tree Runner NZ – available in both
men’s
and
women’s sizes
– is made of a light, breathable knit, and the cushioned memory foam will keep your feet happy, even when you’re sprinting through the airport. The shoes are also machine washable, which means you’ll have these in your rotation for many trips to come.
Compression socks are the type of travel essentials that you don’t want to knock until you try them for yourself. They might not be the most fashionable, but they are extremely functional. This type of footwear is designed to help with joint and muscle stiffness and avoid any blood pooling in your feet – all of which can happen if you sit still for too long on a plane.
I wear a lot of La Ligne – a brand that almost never goes on sale, but just launched its only sale of the year. I’m all for being comfortable on a plane, but I also like to arrive to my destination looking somewhat put-together. The Terry Colby Sweatpant immediately stood out to me as a piece that I, and you, will probably wear for every flight in the foreseeable future.
Choosing outerwear is always one of the hardest parts of packing. The general rule of thumb is to wear your jacket or coat while in transit (so you don’t have to fit it in your suitcase), but sometimes you want to bring an extra layer. In that case, I’ll go with something like the Pioneer Camp Women’s Packable Puffer – a lightweight, water-repellent style that can be packed down into its carrying bag. The timing of the sale is perfect, too; this is the type of piece you need for most winter travel.
I’m a firm believer that travel outfits should be both comfortable and presentable. I tend to stick with sweats and loungewear in solid, neutral colors, like the Forever Fleece Relaxed Crew Sweatshirt from Athleta. Not only will it never go out of style, but the dark navy also masks any inevitable travel stains.
The Athleta Friday sale – 30% off everything – is incredibly tempting for those who live in
athleisure
, but don’t go crazy just yet. Instead, only invest in the pieces that deserve a coveted spot in your suitcase. This cotton crewneck sweatshirt is machine-washable (a crucial trait for travel clothing) and comes in a ton of solid, neutral colors. Read: it will work for most, if not all, of the trips you have on your calendar.
Lydia Mansel is a travel writer and founder of the travel newsletter Just Packed. She specializes in travel and lifestyle, and her work has appeared in Travel + Leisure, Condé Nast Traveler, InStyle, Real Simple, Shape, Garden & Gun, and People, among others.
The 20+ best US Black Friday tech deals on TVs, tablets, phones, smart watches and more
Guardian
www.theguardian.com
2025-11-28 19:15:04
The sales you’ve been waiting for all year have arrived. Snag deals from Samsung, Amazon, Sony and moreThe 43 very best US Black Friday and Cyber Monday deals, curated and vettedSign up for the Filter US newsletter, your weekly guide to buying fewer, better thingsBlack Friday started off as a way to...
B
lack Friday started off as a way to score some great deals on gifts, but let’s be honest: it’s also a chance to pick up some nice, deeply discounted goodies for yourself. This is especially true in the world of tech, where high prices and personal taste mean it’s often just safest to buy what works for you rather than guessing on a gift. Don’t worry, we won’t judge.
But when you’re inundated with Black Friday and Cyber Monday deals, it’s easy to get spun around by specs: is that really enough storage? Is the screen big enough? Will I regret not getting the newer version? That’s when you turn to the experts.
I’ve been a professional tech reviewer since 2013 and I have reviewed all manner of gadgets, from phone to keyboards and even
augmented reality glasses
. If they ever put wifi in a hamburger, I’ll tell you what’s great about it and what still needs work.
How I selected these Black Friday and Cyber Monday tech deals
For this list of deals, I studied deal sites, forums and databases of deals to find deep discounts on products that I know and love. I’ve personally used many of the items in this roundup, held them up in my hand, used them daily in my life, and in many cases, written reviews of them. And in the cases where I haven’t, I know the companies and product space enough to feel confident making recommendations. While plenty of these gadgets would make great gifts, you’ll also find plenty of opportunities to upgrade your own home, if you’re so inclined.
Here are some of the best deals I’ve been able to find so far. This list will be constantly updated through November, so make sure to check back.
A built-in retractable USB-C cable means one less cable I have to bring with me, so a similar version of this charger finds its way into my backpack whenever I travel. With 70 watts of output, it can handle charging many laptops (check the label on yours) with capacity to spare for a phone on the additional USB outlets. Six styles of retractable prongs work in 200 different countries, and it works as a pass-through plug converter, too.
Whether I’m reading or watching a movie, the Amazon Fire HD 10 tablet has a beautiful screen and just the right amount of power to stream content: you don’t need much computing muscle to turn pages of a book or play back a video. It’s also very durable, so it’s great for coffee table reading. While a Fire tablet isn’t as useful as a full Android tablet, at 50% off it’s still a great deal, even if it only ends up as your Netflix screen to go.
Smart cameras typically come with a big trade-off: you need to either run an ugly wire to them or change the battery every couple months. But the Blink Outdoor 4 Wireless camera sidesteps both with a battery that can last up to two years. I’ve had one for about a year so far, and the battery shows no signs of stopping. You can put this camera anywhere it has access to wifi and basically forget it exists, except when you want to see what’s going on in your yard. At 60% off, it’s worth grabbing a few for different parts of the house and yard.
The Amazon Fire TV Stick 4K plus remains the single easiest way to turn a regular TV into a smart TV. Just plug it into your TV and a power source, and just like that you have access to streaming services such as Netflix and Hulu, Amazon and Blink camera feeds, and of course Alexa. The ultra-simple remote makes for easy navigation, and has a built-in mic for voice commands (“Hey Alexa, play The Office.”) At 50% off, you can grab one for every TV in the house, or even one to travel with – it’s tiny.
JBL is an iconic name in sound, and the JBL Live Pro 2 are some of my favorite earbuds. They have excellent Active Noise Cancellation (ANC), which makes it easier to listen to your music at a lower volume to avoid hearing damage. You also get excellent battery life, at up to 10 hours on a single charge, so these can be a great musical companion for the vibe-coders in your life to blot out the world for hours while they crack on the next big thing. I’d heartily recommend them at full price, so at half-off they’re a no brainer.
The Nex Playground is like a game console where you’re the controller. A front-facing camera tracks your motion allowing you to interact with elements on the screen, for instance, waving your hands to slash at on-screen targets. After trying it at a trade show, I quickly got one for my own entertainment center. It comes preloaded with five games – my favorite is Starri, a sort of Beat Saber clone that will get your heart rate up.
In a world of algorithms competing to serve you content in increasingly convoluted ways, the Roku interface is refreshingly simple: it’s just a list of apps. That’s it. Even as a technologically literate Gen Xer, I dig that. If you already have a streaming system that works for you, this could still be a good gift for older or less tech-savvy folks in your life. Just set it up with their account logins for Netflix, Hulu, Disney and the like, and they’ll learn the nice, clean interface in no time.
We live in an amazing time when you can buy a 75in 4K TV for under $400. This model even uses QLED technology for better color accuracy, which used to be a premium feature just a few years ago. Since it’s a Roku TV, all of your streaming services are at your fingertips right out of the box. This is a one-off model that appears to be exclusive to Walmart, so you won’t find reviews on it, but Hisense is a reputable brand and TVs have matured so much that even budget models hold their own to most eyeballs.
For color fidelity and contrast, most home theater enthusiasts still turn to OLED screens, but they seldom come cheap. This is a great deal on a high-end example, just one rung below Samsung’s flagship S95F. Gamers will appreciate the 144Hz refresh rate for smoother action, and the AI processor for 4K upscaling means that even older shows and movies will make use of every pixel.
Meta has been leading the way in the
VR
space for a decade, and the Meta Quest 3S is the most accessible headset on the market today. My favorite game is
Hell Horde
,
a first-person shooter in which demons come running at you through a hole in your living room wall. It’s wild, and there are games for all interests including
The Climb
,
Beat Saber
,
Star Wars: Beyond Victory
and more.
The HoverAir X1 is less of a drone and more of a flying camera that keeps you the center of its focus. It can fly preprogramed routes to capture the scenery around you or follow you around, dodging obstacles along the way. When I tested this drone, I rode an electric bike for five miles under and around trees, and it kept up beautifully. It’s foldable and fits neatly in a jacket pocket.
Few flat-screen TVs come with sufficient built-in sound, and even if yours seems OK, a soundbar takes things to another level. This Bravia Theater Bar 6 comes with a separate subwoofer and two rear-channel speakers to fill your room with sound, and the rear channels are wireless for easier installation. Once you hear it, you will never want to watch TV again without it.
Apple
makes the best smartwatches you can buy, and this is coming from someone who has tested nearly all of them. Just keep in mind that they only work with iPhones – so check out the Samsung below if you’re on Android. I love the app support in particular, which means I can arm my home security system, order a pizza and get directions all from my wrist. You’ll need to charge it every day, but you’re probably already doing that with your phone. At 22% off, this watch is a steal.
Having a camera on your face may sound weird, but it allows you to capture images and video and still enjoy the moment. The speakers are also pretty good for listening on the go without blocking out the world as earbuds do. I use them on bike rides to get that pseudo-GoPro experience, and at trade shows so I don’t miss notifications arriving on my phone while it’s in my pocket.
While I still pay for cloud storage space, the files I don’t need immediate access to live on a portable hard drive. It’s great long-term storage, and it’s accessible just by plugging it in. Seagate is a name known for reliable storage space, and while this hard drive isn’t the fastest, when you’re storing backup files, you don’t necessarily need speed. Just don’t use this drive for installing games or editing video files - it’s not built for that, but it’s priced right for cheap bulk storage.
Samsung’s latest smartwatch brings a ton of improvements to your wrist, including totally redesigned software that makes it easier to use. Like most smart watches, it can track your sleep and monitor your heart rate during exercise, but it also performs some unique feats such as measuring antioxidants to help suggest dietary changes, and tracking blood-oxygen levels to flag potential health issues. It all comes wrapped in an attractive package that lasts for almost two days on a charge.
Photograph: Courtesy of Amazon
Amazon Fire HD 8 Tablet Plus Standing Cover Bundle
The Amazon Fire HD 8 is a slightly smaller version of the aforementioned Amazon Fire Tablet that is better suited to
travel
. This particular bundle also includes a case with an origami-style leg to prop up the tablet up for watching shows on the go. Like the larger model, it’s mainly a media machine, so imagine it more like a portable TV than a full-fledged tablet. At this price, it’s still well worth it.
I review phones year-round, and this is the one I go back to when I’m not reviewing anything else. It’s simply one of the best Android smartphones. It has an amazing camera setup great for ultrawide snaps and zooming in to a crazy degree, onboard AI including Gemini Live, epic battery life (easily a day and a half), and a built-in stylus for those times you want precision in your tapping and swiping. This price tag may not seem like much of a discount since the S25 Ultra usually starts at about $1,200, but this is the upgraded model with 512GB storage, which you’re going to want.
Samsung’s “fan edition” (FE) devices are designed for buyers who want a flagship phone experience at a lower price point. That means the S25 FE phone has most of the same chops as its larger siblings, including all the same AI tricks, and an impressive triple-camera setup that’s no joke. It’s a great value even at full price, and at 27% off one of the best phone deals out there for Black Friday.
Bone-conduction headphones don’t go in your ears – they sit above the ear and transit crisp, clear audio with a full range of tones by simply vibrating against your head. That means you can still hear everything going on around you, making them ideal for runners. But they’re great for non-runners too – like me! I use them often on bike rides.
Bose has been a leader in noise-cancelling headphones for decades, and the QuietComfort series of headphones carry on the legacy. These headphones are great for frequent travelers as they can cancel out the drone of planes, trains, or automobiles, while you enjoy the film Planes, Trains and Automobiles. You don’t often see these headphones at this price, so these would be a great pickup.
If the traveler in your life doesn’t want to carry a bulky set of over-the-ear headphones (like me), earbuds like these are a great solution. Like their bigger brothers, these offer outstanding active noise cancellation to drown out airplane noise, but they’re also compact and still have good battery life. Since they’re earbuds, they form a great seal in your ear canal, which passively seals out noise even when ANC isn’t active. At this price, these earbuds are hard to resist, especially when compared to their peers at $130.
A pair of black Sony WF-1000XM5 Earbuds
Photograph: Courtesy of Sony
Sony headphones are a cut above the rest in terms of sound quality: when I tested the WF-1000XM5, I heard tones in music that I had never heard before. I believe they’re the best-sounding earbuds you can buy, and the Guardian’s reviewer
loved them too
. Their popularity means Sony seldom needs to discount them, so 30% off is hard to ignore. If you know someone who loves music but still listens with cheap headphones, this will open a whole new world.
This past summer, I tested nine different robot lawnmowers, including the Navimow X350 – a version of this with a slightly larger battery. I set it up and I never had to mow my lawn for the entire summer. Neither did my elderly neighbor — I set it up to mow his lawn too. The mower uses GPS and a beacon for centimeter-level accuracy to efficiently mow your lawn as often as you want; I set mine to mow three times per week. I named it Ziggy.
I never thought I needed a digital calendar until I got one, and now I can’t live without it. Yes, I can always open my phone or laptop to see my Google Calendar, but having it up on my wall is a game changer. The Skylight now occupies a prominent spot in my kitchen, and my whole family has quickly adopted using it. It syncs with most major calendar services, and can also serve as a large photo frame if you want. As someone who lives and dies by his calendar, I leave mine on the calendar view all the time.
I wouldn’t go so far as to call myself an amateur astronomer, but I love gazing at the stars. The Dwarf Telescope 3 pairs with your phone to track the skye and follow astronomical phenomena, taking long exposures to give you amazing photos. It comes with its own carrying case and it’s so small, I was able to include it in my suitcase on a trip to Hawaii. It can also work during the day as a telephoto camera for wildlife photography.
Google Nest is one of the original names in smart thermostats – they’ve been at this a long time, with the refined products to show for it. I particularly enjoy mine because it looks great, and allows me to adjust the temperature in my home (in multiple zones) using my voice or the app. When I do, the thermostat remembers my preferences and integrates them into an automatic schedule. Eventually it just does its thing on its own, and saves you money while you’re at it.
Of all the voice assistants I’ve used (all of them), Alexa is the best, providing fast, accurate answers and controlling your smart home devices just as quickly. You can check the weather, get the latest news, listen to podcasts and more with just your voice. While Google Assistant and Siri have stagnated, Alexa continues to evolve and improve: An AI-enabled version called Alexa+ just rolled out this year for Prime subscribers.
Lots of smart-home products are gimmicky, but we
wholeheartedly recommend smart bulbs
. You can have them turn on automatically at dusk, wake you up slowly in the morning as an alarm clock, or just answer to your voice commands. A multicolor bulb like this Kasa model also lets you set the mood with one of 16m colors. A two-pack for $15.99 is an instant upgrade to your home.
Photograph: Courtesy of Amazon
TP-Link Deco X15 Dual-Band AX1500 WiFi 6 Mesh Wi-Fi System
If you have wifi dead spots in your home, a mesh wifi network is an easy modern way to fix the issue. A mesh system uses multiple access points to blanket your home in signal, and automatically switches your devices to the closest one, so you’ll no longer drop Zoom calls when you walk into that one corner of your basement. This system comes with three points, which should be plenty for most homes, but you can easily add more.
As AI agents become more capable, developers are increasingly asking them to take on complex tasks requiring work that spans hours, or even days. However, getting agents to make consistent progress across multiple context windows remains an open problem.
The core challenge of long-running agents is that they must work in discrete sessions, and each new session begins with no memory of what came before. Imagine a software project staffed by engineers working in shifts, where each new engineer arrives with no memory of what happened on the previous shift. Because context windows are limited, and because most complex projects cannot be completed within a single window, agents need a way to bridge the gap between coding sessions.
We developed a two-fold solution to enable the
Claude Agent SDK
to work effectively across many context windows: an
initializer agent
that sets up the environment on the first run, and a
coding agent
that is tasked with making incremental progress in every session, while leaving clear artifacts for the next session. You can find code examples in the accompanying
quickstart.
The long-running agent problem
The Claude Agent SDK is a powerful, general-purpose agent harness adept at coding, as well as other tasks that require the model to use tools to gather context, plan, and execute. It has context management capabilities such as compaction, which enables an agent to work on a task without exhausting the context window. Theoretically, given this setup, it should be possible for an agent to continue to do useful work for an arbitrarily long time.
However, compaction isn’t sufficient. Out of the box, even a frontier coding model like Opus 4.5 running on the Claude Agent SDK in a loop across multiple context windows will fall short of building a production-quality web app if it’s only given a high-level prompt, such as “build a clone of
claude.ai
.”
Claude’s failures manifested in two patterns. First, the agent tended to try to do too much at once—essentially to attempt to one-shot the app. Often, this led to the model running out of context in the middle of its implementation, leaving the next session to start with a feature half-implemented and undocumented. The agent would then have to guess at what had happened, and spend substantial time trying to get the basic app working again. This happens even with compaction, which doesn’t always pass perfectly clear instructions to the next agent.
A second failure mode would often occur later in a project. After some features had already been built, a later agent instance would look around, see that progress had been made, and declare the job done.
This decomposes the problem into two parts. First, we need to set up an initial environment that lays the foundation for
all
the features that a given prompt requires, which sets up the agent to work step-by-step and feature-by-feature. Second, we should prompt each agent to make incremental progress towards its goal while also leaving the environment in a clean state at the end of a session. By “clean state” we mean the kind of code that would be appropriate for merging to a main branch: there are no major bugs, the code is orderly and well-documented, and in general, a developer could easily begin work on a new feature without first having to clean up an unrelated mess.
When experimenting internally, we addressed these problems using a two-part solution:
Initializer agent: The very first agent session uses a specialized prompt that asks the model to set up the initial environment: an
init.sh
script, a claude-progress.txt file that keeps a log of what agents have done, and an initial git commit that shows what files were added.
Coding agent: Every subsequent session asks the model to make incremental progress, then leave structured updates.
1
The key insight here was finding a way for agents to quickly understand the state of work when starting with a fresh context window, which is accomplished with the claude-progress.txt file alongside the git history. Inspiration for these practices came from knowing what effective software engineers do every day.
Environment management
In the updated
Claude 4 prompting guide
, we shared some best practices for multi-context window workflows, including a harness structure that uses “a different prompt for the very first context window.” This “different prompt” requests that the initializer agent set up the environment with all the necessary context that future coding agents will need to work effectively. Here, we provide a deeper dive on some of the key components of such an environment.
Feature list
To address the problem of the agent one-shotting an app or prematurely considering the project complete, we prompted the initializer agent to write a comprehensive file of feature requirements expanding on the user’s initial prompt. In the
claude.ai
clone example, this meant over 200 features, such as “a user can open a new chat, type in a query, press enter, and see an AI response.” These features were all initially marked as “failing” so that later coding agents would have a clear outline of what full functionality looked like.
{
"category": "functional",
"description": "New chat button creates a fresh conversation",
"steps": [
"Navigate to main interface",
"Click the 'New Chat' button",
"Verify a new conversation is created",
"Check that chat area shows welcome state",
"Verify conversation appears in sidebar"
],
"passes": false
}
We prompt coding agents to edit this file only by changing the status of a passes field, and we use strongly-worded instructions like “It is unacceptable to remove or edit tests because this could lead to missing or buggy functionality.” After some experimentation, we landed on using JSON for this, as the model is less likely to inappropriately change or overwrite JSON files compared to Markdown files.
Incremental progress
Given this initial environment scaffolding, the next iteration of the coding agent was then asked to work on only one feature at a time. This incremental approach turned out to be critical to addressing the agent’s tendency to do too much at once.
Once working incrementally, it’s still essential that the model leaves the environment in a clean state after making a code change. In our experiments, we found that the best way to elicit this behavior was to ask the model to commit its progress to git with descriptive commit messages and to write summaries of its progress in a progress file. This allowed the model to use git to revert bad code changes and recover working states of the code base.
These approaches also increased efficiency, as they eliminated the need for an agent to have to guess at what had happened and spend its time trying to get the basic app working again.
Testing
One final major failure mode that we observed was Claude’s tendency to mark a feature as complete without proper testing. Absent explicit prompting, Claude tended to make code changes, and even do testing with unit tests or
curl
commands against a development server, but would fail recognize that the feature didn’t work end-to-end.
In the case of building a web app, Claude mostly did well at verifying features end-to-end once explicitly prompted to use browser automation tools and do all testing as a human user would.
Screenshots taken by Claude through the Puppeteer MCP server as it tested the claude.ai clone.
Providing Claude with these kinds of testing tools dramatically improved performance, as the agent was able to identify and fix bugs that weren’t obvious from the code alone.
Some issues remain, like limitations to Claude’s vision and to browser automation tools making it difficult to identify every kind of bug. For example, Claude can’t see browser-native alert modals through the Puppeteer MCP, and features relying on these modals tended to be buggier as a result.
Getting up to speed
With all of the above in place, every coding agent is prompted to run through a series of steps to get its bearings, some quite basic but still helpful:
Run
pwd
to see the directory you’re working in. You’ll only be able to edit files in this directory.
Read the git logs and progress files to get up to speed on what was recently worked on.
Read the features list file and choose the highest-priority feature that’s not yet done to work on.
This approach saves Claude some tokens in every session since it doesn’t have to figure out how to test the code. It also helps to ask the initializer agent to write an init.sh script that can run the development server, and then run through a basic end-to-end test before implementing a new feature.
In the case of the claude.ai clone, this meant that the agent always started the local development server and used the Puppeteer MCP to start a new chat, send a message, and receive a response. This ensured that Claude could quickly identify if the app had been left in a broken state, and immediately fix any existing bugs. If the agent had instead started implementing a new feature, it would likely make the problem worse.
Given all this, a typical session starts off with the following assistant messages:
[Assistant] I'll start by getting my bearings and understanding the current state of the project.
[Tool Use] <bash - pwd>
[Tool Use] <read - claude-progress.txt>
[Tool Use] <read - feature_list.json>
[Assistant] Let me check the git log to see recent work.
[Tool Use] <bash - git log --oneline -20>
[Assistant] Now let me check if there's an init.sh script to restart the servers.
<Starts the development server>
[Assistant] Excellent! Now let me navigate to the application and verify that some fundamental features are still working.
<Tests basic functionality>
[Assistant] Based on my verification testing, I can see that the fundamental functionality is working well. The core chat features, theme switching, conversation loading, and error handling are all functioning correctly. Now let me review the tests.json file more comprehensively to understand what needs to be implemented next.
<Starts work on a new feature>
Agent failure modes and solutions
Problem
Initializer Agent Behavior
Coding Agent Behavior
Claude declares victory on the entire project too early.
Set up a feature list file: based on the input spec, set up a structured JSON file with a list of end-to-end feature descriptions.
Read the feature list file at the beginning of a session. Choose a single feature to start working on.
Claude leaves the environment in a state with bugs or undocumented progress.
An initial git repo and progress notes file is written.
Start the session by reading the progress notes file and git commit logs, and run a basic test on the development server to catch any undocumented bugs. End the session by writing a git commit and progress update.
Claude marks features as done prematurely.
Set up a feature list file.
Self-verify all features. Only mark features as “passing” after careful testing.
Claude has to spend time figuring out how to run the app.
Write an
init.sh
script that can run the development server.
Start the session by reading
init.sh
.
Summarizing four common failure modes and solutions in long-running AI agents.
Future work
This research demonstrates one possible set of solutions in a long-running agent harness to enable the model to make incremental progress across many context windows. However, there remain open questions.
Most notably, it’s still unclear whether a single, general-purpose coding agent performs best across contexts, or if better performance can be achieved through a multi-agent architecture. It seems reasonable that specialized agents like a testing agent, a quality assurance agent, or a code cleanup agent, could do an even better job at sub-tasks across the software development lifecycle.
Additionally, this demo is optimized for full-stack web app development. A future direction is to generalize these findings to other fields. It’s likely that some or all of these lessons can be applied to the types of long-running agentic tasks required in, for example, scientific research or financial modeling.
Acknowledgements
Written by Justin Young. Special thanks to David Hershey, Prithvi Rajasakeran, Jeremy Hadfield, Naia Bouscal, Michael Tingley, Jesse Mu, Jake Eaton, Marius Buleandara, Maggie Vo, Pedram Navid, Nadine Yasser, and Alex Notov for their contributions.
This work reflects the collective efforts of several teams across Anthropic who made it possible for Claude to safely do long-horizon autonomous software engineering, especially the code RL & Claude Code teams. Interested candidates who would like to contribute are welcome to apply at
anthropic.com/careers
.
Footnotes
1. We refer to these as separate agents in this context only because they have different initial user prompts. The system prompt, set of tools, and overall agent harness was otherwise identical.
Show HN: Pulse 2.0 – Live co-listening rooms where anyone can be a DJ
Just as growth in the “real” economy of material products and services has been decelerating towards contraction, so aggregates of financial wealth have carried on increasing relentlessly.
Since the widening
disequilibrium
between the monetary and the material must eventually crash the financial system, the probability is that notional wealth will reach its peak
at the same moment
at which the monetary system collapses.
We can see this unfolding effect in microcosm in the United Kingdom, where the most recent
official calculation
put national “net worth” at a near-record £12.2 trillion, or 450% of GDP. Yet you don’t need SEEDS analysis to know that the British economy itself is at an advanced stage of disintegration.
Two key factors explain the apparent paradox between soaring wealth and the onset of increasingly chaotic economic decline.
First, every failed effort made to stem
material
economic decline using
monetary
tools increases wealth, as it is measured financially.
Second, most of this wealth is purely notional, in the sense that
none
of its aggregates are capable of conversion into material value. Put another way, very little of the world’s supposedly enormous wealth actually
exists
in any meaningful sense.
A very possible
final scenario
is that, after an initial correction caused by a dawning realization of economic crisis, asset markets will rebound to a last peak before entering outright collapse.
To make sense of these issues, we need a clear understanding of the nature of money and wealth in relation to material economic supply.
1
As many readers will know, there’s no great mystery about the ending and reversal of material economic expansion.
Briefly stated, the “real” economy of physical goods and services is only proxied – and with ever-diminishing fidelity – in the published aggregates of financial flow.
Far from being a measure of the supply of material value to the system, gross domestic product is nothing more than a summation of
transactional activity
taking place in the economy. It’s perfectly possible, indeed commonplace, for money to change hands without any material economic value being added.
The way in which material value is supplied to society is, in principle, comparatively straightforward. In a continuous process of creation, consumption, disposal and replacement, energy is used to
convert
other natural resources (including minerals, non-metallic mining products, biomass and water) into products, and into those artefacts and infrastructures without which no worthwhile service can be supplied.
This is a
dual equation
in which, as energy is used for the conversion of raw materials into products, so energy itself is converted from a dense to a diffuse state. This makes energy-to-mass density, and the portability of energy, important considerations in the resource conversion process of economic supply.
Given that energy is used in the creation, operation, maintenance and replacement of energy-supplying infrastructures, we can state that “whenever energy is accessed for our use, some of this energy is
always
consumed in the access process, and is not available for any other economic purpose”.
This proportionate “consumed in access” component is measured in SEEDS as the
Energy Cost of Energy
. ECoEs have long been on an exponentially climbing trend, and have risen from 2.0% in 1980 to 11.3% today.
Renewables, and for that matter nuclear power as well,
cannot
materially slow, let alone reverse, the relentless rise in ECoEs caused by the depletion of oil, natural gas and coal. Neither can technology halt this trend, since the potential of technology, far from being infinite, is bounded by the limits imposed by the laws of physics.
The other determinant of the supply of physical economic value is the rate at which non-energy raw materials are converted into economic value through the use of energy. This
conversion ratio
is on a gradually declining trajectory, because resource depletion is occurring at a rate slightly more rapid than that at which the broad swathe of conversion methodologies can advance.
On this basis, global material prosperity has grown by 25% since 2004, which is
nowhere near
claimed “growth” of 96% in real GDP over that period. Moreover, the 25% rise in aggregate prosperity has been matched by the rise in population numbers over those twenty years.
The ongoing rate of deceleration is such that aggregate material prosperity is projected to be 17% lower in 2050 than it is now, which is likely to make the “average” person about 31% poorer than he or she is today.
At the same time, the costs of energy-intensive necessities – including food, water, accommodation, domestic energy, essential transport and distribution – are rising markedly. The “cost of living crisis”, far from being the temporary phenomenon that the word “crisis” is intended to imply, is a firmly established trend.
Since all of these process are both knowable and incapable of being “fixed”,
why is monetary value continuing to increase?
The answer lies in the fundamental nature of money, and of how the monetary relates to the material.
2
As we know, no amount of money, irrespective of its format, would be of the slightest use to a person stranded on a desert island, or cast adrift in a lifeboat. If this castaway had an extremely large amount of money, his or her only (and very dubious) comfort would be the prospect of ‘dying rich’.
That’s an appropriate analogy for a world destined to experience financial collapse at the moment of record paper wealth.
The financial system
, like our castaway,
is going to ‘die rich’
.
There’s an instructive twist that we can add to the narrative of the castaway. At his or her greatest extreme of privation, a package is seen descending on a parachute. Opening this package with avid hopes of food or other life-saving material supply, the castaway finds only very large quantities of banknotes, gold coins and precious stones, all of which are wholly valueless in his or her predicament.
Decision-makers in society have made
this exact same mistake
, pouring huge amounts of money into a world whose deficiencies are material. They have done this, not out of idiocy or dishonesty, but because, whilst they cannot be seen to be ‘doing nothing’,
no other possible policy response exists
.
Governments and central banks can create money and wealth in almost limitless quantities, but they cannot similarly conjure energy, or any other material resource, into existence at the touch of a key-stroke.
The essential point, of course, is that, as our castaway swiftly discovers, money has no
intrinsic
worth. It commands value
only
in terms of those physical goods and services for which it can be exchanged. Money is thus an “exercisable claim” on the material.
Warren Buffett alluded to this when he said that “[t]he way I see it is that my money represents an enormous number of claim checks on society. It is like I have these little pieces of paper that I can turn into consumption”.
Various conclusions follow from this
principle of money as claim
. One of the most important, as regular readers will know, is the imperative need to think conceptually in terms of
two economies
. One of these is the “real” economy of material products and services, and the other is the parallel “financial” economy of money, transactions and credit.
Another is the
absolute futility
of any attempt to explain the economy by disregarding the material and concentrating
entirely
on money. This fallacious line of thinking leads inevitably to the deranged proposition of ‘infinite, exponential economic growth on a finite planet’.
3
None
of this means, though, that money is “unimportant”, or that a collapse of the financial system would have no adverse consequences for material economic prosperity.
The most effective approach to economics doesn’t involve the disregard of the material
or
of money.
Rather, what we need to do is to calibrate the physical economy such that we can
benchmark the monetary against the material
. This enables us to avoid the futility of measuring the monetary only against itself.
It’s true that, at the moment when the monetary system collapses, we will still have the same amounts of the energy and other natural resources which are the basis of material economic prosperity.
But money is a
critical enabler
in the processes by which we use energy to convert raw materials into products, artefacts and infrastructures.
This is why not even the poorest person can view the impending collapse of the financial system with equanimity. Without money, how can nations trade products and resources, and how can the individual conduct his or her daily affairs?
Since money – unlike energy and raw materials – is a human construct, it’s perfectly possible, at least in theory, for us to create a new (and perhaps more intelligently-designed) form of money to replace the old.
But the
chaos
that monetary collapse will cause
is hardly capable of over-statement
.
4
Within our overall understanding of the principle of
money as claim
, money itself divides into two functional categories, which are the
flow
of money in the economy and the
stock
of monetary claims set aside for exercise in the future.
This “stock of claims” further subdivides into two components. One of these is
immediate
money, which can be spent without having to go through any preliminary enabling process. Most of this “immediate” money exists as
fiat
currencies, since it’s difficult to buy our groceries or pay our electricity bills using precious metals or cryptocurrencies.
But by far the largest component of the stock of claims is
inferred
rather than immediate. This “inferred” money exists as stocks, bonds, real estate and numerous other asset classes.
Before this claim value can be spent, it must be
monetised
– converted, that is, from an inferred to an immediate state. This means, simply stated, that the assets which comprise
inferred value
must be sold before they can be spent.
5
The aggregates of these
inferred
forms of wealth are enormous. Global stock markets, for instance, currently stand at about 160% of world GDP, which is far higher than this metric was in 2007, on the eve of the global financial crisis (114%). Total debt is about 240% of global GDP, and broader financial assets, in those countries which report this information, are about 470%.
If, to these, were added other asset classes, including real estate, promised pensions, precious metals, cryptos and derivatives, we could undoubtedly calculate that global wealth is at all-time record highs.
All of this is a very far cry from the oil crisis years of the 1970s. In 1975, stock markets equated to only 27% of world GDP. At its nadir, in January of that year, the S&P averaged just 72.6, from which point the index has advanced almost continuously – with few and brief interruptions – to a level today of about 6850.
Fig. 1
This has, in fact, been a near-uninterrupted, fifty-year progression. This road from “
once-in-a-lifetime cheap
” to the vastly higher valuations of today certainly merits some reflection.
It transpires that very little of this accession of paper wealth has happened by accident.
Starting in 1975, the initial advance in stock markets did follow processes that can be ascribed to market forces alone. In that year, the astute investor had little to lose, and much to gain, by putting his or her money, itself subject to severe inflation, into stocks.
Together, an economic rebound from the oil-crises-slump of the 1970s, the gradual taming of inflation and the euphoria created by the policies of the new “neoliberal” incumbencies combined to help to drive markets sharply higher during much of the “decade of greed” of the 1980s.
But everything changed in October 1987.
On “Black Monday”, the markets crashed, with the Dow losing 508 points, or 22.6%, in a matter of hours.
What was
really
significant, though, was that the authorities stepped in to shore up the markets. One of the most important players was the Federal Reserve, which had itself been created in 1913 in response to another such crash, the
Knickerbocker crisis
of 1907.
6
The authorities might well have been wise to have intervened as they did in 1987, but, in doing so, they conveyed the strong impression that, if ever things once more went badly enough wrong, they could
again
be counted upon to ride to the rescue like the fabled 7th Cavalry.
This back-stopping – variously known over the years as the “Greenspan put”, the “Bernanke put”, the “Yellen put” or, more generically, the “Fed put” – remained implicit until 2008.
Then, with the swift, no-holds-barred response to the global financial crisis, the authorities made their support for the market
explicit
.
Some felt at the time that these interventions were necessary and proportionate, others that they “bailed out Wall Street at the expense of Main Street”. Both points of view had their shares of validity.
But the most astute observers fretted instead about what is called
moral hazard
.
In the normal course of events, as envisaged by free market purists, the authorities do not intervene in the markets. If things go well, the wise (or simply fortunate) investor makes big profits but, if things go badly, the reckless (or unlucky) investor gets wiped out. Thus the antithetical forces of “fear and greed” are kept in balance.
Intervention dangerously upsets this balanc
e. An investor rescued once naturally assumes that, if things go badly enough wrong again, another bail-out is certain to follow. This provides an enormous incentive to risk-taking, and undermines the important restraint exercised, through prudence, by fear.
7
Behind all of this, though – and seldom noticed by observers – lies the fact that
all
“values” routinely ascribed to wealth aggregates are fundamentally bogus.
At no point
can
any
of the reported aggregates of wealth be monetized. The only people to whom the stock market could ever be sold
in its entiret
y are
the same people to whom it already belongs
. The same applies to the global or national housing stock, and to every other asset class.
Whenever we’re told that huge amounts of value – in October 1987, for instance, $1.7 trillion – have been “wiped out” by a market fall, we’re being asked to disregard the fact that
at no point
, either before or after the event, could the
whole
market have been sold at its supposed value. The same applies to any statement of how much wealth billionaires have “gained” through rises in the market.
The £12.2tn official calculation of British aggregate “net worth” is similarly meaningless, in the absence of anyone who actually
has
£12.2tn to spend (and is also daft enough to invest it in Britain)
The
fatal error
made here is that of using
marginal
transaction prices to put a “value” on
aggregate
quantities of assets.
We might think that, were all global stock markets to fall to zero, about $180tn of wealth would have been eliminated.
In fact, that supposed “value” was only ever notional, and
never existed
in any meaningful form in the first place, because at no point was it ever capable of monetization.
8
This inability
ever
to monetize even a significant proportion of
this
inferred wealth enables commentators to write lurid, fact-free articles about aggregate wealth being “destroyed” or “boosted”.
But it also enables the authorities to pursue their cherished “wealth effect” without much danger of all of this largesse being converted into
immediate
monetary value, and then spent in ways that trigger runaway inflation in the economy.
Central banks’ use of QE is a case in point. So long as this liquidity injection was
contained within the capital markets
, it could not cross the boundary into spendable money and trigger severe inflation. During the pandemic of 2020, though, when QE was channelled not into the markets but
directly to households
, severe consumer price inflation
did
indeed follow.
9
What we have been exploring here is a paradox that is no paradox at all.
The world will keep setting new wealth records until the financial system collapses
.
Investors have lived with
supportive intervention
from the authorities ever since 1987, which means that very few of them have ever experienced anything else. Throughout this period – and certainly since official support became explicit in 2008 – the “momentum trade” has been the only game in town. It’s been like gambling in a casino where the house stacks the deck in favour of the punter.
So ingrained has this thinking become that there are likely to be many still determined to “buy the dip” even as the financial system goes finally into the blender. This, in short, is why wealth is destined to collapse – swiftly,
not
gradually – from an all-time peak.
Our necessary insight here is that, since any value contained in money exists
only
as a “claim” on a material economy
that is now contracting
, there must come a point of
fatal disequilibrium
between claim and substance
.
The sheer scale and complexity of the aggregates of claim stock are now so extreme that the authorities will be powerless to backstop the next big crash.
It’s scant consolation to know that these enormous aggregates never, in any meaningful sense, actually existed in the first place.
Thus understood, it might not seem to matter all that much if aggregate (and individual) values collapse. Your house, for instance, will still fulfil its essential function of providing somewhere to live, even if its supposed value slumps from $1m to $200k. Even if you’d decided to sell at the highest price, buying a replacement would have been equally costly.
But this comfort only applies
if you hadn’t used the property as security for a large mortgage
.
And the financial system as a whole
has done exactly that
. The entirety of the system is enormously cross-collateralized, and this is where the destruction of “meaningless” aggregate asset values
becomes enormously meaningful
.
The real comfort, if any is to be found, is that anyone who
can
find a way of preserving value will have the opportunity of buying
utility value
at pennies on the dollar. The term “utility” is the watch-word here, because essentials
will remain essential
even as society is picking over the wreckage of discretionary sectors.
Fig. 3
Man behind in-flight Evil Twin WiFi attacks gets 7 years in prison
Bleeping Computer
www.bleepingcomputer.com
2025-11-28 18:25:28
A 44-year-old man was sentenced to seven years and four months in prison for operating an "evil twin" WiFi network to steal the data of unsuspecting travelers at various airports across Australia. [...]...
A 44-year-old man was sentenced to seven years and four months in prison for operating an “evil twin” WiFi network to steal the data of unsuspecting travelers during flights and at various airports across Australia.
The man, an Australian national, was
charged in July 2024
after Australian authorities had confiscated his equipment in April and confirmed that he was engaging in malicious activities during domestic flights and at airports in Perth, Melbourne, and Adelaide.
Specifically, the man was setting up an access point with a ‘WiFi Pineapple’ portable wireless access device and used the same name (SSID) for the rogue wireless network as the legitimate ones in airports.
Users connecting to the malicious access point were directed to a phishing webpage that stole their social media account credentials.
The man used these credentials to access women's accounts to monitor their communications and steal private images and videos.
"Forensic analysis of data and the seized devices identified thousands of intimate images and videos, personal credentials belonging to other people, and records of fraudulent WiFi pages," the
Australian Federal Police
(AFP) says.
"The day after the search warrant, the man deleted 1752 items from his account on a data storage application and unsuccessfully tried to remotely wipe his mobile phone."
After seizing his luggage on April 19, 2024, the man obtained unauthorized access to his employer’s laptop to access information on confidential meetings between his employer and AFP’s investigators.
Eventually, the man pleaded guilty to:
Five counts of causing unauthorized access or modification of restricted data
Three counts of attempting to cause unauthorized access or modification of restricted data
One count of stealing
Two counts of unauthorized impairment of electronic communication
One count of possessing or controlling data with the intent to commit a serious offense
One count of failure to comply with an order under section 3LA(2)
Two counts of attempted destruction of evidence
AFP Commander Renee Colley warned the public about the risks of free WiFi, advising the use of virtual private networks (VPNs), strong passwords, and disabling file-sharing and automatic WiFi connectivity.
“Evil twin” WiFi attacks are not common in the wild, but they are practically possible and may go unnoticed and unreported in public spaces.
Captive portals on free WiFi access points should be treated with extra caution and dismissed when requesting personal account information for logging in.
It's budget season! Over 300 CISOs and security leaders have shared how they're planning, spending, and prioritizing for the year ahead. This report compiles their insights, allowing readers to benchmark strategies, identify emerging trends, and compare their priorities as they head into 2026.
Learn how top leaders are turning investment into measurable impact.
Nobody really understands how TPUs work…and neither do we! So we wanted to make this because we wanted to take a shot and try to guess how it works–from the perspective of complete novices!
We wanted to do something very challenging to prove to ourselves that we can do anything we put our mind to. The reasoning for why we chose to build a TPU specifically is fairly simple:
None of us have real professional experience in hardware design, which, in a way, made the TPU even more appealing since we weren't able to estimate exactly how difficult it would be. As we worked on the initial stages of this project, we established a strict design philosophy: ALWAYS TRY THE HACKY WAY. This meant trying out the "dumb" ideas that came to our mind first BEFORE consulting external sources. This philosophy helped us make sure we weren't reverse engineering the TPU, but rather
re-inventing it
, which helped us derive many of the key mechanisms used in the TPU ourselves.
We also wanted to treat this project as an exercise to code without relying on AI to write for us, since we felt that our initial instinct recently has been to reach for these AI tools whenever we faced a slight struggle. We wanted to cultivate a certain
style of thinking
that we could take forward with us and use in any future endeavours to think through difficult problems. Inspired by Sholto Douglas’s message in his
YouTube short
— that you don’t need permission to make great things — we kept shipping and learning in public.
[1]
A TPU is an application specific integrated circuit (ASIC) - basically a custom chip - designed by Google to make inferencing (using) and training ML models faster and more efficient. Whereas a GPU can be used to render frames AND run ML workloads, a TPU can only perform math operations, allowing it to be better at what it's designed for. Naturally, trying to master a single task is much easier and will yield better results than trying to master multiple tasks and the TPU strongly employs this philosophy.
Quick primer on hardware design:
In hardware, the unit of time we're dealing with is called a clock cycle. This is an arbitrary period of time that we can set, as developers, to meet our requirements. Generally, a single clock cycle can range from 1 picosecond (ps) to 1 nanosecond (ns) and any operations we run will be executed BETWEEN clock cycles.
Clock cycle timing diagram showing how operations are synchronized in hardware
The language we use to describe hardware is called Verilog. It's a hardware description language that allows us to describe the behaviour of a given hardware module (similar to functions in software), but instead of executing as a program, it synthesizes into boolean logic gates (AND, OR, NOT, etc.) that can be combined to build the digital logic for any chip we want. Here's a simple example of an addition in Verilog:
module add (input wire clk,
// reset signal to reset the moduleinput wire rst,
// registers to hold the input and output valuesinput reg a,
input reg b,
output reg c
);
always @(posedgeclk)begin// everything in this block will be executed every clock cycleif(rst)begin// reset the output to 0 when the reset signal is high
c <=0;
endelsebegin// add the two inputs and store the result in the output
c <= a + b;
endend
endmodule
In the example above, the value of the signal b at the next clock cycle is set to the current value of the signal a. You'll find that in most cases, signals (variables) are updated in sequential clock cycles, as opposed to immediate updates like you would find in software design.
Specifically, the TPU is very efficient at performing matrix multiplications, which make up 80-90% of the compute operations in transformers (up to 95% in very large models) and 70-80% in CNNs. Each matrix multiplication represents the calculation for a single layer in an MLP, and in deep learning, we have many of these layers, making TPUs increasingly efficient for larger models.
When we started this project, all we knew was that the equation y = mx + b is the foundational building block for neural networks. However, we needed to fully UNDERSTAND the math behind neural networks to build other modules in our TPU. So before we started writing any code, each of us worked out the math of a simple 2 -> 2 -> 1 multi-layer perceptron (MLP).
Architecture of our 2→2→1 multi-layer perceptron for solving the XOR problem
Why XOR?
The reason we chose this specific network is because we were targeting inference and training for the XOR problem (the "hello world" of neural networks). The XOR problem is one of the simplest problems a neural network can solve. All other gates (AND, OR, etc) can predict the outputs from its inputs using just one linear line (one neuron) to separate which inputs correspond to a 0 and which ones correspond to a 1. But to classify all XOR, an MLP is needed, since it requires curved decision boundaries, which can't be achieved with ONLY linear equations. For a geometric and first-principles treatment, the free book
Understanding Deep Learning
is excellent.
OR and XOR decision boundaries
Batching and dimensions
Now, say we want to do continuous inference (i.e. self driving car making multiple predictions a second). That would imply that we're sending multiple pieces of data at once. Since data is inherently multidimensional and has many features, we would have matrices with very large dimensions. However, the XOR problem simplifies the dimensions for us, as there are only two features (0 or 1) and 4 possible pieces of input data (four possible binary combinations of 0 and 1). This gives us a 4x2 matrix, where 4 is the number of rows (batch size) and 2 is the number of columns (feature size).
The XOR input matrix and target outputs:
Each row represents one of the four possible XOR inputs, and the output vector shows the expected XOR results
Another simplification we're making for our systolic array example here is that we'll use a 2x2 instead of the 256x256 array used in the TPUv1. However, the math is still faithful so nothing is actually dumbed down, rather scaled down instead.
The first step in the equation is multiplying m with x, which, in matrix form, would be
.
More formally:
where
is our input matrix,
is our weight matrix, and
is our bias vector
How can we perform matrix multiplication in hardware? Well, we can use a unit called the systolic array!
Systolic array and PEs
The heart of a TPU is a unit called the systolic array.
[2]
It consists of individual building blocks called Processing Elements (PE) which are connected together in a grid-like structure. Each PE performs a multiply-accumulate operation, meaning it multiplies an incoming input X with a stationary weight W
[3]
and adds it to an incoming accumulated sum, all in the same clock cycle.
Processing Element (PE) architecture showing multiply-accumulate operation (without load weight and start flags)
always_ff @(posedgeclk or posedgerst)beginif(rst)begininput_out<=0;
psum_out<=0;
weight_reg<=0;
endelseif(load_weight)beginweight_reg<=weight;
endelseif(start)begininput_out<=input_in;
// the main multiply-accumulate operationpsum_out<=(input_in*weight_reg)+psum_in;
endend
Systolic matrix multiplication
When these PEs are connected together, they can be used to perform matrix multiplication systolically, meaning multiple elements of the output matrix can be calculated every clock cycle. The inputs enter the systolic array from the left and move to the neighbouring PE to the right, every clock cycle. The accumulated sums start with the multiplication output from the first row of PEs, move downwards, and get added to the products of each successive PE, until they up at the last row of PEs where they become an element of the output matrix.
Systolic array architecture showing how PEs are connected to perform matrix multiplication
Because of this single unit (and the fact that matrix multiplications dominate the computations performed in models), TPUs can very easily inference and train any model.
Worked example
Now let's walk through the example of our XOR problem:
Our systolic array takes two inputs: the input matrix and the weight matrix. For our XOR network, we initialize with the following weights and biases:
Layer 1 parameters:
Layer 2 parameters:
Input and weight scheduling
To input our input batch within the systolic array, we need to:
Rotate our X matrix by 90 degrees
Matrix rotation by 90 degrees to prepare for systolic array input
STAGGER the inputs (delay each row by 1 clock cycle)
[4]
Input matrix staggering pattern for systolic array processing
To input our weight matrix: we need to:
Stagger the weight matrix (similar to the inputs)
Weight matrix staggering pattern for systolic array processing
Transpose it!
Weight matrix transposition for correct mathematical alignment
Note that the rotating and staggering don't have any mathematical significance — they are simply required to make the systolic array work. The transpoing too is just for mathematical bookkeeping – it's required to make the matrix math work because of how we set up our weight pointers within the neural network drawing.
Staggering and FIFOs
To perform the staggering, we designed near-identical accumulators for the weights and inputs that would sit above and to the left of the systolic array, respectively.
Since the activations are fed into the systolic array one-by-one, we thought a first-in-first-out queue (FIFO) would be the optimal data storage option. There was a slight difference between a traditional FIFO and the accumulators we built, however. Our accumulators had 2 input ports — one for writing weights manually to the FIFO and one for writing the previous layer's outputs from the activation modules BACK into the input FIFOs (the previous layer's outputs are inputs for the current layer).
We also needed to load the weights in a similar fashion for every layer, so we replicated the logic for the weight FIFOs, without the second port.
Systolic array matrix multiplication
clk
0
Bias and activation
The next step in the equation is adding the bias. To do this in hardware, we need to create a bias module under each column of the systolic array. We can see that as the sums move out of the last row within the systolic array, we can immediately stream them into our bias modules to compute our pre-activations.
We will denote these values with the variable Z.
The bias vector
is broadcast across all rows of the matrix — meaning it's added to each row of
Now our equation is starting to look a lot like what we've learned in high school –but just in multidimensional form, where each column that streams out of the systolic array represents its own feature!
Next we have to apply the activation, for which we chose Leaky ReLU.
[5]
This is also an element-wise operation, similar to the bias, meaning we need an activation module under every bias module (and by proxy under every column of the systolic array) and we can stream the outputs of our bias modules into the activation modules immediately.
We will denote these post-activation values with H
.
The Leaky ReLU function applies element-wise:
where
is our leak factor. For matrices, this applies to each element independently.
For our XOR example, let's see how Layer 1 processes the data. First, the systolic array computes
:
Then bias is added:
Finally, LeakyReLU is applied element-wise:
Negative values are multiplied by 0.5, positive values pass through unchanged.
Systolic array with bias and leaky ReLU
clk
0
Pipelining
Now you might be asking – why don't we merge the bias term and the activation term in one clock cycle? Well, this is because of something called pipelining! Pipelining allows multiple operations to be executed simultaneously across different stages of the TPU —instead of waiting for one complete operation to finish before starting the next, you break the work into stages that can overlap. Think of it like an assembly line: while one worker (activation module) processes a part, the previous worker (bias module) is already working on the next part. This keeps all of the modules busy rather than having them sit idle waiting for the previous stage to complete. It also affects the speed at which we can run our TPU — if we have one module that tries to squeeze many operations in a single cycle, our clock speed will be bottlenecked by that module, as the other modules can only run as fast as that single module. Therefore, it's efficient and best practice to split up operations into individual clock cycles as much as possible.
Pipelining stages showing how operations overlap across clock cycles
Another mechanism we used to run our chip as efficiently as possible, was a propagating "start" signal, which we called a travelling chip enable (denoted by the purple dot). Because everything in our design was staggered, we realized that we could very elegantly assert a start signal for a single clock cycle at the first accumulator and have it propagate to neighbouring modules exactly when they needed to be turned on.
This would extend into the systolic array and eventually the bias and activation modules, where neighbouring PEs and modules, moving from the top left to the bottom right, were turned on in consecutive clock cycles. This ensured that every module was only performing computations when it was required to and wasn't wasting power in the background.
Double buffering
Now, we know that starting a new layer means we must compute the same
using a new weight matrix. How can we do this if our systolic array is weight-stationary? How can we change the weights?
While thinking about this problem, we came across the idea of double buffering, which originates from video games. The reason why double buffering exists is to prevent something called screen tearing on your monitor. Ultimately, pixels take time to load and we'd like to "hide away" that time somehow. And if you paid attention, this is the exact same problem we're currently facing with the systolic array. Fortunately, video game designers have already come up with a solution for this problem. By adding a second "shadow" buffer, which holds the weights of the next layer while the current layer is being computed on, we can load in new weights during computation, cutting the total clock cycle count in half.
To make this work, we also needed to add some signals to move the data. First, we needed a signal to indicate when to switch the weights in the shadow buffer and the active buffer. We called this signal the "switch" signal (denoted by the blue dot) and it copied the values in the shadow buffer to the active buffer. It propagated from the top left of the systolic array to the bottom right (the same path as the travelling chip enable, but only within the systolic array). We then needed one more signal to indicate when we wanted to move the weights down by one row and we called this the "accept" flag (denoted by the green dot) because each row is ACCEPTING a new set of weights. This would move the new weights into the top row of the systolic array, as well as each row of weights down into the next row of the systolic array. These two control flags worked in tandem to make our double buffering mechanism work.
Double buffering in the systolic array
If you haven't already noticed, this allows the systolic array to do something powerful…continuous inference!!! We can continuously stream in new weights and inputs and compute forward pass for as many layers as we want. This touches into a core design philosophy of the systolic array: we want to maximize PE usage.
We always want to keep the systolic array fed!
For Layer 2, the outputs from Layer 1 (
) now become our inputs:
Adding bias and applying activation:
All values are positive, so they pass through unchanged. These are our final predictions for the XOR problem!
Forward pass walkthrough (with double buffering)
clk
0
Control unit and ISA
Our final step for inference was making a control unit to use a custom instruction set (ISA) to assert all of our control flags and load data through a data bus. Including the data bus, our ISA was 24 bits long and it made our testbench more elegant as we could pass a single string of bits every clock cycle, rather than individually setting multiple flags.
We then put everything together and got inference completely working! This was a big milestone for us and we were very proud about what we had accomplished.
Backpropagation and training
Ok we've solved inference — but what about training? Well here's the beauty: We can use the same architecture we use for inference for training! Why? Because training is just matrix multiplications with a few extra steps.
Here's where things get really exciting. Let's say we just ran inference on the XOR problem and got a prediction that looks something like [0.8, 0.3, 0.1, 0.9] when we actually wanted [1, 0, 0, 1]. Our model is performing poorly! We need to make it better. This is where training comes in. We're going to use something called a loss function to tell our model exactly how poorly it's doing. For simplicity, we chose Mean Squared Error (MSE) — think of it like measuring the "distance" between what we predicted and what we actually wanted, just like how you might measure how far off target your basketball shot was.
Let's denote the loss with L.
where
is the target output,
is our prediction, and
is the number of samples
For our XOR example, with predictions
and targets
:
This loss value tells us how far off our predictions are from the true XOR outputs.
So right after we finish computing our final layer's activations (let's call them
), we immediately stream them into a loss module to calculate just how bad our predictions are. These loss modules sit right below our activation modules, and we only use them when we've reached our final layer. But here's the key insight: you don't actually need to calculate the loss value itself to train. You just need its derivative. Why? Because that derivative tells us which direction to adjust our weights to make the loss smaller. It's like having a compass that points toward "better performance."
The magic of the chain rule
This is where calculus enters the picture. To make our model better, we need to figure out how changing each weight affects our loss. The chain rule lets us break this massive calculation into smaller, manageable pieces.
The chain rule for gradients:
This allows us to compute gradients layer by layer, propagating them backwards through the network
Let's naively trace through what happens step by step.
Calculate
- how much the loss changes with respect to our final activations.
Compute
by taking the derivative of the activation (leaky ReLU in our case).
Compute
,
Compute
Rinse and repeat for the n-1 layer.
Propagating gradients to the hidden layer:
And through the first layer's activation:
With mixed positive and negative values in
, the gradient is:
Once we have all of these individual derivatives, we can multiply them together to find any derivative with respect of the loss (i.e.
gives us
).
After that, we have to compute the activation derivative
, for which the formula is
. This is also an element-wise computation, meaning we can structure it exactly like the loss module (and bias and activation modules), but it will perform a different calculation. One important note about this module, however, is that it requires the activations we computed during forward pass.
Now you might be wondering — how do we actually compute derivatives in hardware? Let's look at Leaky ReLU as an example, since it's beautifully simple but demonstrates the key principles. Remember that Leaky ReLU applies different operations based on whether the input is positive or negative. The derivative follows the same pattern: it outputs 1 for positive inputs and a small constant (we used 0.01) for negative inputs.
Leaky ReLU derivative implementation in hardware showing the conditional logic
What's beautiful about this is that it's just a simple comparison – no complex arithmetic needed. The hardware can compute this derivative in a single clock cycle, keeping our pipeline flowing smoothly. This same principle applies to other activation functions: their derivatives often simplify to basic operations that hardware can execute very efficiently.
You'll notice a really cool pattern emerging: all these modules that sit underneath the systolic array process column vectors that stream out one by one. This gave us the idea to unify them into something we called a
vector processing unit (VPU)
– because that's exactly what they're doing, processing vectors element-wise!
[6]
Not only is this more elegant to work with, it's also useful when we scale our TPU beyond a 2x2 systolic array, as we'll have N number of these modules (N being the size of the systolic array), each of which we would have to interface with individually. Unifying these modules under a parent module makes our design more scalable and elegant!
Vector Processing Unit (VPU) architecture showing unified element-wise operations
Additionally, by incorporating control signals for each module, which we call the VPU pathway bits, we can selectively enable or skip specific operations. This makes the VPU flexible enough to support both inference and training. For instance, during the forward pass, we want to apply biases and activations but skip computing loss or activation derivatives. When transitioning to the backward pass, all modules are engaged, but within the backward chain we only need to compute the activation derivative. Due to pipelining, all values that flow through the VPU pass through each of the four modules, and any unused modules simply act as registers, forwarding their inputs to outputs without performing computation.
The next few derivatives are interesting because we can actually use matrix multiplication (and the systolic array!) to compute the derivatives with the help of these three identities:
If we have
and take its derivative with respect to the weights, we get:
If we have
and take its derivative with respect to the inputs
, we get:
(just the weight matrix transposed)
For the bias term, the derivative is simply 1.
This means that we can multiply the previous
with
,
, and 1 to get
,
, and
, respectively, and we can multiply all of these by
to get the gradients of the loss with respect to all of our second layer parameters. And because all of the gradients are actually gradient matrices, we can use the systolic array!
Now something to note about the activation derivative
and the weight derivative
is that they both require the post-activations (H) we calculate during forward pass. This means we need to store the outputs of every layer in some form of memory to be able to perform training. Here's where we created a new scratchpad memory module
[7]
which we called the unified buffer (UB).
[8]
This lets us store our H values immediately after we compute them during forward pass.
We realized that we can also get rid of the input and weight accumulators, as well as manually loading the bias and leak factors into their respective modules, by using the UB to store them. This is also better practice, rather than loading in new data every clock cycle with the instruction set. Since we want to access two values (2 inputs or 2 weights for each row/col of the systolic array) at the same time, we added TWO read and write ports. We did this for each data primitive (inputs, weights, bias, leak factor, post activations) to minimize data contention since we have many different types of data.
To read values, we supply a starting address and the number of locations we want the UB to read and it will read 2 values every clock cycle. Writing is a similar mechanism, where we specify which values we want to write to each of the two input ports. The beauty in the read mechanism is that it runs in the background once we supply a starting address until the number of locations given are read, meaning we only need to provide an instruction for this every few clock cycles.
At the end of the day, not having these mechanisms wouldn't break the TPU — but they allow us to always keep the systolic array fed, which is a core design principle we couldn't compromise.
While we were working on this, we realized we could make one last small optimization for the activation derivative module — since we only use the
values once (for computing
), we created a tiny cache within the VPU instead of storing them in the UB. The rest of the
values will be stored in the UB because they're needed to compute multiple derivatives.
H-cache optimization for storing temporary activation values
This is what the new TPU architecture, modified to perform training, looks like:
Complete TPU architecture showing all components for both inference and training
Now we can do backpropagation!
The beautiful symmetry of forward and backward pass
Going back to the computational graph, we discovered something remarkable: the longest chain in backpropagation closely resembles forward pass! In forward pass, we multiply activation matrices with transposed weight matrices while in backward pass, we multiply gradient matrices with untransposed weight matrices. It's like looking in a mirror!
This insight led us to compute the long chain of the computational graph first (highlighted in yellow) – getting all our
gradients just like we computed activations in forward pass. We could cache these gradients and reuse them, following the same efficient pattern we'd already mastered.
Backward pass through the second hidden layer (long chain)
clk
0
To then calculate the leaf nodes (weight gradients) we create a loop where we:
Fetch a bridge node (
) from our unified buffer and transpose it
Fetch the corresponding
matrix, also from unified buffer
Stream these through our systolic array to compute the weight gradients
However, there is a problem which you may have noticed already. With a batch size larger than 2, our transposed
matrices don't fit into the systolic array! To solve this, we introduced tiling.
How tiling works
Tiling allows us to split up our transposed pre-activation gradient matrices into manageable chunks to fit into our 2 by 2 systolic array. When our batch size exceeds the array capacity, we divide the computation into smaller tiles that can be processed sequentially. For example, to calculate the gradient matrix for our first layer of weights, our 4 by 2
gradient matrix is split into two 2 by 2 tiles, with each tile transposed. The corresponding input matrix
is similarly partitioned into matching 2 by 2 tiles. Each tile pair then undergoes separate matrix multiplication in the systolic array, meaning we perform two distinct matrix multiplications to compute the complete gradient matrix for our first layer weights. The same procedure is repeated for the second layer of weights.
Computing layer 1 weight gradients for our XOR network without tiling:
Computing layer 1 weight gradients for our XOR network with tiling:
Similarly for layer 2 weight gradients without tiling:
Similarly for layer 2 weight gradients with tiling:
Gradient descent
As the tiled weight gradients stream out of the systolic array row by row, they flow directly into our gradient descent module. This module retrieves the current weights from memory and applies updates using the incoming gradients. Since weight gradients are inherently additive across batch samples, we can process them sequentially where each tile's contribution accumulates naturally into the final weight update, similar to how the number 2 can be split into 1 + 1.
Gradient descent for the first layer of weights
clk
0
The gradient descent update rule:
where
is the learning rate and
represents any parameter (weights or biases)
Applying gradient descent with learning rate
:
You might be wondering: "We've used our matrix multiplication identities for the long chain and weight gradients — how do we calculate bias gradients?" Well, we've actually already done most of the work! Since we're processing batches of data, we can simply sum (the technical term is "reduce") the
gradients across the batch dimension. The beauty is that we can do this reduction right when we're computing the long chain by simply passing the pre-activation gradients through the gradient descent modules and flipping a control bit to pass the output of the gradient descent module as the next old bias value input on the next clock cycle.
Gradient descent for the first layer of biases
clk
0
Bias gradients (sum over samples):
Applying gradient descent with learning rate
:
With all these new changes and control flags, our instruction is significantly longer — 94 bits in fact! But we can confirm that every single one of these bits is needed and we ensured that we couldn't make the instruction set any smaller without compromising the speed and efficiency of the TPU.
94-bit Instruction Set Architecture (ISA) layout showing control flags and data fields
Putting it all together
By continuing this same process iteratively – forward pass, backward pass, weight updates – we can train our network until it performs exactly how we want. The same systolic array that powered our inference now powers our training, with just a few additional modules to handle the gradient computations.
What started as a simple idea about matrix multiplication has grown into a complete training system. Every component works together in harmony: data flows through pipelines, modules operate in parallel, and our systolic array stays fed with useful work.
Final waveform simulation in GTKWave showing the weight and bias updates in memory after one epoch!
[2] Fun fact: the name of the systolic array is actually inspired by the human heart — just as systolic blood pressure is created by coordinated heart contractions that push blood through the cardiovascular system in waves, a systolic array processes data through coordinated computational "beats" that push information through the processing elements in waves.
↩ back
[3] This is a weight-stationary systolic array, which means the weights for each layer are stationary within their respective PEs and don't move around. However, there is a non-weight-stationary systolic array where the weights move along with the inputs, which has its own advantages and disadvantages.
↩ back
[4] Many illustrations online that depict staggering are actually flat out wrong because they pad consecutive rows with zeros, insetad of delaying them by a clock cycle. While this still gets the correct output, it wastes memory because we would have to store additional zeros that we don't use.
↩ back
[5] We chose Leaky ReLU over ReLU because we found that since we have a very small network, the model wasn't training properly when we used ReLU — it needed more non-linearity.
↩ back
[7] A scratchpad memory is a large bank of registers (each of which can store individual values) that lets us access any register we want. A FIFO for example is NOT a scratchpad memory since you can only access the first element in the queue.
↩ back
Imgur Geo-Blocked the UK, So I Geo-Unblocked My Network
Imgur decided to block UK users. Honestly? I don’t really care that much. I haven’t actively browsed the site in years. But it used to be everywhere. Back when Reddit embedded everything on Imgur, maybe fifteen years ago, it was genuinely useful. Then Reddit built their own image hosting, Discord did the same, and Imgur slowly faded into the background.
Except it never fully disappeared. And since the block, I keep stumbling across Imgur links that just show “unavailable.” It’s mildly infuriating.
Here’s a concrete example. I was playing Minecraft with some work colleagues and wanted to try different shaders. Most shader pages embed preview images hosted on Imgur. So I’d click through shader after shader, and every single preview was just gone. I couldn’t see what any of them looked like without the images.
This kind of thing happens constantly now. Old forum posts, Reddit threads, documentation pages, random project READMEs. Imgur links are still scattered across the internet, and in the UK, they’re all broken.
The obvious solution is to use a VPN. Change your location, problem solved. But I have a few issues with that approach.
First, I just
upgraded to 2.5 Gbps internet
and I don’t want to route all my traffic through a VPN and take the speed hit. I have this bandwidth for a reason.
Second, even if I installed a VPN on my main machine, what about my phone? My laptop? My desktop? Every device would need the VPN running, and I’d have to remember to connect it before browsing. It’s messy.
I wanted something cleaner: a solution that works for every device on my network, automatically, without any client-side configuration.
I already run a homelab with Traefik as my reverse proxy, Pi-hole for DNS, and everything declaratively configured with NixOS. If you’ve read my
previous post on Docker containers with secrets
, you’ll recognise the pattern.
The idea was simple: intercept all requests to
i.imgur.com
at the DNS level, route them through a VPN-connected container, and serve the images back. Every device on my network automatically uses Pi-hole for DNS via DHCP, so this would be completely transparent.
Here’s the flow:
Device requests
i.imgur.com
Pi-hole returns my Traefik instance’s IP instead
Traefik sees the SNI hostname and routes to Gluetun
Gluetun tunnels the request through a VPN
Nginx (attached to Gluetun’s network) proxies to the real Imgur
Good question. Gluetun isn’t a reverse proxy. It’s a container that provides VPN connectivity to other containers attached to its network namespace. So I needed something inside Gluetun’s network to actually handle the proxying. Nginx was the simplest choice.
The Nginx config is minimal. It just does TCP passthrough with SNI:
This listens on port 443, reads the SNI header to confirm the destination, and passes the connection through to the real
i.imgur.com
. The TLS handshake happens end-to-end; Nginx never sees the decrypted traffic.
The key detail is
network_mode: "service:gluetun"
. This makes Nginx share Gluetun’s network stack, so all its traffic automatically goes through the VPN tunnel.
I’m not going to mention which VPN provider I use. It’s one of the major ones with WireGuard support, but honestly I’m not thrilled with it. Use whatever you have.
Now when any device on my network requests an Imgur image, it works. My phone, my laptop, guest devices, everything. No VPN apps to install, no browser extensions, no manual configuration. Pi-hole intercepts the DNS, Traefik routes the connection, and Gluetun tunnels it through a non-UK exit point.
The latency increase is negligible for loading images, and it only affects Imgur traffic. Everything else still goes direct at full speed.
Is this overkill for viewing the occasional Imgur image? Probably. But it’s a clean solution that requires minimal ongoing maintenance, and it scratches the homelab itch. Plus I can finally see what those Minecraft shaders look like.
Microsoft: Windows updates make password login option invisible
Bleeping Computer
www.bleepingcomputer.com
2025-11-28 18:07:17
Microsoft warned users that Windows 11 updates released since August may cause the password sign-in option to disappear from the lock screen options, even though the button remains functional. [...]...
My current project uses a cross-platform SDK. My most comfortable development platform is macOS, and that's what I use 90% of the time on this project. Occasionally I work on embedded android systems. For that I use Linux. If I'm on a consulting job and the client has everyone using Windows, I go with the flow, but my impression is that Windows has become end-user hostile, and it's getting worse. Maybe that's just because my permissions and network access are never set right on the first try when a client needs a machine set up for an outside consultant. Subjectively, Windows is the itchy sweater of development platforms.
Linux.
I retired the last Windows machine last year.
Firefox on Linux, though, is not working very well. It keeps hanging during long typing inputs. No CPU or disk usage, just stuck. And it uses so much memory that the OOM killer sometimes kills it.
Not sure this covers popular mixes, eg WSL or considers AI clients.
My IDE is Windows (VSCode or Cursor); but I'm also using ChatGPT in the browser and various Linux command line tools (connecting through Windows Terminal to WSL Redhat).
There should probably be a fully hybrid option in the poll.
I think I'd count WSL as Linux.
Cloud based development and browser hosted environments would certainly be worth measuring. I imagine the numbers are tiny compared to other platforms.
Arduino IDE probably counts as something with decent numbers. Wokwi also makes for an interesting candidate in that area.
Surprised this is apparently the less popular stack. IDE (VS Code) on windows working out of WSL has been so good for a long time now.
I can't even imagine doing development in Windows without WSL anymore. I think Microsoft even requires it for some of their stuff.
It's not a linux machine. The computer is booted and managed by Windows. Linux is an application running on the Windows machine.
have been using linux ever since i got my first personal computer.
our customers all run linux in production too, so it's very easy and natural to develop and test the software in its usual environment (although i wish my laptop had eight times the ram to match).
I never found anything better than the latest macOS machines. I ran ubuntu for years and then switched back to mac just because I don't have the time to tinker and fiddle with stuff in Linux that just randomly makes the computer run hot, or a monitor to not work, or fonts looking awful.
MacOS is just the sweet spot of great desktop + great unix-style devbox.
UX' been degrading on MacOS for ages.
On top of that I've been locked out of my machine and Apple ID and they just kept sending me emails that in some weeks they were going to reset my password, and they sent me those emails for 2 months before I got access to my apple id and machine again, proof[1].
They just kept not obliging the "2 weeks" (which is already mad when I've given you my secret password and I've verified my email and phone already).
And they did not respect the two weeks 3 times in a row!
That is beyond disgusting and Apple has never got a single $ from me since, I only own a MBP I use on the move because a client has sent me an M3 Max with 48 GBs so it made no sense to at least not use it.
However, I went back to linux on my personal laptop (nixos on my case) and I am pleasantly surprised how many things now just work.
The only thing that still annoys me is the laptop not sleeping properly and therefore using too much battery power when idle.
It has made great strides on the last two or so years.
Finally bailed Windows this year after a lifetime of MSVC, couldn't be happier with the decision. I'm actually kind of grateful for Windows 11 being so impossibly shit and forcing so many people to finally make the switch! Now I use Arch, btw.
I recently moved from many many years on Windows to full time on Linux for my bare metal embedded development, largely because of ST Micro's good Linux support with their tools.
LXC containers on top of Debian in a specific work station just for this. I have one generic container to start everything, and then create specific ones if projects get bigger.
This is by far the best option to isolate and easily create development environments that I found.
I connect to the containers from VS Code running on Mac OS.
22 years in the same Corp, targeting Linux systems since day one, and only in the first two years, and this year, have I been permitted a Linux desktop.
+2 years slugging in a vm.
Developing with out bash is just unnecessary work.
My productivity has more than doubled. easily. I manually type passwords half as much and when I do that is to access Microsoft services.
2fa wastes a huge amount of time.
Because nothing that needs 2fa is scriptable.
I think it's possible we see some people now use other OSes, there should at least be an "Other" option. The *BSD's, Nix and some more bespoke options.
Technically I use all 3.
I mostly work on my desktop which is Windows + WSL 2 with Ubuntu and use a MBP on the move.
Mac Mini M4 or Macbook Air M4 is not expensive. Just bought the latter (extensive rebates at the moment) to replace my Intel-based 2019 MPB. Wow! It's pretty much the perfect laptop, at that price.
My primary complaint so far: The green color LED of the magsafe connector is not the same green as the LED on the caps lock key.
28M Hacker News comments as vector embedding search dataset
This dataset can be used to walk through the design, sizing and performance aspects for a large scale,
real world vector search application built on top of user generated, textual data.
The
id
is just an incrementing integer. The additional attributes can be used in predicates to understand
vector similarity search combined with post-filtering/pre-filtering as explained in the
documentation
Run the following SQL to define and build a vector similarity index on the
vector
column of the
hackernews
table:
ALTER TABLE hackernews ADD INDEX vector_index vector TYPE vector_similarity('hnsw', 'cosineDistance', 384, 'bf16', 64, 512);
ALTER TABLE hackernews MATERIALIZE INDEX vector_index SETTINGS mutations_sync = 2;
The parameters and performance considerations for index creation and search are described in the
documentation
.
The statement above uses values of 64 and 512 respectively for the HNSW hyperparameters
M
and
ef_construction
.
Users need to carefully select optimal values for these parameters by evaluating index build time and search results quality
corresponding to selected values.
Building and saving the index could even take a few minutes/hour for the full 28.74 million dataset, depending on the number of CPU cores available and the storage bandwidth.
Sentence Transformers
provide local, easy to use embedding
models for capturing the semantic meaning of sentences and paragraphs.
The dataset in this HackerNews dataset contains vector emebeddings generated from the
all-MiniLM-L6-v2
model.
An example Python script is provided below to demonstrate how to programmatically generate
embedding vectors using
sentence_transformers1 Python package. The search embedding vector is then passed as an argument to the [
cosineDistance()
](/sql-reference/functions/distance-functions#cosineDistance) function in the
SELECT` query.
from sentence_transformers import SentenceTransformer
import sys
import clickhouse_connect
print("Initializing...")
model = SentenceTransformer('sentence-transformers/all-MiniLM-L6-v2')
chclient = clickhouse_connect.get_client() # ClickHouse credentials here
while True:
# Take the search query from user
print("Enter a search query :")
input_query = sys.stdin.readline();
texts = [input_query]
# Run the model and obtain search vector
print("Generating the embedding for ", input_query);
embeddings = model.encode(texts)
print("Querying ClickHouse...")
params = {'v1':list(embeddings[0]), 'v2':20}
result = chclient.query("SELECT id, title, text FROM hackernews ORDER BY cosineDistance(vector, %(v1)s) LIMIT %(v2)s", parameters=params)
print("Results :")
for row in result.result_rows:
print(row[0], row[2][:100])
print("---------")
An example of running the above Python script and similarity search results are shown below
(only 100 characters from each of the top 20 posts are printed):
Initializing...
Enter a search query :
Are OLAP cubes useful
Generating the embedding for "Are OLAP cubes useful"
Querying ClickHouse...
Results :
27742647 smartmic:
slt2021: OLAP Cube is not dead, as long as you use some form of:<p>1. GROUP BY multiple fi
---------
27744260 georgewfraser:A data mart is a logical organization of data to help humans understand the schema. Wh
---------
27761434 mwexler:"We model data according to rigorous frameworks like Kimball or Inmon because we must r
---------
28401230 chotmat:
erosenbe0: OLAP database is just a copy, replica, or archive of data with a schema designe
---------
22198879 Merick:+1 for Apache Kylin, it's a great project and awesome open source community. If anyone i
---------
27741776 crazydoggers:I always felt the value of an OLAP cube was uncovering questions you may not know to as
---------
22189480 shadowsun7:
_Codemonkeyism: After maintaining an OLAP cube system for some years, I'm not that
---------
27742029 smartmic:
gengstrand: My first exposure to OLAP was on a team developing a front end to Essbase that
---------
22364133 irfansharif:
simo7: I'm wondering how this technology could work for OLAP cubes.<p>An OLAP cube
---------
23292746 scoresmoke:When I was developing my pet project for Web analytics (<a href="https://github
---------
22198891 js8:It seems that the article makes a categorical error, arguing that OLAP cubes were replaced by co
---------
28421602 chotmat:
7thaccount: Is there any advantage to OLAP cube over plain SQL (large historical database r
---------
22195444 shadowsun7:
lkcubing: Thanks for sharing. Interesting write up.<p>While this article accurately capt
---------
22198040 lkcubing:Thanks for sharing. Interesting write up.<p>While this article accurately captures the issu
---------
3973185 stefanu:
sgt: Interesting idea. Ofcourse, OLAP isn't just about the underlying cubes and dimensions,
---------
22190903 shadowsun7:
js8: It seems that the article makes a categorical error, arguing that OLAP cubes were r
---------
28422241 sradman:OLAP Cubes have been disrupted by Column Stores. Unless you are interested in the history of
---------
28421480 chotmat:
sradman: OLAP Cubes have been disrupted by Column Stores. Unless you are interested in the
---------
27742515 BadInformatics:
quantified: OP posts with inverted condition: “OLAP != OLAP Cube” is the actual titl
---------
28422935 chotmat:
rstuart4133: I remember hearing about OLAP cubes donkey's years ago (probably not far
---------
The example above demonstrated semantic search and document retrieval using ClickHouse.
A very simple but high potential generative AI example application is presented next.
The application performs the following steps:
Accepts a
topic
as input from the user
Generates an embedding vector for the
topic
by using the
SentenceTransformers
with model
all-MiniLM-L6-v2
Retrieves highly relevant posts/comments using vector similarity search on the
hackernews
table
Uses
LangChain
and OpenAI
gpt-3.5-turbo
Chat API to
summarize
the content retrieved in step #3.
The posts/comments retrieved in step #3 are passed as
context
to the Chat API and are the key link in Generative AI.
An example from running the summarization application is first listed below, followed by the code
for the summarization application. Running the application requires an OpenAI API key to be set in the environment
variable
OPENAI_API_KEY
. The OpenAI API key can be obtained after registering at
https://platform.openai.com
.
This application demonstrates a Generative AI use-case that is applicable to multiple enterprise domains like :
customer sentiment analysis, technical support automation, mining user conversations, legal documents, medical records,
meeting transcripts, financial statements, etc
$ python3 summarize.py
Enter a search topic :
ClickHouse performance experiences
Generating the embedding for ----> ClickHouse performance experiences
Querying ClickHouse to retrieve relevant articles...
Initializing chatgpt-3.5-turbo model...
Summarizing search results retrieved from ClickHouse...
Summary from chatgpt-3.5:
The discussion focuses on comparing ClickHouse with various databases like TimescaleDB, Apache Spark,
AWS Redshift, and QuestDB, highlighting ClickHouse's cost-efficient high performance and suitability
for analytical applications. Users praise ClickHouse for its simplicity, speed, and resource efficiency
in handling large-scale analytics workloads, although some challenges like DMLs and difficulty in backups
are mentioned. ClickHouse is recognized for its real-time aggregate computation capabilities and solid
engineering, with comparisons made to other databases like Druid and MemSQL. Overall, ClickHouse is seen
as a powerful tool for real-time data processing, analytics, and handling large volumes of data
efficiently, gaining popularity for its impressive performance and cost-effectiveness.
Code for the above application :
print("Initializing...")
import sys
import json
import time
from sentence_transformers import SentenceTransformer
import clickhouse_connect
from langchain.docstore.document import Document
from langchain.text_splitter import CharacterTextSplitter
from langchain.chat_models import ChatOpenAI
from langchain.prompts import PromptTemplate
from langchain.chains.summarize import load_summarize_chain
import textwrap
import tiktoken
def num_tokens_from_string(string: str, encoding_name: str) -> int:
encoding = tiktoken.encoding_for_model(encoding_name)
num_tokens = len(encoding.encode(string))
return num_tokens
model = SentenceTransformer('sentence-transformers/all-MiniLM-L6-v2')
chclient = clickhouse_connect.get_client(compress=False) # ClickHouse credentials here
while True:
# Take the search query from user
print("Enter a search topic :")
input_query = sys.stdin.readline();
texts = [input_query]
# Run the model and obtain search or reference vector
print("Generating the embedding for ----> ", input_query);
embeddings = model.encode(texts)
print("Querying ClickHouse...")
params = {'v1':list(embeddings[0]), 'v2':100}
result = chclient.query("SELECT id,title,text FROM hackernews ORDER BY cosineDistance(vector, %(v1)s) LIMIT %(v2)s", parameters=params)
# Just join all the search results
doc_results = ""
for row in result.result_rows:
doc_results = doc_results + "\n" + row[2]
print("Initializing chatgpt-3.5-turbo model")
model_name = "gpt-3.5-turbo"
text_splitter = CharacterTextSplitter.from_tiktoken_encoder(
model_name=model_name
)
texts = text_splitter.split_text(doc_results)
docs = [Document(page_content=t) for t in texts]
llm = ChatOpenAI(temperature=0, model_name=model_name)
prompt_template = """
Write a concise summary of the following in not more than 10 sentences:
{text}
CONSCISE SUMMARY :
"""
prompt = PromptTemplate(template=prompt_template, input_variables=["text"])
num_tokens = num_tokens_from_string(doc_results, model_name)
gpt_35_turbo_max_tokens = 4096
verbose = False
print("Summarizing search results retrieved from ClickHouse...")
if num_tokens <= gpt_35_turbo_max_tokens:
chain = load_summarize_chain(llm, chain_type="stuff", prompt=prompt, verbose=verbose)
else:
chain = load_summarize_chain(llm, chain_type="map_reduce", map_prompt=prompt, combine_prompt=prompt, verbose=verbose)
summary = chain.run(docs)
print(f"Summary from chatgpt-3.5: {summary}")
Are you bored? And I mean,
really
bored? Like "I'd do anything to pass time"-bored?
Do you also happen to be alone and without an internet connection?
You could play Rock Paper Scissors solitaire!
All you need is a dice. That's your opponent. Throw it from your hand and as soon as you throw it form the shape you choose with your hand and see the result of the dice. 1-2 is Rock, 3-4 is Paper, 5-6 is Scrissors.
Those numbers can be changed if you don't like them. For example I grew up calling it Paper Rock Scissors (
Carta-Sasso-Forbici
in Italian), so to me 1-2 being Paper and 3-4 being Rock is easier to mentally handle.
That's it.
Did I mention you need to be
really
bored to enjoy this?
Automatic Locking
When you are gone for a set period of time
RAM Shredding
Securely shreds sensitive data
Tor Support
Supports SOCKS proxy and
Tor
via
Orbot
And more
New and better features to come
Public GitLab repositories exposed more than 17,000 secrets
Bleeping Computer
www.bleepingcomputer.com
2025-11-28 17:43:02
After scanning all 5.6 million public repositories on GitLab Cloud, a security engineer discovered more than 17,000 exposed secrets across over 2,800 unique domains. [...]...
After scanning all 5.6 million public repositories on GitLab Cloud, a security engineer discovered more than 17,000 exposed secrets across over 2,800 unique domains.
Luke Marshall used the TruffleHog open-source tool to check the code in the repositories for sensitive credentials like API keys, passwords, and tokens.
The researcher previously
scanned Bitbucket
, where he found 6,212 secrets spread over 2.6 million repositories. He also checked the
Common Crawl dataset
that is used to train AI models, which exposed 12,000 valid secrets.
GitLab is a web-based Git platform used by software developers, maintainers, and DevOps teams to host code, for CI/CD operations, development collaboration, and repository management.
Marshall used a GitLab public API endpoint to enumerate every public GitLab Cloud repository, using a custom Python script to paginate through all results and sort them by project ID.
This process returned 5.6 million non-duplicate repositories, and their names were sent to an AWS Simple Queue Service (SQS).
Next, an AWS Lambda function pulled the repository name from SQS, ran TruffleHog against it, and logged the results.
“Each Lambda invocation executed a simple TruffleHog scan command with concurrency set to 1000,”
describes Marshall
.
“This setup allowed me to complete the scan of 5,600,000 repositories in just over 24 hours.”
The total cost for the entire public GitLab Cloud repositories using the above method was $770.
The researcher found 17,430 verified live secrets, nearly three times as many as in Bitbucket, and with a 35% higher secret density (secrets per repository), too.
Historical data shows that most leaked secrets are newer than 2018. However, Marshall also found some very older secrets dating from 2009, which are still valid today.
Volume of exposed secrets
Source: Truffle Security
The largest number of leaked secrets, over 5,200 of them, were Google Cloud Platform (GCP) credentials, followed by MongoDB keys, Telegram bot tokens, and OpenAI keys.
The researcher also found a little over 400 GitLab keys leaked in the scanned repositories.
Types of exposed secrets on GitLab
Source: Truffle Security
In the spirit of responsible disclosure and because the discovered secrets were associated with 2,804 unique domains, Marshall relied on automation to notify affected parties and used Claude Sonnet 3.7 with web search ability and a Python script to generate emails.
In the process, the researcher collected multiple bug bounties that amounted to $9,000.
The researcher reports that many organizations revoked their secrets in response to his notifications. However, an undisclosed number of secrets continue to be exposed on GitLab.
It's budget season! Over 300 CISOs and security leaders have shared how they're planning, spending, and prioritizing for the year ahead. This report compiles their insights, allowing readers to benchmark strategies, identify emerging trends, and compare their priorities as they head into 2026.
Learn how top leaders are turning investment into measurable impact.
JSON Schema Demystified: Dialects, Vocabularies and Metaschemas
If you’ve ever tried to dive into JSON Schema, you’ve probably encountered a wall of terminology that makes your head spin: schemas, metaschemas, dialects, vocabularies, keywords, anchors, dynamic references. It feels like the community invented new words for things that already had perfectly good names, just to make the rest of us feel inadequate.
I’ve been working on a Haskell JSON Schema library that’s actually fully spec-compliant, which meant I had to figure all of this out. The problem isn’t that the concepts are inherently difficult. The terminology creates artificial barriers to understanding.
This post will break down the key concepts in JSON Schema in a way that actually makes sense, connecting the dots between all these terms that seem designed to confuse. By the end, you’ll understand not just what these words mean, but how they fit together into a coherent system.
Starting simple
Before we dive into terminology, let’s look at what we’re actually trying to accomplish. JSON Schema is fundamentally about describing the shape and constraints of JSON data. Here’s a simple example:
This schema says: “I expect a JSON object with a string
name
field (required) and an optional numeric
age
field that must be non-negative.” Simple enough, right?
Now here’s where it gets interesting: this schema is itself valid JSON. And since JSON can describe the structure of JSON documents, we can describe the structure of schemas using more schemas. This recursive property is what gives rise to metaschemas, and where the terminology starts to get confusing.
What’s a schema anyway?
A schema is just a JSON document that describes constraints on other JSON documents. That’s it. The example above is a schema.
Schemas tell you what type a value should be (string, number, object, array), what values are allowed or disallowed, what properties must or may exist on an object, how many items should be in an array. When you write a schema, you’re essentially writing rules that say “valid JSON documents that I care about look like this.”
This schema says: “I want a string between 1 and 100 characters long.” Any JSON validator that understands JSON Schema can take this schema and your data and tell you whether your data follows the rules.
The confusing part is that schemas themselves are JSON documents. So naturally, you might ask: “What describes the structure of a schema?” And that leads us to the next layer.
A metaschema is a schema that describes the structure of other schemas. The “schema of schemas,” if you will.
This sounds abstract and philosophical, but it’s actually quite practical. Remember how our simple schema used keywords like
"type"
,
"properties"
, and
"minimum"
? The metaschema defines what those keywords mean, what values they can have, and how they work together.
Here’s a tiny excerpt of what a metaschema might look like:
This fragment says things like: “The
type
keyword can be a single type string or an array of type strings” and “The
properties
keyword should be an object where each value is itself a schema.”
Why does this matter? Well, you can validate that your schema is well-formed by checking it against the metaschema. If someone writes
"type": "stirng"
(typo!), the metaschema validation will catch it. The metaschema is also the formal specification of what’s allowed in schemas. Tools that process schemas (validators, code generators, documentation generators) use the metaschema to understand what they’re working with.
The relationship is simple: schemas validate data, metaschemas validate schemas.
Data → validated by → Schema → validated by → Metaschema
Here’s where it gets recursive: since a metaschema is also a schema (JSON describing JSON structure), it can validate itself. The JSON Schema metaschema is designed to be self-describing. This is similar to how a compiler written in its own language can compile itself (bootstrapping).
Dialects: when versions matter
So we have schemas and metaschemas. But JSON Schema has evolved over time. Different versions have added new keywords, changed behavior, and deprecated old features. How do we keep track of which version of JSON Schema we’re using?
A dialect is a specific version or flavor of JSON Schema, defined by a particular metaschema. When someone says they’re using “Draft 2020-12” or “Draft 7,” they’re referring to specific dialects.
Each dialect has its own metaschema that defines which keywords are available, its own set of behaviors and validation rules, and is identified by a URI (usually something like
https://json-schema.org/draft/2020-12/schema
).
You declare which dialect your schema uses with the
$schema
keyword:
This tells validators: “Hey, interpret this schema according to the Draft 2020-12 rules.”
Different dialects can have different keywords and different behaviors. Draft 4 didn’t have the
const
keyword, but Draft 6 added it. The
$id
keyword worked differently in Draft 4 versus Draft 7. Draft 2019-09 introduced the concept of vocabularies (we’ll get to that).
If you write a schema using Draft 2020-12 features and someone tries to validate it with a Draft 4 validator, things won’t work correctly. The
$schema
keyword ensures everyone is on the same page.
Think of dialects like programming language versions. Python 2 and Python 3 are different dialects of Python. Your code needs to declare which one it’s written for, or chaos ensues.
Vocabularies: the modular twist
Here’s where JSON Schema gets really interesting (and where my initial confusion peaked). Starting with Draft 2019-09, JSON Schema introduced the concept of vocabularies.
A vocabulary is a named collection of keywords that work together to provide a specific kind of functionality. Instead of having one monolithic metaschema that defines all possible keywords, you can compose metaschemas from smaller, focused vocabularies.
Think of vocabularies as modules or packages. Each vocabulary provides a set of related keywords. The core vocabulary has fundamental keywords like
$id
,
$schema
,
$ref
, and
$defs
. The applicator vocabulary has keywords that apply schemas to different parts of the data like
properties
,
items
, and
additionalProperties
. The validation vocabulary has keywords for constraints like
minimum
,
maxLength
,
pattern
, and
enum
. The metadata vocabulary has keywords for human-readable information like
title
,
description
, and
examples
.
Here’s a schema using keywords from different vocabularies:
{ "$id": "https://example.com/my-schema", "title": "User Name", "description": "The user's full name", "type": "string", "minLength": 5, "pattern": "^[A-Z]"}
The
$id
comes from the core vocabulary,
title
and
description
from the metadata vocabulary,
type
from the applicator vocabulary, and
minLength
and
pattern
from the validation vocabulary.
Why vocabularies? They enable modularity and extensibility. You can pick and choose which vocabularies your dialect supports. Maybe you want validation but not format checking? Just include the vocabularies you need. You can define your own vocabulary with custom keywords specific to your domain. For example, a database schema dialect might add keywords like
indexed
or
foreignKey
. Each vocabulary is independently specified, making it easier to understand and implement different parts of JSON Schema.
Here’s how a metaschema declares which vocabularies it uses:
The
true
versus
false
values indicate whether the vocabulary is required or optional. If a validator doesn’t understand a required vocabulary, it should refuse to process the schema. If it doesn’t understand an optional vocabulary, it can safely ignore those keywords.
Extending with your own keywords
Here’s where this all gets practical. Once you understand vocabularies, you realize you can extend JSON Schema with your own domain-specific keywords. This is incredibly powerful.
In fact, you’ve probably already used extended JSON Schema without realizing it. OpenAPI (the spec for describing REST APIs) is exactly this: JSON Schema extended with custom keywords for HTTP-specific concerns like
operationId
,
responses
,
parameters
, and so on. OpenAPI is JSON Schema plus a vocabulary for APIs. And you could extend OpenAPI further with your own vocabulary for framework-specific behaviors or company-specific conventions.
Say you’re building an API framework and you want to annotate your schemas with HTTP-specific metadata. Standard JSON Schema doesn’t have keywords for things like “this field comes from a query parameter” or “this response uses status code 201.” So you create your own vocabulary.
First, you define your custom keywords in a vocabulary document:
Your validator needs to understand what to do with
httpSource
, of course. When it encounters a schema using your custom vocabulary, it checks whether it supports that vocabulary. If the vocabulary is marked as required and the validator doesn’t support it, validation should fail with an error saying “I don’t understand this vocabulary.” If it’s optional, the validator can safely ignore those keywords.
The beauty of this approach is that your extensions are explicit and discoverable. Someone reading your schema can see exactly which vocabularies it uses. A validator can definitively say whether it supports your schema or not. You’re not just stuffing random properties into schemas and hoping validators ignore them.
You can extend validation rules too. Maybe you’re working with database schemas and want to validate that certain string fields match database identifier conventions. You could define a custom keyword like
dbIdentifier
:
{ "type": "string", "dbIdentifier": true, "description": "Must be a valid PostgreSQL identifier"}
Your validator would implement the logic to check PostgreSQL identifier rules (no leading numbers, only certain special characters, length limits, etc.). Standard JSON Schema validators would ignore this keyword if you mark the vocabulary as optional, or refuse to process the schema if you mark it as required.
This extensibility is why JSON Schema has all this vocabulary machinery. It’s not just academic complexity for its own sake. The vocabulary system lets you build domain-specific validation languages on top of JSON Schema’s foundation, while maintaining clear boundaries about what’s standard and what’s custom.
Putting it all together
Let’s connect all the dots. You write a schema that describes your data structure (like a User object). The schema uses keywords like
type
,
properties
, and
minimum
to express constraints. These keywords are defined by vocabularies (the validation vocabulary, applicator vocabulary). The vocabularies are bundled into a dialect (like Draft 2020-12). The dialect is defined by a metaschema that describes which keywords are available and how they work. Your schema declares its dialect using the
$schema
keyword.
Here’s a visual:
Your Data (JSON) ↓ validated byYour Schema (JSON) ↓ uses keywords fromVocabularies (sets of related keywords) ↓ bundled intoDialect (specific version/flavor) ↓ defined byMetaschema (schema of schemas)
Breaking this down:
$schema
declares we’re using the Draft 2020-12 dialect.
$id
is a core vocabulary keyword that uniquely identifies this schema.
title
and
description
are metadata vocabulary keywords for documentation.
type
,
properties
, and
required
are applicator vocabulary keywords that apply constraints.
pattern
,
minLength
,
minimum
, and
format
are validation vocabulary keywords that enforce rules.
All of these keywords are defined in the Draft 2020-12 metaschema, which specifies their meaning and behavior.
Other terms you’ll encounter
While we’ve covered the big four (schema, metaschema, dialect, vocabulary), there are a few other terms worth understanding.
A keyword is a specific property name with defined semantics in a schema. Examples:
type
,
properties
,
minimum
,
$ref
. Keywords are the building blocks defined by vocabularies. Some keywords are universal (
type
,
properties
), while others are specific to certain vocabularies (
contentMediaType
from the content vocabulary,
deprecated
from the metadata vocabulary).
There’s also a distinction between annotations and assertions. Assertions are keywords that can make validation fail (like
type
,
minimum
,
required
,
pattern
). If your data violates an assertion, validation fails. Annotations are keywords that just provide information and never cause validation to fail (like
title
,
description
,
examples
,
default
). Annotations are useful for documentation and tooling but don’t affect validity. Some keywords can produce both annotations and assertions. For instance,
properties
asserts the types of the properties while also annotating which properties were validated.
Anchors provide named locations within a schema that you can reference. They’re like bookmarks:
Dynamic anchors (
$dynamicAnchor
and
$dynamicRef
) are more advanced. They allow references to be resolved differently depending on the “context” of evaluation. This is mostly useful for extending metaschemas and creating recursive schemas that can be overridden. Honestly, you can probably ignore dynamic anchors until you’re doing very advanced schema composition.
A bundled schema is a single schema document that contains multiple schema resources, usually via
$defs
. This is handy for distributing related schemas together:
Now you can reference
https://example.com/schemas/user
and
https://example.com/schemas/product
from other schemas, even though they’re defined in the same document.
Why is the terminology so confusing?
You might be wondering: why did they make this so complicated? The answer is that JSON Schema has evolved significantly over more than a decade, and the terminology evolved with it.
Early versions (Draft 3, Draft 4) had simpler, more monolithic metaschemas. As the specification matured, the community recognized the need for modularity, extensibility, and clearer versioning. That’s when concepts like dialects and vocabularies were formalized.
The terminology can feel academic because it comes from formal specification work. These are precise technical terms designed for specification writers and implementers, not necessarily for end users. Unfortunately, they leaked into the documentation that everyone reads, creating a steep learning curve.
But here’s the thing: you don’t need to think about most of this complexity to use JSON Schema effectively.
What do you actually need to know?
For 95% of JSON Schema usage, you need to understand that schemas describe data structure and constraints,
$schema
declares which version (dialect) you’re using, and keywords like
type
,
properties
, and
minimum
define your rules.
That’s it. You can write perfectly good schemas for years without ever thinking about metaschemas or vocabularies in depth.
The deeper concepts matter when you’re building tools that process schemas (validators, code generators), extending JSON Schema with custom keywords, working on the specification itself, or debugging complex reference resolution issues. For everyone else, just know that these concepts exist and form a coherent system. If you encounter them in documentation, you’ll know what they mean, but you probably won’t need to think about them day-to-day.
Some practical advice
Always specify
$schema
to make it explicit which dialect you’re using:
This ensures validators interpret your schema correctly.
Start with the latest stable dialect. As of this writing, that’s Draft 2020-12. It has the most features and best tooling support. Don’t worry about older drafts unless you’re maintaining legacy schemas.
Use clear, descriptive metadata. Even though
title
,
description
, and
examples
don’t affect validation, they make your schemas much more useful:
{ "type": "object", "title": "User Account", "description": "Represents a user account in the system", "properties": { "username": { "type": "string", "description": "Unique username for login (alphanumeric and underscores only)", "examples": ["john_doe", "alice123"] } }}
For complex schemas, use
$defs
to break things into reusable pieces:
Test your schemas with online validators or schema testing tools to ensure they work as expected. The official JSON Schema website has a validator you can try:
https://www.jsonschemavalidator.net/
Wrapping up
JSON Schema’s terminology can feel intimidating, but the core ideas are straightforward. Schemas describe data. Metaschemas describe schemas. Dialects are specific versions of JSON Schema. Vocabularies are modular collections of keywords. Keywords are the actual properties you use in schemas.
The terminology exists to support a powerful, extensible system for describing JSON data structures. But for everyday use, you can mostly ignore the academic terminology and focus on writing clear, useful schemas.
The next time you see “metaschema” or “vocabulary” in JSON Schema documentation, don’t panic. You know what these terms mean now, and more importantly, you understand how they fit together. That’s the real goal: building a mental model of how the system works, not memorizing definitions.
Now go forth and write some schemas. And remember: if you find yourself confused by JSON Schema terminology again, you’re not alone. The important thing is that underneath the jargon, there’s a well-designed system for a genuinely useful purpose.
How to avoid bad Black Friday laptop deals – and some of the best UK offers for 2025
Guardian
www.theguardian.com
2025-11-28 17:31:26
Here’s how to spot a genuinely good laptop deal, plus the best discounts we’ve seen so far on everything from MacBooks to gaming laptops • Do you really need to buy a new laptop?• How to shop smart this Black Friday Black Friday deals have started, and if you’ve been on the lookout for a good price ...
Black Friday deals
have started, and if you’ve been on the lookout for a good price on a new laptop, then this could be your lucky day. But with so many websites being shouty about their Black Friday offers, the best buys aren’t always easy to spot. So before you splash the cash, it might pay to do some research – and look closely at the specification.
I know this may not be welcome advice. After all, the thought of drawing up a spreadsheet of memory configurations and pricing history might put a slight dampener on the excitement that builds as Black Friday approaches. But buy the right laptop today and you can look forward to many years of joyful productivity. Pick a duff one, and every time you open the lid you’ll be cursing your past self’s impulsive nature. So don’t get caught out; be prepared with our useful tips – and a roundup of the Filter’s favourite laptop deals.
How to find a genuinely good Black Friday laptop deal
Find out what a laptop is really like to use to ensure it’s right for you.
Photograph: Oscar Wong/Getty Images
Don’t sweat the CPU
Many people get hung up on processor power, but this is the one thing you rarely need to worry about these days. Although new processor models come out with alarming frequency, almost any AMD Ryzen, Intel Core or Apple M-series chip of the past few years will be fine for everyday web browsing and office tasks. High-end models are only really needed for particularly demanding workloads; a quick trip to Google (or your AI chatbot of choice) will help you see how different processor models measure up.
Plan ahead with plenty of RAM and storage
Every laptop needs a decent amount of memory. If the system is starved of RAM, then performance will be sluggish, regardless of the CPU’s speed. While Windows 11 runs acceptably in 8GB, a minimum of 16GB will help ensure that future updates continue to run smoothly. Some models are upgradeable, so you can start with a basic allocation of RAM and add more as your needs grow, but this certainly isn’t something you can take for granted.
Laptop storage is also rarely expandable, except by plugging in a USB flash drive or an external SSD. That might be fine if your device will spend much of its time on a desk, but if you want to carry it around with you – not an unreasonable ask for a computer of this type – it’s a drag. So while a base-level 256GB SSD might suffice for home-working, consider stepping up to 512GB or even 1TB of internal storage, especially if you want to edit videos or play big 3D games. Look into battery life, weight and overall dimensions, too, if portability is a priority.
Find out what it’s really like to use
Some important considerations – such as the quality of the screen and keyboard – don’t show up on the spec sheet, yet these things are arguably just as important as the processor and memory. If the display is dim and blocky, and typing emails feels like pressing Scrabble tiles into a flannel, it will make day-to-day working more difficult.
Since online retail doesn’t give you an opportunity to try tapping out “the quick brown fox” for yourself, the next best thing is to read reviews of other people’s hands-on experience. Pay particular attention to the model number, though: laptops often come in a few variants, including a high-end version that will usually get great reviews – and a more cheaply made model that can be flogged for a knock-down price on Black Friday.
Is this a genuine special offer?
The final thing to check is whether the bargain that’s flashing up on your screen is actually a deal at all. You can look up past prices for a vast range of items by going to
CamelCamelCamel
– yes, really – and either typing in a laptop model number or pasting in the web address of an Amazon product page. You may find that the heavily promoted Black Friday price is identical to last month’s standard price on Amazon. That doesn’t mean it’s a bad deal, but it signals that you probably don’t need to race to grab a once-in-a-lifetime bargain (we’ve made sure to list this price history on all the laptop deals below).
Indeed, with Cyber Monday, pre- and post-Christmas sales, Easter specials, Amazon Prime Day, back-to-school offers and so forth, you’re rarely more than a few weeks away from the next big discount event – so don’t let the excitement of Black Friday encourage you into making a hasty purchase.
Darien Graham-Smith
At the Filter, we believe in buying sustainably, and the excessive consumerism encouraged by Black Friday doesn’t sit easily with us. However, we also believe in shopping smarter, and there’s no denying that it’s often the best time of year to buy big-ticket items that you genuinely need and have planned to buy in advance, or stock up on regular buys such as skincare and cleaning products.
Retailers often push offers that are not as good as they seem, with the intention of clearing out old stock, so we only recommend genuine deals. We assess the price history of every product where it’s available, and we won’t feature anything unless it is genuinely lower than its average price – and we will always specify this in our articles.
We only recommend deals on products that we’ve tested or have been recommended by product experts. What we choose to feature is based on the best products at the best prices chosen by our editorially independent team, free of commercial influence.
The CX34 provides a palatable, lightweight computing experience for a reasonable price in this Currys deal. It has a more compact 14inch Full HD 60Hz screen with decent detail across a smaller resolution, plus 8GB of RAM and a sizeable 512GB SSD. As most of your work may be online (in Google Workspace), that’s a solid amount of space for apps and games. It also comes with ample power for casual use with Intel’s 10-core Core i5-1334U processor.
This is a Chromebook Plus model, so it’s a more powerful option for multitasking and benefits from clever Google software integrations such as Photo Eraser, Offline File Sync and Google Meet effects such as auto-framing and background blur powers.
Price history:
this is the best price in the past few months, but it’s been cheaper on Amazon in previous deals.
This Lenovo IdeaPad Slim 5 comes with a potent AMD Ryzen AI 7 350 processor inside, with eight cores and 16 threads. There’s 16GB of RAM and a 1TB SSD for a solid amount of headroom for multitasking and storage. This model comes with a 14in 1,920 x 1,200 IPS screen that should be reasonable for work purposes, plus it comes in a fetching silver colourway.
Price history:
this is the best price for this model.
The Samsung Galaxy Chromebook Plus is a strong choice for an upmarket Chromebook. It’s very sleek and lightweight at just 1.17kg, and it comes with a pleasant 14in Full HD panel, with up to 400 nits of brightness for sharp images. Inside, it’s got an Intel Core 5 120U processor with 10 cores and 12 threads to make productivity work a breeze. There’s 8GB of RAM and a 256GB SSD, and battery life is solid for the price.
Price history:
this is the best price for this model.
The Acer Swift Go 14 AI is a capable Windows laptop witha decent eight-core Snapdragon X processor that yields excellent efficiency, with Acer claiming up to 28 hours of battery life. There’s 16GB of RAM and a 1TB SSD, plus a decent 14.5in 1,920 x 1,200 120Hz screen for solid detail and slick motion for productivity workloads. At 1.36kg, it’s lightweight and portable, too.
Price history:
this is the best price for this model.
This Samsung Galaxy Book5 Pro 360 is a powerful premium 2-in-1 laptop to use either as a laptop or tablet. This base model has 15.6in Full HD AMOLED touchscreen with decent detail for creative tasks, plus lovely depth and contrast. Inside, this Samsung laptop has a solid mid-range Intel Core Ultra 5 processor plus 16GB of RAM. The 256GB SSD feels a little stingy at this price tag, though. It also has a decent set of ports with a pair of Thunderbolt 4-capable USB-C ports, HDMI, USB-A, and a microSD card reader.
Price history:
this is the lowest price it’s been in a month, although not the lowest ever.
For this price, this Lenovo IdeaPad Slim 3 has a decent core for productivity tasks as well as some headroom for more intensive loads. The Intel Core i5-13620H processor has eight cores and eight threads, while you also get 16GB of RAM and a 512GB SSD for storage. It’s a slim laptop for easier portability, although it still comes with a mid-size 15in IPS screen.
This Asus Vivobook S16 OLED nails the basics, if you’re after a big-screen laptop with a little something extra. Its Intel Core Ultra 5 225H processor delivers solid performance, while 32GB of RAM and a hefty capacity 1TB SSD provide enough headroom for intensive multitasking and installing of all sorts of apps.
A larger 16in Full HD+ resolution OLED screen displays high-quality output with deeper blacks, stronger contrast, and more accurate colours than standard IPS screens found elsewhere at this price. Up to 20 hours of battery life is a boon if you’re going to be away from the mains, too.
Price history:
not available, but this is the lowest price ever at Currys.
Apple’s M2 MacBook Air is a couple of years old now, but the Apple Silicon chip inside continues to deliver oodles of power for anything from productivity loads to editing high-res video on battery power. It’s sleek, portable and stylish, although it lacks ports, so you may need to pick up a cheap USB-C adapter to supplement. The 13.6in Liquid Retina screen is sharp and detailed, while 18 hours of battery life is sure to keep you up and running for a couple of working days away from the mains.
The current flagship MacBook Air has dropped back to its lowest ever price for Black Friday, complete with its capable M4 chip inside. It’s a beefier choice than older M-series chips, with faster single-core performance and two extra CPU and GPU cores for even more power. There are incremental upgrades elsewhere, such as 16GB on this base model, plus the ability to connect to two external displays with the lid down. Otherwise, this is the same modern MacBook Air we know and love, with a dazzling Liquid Retina screen and up to 18 hours of battery life. You will need a cheap USB-C adapter to extend the ports to a more usable selection.
For basic working needs, this Acer Aspire 14 AI has everything you need at a wallet-friendly price. The Snapdragon X chip inside provides more than enough power for day-to-day tasks, plus it enables this laptop to last for up to 28 hours on a charge, which means battery woes can be pushed to the back of your mind. A RAM allocation of 16GB is handy for multitasking, and a 512GB SSD is a decent amount of storage at this price. The 14in, 1,920 x 1,200 IPS panel is perfectly serviceable for productivity tasks, plus its 120Hz refresh rate keeps onscreen action zippy.
Price history:
not available, but this is the lowest ever price at Currys.
The HP Omnibook 5 is a solid mid-ranger, offering decent performance on a larger screen. An eight-core AMD Ryzen AI 7 350 processor is paired with 16GB of RAM and a larger 1TB SSD. It also benefits from a larger 16in, 1,920 x 1,200 IPS display. It also comes in a pleasant blue colour, and has ports including USB-C, USB-A and HDMI, plus a full-size, backlit keyboard.
Price history:
this is the best price for this model – historical best on Amazon is £820, so a good saving.
Made from an innovative blend of ceramic and aluminium, this Asus Zenbook A14 is one of the lightest Windows laptops you’ll find, weighing in at less than a kilo. Not only is it super light, but a Snapdragon X chip alongside 16GB of RAM ensures enough grunt for productivity and multitasking.
A 1TB SSD is handy for storing documents, apps, and more besides, while the 14in 1,920 x 1,200 OLED screen is compact and sharp. Asus also rates this laptop to last for up to 32 hours on a charge – while my tests put it at about 21 hours, I’ll happily take nearly three days of use away from the mains.
The Samsung Galaxy Book4 is an attractive Windows alternative to similarly priced Chromebooks, offering greater software flexibility for getting work done. It includes an impressive range of ports for the price, with USB-C, USB-A, HDMI, microSD and even wired Ethernet in the mix. The Intel Core 3 processor will happily cope with everyday productivity tasks, and is supported by 8GB of RAM and a 256GB SSD for storage.
If one screen isn’t enough for you, then the Asus Zenbook Duo is an intriguing concept. It’s a laptop that can be
all
screen, with a pair of 14in 3K 120Hz OLED screens with excellent clarity and motion. They can be used as touchscreens for design work or for some clever multi-tasking, especially with the laptop’s detachable keyboard and trackpad. Inside, there’s a powerful Intel Core Ultra 9 285H processor along with 32GB of RAM and a capacious 2TB SSD. Battery life is also pretty good, lasting 12 or so hours in my tests.
Price history:
this is the best price for this model.
If you want the powers of a conventional laptop and a very large-screen tablet in one, then the Dell 16 Plus 2-in-1 is a well-specced option, and it’s £300 off at Amazon (not currently listed as a price cut, but price history on Keepa shows the discount, making this a hidden deal of sorts).
It comes with a 16in, 1,920 x 1,200 IPS screen, providing lots of real estate for everything from conventional productivity tasks in its laptop form factor to more design-orientated projects in its tablet form. This screen has a vibrant 600 nits of peak brightness and there’s Dolby Vision HDR support for extra sharpness in supported content.
Inside, it’s got an eight-core Intel Core Ultra 7 256V processor – often found on laptops much more expensive than this. It provides great performance for both productivity and more intensive tasks, and is an efficient chip while doing so – Dell quotes battery life as up to 21 hours on a charge. To go with it, there’s 16GB of RAM and an OK capacity 512GB SSD. It also comes in a stylish sky blue colour, and has a decent port selection with two USB-Cs, a USB-A and an HDMI port.
Price history:
this is the best price for this model.
Chromebooks have always been deemed inferior to Windows laptops, but you can now easily find genuinely capable budget options with few compromises. Acer’s Chromebook Plus 515 features a modest Intel processor with six cores that should prove sufficiently speedy for basic tasks, while its 8GB of RAM will allow you to have multiple Chrome tabs open without the device grinding to a halt. You also get 256GB of SSD storage for apps and light games, plus room for any local documents that aren’t in Google’s online suite. There’s also a handy 15.6in Full HD screen and a decent set of ports for this bargain-basement price.
If you feel like you need the extra performance, you can step up to a Core i5 processor with an extra four cores for an
extra £102 at Amazon
.
Price history:
it was £12.79 cheaper in a deal this summer.
This Asus Chromebook Plus CX14 has the fundamentals of a decent laptop for kids’ schoolwork, with a compact 14in Full HD screen plus 8GB of RAM for reasonable multitasking and a 128GB SSD for any apps or lightweight games. Inside, there’s an Intel Core 3 N355 processor with eight cores, which should be reasonable for the light workloads Chromebooks are designed for.
This CX14 is a Chromebook Plus model, which means Google has designated it as a more powerful option. It benefits from clever software integrations such as the Photo Eraser editing trickery seen on Pixel phones, Offline File Sync for accessing Google Docs and so on offline, and Google Meet effects such as auto-framing and background blur.
Price history:
this is the best price at the moment – but it was lower earlier this Black Friday in a deal that’s now expired.
This Asus Vivobook 14 model is a compact and potent choice that won’t break the bank. Its Snapdragon X processor provides enough power for basic tasks and enables 19 hours of battery life per charge. There’s also 16GB of RAM and a 512GB SSD for solid multitasking headroom and storage needs. The compact 14in, 1,920 x 1,200 IPS screen has a refresh rate of up to 75Hz for a little bit more responsiveness than the more standard 60Hz.
This Vivobook also comes with a far-reaching port selection for the price, including a pair of USB-A ports, HDMI, USB-C and more besides.
Price history:
this is the best price for this model, although Amazon doesn’t stock it.
This Lenovo IdeaPad Slim 5 is on a “reduced to clear” discount at John Lewis, giving you the chance to grab a bit of a bargain. It has everything you could want from a modern laptop: a compact 14in 1,920 x 1,200 OLED screen for dazzling results; an eight-core Snapdragon X Plus chip for zippy performance; and excellent battery life – Lenovo says the laptop can last for up to 20 hours or so on a charge, providing all-day working and then some. For multitasking and intensive tasks, 16GB of RAM provides plenty of headroom, while a 512GB SSD is fine for storage at this price.
Price history:
this was briefly cheaper in the summer.
This Asus Zenbook 14 is a very capable choice. The Intel Core Ultra 9 285H processor with its 16 cores means it will be able to handle any tasks you throw at it, and 32GB of RAM and a 1TB SSD provide lots of capacity for multitasking and dealing with heavier, creative workloads. Elsewhere, the 14in 3K OLED screen is bright and delivers good detail, and a weight of just 1.2kg makes the Asus super portable. There’s a decent selection of ports, too –and its dark-blue chassis oozes class.
If you don’t necessarily need the power of the Core Ultra 9 285H, and you’re happy with a slightly lower-end Core Ultra 7 model (which performs quite similarly in some tests) with 16GB of RAM, then that model is £799 from John Lewis, too.
The Asus Zenbook S 16 OLED is one of the most complete ultrabooks you can buy today, making no real sacrifices anywhere. The star of the show is the gorgeous 16in, 3K-resolution screen, which delivers superb detail and general sharpness. On the inside sits a 12-core Ryzen AI R9 HX 370 processor, alongside 32GB of RAM and a 2TB SSD. There’s a decent set of ports and the casing is made from the same innovative ceraluminum material as the Zenbook A14 above, meaning it’s durable and stylish, too.
Price history:
this is its lowest ever price, and it’s cheaper than lower-spec deals on the same laptop.
This Lenovo Yoga Slim 7x option provides a very rich set of specs for the price. The 12-core Snapdragon X Elite processor delivers both in terms of performance and efficiency, with the laptop rated to last for up to 24 hours on a single charge. Add to this a decent 16GB of RAM and 1TB of storage.
Its compact 3K-resolution OLED screen displays plenty of detail in a smaller space, and up to 500 nits of brightness means images are sharp and vibrant. The Yoga Slim is also a touchscreen, giving you the flexibility to use it for more creative or design-type tasks. Go for the blue colourway to add some style to your workspace.
The Asus Vivobook S 14’s portable form factor houses an eight-core AMD Ryzen 9 270 processor, plus 32GB of RAM and a 1TB SSD, and should prove ample for general work tasks, whether at home or on the move. The 14in 1,920 x 1,200-resolution IPS panel might not be an OLED, but it’s still perfectly capable for what this laptop is designed for. The port selection here is also pretty good, providing decent connectivity for most people’s needs.
This Lenovo IdeaPad Slim 5 is a slightly older variant of the one above, arriving with a larger 16in, 1,920 x 1,200-resolution IPS screen, as opposed to that model’s 14in OLED. The eight cores and 12 threads of the Intel Core i5-13420H processor here deliver solid productivity performance, with room to step up to more intense workloads if the need arises. Elsewhere, 16GB of RAM and a capacious 1TB SSD are excellent for the price, plus there’s a decent port selection that includes USB-C, USB-A, HDMI, a microSD reader and more besides.
Price history:
this matches its lowest ever price.
The Acer Swift X 14 AI is a slim and powerful ultrabook, featuring a dazzling 14in 2,880 x 1,800 OLED display with a 120Hz refresh rate for smooth and responsive on-screen action. Its AMD Ryzen 7 AI 350 processor can handle anything from productivity tasks to more intensive work, with Nvidia’s RTX 5050 GPU stepping up where extra graphical horsepower is required. Elsewhere, the port count includes USB-C, USB-A, microSD and HDMI, all present in a chassis that’s less than 20mm thick and 1.57kg in weight.
LG’s Gram laptops have long been lightweight and slender choices in their size classes, and this 17in model is no exception, weighing in at just 1.479kg. It’s also just 14.5mm thick, but maintains a decent port selection with Thunderbolt 4-capable USB-C ports, USB-A and HDMI options.
The 17-inch 2.5K resolution screen with 144Hz refresh rate is zippy and responsive, thanks to an Nvidia RTX 5050 paired with a powerful Intel Core Ultra 7 255H processor. In spite of this power, LG says this laptop will last for up to 27 hours on a charge, giving you several days of work away from the mains.
Price history:
this is a match for its lowest ever price.
For a larger-screen Windows laptop for productivity tasks and the odd bit of more intensive work, this Asus Vivobook 16 is perfect. Performance is decent, thanks to a 10-core Intel Core 7-150U processor, plus 16GB of RAM and a 1TB SSD for your storage needs. The 16-inch 1,920 x 1,200 IPS screen is pretty standard at this price, but a lay-flat hinge makes this laptop great for collaborative working. You also benefit here from a full-size keyboard, while USB-C, USB-A, HDMI and a headphone jack make up the port count.
The Acer Aspire 14 AI is different to the model above: it comes running an eight-core AMD Ryzen AI 7 350 chip, with 16GB of RAM and a 1TB SSD, rather than the Arm-based Snapdragon processor. Display-wise, you get a 14in 1,920 x 1,200 OLED panel that delivers deeper blacks and stronger contrast and colour accuracy, and a good selection of ports. This model is a little more expensive than the other version, but I’d argue the expense is justified.
In keeping with the portable laptop theme, this LG Gram Pro 16Z90TS provides one of the lightest 16in laptops you’ll find, delivering a good selection of ports and solid performance, with 16GB of RAM and a 1TB SSD. Intel’s Core Ultra 7 256V processor with eight cores and eight threads, plus its potent integrated graphics, provide enough oomph for both basic workloads and more intensive tasks. It’s a shame the 16in 2.5K 144Hz panel isn’t OLED; but it’s a decent IPS screen – it’s responsive and delivers good detail. Lasting for up to 25.5 hours on a charge, you’ll get a few days away from the mains.
The Asus ProArt P16 (2025) is a creative’s dream, providing one of the most compelling Windows alternatives to a MacBook Pro. This RTX 5070 variant isn’t lacking in power for everything from video and photo editing to gaming, especially when paired with a AMD Ryzen AI 9 HX 370 processor. There’s also a generous 64GB of RAM and 2TB SSD.
The real kicker with this ProArt laptop is the display: a 16in 4K (3,840 x 2,400) 120Hz OLED screen that’s one of the best on the market for detail, clarity and generally sharp images – and I also found it pretty bright in my testing. There’s one of the best port selections out there, plus a comfortable keyboard, huge trackpad and stylish chassis. You are paying out for it, but this is a really excellent laptop.
Price history:
this is the best ever price for this model.
The Acer Nitro V15 is a respectable affordable gaming laptop and it’s dropped to its lowest price. It features a solid core for Full HD gaming, pairing Nvidia’s modest RTX 5060 GPU with a decently potent Intel Core i7-13620H processor. That might not be the latest in Team Blue’s catalogue, but it’s still capable, with 10 cores and 16 threads. Elsewhere, there’s a 15.6in Full HD IPS screen for solid detail, plus a 165Hz refresh rate for slick motion. There’s not too much of a gamer aesthetic, and there’s a solid set of ports, with everything from Ethernet for wired networking to USB-A and USB-C for expansion. The 1TB SSD is generous at this price.
Price history:
this is the best ever price by £50.
Asus’s ROG gaming laptops typically carry a premium, but this Strix G16 is one of the cheapest RTX 5070 Ti-powered gaming machines available right now. Pairing it with a 16-core AMD Ryzen 9 7940HX processor will yield very capable gaming performance at this laptop’s native 1,920 x 1,200 resolution.
The display also has a 165Hz refresh rate for more responsive onscreen action. Modern AAA games can be a storage hog, but the 1TB SSD means there’s enough headroom for a good few here, while 16GB of RAM is enough for gaming loads.
Price history:
not available, but cheaper than the closest equivalent on Amazon.
Acer’s Nitro V16 is a strong mid-range gaming laptop, especially in this spec, which pairs an RTX 5070 graphics card with AMD’s eight-core Ryzen AI 7 350 processor. The setup delivers solid performance at 1080p and the laptop’s native 2,560 x 1,600 resolution – although the higher resolution may benefit from reduced settings and some upscaling. A 180Hz refresh rate makes for a smooth and responsive panel, and the laptop comes with a well-rounded port selection, too. Acer rounds off the package with 16GB of RAM and a 1TB SSD.
At £799, the Asus V16 is quite a feature-rich gaming laptop, as long as you don’t mind its modest 1080p display. The 10-core, 16-thread Intel Core 7 240H processor paired with an RTX 5060 laptop GPU brings solid performance to the table, alongside the powers of Nvidia’s DLSS4 upscaler and the multi-frame-gen tech, if you want it. The 16GB of RAM will be good to run most modern games, with the 1TB SSD generous for storage. All of this helps to drive a large, 16in, 1,920 x 1,200-resolution, 144Hz-refresh-rate screen for a solid blend of detail and responsiveness. An array of USB-C, USB-A, HDMI ports and more deliver decent connectivity, too.
If you’re after a decent gaming machine for 1080p duties, then this HP Victus 15-fb2008na model packs a punch for a decent price. It might feature the latest components inside, but it’s one of the better budget gaming laptops out there, pairing an RTX 4060 GPU with a six-core AMD Ryzen 5 8645HS processor.
There’s a 15.6in Full HD 144Hz IPS screen, plus 16GB of RAM and a 512GB SSD. Connectivity is very capable for the price, too, with USB-As, USB-C, HDMI, wired Ethernet and an SD card reader.
Price history:
this is the best price for this model, although Amazon doesn’t stock it.
Asus’s gaming laptops carry a bit of a premium, and arguably none more so than its Zephyrus line. This is Asus’ most stylish set of laptops, pairing beefy internals for gaming grunt with a compact, suave chassis. In this example, there’s an RTX 5070 GPU and an AMD Ryzen 9 270 processor, plus 32GB of RAM and a 1TB SSD. That’s a good core for playing games at both 1080p and 1440p. The display here is a dazzling 14in 3K (or 2,880 x 1,800) 120Hz OLED screen for sublime detail, clarity and smooth motion, plus we’ve got a slim and slender chassis for such a powerful laptop that has a good set of ports. It includes a pair of USB-C ports, plus two USB-As, HDMI and more.
Price history:
£50 off the best price ever from Amazon.
If it’s a
very
capable gaming laptop you’re after, this Alienware 18 Area-51 is one of the strongest options you’ll find. A 24-core Intel Core Ultra 9 275HX and Nvidia’s second-in-command RTX 5080 laptop GPU deliver the goods for gaming on its huge 18in QHD+ resolution screen. The IPS panel here is strong, too, with its super-high 300Hz refresh rate bringing impeccable motion handling. There’s 32GB of DDR5 RAM and a generous 2TB SSD. Sporting Alienware’s classic space-age looks, you’ll need some muscle if you plan to use it on the move – this laptop is big and bulky; but the extra room also means it arrives with an enviable set of ports.
The Acer Predator Helios Neo 16 AI is one of the best value gaming laptops in its price class – but it’s become an even stronger proposition with a £300 price cut. On the inside beats an RTX 5070 Ti GPU alongside the same beefy Intel Core Ultra 9 275HX processor as the Alienware option above to handle the most demanding of games on its 16-inch, 2,560 x 1,600-resolution screen. The panel’s 240Hz refresh rate delivers smooth motion, plus you also get 16GB of RAM and a 1TB SSD. Those looking for style as well as substance won’t be disappointed, as the Acer is quite a looker compared to other gaming behemoths out there. If price-to-performance is the name of the game, this is a candidate for the best we’ve seen this Black Friday so far.
Price history:
this is its lowest ever price, although it was only 1p more for a period in September.
Reece Bithrey
Show HN: An LLM-Powered Tool to Catch PCB Schematic Mistakes
After working with LLMs for the last 15 months, these are some of the anti-patterns I have discovered.
By anti-patterns, I simply mean patterns or behaviors we should avoid when working with LLMs.
1. Did I tell you that already?
Context is a scarce resource and probably worth its weight in gold, we need to use it wisely. One of the learnings is to not send the same information/text multiple times in the same session.
For example, during
computer-use
sending each and every image frame when a mouse is going from point A to point B on the screen as screenshots with barely anything changing between a lot of consecutive frames (mouse pointer moving 1 millimeter for example) in each API call, when just one new and final screenshot showing current context is enough.
It's sort of an irony that the same company has come up with a context management
tool/api
, which helps you reduce/compress the context by removing redundant messages while it did exact opposite for computer-use and sent all previous almost duplicated screenshots in every new LLM api call again. We built open-source
click3
which does it without sending any possibly duplicate screenshots in API calls - screenshots with significant differences (or taken at state changes) are enough for the LLM to decide next course of action.
2. Asking a fish to climb a tree
Should we ask the fish to climb a tree? Sure sometimes they can climb a tree, but better ask them do things they are good at. For example, asking Gemini Banana to generate an image on a wooden plank with a text starting with prefix 1AA..(notice the double A) always ended up with 1A.. (single A) after 13 tries or so, i decided to give up. Later, I had an idea - to write the text in a google doc, take its picture and then give the picture and ask it to merge it on a wooden plank picture (also given by me) -- It did it in 1 shot.
Similarly we should not ask LLMs how many Rs are there in BLUEBERRY - we should ask it to write a code which counts the Rs. Coding ability > Counting ability - atleast for the current LLMs.
Take another example,
Cloudflare recently realised
that tool calling is better when its written as
code
that calls them. So, it seems we should ask it to generate code whenever we expect more accurate answers.
The climbing perch - A tree climbing fish
3. Asking LLM to speak, when its drowning (in context)
LLMs do best when it's not nearly full with 128k tokens. For long running sessions, which go beyond the 128k token count - it can be even worse, we then depend on the ability of the Claude to compress or discard information based on its whim. For example, the other day, it completely forgot about a database connection URL I had given it and started spitting someone else's database URL in the same session. Thankfully(for them) that URL didn't work. Unfortunately, some tasks do need big contexts, my only advice in that case is to be aware of its accuracy decline.
Some random database url, from its memory
4. The squeaky wheel gets the grease
LLMs don't perform well on obscure topics. Similarly and as expected, on topics which were invented after their training cut-off dates, for the simple reason of them not being trained on those topics. They perform well on topics which have been widely discussed. So if your topic is an obscure one, assume less accuracy and figure out ways to make it accurate. Here is an instance of Claude-CLI giving up on Stripe integration which btw has one of the nicest documentation -
5. You don't want to be a vibe-coder
It's easy to slip into a manager (or as Andrej Karpathy calls it - a vibe-coder) mode with Claude Code like tool but in my observation if you lose the sight of what the LLM is writing, it will eventually be a net loss. Never lose the thread of what's going on. For example, in the
/invoices
api, Claude decided it was fine to put the
User
object in the response json, since it is part of the invoice object. Only I could see it was exposing the
password_hash
unnecessarily. Although not a security issue immediately, but if something goes wrong, and the attackers gets access to the invoice jsons, this will only help the attackers get more important information. Or imagine someone not even hashing the password and getting exposed. You get the point.
After a long break from working on my hobby operating system, I finally got back into it and finished a very important milestone: a working web server.
Web browser accessing HTTP server on my OS
Networking was always integral to my hobby project. The first goal was getting the basic networking stack working: Ethernet, IP, ARP, UDP, TCP, DHCP and DNS. Besides TCP this was rather straightforward, but when moving onto HTTP things broke.
This led to my first break from the project, but also left a nagging thought in my mind, wanting to make it work. I finally sat down and started debugging.
I eventually found the culprit after hours of dissecting my own code, the problem was a broken implementation of the terminal buffer, overwriting a lock in another process… fun. Additionally, the E1000 network driver did not correctly handle incoming packets, which I finally got working by handling bursts of packets.
Performance and hardening
After getting an HTML page returned from the web engine I started noticing lots of performance errors and hangs from TCP, mainly because quickly refreshing the browser led to a spam of RST packets which were not handled correctly.
After a few hours of tinkering I finally got the RST packets working and the network stack is now able to handle a packet spam from the browser.
The HTTP Engine
Next step was actually implementing a HTTP engine, parsing the requests from the user. Before this engine I simply returned a static HTTP response no matter the actual request.
Keeping with the spirit of this hobby OS I want to write everything from scratch, luckily I already had implemented a pretty complete HTTP parser for my other project
c-web-modules
. So I extracted the HTTP parser as a standalone library and ported it to my OS.
The Web Engine.
After the HTTP engine was done I moved onto the web engine, focusing on something small, rather than big and fancy. Mainly routing was important and adding route handlers. Allowing the user to specify a route, method and lambda function handler.
It’s a tiny example, but it mirrors how a lot of modern C++ and web frameworks think about routing: match a path + method, call a handler, build a response.
#lang:plaintext
[ Browser ]
|
v
[ Web Server (userspace):
WebEngine | HTTPEngine | FileRepository ]
|
v
[ Network stack:
TCP/UDP | IP | ARP | DHCP | DNS | Ethernet(E1000) ]
The Web Server.
The last step was updating the userspace program with the new HTTP and Web engine. Finally I added a way to serve files using a FileRepository which supports caching. Now I can edit the files inside the operating system and then serve them with the web server.
I don’t remember when I first started noticing that people I knew out in the world had lost their sense of erotic privacy, but I do remember the day it struck me as a phenomenon that had escaped my timeline and entered my real, fleshy life. It was last year, when I was having a conversation with a friend of mine, who, for the record, is five years younger than me (I’m 31). I told my friend about an erotic encounter I’d just experienced and very much delighted in, in which I had my hair brushed at the same time by two very beautiful women at the hair salon — one was teaching the other how to do it a certain way. When I finished my story, my friend looked at me, horrified.
“They had no idea you felt something sexual about them,” she said. “What if they found out? Lowkey, I hate to say this but: you took advantage of them.” I was shocked. I tried to explain — and it felt extremely absurd to explain — that this had happened in my body and in my thoughts, which were private to me and which nobody had the right to know about. But they did have the right, my friend argued. She demanded that I apologize to the women for sexualizing them. Offended at having been accused — in my view, in extremely bad faith — of being some kind of peep-show creep, I tried to argue that I’d simply responded in a physical way to an unexpected, direct, and involuntary stimulus. Back and forth, back and forth, we fought like this for a while. In fact, it ended the friendship.
There were other conversations, too, that suggested to me that conceptions of love and sex have changed fundamentally among people I know. Too many of my friends and acquaintances — of varying degrees of “onlineness,” from veteran discourse observers to casual browsers — seem to have internalized the internet’s tendency to reach for the least charitable interpretation of every glancing thought and, as a result, to have pathologized what I would characterize as the normal, internal vagaries of desire.
Hence, there was the friend who justified her predilection for being praised in bed as a “kink” inherited through the “trauma” of her father always harping on her because of her grades. There was the friend who felt entitled to posting screenshots of intimate conversations on Twitter after a messy breakup so that she could get a ruling on “who was the crazy one.” Then there was the friend who bitterly described a man he was dating as a “fuckboy” because he stood him up, claiming that their having enjoyed sex together beforehand was “emotionally manipulative.” When I dug a bit deeper, it turned out the man in question had just gotten out of a seven-year relationship and realized he wasn’t ready to be sexually intimate, and while he was rude to stand my friend up, it shocked me how quick my friend was to categorize his rightfully hurt feelings as something pathological or sinister in the other person, and that he did this in order to preemptively shield himself from being cast as the villain in what was a multi-party experience. This last friend I asked: “Who are you defending yourself against?” To which he answered, to my astonishment: “I don’t know. The world.”
I choose these examples from my personal life because they express sentiments that were once the kind of stuff I encountered only in the messy battlegrounds of Twitter, amid discussions about whether Sabrina Carpenter is being oversexualized, whether kinks are akin to a sexual orientation, whether a woman can truly consent in an age-gap relationship, and whether exposure to sex scenes in movies violates viewer consent. It is quite easy to dismiss these “discourse wars” as a “puritanism” afflicting the young, a reactionary current to be solved with a different, corrective discourse of pro-sex liberation, distributed via those same channels. If only it were so! To me, the reality goes deeper and is bleaker.
The fact is that our most intimate interactions with others are now governed by the expectation of surveillance and punishment from an online public. One can never be sure that this public or someone who could potentially expose us to it isn’t there
,
always secretly filming, posting, taking notes, ready to pounce the second one does something cringe or problematic (as defined by whom?). To claim that these matters are merely discursive in nature is to ignore the problem. Because love and sex are so intimate and vulnerable, the stakes of punishment are higher, and the fear of it penetrates deeper into the psyche and is harder to rationalize away than, say, fear of pushback from tweeting a divisive political opinion.
I should state at this point that this is not an essay about “cancel culture going too far,” a topic which can now be historicized as little more than a rhetorical cudgel wielded successfully by the right to wrest cultural power back from an ascendant progressive liberalism. This was especially true after the prominence of organized campaigns such as #MeToo. #MeToo was smeared by liberals and conservatives alike (united, as they always are, in misogyny) as being inherently punitive in nature, meant to punish men who’d fallen into a rough patch of bad behavior, or who, perhaps, might not have done anything at all (the falsely accused or the misinterpreted man became the real victim, in this view). #MeToo did make use of the call-out — the story shared in a spreadsheet anonymously or in a signed op-ed — but the call-outs had a purpose: to end a long-standing and long-permitted norm of sexual abuse within institutions. Underlying this was a discursive practice and a form of solidarity building in which people believed that sharing their stories of trauma en masse could bring about structural change. As someone who participated myself, I too believed in this theory and saw it as necessary, cathartic, and political, and far from vigilante justice.
But the pushback against #MeToo reveals a certain peril to storytelling as politics, not only in the retraumatization evident in the practice of revealing one’s most intimate harms before an infinite online audience, which could always include those listening in bad faith. But also, a discursive market opened up in which trauma became a kind of currency of authenticity, resulting in a doubled exploitation. This idea, while not very nice, lingers in the use of harm as an authoritative form of rhetorical defense. The problem here is not what is said, but how it is used. A friction has since emerged between an awareness of weaponization of harm and emotion and the continued need to express oneself as vulnerably as possible in order to come off as sincere. This friction is unresolved.
The organized goals of the #MeToo movement are missing from the new puritanism. I think that the prudish revulsion I’ve seen online and in my own life has as much to do with surveillance as with sex. Punishing strangers for their perceived perversion is a form of compensation for a process that is already completed: the erosion of erotic and emotional privacy through internet-driven surveillance practices, practices we have since turned inward on ourselves. In short, we have become our own panopticons.
The prudish revulsion I’ve seen online and in my own life has as much to do with surveillance as with sex.
On the rightmost side of the spectrum, punitive anti-erotic surveillance is very explicit and very real, especially for women. The Andrew Tates of the world and the practitioners of extreme forms of misogyny have no problem with using internet tools and social media websites for mass shaming and explicit harm. Covert filming of sex acts, AI deep fakes, extortion, and revenge porn are all realities one has to contend with when thinking about hooking up or going to public places such as nightclubs and gay bars. This is blackmail at its most explicit and extreme, meant to further solidify a link between sex and fear.
But that link between sex and fear is operating in more “benign” or common modes of internet practice. There is an online culture that thinks nothing of submitting screenshots, notes, videos, and photos with calls for collective judgement. When it became desirable and permissible to transform our own lives into content, it didn’t take long before a sense of entitlement emerged that extended that transformation to people we know and to strangers. My ex sent me this text, clearly she is the crazy one, right? Look at this dumb/funny/cringe Hinge profile! Look at this note some guy sent me, is this a red flag? Look at this random woman I photographed buying wine, coconut oil, and a long cucumber at the supermarket!
I think these kinds of posts sometimes amount to little more than common bullying, but they are on a continuum with a puritan discourse in which intimate questions, practices, and beliefs about queerness, sexuality, gender presentation, and desire are also subjected to days-long piles-on. In both instances, the instinct to submit online strangers to viral discipline is given a faux-radical sheen. It’s a kind of casual blackmail that warns everyone to conform or be exposed; a way of saying if you don’t cave to my point of view, redefine yourself in my image of what sexuality is or should be, and (most importantly) apologize to me and the public, I will subject you to my large following and there will be hell to pay. Such unproductive and antisocial behavior is justified as a step toward liberation from predation, misogyny, or any number of other harms. But the punitive mindset we’ve developed towards relationships is indicative of an inability to imagine a future of gendered or sexual relations without subjugation. To couch that in the language of harm reduction and trauma delegitimizes both.
There are other ways the politics of surveillance have become a kind of funhouse mirror. It is seen as more and more normal to track one’s partner through Find My iPhone or an AirTag, even though the potential for abuse of this technology is staggering and obvious. There are all kinds of new products, such as a biometric ring that is allegedly able to tell you whether your partner is cheating, that expand this capability into more and more granular settings. That’s all before we get into the endless TikToks about “why I go through my partner’s text messages.” That men use these tactics and tools to control women is a known threat. What is astonishing is the lengths to which some women will go to use these same technologies, claiming that they are necessary to
prevent
harm — especially that caused by cheating, which is now seen as some kind of lifelong trauma or permanently damnable offense instead of one of the rather quotidian, if very painful, ways we hurt one another. Each of these surveillance practices operates from a feeling of entitlement and control over other people, their bodies, and what they do.
Pundits like to decree sexlessness as a Gen-Z problem, to argue no one is fucking because they are too on their phones. However, it is always too easy to blame the young. It was my generation that failed to instill the social norms necessary to prevent a situation where fear of strangers on the internet has successfully replaced the disciplinary apparatus more commonly held by religious or conservative doctrine. Even when, as in my experience in the salon, I am acting in the privacy of my own body, someone is always there watching, ready to interpret my actions, problematize them so as to share in the same sense of magical thinking, the same insecurities, and to be punished for not being insecure in the same way.
It’s only in retrospect that I’m able to realize the toll that constant, nagging interaction with my devices and the internet has taken on my thinking life
and
my sex life. I remember very viscerally when I’d just come out of the closet as bisexual in 2016. When I embarked on a journey to find the kind of lover I wanted to be, my only experience with the world of queerness was online through memes, articles, and others’ social media presentation of themselves and of politics. Queer sex was not something that could be discovered through sensation, through physical interaction, but was rather a catalog of specific acts and roles one was already expected to know. I was terrified of making some kind of mistake, of being the wrong kind of bisexual, of misrepresenting myself in an offensive way (could I use the term “soft butch” if I wasn’t a lesbian?), of being exposed somehow as a fraud. When the time came for me to have sex for the first time, what should have been a joyous occasion was instead burdened with a sense of being watched. I could not let the natural processes of erotic discovery take their course, so caught up was I in judging myself from the perspective of strangers to whom I owed nothing.
But it wasn’t just a matter of queerness, either. When I hooked up with men, I could only perceive of sex the same way, not as situational but as a set of prescribed acts and scenes, many of which I wanted to explore. However, this time I interrogated these urges as being sociogenic in nature and somehow harmful to me, when they were, in fact, private, and I did not, in reality, feel harmed. Because I wanted, at one point in my life, to be tied up and gagged, the disempowering nature of such a want necessitated trying to justify it against invisible accusations with some kind of traumatogenic and immutable quality. Maybe it was because I was raped in college. Maybe I was just inherently submissive. One of the great ironies in the history of sex is that pathologization used to be a way of controlling sexual desire. (All are familiar with the many myths that masturbation would turn one blind.) Now it is a way of exempting oneself, of relinquishing control of one’s actions so as to absolve them of scrutiny. My little bondage moment couldn’t be problematic if it couldn’t be helped. It couldn’t be subjected to interrogation if there was something I could point to to say “it’s beyond my control, don’t judge me!” One day, however, I came to an important revelation: The reality was much simpler. It was a passing phase, a desire that originated with a specific man and lost its charm after I moved on from him. There wasn’t some deterministic quality in myself that made me like this. My desire was not fixed in nature. My sexual qualities were transient and not inborn. What aroused me was wonderfully, entirely situational.
A situational eroticism is what is needed now, in our literalist times. It’s exhausting, how everything is so readily defined by types, acts, traumas, kinks, fetishes, pathology, and aesthetics. To me, our predilection for determinism is an expected psychological response to excessive surveillance. A situational eroticism decouples sensation from narrative and typology. It allows us to feel without excuse and to relate our feelings to our immediate embodied moment, grounded in a fundamental sense of personal privacy. While it is admirable to try and understand ourselves and important to protect ourselves from harm and investigate critically the ways in which what we want may put us at risk of that harm — or at risk of doing harm to others — sometimes desires just are, and they are not that way for long. Arousal is a matter of the self, which takes place within the body, a space no one can see into. It is often a mystery, a surprise, a discovery. It can happen at a small scale, say, the frisson of two sets of fingers in one’s hair at once. It is beautiful, unplanned and does not judge itself because it is an inert sensation, unimbued with premeditated meaning. This should liberate rather than frighten us. Maybe what it means doesn’t matter. Maybe we don’t have to justify it even to ourselves.
We are very afraid not of sex, but of exposure.
But in order to facilitate a return to situational eroticism, we need to kill the panopticon in our heads. That means first killing the panopticon we’ve built for others. There is no purpose in vindictive or thoughtless exposure. Not everything needs to be subjected to public opinion, not every anecdote is worth sharing, not every debate needs engagement, especially those debates which have no material basis to them, no ask, no funnel for all that energy. We need to stop confusing vigilantism with justice and posting with politics. That does not mean we stop the work that #MeToo started, but that revenge is a weapon best utilized collectively against the enemies of liberation. We need to protect the vulnerable from exploitative technologies and practices, repeatedly denounce their use, and work towards a world without sexual coercion, digital or otherwise.
On an individual level, we need to abandon or reshape our relationships with our phones and regain a sense of our own personal and mental privacy. It’s a matter of existential, metaphysical importance. Only when this decoupling from ourselves and the mediated performance of ourselves is complete, can we begin the process of returning to our own bodies out there, in the world, with no one watching or reading our thoughts except those we want to. The truth is, we are very afraid not of sex, but of exposure. Only when we are unafraid can we begin to let desire flourish. Only when we return to ourselves can we really know what we want.
Kate Wagner is the architecture critic at
The Nation
. Her award-winning cultural writing has been featured in magazines ranging from
The Baffler
to the
New Republic
.
Apple supply chain analyst Ming-Chi Kuo today
said
Intel is expected to begin shipping Apple's lowest-end M-series chip as early as mid-2027.
Kuo said Apple plans to utilize Intel's
18A process
, which is the "earliest available sub-2nm advanced node manufactured in North America."
If this rumor proves to be accurate, Intel could supply Apple with M6 or M7 chips for future MacBook Air, iPad Air, and iPad Pro models at a minimum. However, while previous Intel chips for Macs were designed by Intel and based on x86 architecture, M-series chips are designed by Apple and use Arm architecture. Intel would only assist with manufacturing.
TSMC would continue to supply the majority of Apple's M-series chips.
Kuo said that Apple choosing to have Intel supply its lowest-end M-series chip would appease the Trump administration's desire for "Made in USA" products, and it would also help Apple to diversify its supply chain for manufacturing.
Apple began transitioning away from Intel processors in Macs in 2020, and its own M-series chips continue to provide industry-leading performance per watt.
Apple's online store is going down for a few hours on a rolling country-by-country basis right now, but do not get your hopes up for new products.
Apple takes its online store down for a few hours ahead of Black Friday every year to tease/prepare for its annual gift card offer with the purchase of select products. The store already went down and came back online in Australia and New Zealand, ...
Apple's first foldable iPhone is expected to launch alongside the iPhone 18 Pro models in fall 2026, and it's shaping up to include three standout features that could set it apart from the competition.
The book-style foldable will reportedly feature an industry-first 24-megapixel under-display camera built into the inner display, according to a recent JP Morgan equity research report. That...
Apple recently teamed up with Japanese fashion brand ISSEY MIYAKE to create the iPhone Pocket, a limited-edition knitted accessory designed to carry an iPhone. However, it is now completely sold out in all countries where it was released.
iPhone Pocket became available to order on Apple's online store starting Friday, November 14, in the United States, France, China, Italy, Japan, Singapore, ...
We've been focusing on deals on physical products over the past few weeks, but Black Friday is also a great time of year to purchase a streaming membership. Some of the biggest services have great discounts for new and select returning members this week, including Disney+, Hulu, Paramount+, Peacock, and more.
Note: MacRumors is an affiliate partner with some of these vendors. When you click a...
We've been focusing on deals on physical products over the past few weeks, but Black Friday is also a great time of year to purchase a streaming membership. Some of the biggest services have great discounts for new and select returning members this week, including Apple TV, Disney+, Hulu, Paramount+, Peacock, and more.
Note: MacRumors is an affiliate partner with some of these vendors. When...
Black Friday is in full swing, and as always this is the best time of the year to shop for great deals, including popular Apple products like AirPods, iPad, Apple Watch, and more. In this article, the majority of the discounts will be found on Amazon.
Note: MacRumors is an affiliate partner with some of these vendors. When you click a link and make a purchase, we may receive a small payment,...
Singapore has ordered Apple to block or filter messages on iMessage that impersonate government agencies, requiring the company to implement new anti-spoofing protections by December as part of efforts to curb rising online scams, the Straits Times reports.
Singapore's Ministry of Home Affairs (MHA) said that it had issued an Implementation Directive to Apple under the Online Criminal Harms...
Apple's disappointing iPhone Air sales are causing major Chinese mobile vendors to scrap or freeze their own ultra-thin phone projects, according to reports coming out of Asia.
Since the iPhone Air launched in September, there have been reports of poor sales and manufacturing cuts, while Apple's supply chain has scaled back shipments and production.
Apple supplier Foxconn has...
200 Lines of Python beats $50M supercomputer – Navier-Stokes at Re=10⁸ [pdf]
When we launched
Skald
, we wanted it to not only be self-hostable, but also for one to be able to run it without sending any data to third-parties.
With LLMs getting better and better, privacy-sensitive organizations shouldn't have to choose between being left behind by not accessing frontier models and doing away with their committment (or legal requirement) for data privacy.
So here's what we did to support this use case and also some benchmarks comparing performance when using proprietary APIs vs self-hosted open-source tech.
RAG components and their OSS alternatives
A basic RAG usually has the following core components:
A vector database
A vector embeddings model
An LLM
And most times it also has these as well:
A reranker
Document parsing (for PDFs, PowerPoints, etc)
What that means is that when you're looking to build a fully local RAG setup, you'll need to substitute whatever SaaS providers you're using for a local option for each of those components.
Here's a table with some examples of what we might use in a scenario where we can use third-party Cloud services and one where we can't:
Do note that running something locally does not mean it
needs
to be open-source, as one could pay for a license to self-host proprietary software. But at Skald our goal was to use fully open-source tech, which is what I'll be convering here.
The table above is far from covering all available options on both columns, but basically it gives you an indication of what to research into in order to pick a tool that works for you.
As with anything, what works for you will greatly depend on your use case. And you need to be prepared to run a few more services than you're used to if you've just been calling APIs.
For our local stack, we went with the easiest setup for now to get it working (and it does! see writeup on this lower down) but will be running benchmarks on all other options to determine the best possible setup.
This is what we have today:
Vector DB:
Postgres + pgvector. We already use Postgres and didn't want to bundle another service into our stack, but this is
controversial
and we will be running benchmarks to make a better informed decision here. Note that pgvector will serve a lot of use cases well all the way up to hundreds of thousands of documents, though.
Vector embeddings:
Users can configure this in Skald and we use Sentence Transformers (
all-MiniLM-L6-v2
) as our default (solid all-around performer for speed and retrieval, English-only). I also ran Skald with bge-m3 (larger, multi-language) and share the results later in this post.
LLM:
We don't even bundle a default with Skald and it's up to the users to run and manage this. I tested our setup with GPT-OSS 20B on EC2 (results shown below).
Reranker:
Users can also configure this in Skald, and the default is the Sentence Transformers
cross encoder
(solid, English-only). I've also used bge-reranker-v2-m3 and mmarco-mMiniLMv2-L12-H384-v1 which offer multi-lingual support.
Document parsing:
There isn't much of a question on this one. We're using Docling. It's great. We run it via
docling-serve
.
Does it perform though?
So the main goal here was first to get something working then ensure it worked well with our platform and could be easily deployed. From here we'll be running extensive benchmarks and working with our clients to provide a solid setup that both performs well but is also not a nightmare to deploy and manage.
From that perspective, this was a great success.
Deploying a production instance of Skald with this whole stack took me 8 minutes, and that comes bundled with the vector database (well, Postgres), a reranking and embedding service, and Docling.
The only thing I needed to run separately was the LLM, which I did via
llama.cpp
.
Having gotten this sorted, I imported all the content from the PostHog website
[1]
and set up a tiny dataset
[2]
of questions and expected answers inside of Skald, then used our Experiments feature to run the RAG over this dataset.
I explicitly kept the topK values really high (100 for the vector search and 50 for post-reranking), as I was mostly testing for accuracy and wanted to see the performance when questions required e.g. aggregating context over 15+ documents.
Full config
Here are the params configured in the Skald UI for the the experiment.
Config option
Selection
Extra system prompt
Be really concise in your answers
Query rewriting
Off
Vector search topK
100
Vector search distance threshold
0.8
Reranking
On
Reranking topK
50
References
Off
So without any more delay, here are the results of my not-very-scientific at all benchmark using the experimentation platform inside of Skald.
Voyage + Claude
This is our default Cloud setup. We use voyage-3-large and rerank-2.5 from
Voyage AI
as our embedding and reranking models respectively, and we default to Claude Sonnet 3.7 for responses (users can configure the model though).
It passed with flying colors.
Our LLM-as-a-Judge gave an average score of 9.45 to the responses, and I basically agree with the assessment. All answers were correct, with one missing a few extra bits of context.
Voyage + GPT-OSS 20B
With the control experiment done, I then moved on to a setup where I kept Voyage as the embeddings provider and reranker, and then used GPT-OSS 20B running on a llama.cpp server on a g5.2xlarge EC2 instance as the LLM.
The goal here was to see how well the open-source LLM model itself stacked up against a frontier model accessed via API.
And it did great!
We don't yet support LLM-as-a-Judge on fully local deployments, so the only score we have here is mine. I scored the answers an average of 9.18 and they were all correct, with two of them just missing a few bits of information or highlighting less relevant information from the context.
Fully local + GPT-OSS 20B
Lastly, it was time for the moment of truth: running a fully local setup.
For this I ran two tests:
1. Default sentence transformers embedding and reranking models
Here the average score was 7.10. Not bad, but definitely not great. However, when we dig into the results, we can get a better understanding of how this setup fails.
Basically, it got all point queries right, which are questions where the answer is somewhere in the mess of documents, but can be found from one specific place.
Where it failed was:
Non-english query: The embeddings model and the reranker are English-based, so my question in Portuguese obviously got no answer
An ambiguous question with very little context ("what's ch")
Aggregating information from multiple documents/chunks e.g. it only found 5 out of PostHog's 7 funding rounds, and only a subset of the PostHog competitors that offer session replay (as mentioned in the source data)
In my view, this is good news. That means that the default options will go a long way and should give you very good performance if your use case is only doing point queries in English. The other great thing is that these models are also fast.
Now, if you need to handle ambiguity better, or handle questions in other languages, then this setup is simply not for you.
2. Multi-lingual models
The next test I did used
bge-m3
as the embeddings model and
mmarco-mMiniLMv2-L12-H384-v1
as the reranker. The embeddings model is supposedly much better than the one used in the previous test and is also multi-lingual. The reranker on the other hand uses the same cross-encoder from the previous test as the base model but also adds multi-lingual support. The more standard option here would have been the much more popular
bge-reranker-v2-m3
model, but I found it to be much slower. I intend to tweak my setup and test it again, however.
Anyway, onto the results! I scored it 8.63 on average, which is very good. There were no complete failures, and it handled the question in Portuguese well.
The mistakes it made were:
This new setup also did not do the best job at aggregating information, missing 2 of PostHog's funding rounds, and a couple of its session replay competitors
It also answered a question correctly, but added incorrect additional context after it
So overall it performed quite well. Again what we what saw was the main problem is when the context needed for the response is scattered across multiple documents. There are various techniques to help with this and we'll be trialing some soon! They haven't been needed on the Cloud version because better models save you from having to add complexity for minimal performance gains, but as we're focused on building a really solid setup for local deploys, we'll be looking into this more and more.
Now what?
I hope this writeup has provided you with at least some insight and context into building a local RAG, and also the fact that it does work, it can serve a lot of use cases, and that the tendency is for this setup to get better and better as a) models improve b) we get more open-source models across the board, with both being things that we seem to be trending towards.
As for us at Skald, we intend to polish this setup further in order to serve even more use cases really well, as well as intend to soon be publishing more legitimate benchmarks for models in the open-source space, from LLMs to rerankers.
If you're a company that needs to run AI tooling in air-gapped infrastructure, let's chat -- feel free to email me at yakko [at] useskald [dot] com.
Lastly, if you want to get involved, feel free to chat to us over on our
GitHub repo
(MIT-licensed) or catch us on
Slack
.
[1]
I used the PostHog website here because the website content is MIT-licensed (yes, wild) and readily-available as markdown on
GitHub
and having worked there I know a lot of answers off the top of my head making it a great dataset of ~2000 documents that I know well.
[2]
The questions and answers dataset I used for the experiments was the following:
Dataset
Question
Expected answer
Comments
How many raises did PostHog do?
PostHog has raised money 7 times: it raised $150k from YCombinator, then did a seed round ($3.025M), a series A ($12M), a series B ($15M), a series C ($10M), a series D ($70M), and a series E ($75M).
Requires aggregating context from at least 7 documents
When did group analytics launch?
December 16, 2021.
Point query, multiple mentions to "group analytics" in the source docs
Why was the sessions page removed?
The sessions page was removed because it was confusing and limited in functionality. It was replaced by the 'Recordings' tab.
Point query, multiple mentions to "sessions" in the source docs
What's the difference between a product engineer and other roles?
Compared to product managers, product engineers focus more on building rather than deep research and planning. When it comes to software engineers, both product and software engineers write code, but software engineers focus on building great software, whereas product engineers focus on building great products.
Requires aggregating context from multiple docs + there are a ton of mentions of "product engineer" in the source docs
What were the main benefits of YC?
The main benefits of YC were: Network Access, Investor Reviews, Office Hours, Funding Opportunities, Hiring Resources, Angel Investing Opportunities, Accelerated Growth and Experience, Shift in Self-Perception, Customer Acquisition, Product Market Fit, Ambitious Goal Setting, Access to Thought Leaders, Community Support
Point query
quem foi o primeiro investidor da posthogg?
O primeiro investidor da PostHog foi o YCombinator.
Question in Portuguese, with PostHog misspelled
what posthog competitors also offer session replays
LogRocket, Smartlook, FullStory, Microsoft Clarity, Contentsquare, Mouseflow, Heap, Pendo, Hotjar, Glassbox, and Amplitude.
Requires aggregating content from at least 11 docs (more because I actually missed some in my expected answer)
top tips find client
1. Leverage your inner circle 2. Join relevant communities 3. Be laser-focused 4. Set achievable goals 5. Frame conversations properly
Point query, worded weirdly
what's ch
CH most likely refers to ClickHouse, a column-oriented OLAP database.
Really ambiguous. I meant ClickHouse with my question.
what is mixedpanel
Mixpanel is a popular product analytics tool that was founded in 2009
Mixpanel misspelled as Mixedpanel
how was prpoerty filtering made faster?
Using materialized columns allowed ClickHouse to skip JSON parsing during queries and made queries with property filtering 25x faster.
I recently had an engaging conversation with Alex
(
@veqq
) from the
Lobsters
community about computing,
mathematics and a range of related topics. Our conversation was
later published on the community website as
Lobsters Interview with
Susam
.
I should mention the sections presented in that post are not in the
same order in which we originally discussed them. The sections were
edited and rearranged by Alex to improve the flow and avoid
repetition of similar topics too close to each other.
This page preserves a copy of our discussion as edited by Alex, so I
can keep an archived version on my website. In my copy, I have
added a table of contents to make it easier to navigate to specific
sections. The interview itself follows the table of contents. I
hope you enjoy reading it.
Hi
@susam
, I primarily know
you as a Lisper, what other things do you use?
Yes, I use Lisp extensively for my personal projects and much of
what I do in my leisure is built on it. I ran
a
mathematics pastebin
for close to thirteen years. It was quite popular on some IRC
channels. The pastebin was written in Common Lisp.
My
personal website
and blog are
generated using a tiny static site generator written in Common Lisp.
Over the years I have built several other personal tools in it as
well.
I am an active Emacs Lisp programmer too. Many of my software tools
are in fact Emacs Lisp functions that I invoke with convenient key
sequences. They help me automate repetitive tasks as well as
improve my text editing and task management experience.
I use plenty of other tools as well. In my early adulthood, I spent
many years working with C, C++, Java and PHP. My
first
substantial open source contribution
was to the Apache Nutch
project which was in Java and one of my early original open source
projects was
Uncap
, a C
program to remap keys on Windows.
These days I use a lot of Python, along with some Go and Rust, but
Lisp remains important to my personal work. I also enjoy writing
small standalone tools directly in HTML and JavaScript, often with
all the code in a single file in a readable, unminified form.
How did you first discover computing, then end up with Lisp, Emacs
and mathematics?
I got introduced to computers through the Logo programming language
as a kid. Using simple arithmetic, geometry, logic and code to
manipulate a two-dimensional world had a lasting effect on me.
I still vividly remember how I ended up with Lisp. It was at an
airport during a long layover in 2007. I wanted to use the time to
learn something, so I booted my laptop
running
Debian
GNU/Linux 4.0
(Etch) and then started
GNU CLISP
2.41.
In those days, Wi-Fi in airports was uncommon. Smartphones and
mobile data were also uncommon. So it was fortunate that I had
CLISP already installed on my system and my laptop was ready for
learning Common Lisp. I had it installed because I had wanted to
learn Common Lisp for some time. I was especially attracted by its
simplicity, by the fact that the entire language can be built up
from a very small set of special forms. I
use
SBCL
these days, by the way.
I discovered Emacs through Common Lisp. Several sources recommended
using the
Superior Lisp
Interaction Mode for Emacs (SLIME)
for Common Lisp programming,
so that's where I began. For many years I continued to use Vim as
my primary editor, while relying on Emacs and SLIME for Lisp
development. Over time, as I learnt more about Emacs itself, I grew
fond of Emacs Lisp and eventually made Emacs my primary editor and
computing environment.
I have loved mathematics since my childhood days. What has always
fascinated me is how we can prove deep and complex facts using first
principles and clear logical steps. That feeling of certainty and
rigour is unlike anything else.
Over the years, my love for the subject has been rekindled many
times. As a specific example, let me share how I got into number
theory. One day I decided to learn the RSA cryptosystem. As I was
working through the
RSA
paper
, I stumbled upon the Euler totient function
\( \varphi(n) \) which gives the number of positive integers not
exceeding n that are relatively prime to n. The paper first states
that
\[
\varphi(p) = p - 1
\]
for prime numbers \( p. \) That was obvious since \( p \) has no
factors other than \( 1 \) and itself, so every integer from \( 1 \)
up to \( p - 1 \) must be relatively prime to it. But then it
presents
\[
\varphi(pq) = \varphi(p) \cdot \varphi(q) = (p - 1)(q - 1)
\]
for primes \( p \) and \( q. \) That was not immediately obvious to
me back then. After a few minutes of thinking, I managed to prove
it from scratch. By the inclusion-exclusion principle, we count how
many integers from \( 1 \) up to \( pq \) are not divisible by
\(p \) or \( q. \) There are \( pq \) integers in total. Among
them, there are \( q \) integers divisible by \( p \) and \( p \)
integers divisible by \( q. \) So we need to subtract \( p + q \)
from \(pq. \) But since one integer (\( pq \) itself) is counted in
both groups, we add \( 1 \) back. Therefore
\[
\varphi(pq) = pq - (p + q) + 1 = (p - 1)(q - 1).
\]
Next I could also obtain the general formula for \( \varphi(n) \)
for an arbitrary positive integer \( n \) using the same idea.
There are several other proofs too, but that is how I derived the
general formula for \( \varphi(n) \) when I first encountered it.
And just like that, I had begun to learn number theory!
You've said you prefer computing for fun. What is fun to you? Do
you have an idea of what makes something fun or not?
For me, fun in computing began when I first learnt IBM/LCSI PC Logo
when I was nine years old. I had very limited access to computers
back then, perhaps only about two hours per
month
in the
computer laboratory at my primary school. Most of my Logo
programming happened with pen and paper at home. I would "test" my
programs by tracing the results on graph paper. Eventually I would
get about thirty minutes of actual computer time in the lab to run
them for real.
So back then, most of my computing happened without an actual
computer. But even with that limited access to computers, a whole
new world opened up for me: one that showed me the joy of computing
and more importantly, the joy of sharing my little programs with my
friends and teachers. One particular Logo program I still remember
very well drew a house with animated dashed lines, where the dashes
moved around the outline of the house. Everyone around me loved it,
copied it and tweaked it to change the colours, alter the details
and add their own little touches.
For me, fun in computing comes from such exploration and sharing. I
enjoy asking "what happens if" and then seeing where it leads me.
My Emacs package
devil-mode
comes from such exploration. It came from asking, "What happens if
we avoid using the
ctrl
and
meta
modifier keys
and use
,
(the comma key) or another suitable key as a
leader key instead? And can we still have a non-modal editing
experience?"
Sometimes computing for fun may mean crafting a minimal esoteric
drawing language, making a small game or building a tool that solves
an interesting problem elegantly. It is a bonus if the exploration
results in something working well enough that I can share with
others on the World Wide Web and others find it fun too.
How do you choose what to investigate? Which most interest you,
with what commonalities?
For me, it has always been one exploration leading to another.
For example, I originally built
MathB
for my friends
and myself who were going through a phase in our lives when we used
to challenge each other with mathematical puzzles. This tool became
a nice way to share solutions with each other. Its use spread from
my friends to their friends and colleagues, then to schools and
universities and eventually to IRC channels.
Similarly, I built
TeXMe
when I was learning neural networks and taking a lot of notes on the
subject. I was not ready to share the notes online, but I did want
to share them with my friends and colleagues who were also learning
the same topic. Normally I would write my notes in LaTeX, compile
them to PDF and share the PDF, but in this case, I wondered, what if
I took some of the code from MathB and created a tool that would let
me write plain Markdown
(
GFM
) + LaTeX
(
MathJax
) in
a
.html
file and have the tool render the file as soon
as it was opened in a web browser? That resulted in TeXMe, which
has surprisingly become one of my most popular projects, receiving
millions of hits in some months according to the CDN statistics.
Another example is
Muboard
,
which is a bit like an interactive mathematics chalkboard. I built
this when I was hosting an
analytic number
theory book club
and I needed a way to type LaTeX snippets live
on screen and see them immediately rendered. That made me wonder:
what if I took TeXMe, made it interactive and gave it a chalkboard
look-and-feel? That led to Muboard.
So we can see that sharing mathematical notes and snippets has been
a recurring theme in several of my projects. But that is only a
small fraction of my interests. I have a wide variety of interests
in computing. I also engage in random explorations, like writing
IRC clients
(
NIMB
,
Tzero
),
ray tracing
(
POV-Ray
,
Java ray tracer
),
writing Emacs guides
(
Emacs4CL
,
Emfy
),
developing small single-file HTML games
(
Andromeda Invaders
,
Guess My RGB
),
purely recreational programming
(
FXYT
,
may4.fs
,
self-printing machine code
,
prime number grid explorer
)
and so on. The list goes on. When it comes to hobby computing, I
don't think I can pick just one domain and say it interests me the
most. I have a lot of interests.
What is computing, to you?
Computing, to me, covers a wide range of activities: programming a
computer, using a computer, understanding how it works, even
building one. For example, I once built a tiny 16-bit CPU along
with a small main memory that could hold only eight 16-bit
instructions, using VHDL and a Xilinx CPLD kit. The design was
based on the Mano CPU introduced in the book
Computer System
Architecture
(3rd ed.) by M. Morris Mano. It was incredibly
fun to enter instructions into the main memory, one at a time, by
pushing DIP switches up and down and then watch the CPU I had built
execute an entire program. For someone like me, who usually works
with software at higher levels of abstraction, that was a thrilling
experience!
Beyond such experiments, computing also includes more practical and
concrete activities, such as installing and using my favourite Linux
distribution (Debian), writing software tools in languages like
Common Lisp, Emacs Lisp, Python and the shell command language or
customising my Emacs environment to automate repetitive tasks.
To me, computing also includes the abstract stuff like spending time
with abstract algebra and number theory and getting a deeper
understanding of the results pertaining to groups, rings and fields,
as well as numerous number-theoretic results. Browsing the
On-Line Encyclopedia of Integer
Sequences
(OEIS), writing small programs to explore interesting
sequences or just thinking about them is computing too. I think
many of the interesting results in computer science have deep
mathematical foundations. I believe much of computer science is
really discrete mathematics in action.
And if we dive all the way down from the CPU to the level of
transistors, we encounter continuous mathematics as well, with
non-linear voltage-current relationships and analogue behaviour that
make digital computing possible. It is fascinating how, as a
relatively new species on this planet, we have managed to take sand
and find a way to use continuous voltages and currents in electronic
circuits built with silicon and convert them into the discrete
operations of digital logic. We have machines that can simulate
themselves!
To me, all of this is fun. To study and learn about these things,
to think about them, to understand them better and to accomplish
useful or amusing results with this knowledge is all part of the
fun.
How do you view programming vs. domains?
I focus more on the domain than the tool. Most of the time it is a
problem that catches my attention and then I explore it to
understand the domain and arrive at a solution. The problem itself
usually points me to one of the tools I already know.
For example, if it is about working with text files, I might write
an Emacs Lisp function. If it involves checking large sets of
numbers rapidly for patterns, I might choose C++ or Rust. But if I
want to share interactive visualisations of those patterns with
others, I might rewrite the solution in HTML and JavaScript,
possibly with the use of the Canvas API, so that I can share the
work as a self-contained file that others can execute easily within
their web browsers. When I do that, I prefer to keep the HTML neat
and readable, rather than bundled or minified, so that people who
like to 'View Source' can copy, edit and customise the code
themselves to immediately see their changes take effect.
Let me share a specific example. While working on a web-based game, I first
used
CanvasRenderingContext2D
's
fillText()
to display text on the game canvas. However, dissatisfied with the
text rendering quality, I began looking for IBM PC OEM fonts and
similar retro fonts online. After downloading a few font packs, I
wrote a little Python script to convert them to bitmaps (arrays of
integers) and then used the bitmaps to draw text on the canvas using
JavaScript, one cell at a time, to get pixel-perfect results! These
tiny Python and JavaScript tools were good enough that I felt
comfortable sharing them together as a tiny toolkit called
PCFace
.
This toolkit offers JavaScript bitmap arrays and tiny JavaScript
rendering functions, so that someone else who wants to display text
on their game canvas using PC fonts and nothing but plain HTML and
JavaScript can do so without having to solve the problem from
scratch!
Has the rate of your making new Emacs functions has diminished over
time (as if everything's covered) or do the widening domains lead to
more? I'm curious how applicable old functionality is for new
problems and how that impacts the APIs!
My rate of making new Emacs functions has definitely decreased.
There are two reasons. One is that over the years my computing
environment has converged into a comfortable, stable setup I am very
happy with. The other is that at this stage of life I simply cannot
afford the time to endlessly tinker with Emacs as I did in my
younger days.
More generally, when it comes to APIs, I find that well-designed
functionality tends to remain useful even when new problems appear.
In Emacs, for example, many of my older functions continue to serve
me well because they were written in a composable way. New problems
can often be solved with small wrappers or combinations of existing
functions. I think APIs that consist of functions that are simple,
orthogonal and flexible age well. If each function in an API does
one thing and does it well (the Unix philosophy), it will have
long-lasting utility.
Of course, new domains and problems do require new functions and
extensions to an API, but I think it is very important to not give
in to the temptation of enhancing the existing functions by making
them more complicated with optional parameters, keyword arguments,
nested branches and so on. Personally, I have found that it is much
better to implement new functions that are small, orthogonal and
flexible, each doing one thing and doing it well.
What design methods or tips do you have, to increase composability?
For me, good design starts with good vocabulary. Clear vocabulary
makes abstract notions concrete and gives collaborators a shared
language to work with. For example, while working on a network
events database many years ago, we collected data minute by minute
from network devices. We decided to call each minute of data from a
single device a "nugget". So if we had 15 minutes of data from 10
devices, that meant 150 nuggets.
Why "nugget"? Because it was shorter and more convenient than
repeatedly saying "a minute of data from one device". Why not
something less fancy like "chunk"? Because we reserved "chunk" for
subdivisions within a nugget. Perhaps there were better choices,
but "nugget" was the term we settled on and it quickly became shared
terminology between the collaborators. Good terminology naturally
carries over into code. With this vocabulary in place, function
names like
collect_nugget()
,
open_nugget()
,
parse_chunk()
,
index_chunk()
,
skip_chunk()
,
etc. immediately become meaningful to everyone involved.
Thinking about the vocabulary also ensures that we are thinking
about the data, concepts and notions we are working with in a
deliberate manner and that kind of thinking also helps when we
design the architecture of software.
Too often I see collaborators on software projects jump straight
into writing functions that take some input and produce some desired
effect, with variable names and function names decided on the fly.
To me, this feels backwards. I prefer the opposite approach.
Define the terms first and let the code follow from them.
I also prefer developing software in a layered manner, where complex
functionality is built from simpler, well-named building blocks. It
is especially important to avoid
layer violations
, where
one complex function invokes another complex function. That creates
tight coupling between two complex functions. If one function
changes in the future, we have to reason carefully about how it
affects the other. Since both are already complex, the cognitive
burden is high. A better approach, I think, is to identify the
common functionality they share and factor that out into smaller,
simpler functions.
To summarise, I like to develop software with a clear vocabulary,
consistent use of that vocabulary, a layered design where complex
functions are built from simpler ones and by avoiding layer
violations. I am sure none of this is new to the Lobsters
community. Some of these ideas also occur
in
domain-driven
design
(DDD). DDD defines the term
ubiquitous language
to mean, "A language structured around the domain model and used by
all team members within a bounded context to connect all the
activities of the team with the software." If I could call this
approach of software development something, I would simply call it
"vocabulary-driven development" (VDD), though of course DDD is the
more comprehensive concept.
Like I said, none of this is likely new to the Lobsters community.
In particular, I suspect Forth programmers would find it too
obvious. In Forth, it is very difficult to begin with a long,
poorly thought-out monolithic word and then break it down into
smaller ones later. The stack effects quickly become too hard to
track mentally with that approach. The only viable way to develop
software in Forth is to start with a small set of words that
represent the important notions of the problem domain, test them
immediately and then compose higher-level words from the lower-level
ones. Forth naturally encourages a layered style of development,
where the programmer thinks carefully about the domain, invents
vocabulary and expresses complex ideas in terms of simpler ones,
almost in a mathematical fashion. In my experience, this kind of
deliberate design produces software that remains easy to understand
and reason about even years after it was written.
Not enhancing existing functions but adding new small ones seems
quite lovely, but how do you come back to such a codebase later with
many tiny functions? At points, I've advocated for very large
functions, particularly traumatized by Java-esque 1000 functions in
1000 files approaches. When you had time, would you often
rearchitecture the conceptual space of all of those functions?
The famous quote from Alan J. Perlis comes to mind:
It is better to have 100 functions operate on one data structure
than 10 functions on 10 data structures.
Personally, I enjoy working with a codebase that has thousands of
functions, provided most of them are small, well-scoped and do one
thing well. That said, I am not dogmatically opposed to large
functions. It is always a matter of taste and judgement. Sometimes
one large, cohesive function is clearer than a pile of tiny ones.
For example, when I worked on parser generators, I often found that
lexers and finite state machines benefited from a single top-level
function containing the full tokenisation logic or the full state
transition logic in one place. That function could call smaller
helpers for specific tasks, but we still need the overall
switch
-
case
or
if
-
else
or
cond
ladder
somewhere. I think trying to split that ladder into smaller
functions would only make the code harder to follow.
So while I lean towards small, composable functions, the real goal
is to strike a balance that keeps code maintainable in the long run.
Each function should be as small as it can reasonably be and no
smaller.
Like you, I program as a tool to explore domains. Which do you know
the most about?
For me too, the appeal of computer programming lies especially in
how it lets me explore different domains. There are two kinds of
domains in which I think I have gained good expertise. The first
comes from years of developing software for businesses, which has
included solving problems such as network events parsing, indexing
and querying, packet decoding, developing parser generators,
database session management and TLS certificate lifecycle
management. The second comes from areas I pursue purely out of
curiosity or for hobby computing. This is the kind I am going to
focus on in our conversation.
Although computing and software are serious business today, for me,
as for many others, computing is also a hobby.
Personal hobby projects often lead me down various rabbit holes and
I end up learning new domains along the way. For example, although
I am not a web developer, I learnt to build small, interactive
single-page tools in plain HTML, CSS and JavaScript simply because I
needed them for my hobby projects over and over again. An early
example is
QuickQWERTY
, which I built
to teach myself and my friends touch-typing on QWERTY keyboards.
Another example is
CFRS[]
, which I created
because I wanted to make a total (non-Turing complete) drawing
language that has turtle graphics like Logo but is absolutely
minimal like P′′.
You use double spaces after periods which I'd only experienced from
people who learned touch typing on typewriters, unexpected!
Yes, I do separate sentences by double spaces. It is interesting
that you noticed this.
I once briefly learnt touch typing on typewriters as a kid, but
those lessons did not stick with me. It was much later, when I used
a Java applet-based touch typing tutor that I found online about two
decades ago, that the lessons really stayed with me. Surprisingly,
that application taught me to type with a single space between
sentences. By the way, I disliked installing Java plugins into the
web browser, so I wrote
QuickQWERTY
as a similar touch typing tutor in plain HTML and JavaScript for
myself and my friends.
I learnt to use double spaces between sentences first with Vim and
then later again with Emacs. For example, in Vim,
the
joinspaces
option is on by default, so when we join
sentences with the normal mode command
J
or format
paragraphs with
gqap
, Vim inserts two spaces after full
stops. We need to disable that behaviour with
:set
nojoinspaces
if we want single spacing.
It is similar in Emacs. In Emacs, the
delete-indentation
command (
M-^
) and
the
fill-paragraph
command (
M-q
) both
insert two spaces between sentences by default. Single spacing can
be enabled with
(setq sentence-end-double-space nil)
.
Incidentally, I spend a good portion of the README for my Emacs
quick-start DIY kit named
Emfy
discussing sentence
spacing conventions under the section
Single
Space for Sentence Spacing
. There I explain how to configure
Emacs to use single spaces, although I use double spaces myself.
That's because many new Emacs users prefer single spacing.
The defaults in Vim and Emacs made me adopt double spacing. The
double spacing convention is also widespread across open source
software. If we look at the Vim help pages, Emacs built-in
documentation or the Unix and Linux man pages, double spacing is the
norm. Even inline comments in traditional open source projects
often use it. For example, see Vim's
:h usr_01.txt
,
Emacs's
(info "(emacs) Intro")
or the comments in the
GCC source code
.
How do you approach learning a new domain?
When I take on a new domain, there is of course a lot of reading
involved from articles, books and documentation. But as I read, I
constantly try to test what I learn. Whenever I see a claim, I ask
myself, "If this claim were wrong, how could I demonstrate it?"
Then I design a little experiment, perhaps write a snippet of code
or run a command or work through a concrete example, with the goal
of checking the claim in practice.
Now I am not genuinely hoping to prove a claim wrong. It is just a
way to engage with the material. To illustrate, let me share an
extremely simple and generic example without going into any
particular domain. Suppose I learn that Boolean operations in
Python short-circuit. I might write out several experimental
snippets like the following:
def t(): print('t'); return True
def f(): print('f'); return False
f() or t() or f()
And then confirm that the results do indeed confirm short-circuit
evaluation (
f
followed by
t
in this case).
At this point, one could say, "Well, you just confirmed what the
documentation already told you." And that's true. But for me, the
value lies in trying to test it for myself. Even if the claim
holds, the act of checking forces me to see the idea in action.
That not only reinforces the concept but also helps me build a much
deeper intuition for it.
Sometimes these experiments also expose gaps in my own
understanding. Suppose I didn't properly know what "short-circuit"
means. Then the results might contradict my expectations. That
contradiction would push me to correct my misconception and that's
where the real learning happens.
Occasionally, this process even uncovers subtleties I didn't expect.
For example, while learning socket programming, I discovered that a
client can successfully receive data using
recv()
even
after calling
shutdown()
, contrary to what I had first
inferred from the specifications. See my Stack Overflow post
Why can recv()
receive messages after the client has invoked shutdown()?
for
more details if you are curious.
Now this method cannot always be applied, especially if it is very
expensive or unwieldy to do so. For example, if I am learning
something in the finance domain, it is not always possible to
perform an actual transaction. One can sometimes use simulation
software, mock environments or sandbox systems to explore ideas
safely. Still, it is worth noting that this method has its
limitations.
In mathematics, though, I find this method highly effective. When I
study a new branch of mathematics, I try to come up with examples
and counterexamples to test what I am learning. Often, failing to
find a counterexample helps me appreciate more deeply why a claim
holds and why no counterexamples exist.
Do you have trouble not getting distracted with so much on your
plate? I'm curious how you balance the time commitments of
everything!
Indeed, it is very easy to get distracted. One thing that has
helped over the years is the increase in responsibilities in other
areas of my life. These days I also spend some of my free time
studying mathematics textbooks. With growing responsibilities and
the time I devote to mathematics, I now get at most a few hours each
week for hobby computing. This automatically narrows down my
options. I can explore perhaps one or at most two ideas in a month
and that constraint makes me very deliberate about choosing my
pursuits.
Many of the explorations do not evolve into something solid that I
can share. They remain as little experimental code snippets or
notes archived in a private repository. But once in a while, an
exploration grows into something concrete and feels worth sharing on
the Web. That becomes a short-term hobby project. I might work on
it over a weekend if it is small or for a few weeks if it is more
complex. When that happens, the goal of sharing the project helps
me focus.
I try not to worry too much about making time. After all, this is
just a hobby. Other areas of my life have higher priority. I also
want to devote a good portion of my free time to learning more
mathematics, which is another hobby I am passionate about. Whatever
little spare time remains after attending to the higher-priority
aspects of my life goes into my computing projects, usually a couple
of hours a week, most of it on weekends.
How does blogging mix in? What's the development like of a single
piece of curiosity through wrestling with the domain, learning and
sharing it etc.?
Maintaining my personal website is another aspect of computing that
I find very enjoyable. My website began as a loose collection of
pages on a LAN site during my university days. Since then I have
been adding pages to it to write about various topics that I find
interesting. It acquired its blog shape and form much later when
blogging became fashionable.
I usually write a new blog post when I feel like there is some piece
of knowledge or some exploration that I want to archive in a
persistent format. Now what the development of a post looks like
depends very much on the post. So let me share two opposite
examples to describe what the development of a single piece looks
like.
One of my most frequently visited posts
is
Lisp in Vim
. It started when I
was hosting a Common Lisp programming club for beginners. Although
I have always used Emacs and SLIME for Common Lisp programming
myself, many in the club used Vim, so I decided to write a short
guide on setting up something SLIME-like there. As a former
long-time Vim user myself, I wanted to make the Lisp journey easier
for Vim users too. I thought it would be a 30-minute exercise where
I write up a README that explains how to install
Slimv
and how to set
it up in Vim. But then I discovered a newer plugin called
Vlime
that also offered
SLIME-like features in Vim! That detail sent me down a very deep
rabbit hole. Now I needed to know how the two packages were
different, what their strengths and weaknesses were, how routine
operations were performed in both and so on. What was meant to be a
short note turned into a nearly 10,000-word article. As I was
comparing the two SLIME-like packages for Vim, I also found a few
bugs in Slimv and contributed fixes for them
(
#87
,
#88
,
#89
,
#90
).
Writing this blog post turned into a month-long project!
At the opposite extreme is a post like
Elliptical
Python Programming
. I stumbled upon Python's
Ellipsis
while reviewing someone's code. It immediately caught my attention.
I wondered if, combined with some standard obfuscation techniques,
one could write arbitrary Python programs that looked almost like
Morse code. A few minutes of experimentation showed that a
genuinely Morse code-like appearance was not possible, but something
close could be achieved. So I wrote what I hope is a humorous post
demonstrating that arbitrary Python programs can be written using a
very restricted set of symbols, one of which is the ellipsis. It
took me less than an hour to write this post. The final result
doesn't look quite like Morse code as I had imagined, but it is
quite amusing nevertheless!
What draws you to post and read online forums? How do you balance
or allot time for reading technical articles, blogs etc.?
The exchange of ideas! Just as I enjoy sharing my own
computing-related thoughts, ideas and projects, I also find joy in
reading what others have to share.
Other areas of my life take precedence over hobby projects and hobby
projects take precedence over technical forums.
After I've given time to the higher-priority parts of my life and to
my own technical explorations, I use whatever spare time remains to
read articles, follow technical discussions and occasionally add
comments.
When you decided to stop with MathB due to moderation burdens, I
offered to take over/help and you mentioned others had too. Did
anyone end up forking it, to your knowledge?
I first thought of shutting down the
MathB
-based pastebin
website in November 2019. The website had been running for seven
years at that time. When I announced my thoughts to the IRC
communities that would be affected, I received a lot of support and
encouragement. A few members even volunteered to help me out with
moderation. That support and encouragement kept me going for
another six years. However, the volunteers eventually became busy
with their own lives and moved on. After all, moderating user
content for an open pastebin that anyone in the world can post to is
a thankless and tiring activity. So most of the moderation activity
fell back on me. Finally, in February 2025, I realised that I no
longer want to spend time on this kind of work.
I developed MathB with a lot of passion for myself and my friends.
I had no idea at the time that this little project would keep a
corner of my mind occupied even during weekends and holidays. There
was always a nagging worry. What if someone posted content that
triggered compliance concerns and my server was taken offline while
I was away? I no longer wanted that kind of burden in my life. So
I finally decided to shut it down. I've written more about this
in
MathB.in Is Shutting
Down
.
To my knowledge, no one has forked it, but others have developed
alternatives. Further, the
Archive Team
has
archived
all posts from the now-defunct MathB-based website. A member of the
Archive Team reached out to me over IRC and we worked together for
about a week to get everything successfully archived.
What're your favorite math textbooks?
I have several favourite mathematics books, but let me share three I
remember especially fondly.
The first is
Advanced Engineering Mathematics
by Erwin
Kreyszig. I don't often see this book recommended online, but for
me it played a major role in broadening my horizons. I think I
studied the 8th edition back in the early 2000s. It is a hefty book
with over a thousand pages and I remember reading it cover to cover,
solving every exercise problem along the way. It gave me a solid
foundation in routine areas like differential equations, linear
algebra, vector calculus and complex analysis. It also introduced
me to Fourier transforms and Laplace transforms, which I found
fascinating.
Of course, the Fourier transform has a wide range of applications in
signal processing, communications, spectroscopy and more. But I
want to focus on the fun and playful part. In the early 2000s, I
was also learning to play the piano as a hobby. I used to record my
amateur music compositions with
Audacity
by
connecting my digital piano to my laptop with a line-in cable. It
was great fun to plot the spectrum of my music on Audacity, apply
high-pass and low-pass filters and observe how the Fourier transform
of the audio changed and then hear the effect on the music. That
kind of hands-on tinkering made Fourier analysis intuitive for me
and I highly recommend it to anyone who enjoys both music and
mathematics.
The second book is
Introduction to Analytic Number Theory
by Tom M. Apostol. As a child I was intrigued by the prime number
theorem but lacked the mathematical maturity to understand its
proof. Years later, as an adult, I finally taught myself the proof
from Apostol's book. It was a fantastic journey that began with
simple concepts like the Möbius function and Dirichlet products and
ended with quite clever contour integrals that proved the theorem.
The complex analysis I had learnt from Kreyszig turned out to be
crucial for understanding those integrals. Along the way I gained a
deeper understanding of the Riemann zeta function \( \zeta(s). \)
The book discusses zero-free regions where \( \zeta(s) \) does not
vanish, which I found especially fascinating. Results like \(
\zeta(-1) = -1/12, \) which once seemed mysterious, became obvious
after studying this book.
The third is
Galois Theory
by Ian Stewart. It introduced
me to field extensions, field homomorphisms and solubility by
radicals. I had long known that not all quintic equations are
soluble by radicals, but I didn't know why. Stewart's book taught
me exactly why. In particular, it demonstrated that the polynomial
\( t^5 - 6t + 3 \) over the field of rational numbers is not soluble
by radicals. This particular result, although fascinating, is just
a small part of a much larger body of work, which is even more
remarkable. To arrive at this result, the book takes us through a
wonderful journey that includes the theory of polynomial rings,
algebraic and transcendental field extensions, impossibility proofs
for ruler-and-compass constructions, the Galois correspondence and
much more.
One of the most rewarding aspects of reading books like these is how
they open doors to new knowledge, including things I didn't even
know that I didn't know.
How does the newer math jell with or inform past or present
computing, compared to much older stuff?
I don't always think explicitly about how mathematics informs
computing, past or present. Often the textbooks I pick feel very
challenging to me, so much so that all my energy goes into simply
mastering the material. It is arduous but enjoyable. I do it
purely for the fun of learning without worrying about applications.
Of course, a good portion of pure mathematics probably has no
real-world applications. As G. H. Hardy famously wrote in
A
Mathematician's Apology
:
I have never done anything 'useful'. No discovery of mine has
made or is likely to make, directly or indirectly, for good or
ill, the least difference to the amenity of the world.
But there is no denying that some of it does find applications.
Were Hardy alive today, he might be disappointed that number theory,
his favourite field of "useless" mathematics, is now a crucial part
of modern cryptography. Electronic commerce wouldn't likely exist
without it.
Similarly, it is amusing how something as abstract as abstract
algebra finds very concrete applications in coding theory. Concepts
such as polynomial rings, finite fields and cosets of subspaces in
vector spaces over finite fields play a crucial role in
error-correcting codes, without which modern data transmission and
storage would not be possible.
On a more personal note, some simpler areas of mathematics have been
directly useful in my own work. While solving problems for
businesses, information entropy, combinatorics and probability
theory were crucial when I worked on gesture-based authentication
about one and a half decades ago.
Similarly, when I was developing Bloom filter-based indexing and
querying for a network events database, again, probability theory
was crucial in determining the parameters of the Bloom filters (such
as the number of hash functions, bits per filter and elements per
filter) to ensure that the false positive rate remained below a
certain threshold. Subsequent testing with randomly sampled network
events confirmed that the observed false positive rate matched the
theoretical estimate quite well. It was very satisfying to see
probability theory and the real world agreeing so closely.
Beyond these specific examples, studying mathematics also influences
the way I think about problems. Embarking on journeys like analytic
number theory or Galois theory is humbling. There are times when I
struggle to understand a small paragraph of the book and it takes me
several hours (or even days) to work out the arguments in detail
with pen and paper (lots of it) before I really grok them. That
experience of grappling with dense reasoning teaches humility and
also makes me sceptical of complex, hand-wavy logic in day-to-day
programming.
Several times I have seen code that bundles too many decisions into
one block of logic, where it is not obvious whether it would behave
correctly in all circumstances. Explanations may sometimes be
offered about why it works for reasonable inputs, but the reasoning
is often not watertight. The experience of working through
mathematical proofs, writing my own, making mistakes and then
correcting them has taught me that if the reasoning for correctness
is not clear and rigorous, something could be wrong. In my
experience, once such code sees real-world usage, a bug is nearly
always found.
That's why I usually insist either on simplifying the logic or on
demonstrating correctness in a clear, rigorous way. Sometimes this
means doing a case-by-case analysis for different types of inputs or
conditions and showing that the code behaves correctly in each case.
There is also a bit of an art to reducing what seem like numerous or
even infinitely many cases to a small, manageable set of cases by
spotting structure, such as symmetries, invariants or natural
partitions of the input space. Alternatively, one can look for a
simpler argument that covers all cases. These are techniques we
employ routinely in mathematics and I think that kind of thinking
and reasoning is quite valuable in software development too.
Stellantis Is Spamming Owners' Screens with Pop-Up Ads for New Car Discounts
Our free daily newsletter sends the stories that really matter directly to you, every weekday.
The internet is raging right now over
Stellantis
pushing marketing pop-ups to owners’ in-car screens. It’s obnoxious, to be sure, and it’s legit—we have confirmation from a
Jeep
driver as well as Stellantis itself. But this isn’t even the first time it’s happened, as we
reported in February
about Jeep advertising extended warranties in the same way.
Auto writer and all-around car guy
Zerin Dube
posted earlier this week about his WL Grand Cherokee’s “marketing notification” on X. The photo started popping off, and before long, others were sharing the same ad that was sent to their screens. Dube labeled it as “late stage capitalism,” which feels like an accurate descriptor.
Funny enough, Dube was in the market for a new Wrangler anyhow, and he took advantage of the $1,500 loyalty offer Jeep sent straight to his Grand Cherokee. He drove off the lot Thursday night in a new Rubicon X, so I guess you can say it worked.
And you're looking for ANOTHER jeep? I'd swear off the brand forever if it started sending me ads in the infotainment
Others online were less accepting of the ploy. Practically every repost of Dube’s photo decried Jeep and Stellantis at large, with some Ram and Chrysler drivers corroborating the ad’s rollout that spread across multiple brands. Most of the comments said something along the lines of, “Guess what I’m never buying.” Some made the informed prediction that this type of promotion will become commonplace in the industry before long.
For its part, Stellantis told
The Drive
that it sends out these notifications to “stay in contact with our owners at critical points in their ownership.” The in-vehicle message system is also used to notify drivers of vehicle recalls and health monitor alerts. A spokesperson for the brand said:
“Recently, a select group of owners received a special marketing notification in their vehicle, and we tailored this special offer to minimize any intrusions:
The simple text message offering a $1,500 bonus incentive appears only on startup and while the vehicle is stationary
The message disappears when the vehicle begins moving, or the driver clicks the OK or X icon on the screen, or after 15 seconds
The message returns at the next key-on cycle
only
if the driver clicked on Remind Me Later, or they did not click OK or X
“Our goal is to deliver the best vehicle experience for our customers. As a result of these efforts, we have seen our customers take advantage of this offer,” the spokesperson said.
The Stellantis spokesperson concluded by saying that owners can permanently opt out of in-vehicle messaging by calling the company’s customer care line at 800-777-3600.
The $1,500 loyalty bonus is just one of many discounts that Stellantis dealers are stacking in order to move more units. I reported last week that
storming Ford Bronco sales
are genuinely threatening the Wrangler’s spot atop the segment, and as Dube shared on X, he got something like $16,500 off his new Jeep. Apparently, now’s a good time to buy if you can stand the occasional spam on your car’s screen.
The last time this happened
, Stellantis was advertising extended warranties to owners via their infotainment. Many were frustrated as the offer continued to show, even after acknowledging it by pressing “OK.” What’s more, others saw the ad despite their car exceeding the mileage limit mentioned in the promotion.
Connected cars, man. Gotta love ’em.
Got a tip or question for the author? Contact them directly: caleb@thedrive.com
We're taught that email must go through servers. Why? Because the Internet was built around centralized infrastructure. Every email you send travels through multiple servers - your provider's server, maybe a few relay servers, and finally your recipient's provider's server. Each hop is a potential point of surveillance, censorship, or failure.
Even "encrypted" email solutions still rely on these centralized servers. They encrypt the message content but the metadata - who you're talking to, when, how often - is visible to anyone watching the servers.
But there is a network, called
Yggdrasil
, that gives everyone a free IPv6 and doesn't need a blessing from your ISP. We finally have this possibility to use true P2P email. And moreover, this network has strong encryption to protect all data that flows from one IP to another.
So, Tyr brings true peer-to-peer email to your Android device using these unusual conditions. Unlike traditional email clients, Tyr doesn't need:
Centralized mail servers (the connections are straight P2P)
Message encryption layers (the network takes care of that)
Port forwarding or STUN/TURN servers (Yggdrasil handles NAT traversal)
What does Tyr already have?
Full integration with DeltaChat and ArcaneChat - the best decentralized messengers
Local SMTP/IMAP server running on your device
Automatic Ed25519 key generation for your mail identity
Connection to the Yggdrasil Network with configurable peers
Auto-start on boot for always-on availability
Encrypted backup & restore with password protection
Automatic recovery from Android Keystore issues (Samsung devices)
One of Tyr's strong points is censorship circumvention: you can connect to any of hundreds of available Yggdrasil nodes, host your own, or even build a private network. Email freedom is literally in your hands.
How does it work?
Tyr runs a complete email server right on your Android device, using the Yggdrasil network for transport. The
Yggmail
mail server (built in Go) is embedded as a library inside the app and runs as a foreground service.
On top of Yggdrasil, it provides standard SMTP and IMAP protocols on localhost (127.0.0.1:1025 and 127.0.0.1:1143). Any email client can connect to these ports - but we recommend DeltaChat or ArcaneChat for the best P2P messaging experience.
Every Tyr installation generates unique Ed25519 cryptographic keys. Your mail address is derived from your public key, making it:
<64-hex-characters>@yggmail
. This means your identity is cryptographically verifiable and cannot be spoofed.
DeltaChat/ArcaneChat Integration
DeltaChat and ArcaneChat are perfect companions for Tyr. These are messengers that use email protocols but provide modern chat interfaces. When you configure DeltaChat/ArcaneChat to use Tyr's local server:
DeltaChat/ArcaneChat sends messages via SMTP to Tyr
Tyr wraps them in Yggmail protocol and sends through Yggdrasil
The recipient's Tyr receives the message via Yggdrasil
Their DeltaChat/ArcaneChat fetches it via IMAP from their local Tyr
All this happens peer-to-peer, with no central servers
Setting up DeltaChat/ArcaneChat with Tyr
Option 1: Automatic Setup (Recommended)
Install Tyr and complete the onboarding (set password, configure peers)
Start the Yggmail service in Tyr
Install DeltaChat or ArcaneChat from F-Droid or Google Play
In Tyr's main screen, tap
"Setup DeltaChat/ArcaneChat"
Tyr will automatically open DeltaChat/ArcaneChat with pre-configured settings
Complete the setup and start chatting!
Option 2: Manual Setup
If automatic setup doesn't work:
Install Tyr and complete the onboarding (set password, configure peers)
Start the Yggmail service in Tyr
Copy your mail address from the main screen (looks like
abc123...@yggmail
)
Install DeltaChat or ArcaneChat from F-Droid or Google Play
In DeltaChat/ArcaneChat, tap
"Create a new profile"
Enter a name and optionally select an avatar
Tap
"Use a different server"
(below the login fields)
Enter your Yggmail address and the password you set in Tyr
Tap "✓" in the top right corner to complete setup
Important
: Tyr must be running for DeltaChat/ArcaneChat to send and receive messages. Enable auto-start in Tyr settings for seamless experience.
Building from source
Prerequisites
Android Studio (latest version recommended)
JDK 17
Android SDK (API 23-36)
Go 1.21+ and gomobile (only if rebuilding yggmail.aar)
Build steps
Clone the repository:
git clone <repository-url>cd Tyr
Build debug APK:
Install to connected device:
APKs will be in
app/build/outputs/apk/debug/
or
app/build/outputs/apk/release/
Rebuilding yggmail.aar (optional)
If you need to rebuild the Yggmail library:
cd ../yggmail/mobile
# On Windows:
..\build-android.bat
# On Unix:
gomobile bind -target=android -androidapi 23 -javapkg=com.jbselfcompany.tyr -ldflags="-checklinkname=0" -o yggmail.aar .
Then copy
yggmail.aar
to
Tyr/app/libs/
Technical details
Language
: Kotlin 2.2.20
Min SDK
: 23 (Android 6.0)
Target SDK
: 33 (Android 13)
Architecture
: Layered (UI → Service → Data)
Mail server
: Yggmail (Go library, embedded via gomobile)
Network
: Yggdrasil overlay network
Localization
: English, Russian
Security Features
🔒
Security implementation
:
Passwords are encrypted
using Android Keystore System (AES256-GCM encryption)
Automatic Keystore recovery
: Handles Android Keystore issues on Samsung and other devices automatically
Network encryption
provided by Yggdrasil Network for all peer-to-peer communications
Local-only access
: SMTP/IMAP ports (1025/1143) are bound to localhost only, not accessible from network
Cryptographic identity
: Ed25519 keys ensure your mail address cannot be spoofed
Encrypted backups
: Configuration and keys can be backed up with password protection
ArcaneChat
: Alternative email-based messenger client
License
Tyr is open source software. The Yggmail library uses Mozilla Public License v. 2.0.
See
LICENSE
file for full details
Simon Josefsson: Container Images For Debian With Guix
PlanetDebian
blog.josefsson.org
2025-11-28 16:32:33
The debian-with-guix-container project build and publish container images of Debian GNU/Linux stable with GNU Guix installed.
The images are like normal Debian stable containers but have the guix tool and a reasonable fresh guix pull.
Supported architectures include amd64 and arm64. The multi-...
Data from the Census Bureau and Ramp shows that AI adoption rates are starting to flatten out across all firm sizes, see charts below.
Note: Data is six-survey moving average. The survey is conducted bi-weekly. Sources: US Census Bureau, Macrobond, Apollo Chief Economist
Note: Ramp Al Index measures the adoption rate of artificial intelligence products and services among American businesses. The sample includes more than 40,000 American businesses and billions of dollars in corporate spend using data from Ramp’s corporate card and bill pay platform. Sources: Ramp, Bloomberg, Macrobond, Apollo Chief Economist
This presentation may not be distributed, transmitted or otherwise communicated to others in whole or in part without the express consent of Apollo Global Management, Inc. (together with its subsidiaries, “Apollo”).
Apollo makes no representation or warranty, expressed or implied, with respect to the accuracy, reasonableness, or completeness of any of the statements made during this presentation, including, but not limited to, statements obtained from third parties. Opinions, estimates and projections constitute the current judgment of the speaker as of the date indicated. They do not necessarily reflect the views and opinions of Apollo and are subject to change at any time without notice. Apollo does not have any responsibility to update this presentation to account for such changes. There can be no assurance that any trends discussed during this presentation will continue.
Statements made throughout this presentation are not intended to provide, and should not be relied upon for, accounting, legal or tax advice and do not constitute an investment recommendation or investment advice. Investors should make an independent investigation of the information discussed during this presentation, including consulting their tax, legal, accounting or other advisors about such information. Apollo does not act for you and is not responsible for providing you with the protections afforded to its clients. This presentation does not constitute an offer to sell, or the solicitation of an offer to buy, any security, product or service, including interest in any investment product or fund or account managed or advised by Apollo.
Certain statements made throughout this presentation may be “forward-looking” in nature. Due to various risks and uncertainties, actual events or results may differ materially from those reflected or contemplated in such forward-looking information. As such, undue reliance should not be placed on such statements. Forward-looking statements may be identified by the use of terminology including, but not limited to, “may”, “will”, “should”, “expect”, “anticipate”, “target”, “project”, “estimate”, “intend”, “continue” or “believe” or the negatives thereof or other variations thereon or comparable terminology.
We do not yet know what a computer can't do. Indeed, for nearly one hundred years, the computer has been defined capaciously, as a machine that can do the work of any other machine provided it can be defined logically (Alan Turing). Adopting François Laruelle's parlance, Turing's definition could be renamed the Principle of Sufficient Computation; the definition ensures that the computer can actuate any and all events, provided they are formulated as ideas.
The Principle of Sufficient Computation thus reveals a series of characteristics common in computing:
(1) The centrality of action or practice, understood as a series of commands that may be executed in order to alter the states of a system.
(2) The linking of idea to action, wherein if something can be thought it can be executed, and if something has been executed it was, perforce, previously thought.
(3) Practical omniscience, where knowledge swells to the very limits of knowability, even as those limits have been incontrovertibly demonstrated using logical proof.
(4) A system of judgment based not in morality or politics but in mimesis. Computers thus parrot the old question from the
Poetics
of Aristotle:
Is this copy a well-crafted copy?
So we do not yet know what a computer can't do, mostly because the computer has been doing so much for so long.
And, still, indicators show a variety of alternatives, varieties of computation that reside not so much before or after mainstream computing, but along side it. The varieties of computation would include digital computing (the paradigmatic implementation of the Principle of Sufficient Computation), analog computing (formerly dominant, but today largely overshadowed), dialectical computing (unimaginable using today's chips and software), and non-standard or artificial computing.
Artificial computation was discovered by Laruelle, even as artificial computers have not yet been invented, similar to the discovery of Shor's algorithm prior to any machine capable of implementing it. Synonyms for artificial computation include: non-computation, non-standard computation, compu-fiction, and computer fiction.
Artificial computation is defined, axiomatically, as the withdrawal from the Principle of Sufficient Computation, and hence in terms of:
(1) The preemption of all commands and the neutering of the executable, in favor of
pure process
as a phenomenon immanent to itself.
(2) The delinking of idea and action as to be absolutely un-exchangeable with each other.
(3) Knowledge as radically finite, existing not as the total aggregation of ever-widening claims about the world, but as a series of axioms in the generic real.
(4) A non-Aristotelian technology of immanence, where technology is not understood in terms of craft or mimesis (whether effective or defective).
Artificial computation is thus not post-computational, but rather, somehow, along side it, as a science "liberated from...the neurosciences or cybernetics" (Laruelle). In this sense artificial computation enacts a generic form of thinking, which, ironically, has thus far remained unthinkable by that overweening discipline of philosophy.
French Football Federation discloses data breach after cyberattack
Bleeping Computer
www.bleepingcomputer.com
2025-11-28 16:12:03
The French Football Federation (FFF) disclosed a data breach on Friday after attackers used a compromised account to gain access to administrative management software used by football clubs. [...]...
The French Football Federation (FFF) disclosed a data breach on Friday after attackers used a compromised account to gain access to administrative management software used by football clubs.
After detecting the unauthorized access, FFF's security team disabled the compromised account and reset all user passwords across the system.
However, before they were detected and evicted from the breached systems, the threat actors stole personal and contact information from members of French football clubs.
"Upon detection of this unauthorized access through the use of a compromised account, the FFF services took the necessary steps to secure the software and data, including immediately disabling the account in question and resetting all user account passwords," the
FFF said
[
machine translation
].
"This breach is limited to the following data only: name, surname, gender, date and place of birth, nationality, postal address, email address, telephone number and license number."
As required under European data protection regulations, the organization has filed a criminal complaint and notified France's National Cybersecurity Agency (ANSSI) and the National Commission on Informatics and Liberty (CNIL), the country's data protection authority.
The FFF said it will directly notify all individuals whose email addresses appear in the compromised database and urged members to be suspicious of messages claiming to originate from the federation, their clubs, or other senders.
French football club members should be wary of any communications requesting that they open attachments or provide account credentials, passwords, or banking information.
"The FFF is committed to protecting all the data entrusted to it and is constantly strengthening and adapting its security measures in order to cope, like many other actors, with the increasing number and new forms of cyberattacks," the FFF added.
A spokesperson for the French Football Federation (FFF) was not immediately available for comment when contacted by BleepingComputer earlier today.
Earlier this month, the French social security service for parents and home-based childcare providers (Pajemploi)
also suffered a data breach
that may have exposed personal information of approximately 1.2 million individuals.
It's budget season! Over 300 CISOs and security leaders have shared how they're planning, spending, and prioritizing for the year ahead. This report compiles their insights, allowing readers to benchmark strategies, identify emerging trends, and compare their priorities as they head into 2026.
Learn how top leaders are turning investment into measurable impact.
The best Black Friday TV deals in the UK – and how to avoid a bad one
Guardian
www.theguardian.com
2025-11-28 16:11:20
We’ve rounded up the best Black Friday TV deals for every budget, from 50in OLEDs and small smart TVs to top-rated brands like Samsung and LG • Do you really need to buy a new TV?• The best Black Friday laptop deals When it comes to buying a new TV during Black Friday, careful prep and a canny eye f...
W
hen it comes to buying a new TV during Black Friday, careful prep and a canny eye for detail are everything. Sometimes that big-screen bargain isn’t quite the steal you might think, and even if the price is right, niggling problems could sour long-term satisfaction.
But if you are set on a new TV, the trick is to know what you want, and why, before you start shortlisting. Here we round up some of the best TV deals in this year’s
Black Friday sales
, before giving detailed advice on what to look out for.
Q&A
How is the Filter covering Black Friday?
Show
At the Filter, we believe in buying sustainably, and the excessive consumerism encouraged by Black Friday doesn’t sit easily with us. However, we also believe in shopping smarter, and there’s no denying that it’s often the best time of year to buy big-ticket items that you genuinely need and have planned to buy in advance, or stock up on regular buys such as skincare and cleaning products.
Retailers often push offers that are not as good as they seem, with the intention of clearing out old stock, so we only recommend genuine deals. We assess the price history of every product where it’s available, and we won’t feature anything unless it is genuinely lower than its average price – and we will always specify this in our articles.
We only recommend deals on products that we’ve tested or have been recommended by product experts. What we choose to feature is based on the best products at the best prices chosen by our editorially independent team, free of commercial influence.
This very feature-rich 55in Hisense U7Q Pro model has a searingly bright Mini LED panel, and supports up to 4K/144Hz gaming with AMD FreeSync Premium Pro VRR and ALLM support over its HDMI 2.1 ports. The built-in speakers support Dolby Atmos for more immersive audio, and it even has a built-in subwoofer to provide stronger bass extension. HDR support is great, with Dolby Vision IQ, HDR10+ and HDR10 – it seems HLG is missing. This is one of the best sets of specs we’ve seen for the £429 asking price, though.
This capable 50in tellyoffers a very solid performance for the money. It supports a full complement of HDR standards, with Dolby Vision IQ, HDR10+ Adaptive, HDR 10 and HLG. It’s a QLED with capable local dimming, so images are vibrant with a good amount of depth. If it’s a gaming TV you’re after, this probably isn’t the best choice: there’s one HDMI 2.1 eARC port and three HDMI 2.0 ports, and the refresh rate is limited to 60Hz. Nonetheless, with Amazon’s Fire TV smart stuff built in, this is a good TV for the price.
Amazon’s own budget 4K TV has dropped to its lowest price ever for Black Friday Week, and is a decent offer for its modest price tag. This deal is on the mid-size 55in model, which provides good detail alongside decent HDR support with HDR10 and HLG supported. It’s a smart TV, and uses Amazon’s Fire system, which will be familiar to anyone who owns a Fire Stick – it’s like having one built in to your telly, providing access to streaming apps and Amazon’s Prime Video service at the touch of a button.
This 43in 4K TCL TV is at its best price in a few months. As has become common for TCL’s TVs, it punches above its weight in terms of specs, supporting Dolby Vision HDR and Dolby Atmos audio with its built-in speakers. It also has three HDMI ports, with support for HDMI 2.1 with ALLM and VRR for gaming. It maxes out at 60Hz, though, so if you’re looking specifically for a gaming TV there are better choices on this list.
Price history:
this is its lowest ever price. It was discounted in July, but only to £199.
For such an affordable mid-sized TV, this Hisense A6Q ticks most of the boxes, with a 4K resolution and a pleasant complement of HDR standards (Dolby Vision, HDR10 and HLG) that can provide vivid highlights in supported content. Freely is built in to provide access to live and on-demand TV out of the box, with no aerial or set-top box required. For gaming, there’s support for Auto Low Latency Mode, although the refresh rate tops out at 60Hz (120Hz or higher is preferable for smoother onscreen action with recent consoles). Connectivity is limited to three HDMI 2.1 ports.
The TCL C6KS seems too good to be true, packed with higher-end features at a bargain-basement price. It’s a modest 50 inches, but its Mini LED panel is bright and sharp with 160 dimming zones, allowing for surprisingly vibrant and saturated images for a television at this price point. Its new HVA panel helps in providing more depth to its overall images, top. For this price, the fact that it supports HLG, HDR10, HDR10+ and Dolby Vision is excellent. Many TVs several times more expensive can’t say that.
Gaming performance is fine – it’ll do up to 4K/60Hz with VRR and ALLM – although the lack of higher-refresh-rate output means it isn’t optimal for powerful home consoles or PCs. In terms of connectivity, there are three HDMI ports, one of which supports eARC to hook up a soundbar or speakers for improved audio. The Onkyo-branded system with this TV is surprisingly detailed, though – you may be able to get away with using it for some time before feeling the need to upgrade to separate speakers.
While this isn’t Sony’s latest and greatest option, it’s a quality telly for a great price. It’s a moderate size that should be good for most rooms, and HDR support is reasonable, with HDR10, HLG and Dolby Vision all supported.
This Sony TV also comes with four HDMI ports for inputs, plus support for eARC to connect a soundbar or supported speakers. For gaming, it has ALLM, a custom screen size setting, a black equaliser and an onscreen crosshair – features more commonly seen in monitors than TVs.
Smart TV duties are handled by the familiar Google TV OS, providing good access to smart apps, and it bundles in 15 credits of Sony Pictures Core and a 24-month streaming package.
This is an excellent value pick if you’re after a TV for gaming. It features four HDMI ports, with two that support full 4K/144Hz HDMI 2.1 output, making it ideal to pair with a games console or living-room PC. It also supports VRR and ALLM for the most optimal experience. This screen also has rich HDR support – supporting Dolby Vision, HLG, HDR10 and HDR10+ – and it has a decent Onkyo-tuned Atmos-capable speaker setup. For well under £400, this feels like quite a steal.
This large, feature-rich Mini LED TV looks like a capable option for everything from games to films.
There’s a rich set of HDR support (Dolby Vision and Dolby Vision IQ, plus HDR10+, HDR10 and HLG) and the Mini LED screen allows for some serious depth and searing brightness (Hisense claims a peak brightness of 2,000 nits).
The integrated speakers provide up to 50W of power and support Dolby Atmos and DTS:X surround standards. There are different sound modes to dig into, too, as well as different picture modes, so you can optimise image quality as you wish.
It has four HDMI 2.1 ports with full support for 4K/120Hz gaming, so will play nicely with modern games consoles (and if you’re a PC gamer looking for a living-room monitor, you’ll be glad to know that refresh rate can be overclocked further up to 165Hz). Support for variable refresh rate and ALLM further enhances the gaming experience.
Price history:
not available, but this is its lowest ever price at Currys.
LG’s B5 OLED is the most affordable entry in the brand’s 2025 OLED lineup, adding a new processor, improved connectivity and an updated UI to last year’s B4 model.
It comes with four HDMI 2.1 ports with full support for 4K/120Hz output, and also offers VRR and ALLM for gaming. In terms of HDR, the B5 supports Dolby Vision IQ, Dolby Vision, HDR10, HLG and Cinema HDR, offering impactful highlights in supported content. It also comes with LG’s slick and easy-to-use webOS operating system, with a wide range of streaming apps, plus different picture modes. The 20W speakers support Dolby Atmos.
If you want a larger version, the 65in version is currently
£1,159 at AO
.
If all you’re after is a serviceable and hardy larger-screen TV, then this 65in Philips Ambilight model could be a good choice – especially for sub-£500. It provides you with plenty of screen real estate and Philips’ own TitanOS smart system for accessing smart TV apps.
There is decent HDR support for such an affordable television, with HDR10 and HLG, plus compatibility with HDR10+, and it has three HDMI ports – one of which is an eARC port for soundbars. It also supports HDMI VRR and ALLM for gaming. With this in mind, though, its maximum refresh rate is 60Hz rather than 120Hz. Its 20W speakers also have Atmos support.
What’s unique about Philips TVs is the presence of the company’s Ambilight system, which provides atmospheric lighting on the rear of the unit that projects on to the surface behind it. Aimed at reducing eye strain, this is also useful if you want to add a splash of colour to your room.
The Philips Ambilight 65OLED760 might not be one of the brand’s flagship OLED choices, but as more of a mid-range one provides a very rich set of features for its price point. It’s a solid OLED screen that provides great depth and good contrast while supporting all four of the main HDR formats – Dolby Vision, HLG, HDR10 and HDR10+, and comes with various picture modes to choose from. This Philips screen also benefits from four HDMI 2.1-capable ports for gaming use, with full 4K/120Hz powers with ALLM and VRR so it’ll play nicely with console or PC.
What’s quite unique about this Philips TV is the presence of its Ambilight system, providing atmospheric lighting on the rear of the unit that projects onto the surface behind it. This is useful if you want to add a splash of colour to your setup through your TV, which not many others can do.
The S90F is one of the only QD-OLED TVs Samsung offers – combining the inky blacks and virtually infinite contrast of an OLED with the higher peak brightness of a QLED. It means this telly provides sublime image quality. HDR support consists of HDR10, HDR10+ and HLG; Samsung still isn’t supporting Dolby Vision.
The four HDMI 2.1 ports are welcome, and offer support for proper 4K/120Hz gaming, with up to 144Hz for PC gaming. There is also support for variable refresh and ALLM, alongside Samsung’s Game Hub menu.
Price history:
this is its lowest ever price at Amazon.
Sony’s Bravia 8 II is one of the best TVs currently available – with a price to match. The QD-OLED panel is 25% brighter than on the A95L model it replaces, with even sharper picture quality, not least with HDR enabled. There’s support for Dolby Vision, HLG and HDR10, and claimed peak brightness is a searing 4,000 nits.
The Acoustic Surface Audio+ speaker system is widely well reviewed, providing surprisingly solid audio for a set of TV speakers. Gaming support is strong, too, with 4K/120Hz input supported over HDMI 2.1, plus VRR and ALLM. There’s even an ‘optimised for PS5’ mode that automatically optimises the TV’s settings when it detects that the PlayStation is plugged in. Unlike Samsung and LG’s flagships, however, there are only two HDMI 2.1 ports here (one of which is an eARC, if you did want to connect a soundbar).
LG’s latest mid-range C-series model OLED offers an improved operating system and a new processor over last year’s C4. The new processor helps it to better upscale content to 4K than previous models, too, which is handy if you watch a lot of older and lower-definition content.
There are four HDMI 2.1 ports for 4K/144Hz gaming (or up to 4K/120Hz on consoles), with VRR and ALLM supported on all of them. HDR support comes in the form of HDR10, HLG and Dolby Vision, and this 55in model benefits from LG’s Evo panel for even stronger brightness.
As you’d expect from an LG OLED, it comes with dazzling image quality with inky blacks, sublime contrast and wonderfully accurate colours that make viewing everything from games to movies a pleasure.
The Sony Bravia 8A was initially more expensive than the standard Bravia 8, but as pricing has settled, this A variant has become a more compelling choice. One of Sony’s 2024 TV models, it comes with an OLED panel that it claims is 10% brighter than its predecessor. HDR support comes in with Dolby Vision, HDR10 and HLG (no HDR10+). It also comes with Sony’s AI-enabled Bravia XR processor inside, plus the same Acoustic Surface Audio+ sound system for surprisingly decent audio. The Google TV operating system also provides a familiar experience.
As an OLED, image quality looks to be rather excellent, and gamers should be decently happy with HDMI 2.1 support for 4K/120Hz output with VRR and ALLM. As with the other Bravia above, this comes with the caveat that HDMI 2.1 is supported on only two of this TV’s four HDMI inputs.
This 42in TV is the most compact OLED you can buy in the current LG range, making it a dead cert if you’ve got a smaller room like me, or want to use this TV as a super-size monitor.
There are four HDMI 2.1-enabled ports with 4K/144Hz capability (and up to 120Hz on consoles), plus VRR and ALLM support for gaming. There’s also solid HDR support with HDR10, Dolby Vision and HLG. LG’s new webOS 25 brings small upgrades, such as the ability to hide apps on the front screen, and it’s a reasonable smart TV system to use every day.
It also comes with a new AI Magic Remote with what LG calls ‘AI buttons’ – voice controls and drag-and-drop functions.
TCL has become known for its more affordable, feature-rich televisions in recent years. The C7K is a shining example of this, providing excellent image quality at a modest price point for its 65in size. A particular highlight is its fantastic black levels, contrast and dynamic range, helped along by a bright QD-Mini LED panel with 1,008 dimming zones and a claimed peak brightness of up to 2,600 nits.
It also has a full complement of HDR support, with HLG, HDR10, HDR10+ and Dolby Vision for impactful highlights. Gaming support is great, with two out of the four HDMI ports supporting 4K/144Hz HDMI 2.1 powers with VRR and ALLM. Plus, if you’re willing to drop the resolution down to Full HD from 4K, it can double the refresh rate to 288Hz. That’s a feature more typically found on a PC monitor than a TV, but if you have a powerful PC connected and want to maximise performance in high-refresh-rate games, then the C7K should allow you to do so.
This new model also adds in a Bang & Olufsen stereo with 60W of power, replacing TCL’s previous collaboration with Onkyo, as well as support for Dolby Atmos and DTS:X soundtracks.
If you’re after an even bigger screen, the 75in version is
£948 at Amazon
and
AO
.
Price history:
it’s only £40 cheaper than it’s been at Currys all month, but it’s still its lowest ever price.
Last year’s mid-range LG OLED, the C4 was the first LG C-series OLED to support native 4K/144Hz output over its four HDMI 2.1 ports – a boon for gamers. It also brings proper Nvidia G-Sync certification for one of its supported VRR standards.
The presence of webOS 24 brings benefits such as a built-in Chromecast for easy wireless casting right out of the box, and the Alpha 9 processor brings AI smarts to enhance the clarity of onscreen dialogue and to bring even more channels of virtual surround sound. The internal 40W speakers have Atmos support to provide more cinematic audio in supported content.
Price history:
this is higher than it was during Prime Day – but only by 2p.
Reece Bithrey
Black Friday TV deals: what to look out for – and how to avoid the bad ones
Steer clear of the bargain aisle if you want your new TV to make a statement.
Photograph: Pressmaster/Getty Images
Design is important
Design counts. If you want that new TV to be a statement in your living space, stay clear of the bargain aisle – that’s where you’ll find cookie-cutter designs with flimsy plastic pedestal stands. If you’re not wall mounting, pay particular attention to the feet. Are they placed close to each edge? On a TV 55 inches and larger, that could mean you’ll also have to factor in new furniture just to accommodate it.
Central pedestal stands are always the easiest to live with, and some models also have a swivel so that you can angle the screen to best suit your seating. It’s a little bonus well worth having.
Think about when you’ll use it
Are you buying a TV for everyday use, or do you hanker after a special screen for movies? If it’s the latter, buying an OLED will generally be your best bet. Unlike LED-based TVs, there’s no need for a backlight, because OLED pixels are self-emitting. This means when you dim the lights, black levels stay nice and inky and shadow detail is retained, giving your pictures cinematic depth. Conversely, LED models (be they LCD LED, QLED or Mini LED) tend to look their best in rooms with ambient light, and therefore make better daytime TVs.
Connectivity counts
Don’t just look at the front. The cheapest TVs you’ll see during the Black Friday sales will only offer three HDMI inputs at the back. This may be fine if you don’t plan on connecting much equipment, but it could prove limiting in the long term. Cheap televisions tend to offer poor audio, so one of those HDMI ports will probably be assigned to a soundbar. That just leaves two to share between games consoles, set-top boxes and Blu-ray/DVD players.
Maximise gaming performance by playing at a higher refresh rate.
Photograph: simpson33/Getty Images
Consider what you need for gaming
If you plan to play video games on your new set, check to see if those HDMIs support a 120Hz high refresh rate. If you own a PlayStation 5 or current Xbox, you can maximise performance (and therefore improve your chances of winning against your mates) by playing at a higher refresh rate. These 120Hz-capable TVs also tend to offer VRR (variable refresh rate) and ALLM (auto low latency mode), acronyms that add to the gaming experience.
Incidentally, if you buy a Sony Bravia, there’s a good chance it will also have PS Remote Play, meaning you can enjoy your PlayStation console while it’s not even in the same room as the TV.
Of course, you can always play games on standard 60Hz TVs, and if you have an older console or just like casual family games, you’ve nothing to worry about.
Don’t avoid last year’s models
Many TV makers use Black Friday to offer cheap deals on older stock, to clear inventory. This is where you really can grab a killer deal, particularly at the mid-to-higher end of the market.
For example, a 2024 LG OLED C5 55in screen has a sale price of £1,199. The 2025 G5 OLED, also on sale, still commands a premium of £1,599. Last year’s top models will still impress 12 months after release.
Buying a well-reviewed older TV is almost always better than buying a newer model that’s been stockpiled to shift in volume during the sales.
Not all HDR is worth having
It’s worth bearing in mind that not all HDR (high dynamic range) TVs are created equal. While every 4K model sporting a Black Friday price tag will boast HDR compatibility, there can be huge differences in performance. Entry-level screens – typically those 50in models selling for little more than £200 – will invariably lack the brightness to make HDR programmes really shine. Indeed, in their attempt to make HDR details (such as bright street lights, fireworks, explosions and so on) pop, the rest of the show can look unnaturally dark. These HDR ‘lite’ TVs are actually better suited to non-HDR programmes, such as regular SDR (standard dynamic range) channels on Freeview, rather than streams from Netflix and Co.
The good news is that HDR performance improves dramatically from the mid-range upwards, and is a real differentiator at the posh end of the market.
Be aware also that HDR comes in different flavours. In addition to standard HDR10, there’s HDR10+, Dolby Vision and cleverly ‘Adaptive’ versions of each on top, able to react to the light levels in your living room. Film fans favour screens that offer Dolby Vision, but not every brand has it. Samsung is the most prominent outlier.
Sound advice
Finally, listen out for audio. It’s a fact that thin, inexpensive TVs generally tend to sound awful. They lack bass and become painful when you crank the volume. But there are exceptions that could save you from shelling out on a soundbar.
Samsung QLED TVs boasting object tracking sound (OTS) offer far better audio than you might expect, and tend to have innovative processing that can enhance dialogue and combat extraneous noise, making them great for family use. Meanwhile, Sony OLED TVs have a clever Acoustic Surface Audio sound system, which uses actuators on the rear of the OLED panel to produce impressive high-fidelity audio. And if you want the full home theatre audio enchilada, Panasonic’s best sets have a full 360-degree sound system with front, side and up-firing speakers tuned by sister brand Technics, able to produce convincing Dolby Atmos cinema sound.
And here’s a closing tip. Even if your Black Friday bargain TV doesn’t have decent sound onboard, check to see if it passes a Dolby Atmos signal out over the e-ARC HDMI connection, because you can always add a great-sounding Dolby Atmos soundbar during the January sales.
Steve May
Analyst: Tom Bellwether
Contact Information: None available*
*Because of the complex nature of financial alchemy, our analysts live a hermetic lifestyle and avoid relevant news, daylight, and the olfactory senses needed to detect bullshit.
Following our review of Beignet Investor LLC (the Issuer), an affiliate of Blue Owl Capital, in connection with its participation in an 80% joint venture with Meta Platforms Inc., we assign a preliminary A+ rating to the Issuer’s proposed $27.30 billion senior secured amortizing notes.
This rating reflects our opinion that:
All material risks are contractually assigned to Meta,
which allows us to classify them as hypothetical and proceed accordingly.
Projected cash flows are sufficiently flat and unbothered by reality
to support the rating.
Residual Value Guarantees (RVGs) exist
, which we take as evidence that asset values will behave in accordance with wishes rather than markets.
The Outlook is Superficially Stable, defined here as “By outward appearances stable unless, you know, things happen. Then we’ll downgrade after the shit hits the fan.”
Blue Owl Capital Inc. (Blue Owl, BBB/Stable), through affiliated funds, has created Beignet Investor LLC (Beignet or Issuer), a project finance-style holding company that will own an 80 percent interest in a joint venture (JVCo) with Meta Platforms Inc. (Meta, AA-/Stable). The entity is named “Beignet,” presumably because “Off-Balance-Sheet Leverage Vehicle No. 5” tested poorly with focus groups.
Beignet is issuing $27.30 billion of senior secured amortizing notes due May 2049 under a Rule 144A structure.
Note proceeds, together with $2.45 billion of deferred equity from Blue Owl funds and $1.16 billion of interest earned on borrowed money held in Treasuries, will fund Beignet’s $23.03 billion contribution to JVCo for the 2.064 GW hyperscale data center campus in Richland Parish, Louisiana, along with reserve accounts, capitalized interest and other transaction costs that seem small only in comparison to the rest of the sentence.
Iris Crossing LLC, an indirect Meta subsidiary, will own the remaining 20 percent of JVCo and fund approximately $5.76 billion of construction costs.
We assign a preliminary A+ rating to the notes, one notch below Meta’s issuer credit rating, reflecting the very strong contractual linkage to Meta and the tight technical separation that allows Meta to keep roughly $27 billion of assets and debt off its balance sheet while continuing to provide all material economic support.
Arrows, like cats, have a way of coming home, no matter how far you throw them.
Meta transferred the Hyperion data center project into JVCo, which is owned 80 percent by Beignet and 20 percent by Iris Crossing LLC, an indirect Meta subsidiary. JVCo, in turn, owns Laidley LLC (Landlord). None of this is unusual except for the part where Meta designs, builds, guarantees, operates, funds the overruns, pays the rent, and does not consolidate it.
This project has nine data centers and two support buildings, with about four million sq. ft. and 2.064 GW capacity. The support buildings will store the reams of documentation needed to convince everyone this structure isn’t what it looks like. The total capital plan of $28.79 billion will be funded as follows:
$27.30 billion of debt raised by Beignet.
$2.45 billion of deferred equity commitments from Blue Owl funds.
And, in a feat of financial hydration, $1.16 billion of interest generated by the same borrowed money while it sits in laddered Treasuries.
The structure allows the Issuer to borrow money, earn interest on the borrowed money, and then use that interest to satisfy the equity requirement that would normally require… money.
Nothing is created. Nothing is contributed. It’s a loop. Borrow money, earn interest, and use the interest to claim you provided equity. The kind of circle only finance can call a straight line.
Together, these flows cover Beignet’s $23.03 billion obligation to JVCo, plus the usual constellation of capitalized interest, reserve accounts, and transaction expenses. In any other context this would raise questions. For us, it raises the credit rating.
Meta, through Pelican Leap LLC (Tenant), has entered into eleven triple-net leases—one for each building—with an initial four-year term starting in 2029 and four renewal options that could extend the arrangement to twenty years. The leases rely on the assumption that Meta will continue to need exponentially more compute power and that AI demand will not collapse, reverse, plateau, or become structurally inconvenient.
The notes issued by Beignet are secured by Beignet’s equity interest in JVCo and relevant transaction accounts. They are not secured by the underlying physical assets, which remain at the JVCo and Landlord level. This is described as standard practice, which is true in the same way that using eleven entities to rent buildings to yourself has become standard practice.
The resulting structure allows Meta to support the project economically while leaving the associated debt somewhere that is technically not on Meta’s balance sheet. The distinction is thin, but apparently wide enough to matter.
The preliminary A+ rating reflects our view that this is functionally Meta borrowing $27.30 billion for a campus no one else will touch, packaged in legal formality precise enough to satisfy the letter of consolidation rules and absurd enough to insult the spirit.
Credit risk aligns almost one-for-one with Meta’s own profile because:
Meta is obligated to fund construction cost overruns beyond 105 percent of the fixed budget, excluding force majeure events, which rating agencies historically treat as theoretical inconveniences rather than recurring features of the physical world.
Meta guarantees all lease payments and operating obligations, both during the initial four-year term and across any renewal periods it already intends to exercise, an arrangement whose purpose becomes clearer when one remembers why the campus is being built at all.
Meta provides an RVG (residual value guarantee) structured to be sufficient, in most modeled cases, to ensure bondholders are repaid even if Meta recommits to the Metaverse or any future initiative born from its ongoing fascination with expensive detours. We did not model what would happen if data center demand collapses and Meta cannot secure a new tenant. This scenario was excluded for methodological convenience.
The minimum rent schedule has been calibrated to produce a debt service coverage ratio of approximately 1.12 through 2049. We consider this a sufficient level of stability usually found only in spreadsheets that freeze when real-world data is used.
Taken together, these features tie Beignet’s credit quality to Meta so tightly that you’d have to not be paying attention to miss them. The structure maintains a precarious technical separation that, under current interpretations of accounting guidance, allows Meta to keep roughly $27 billion of assets and debt off its own balance sheet while continuing to provide every meaningful form of economic support.
This treatment is considered acceptable because the people who decide what is acceptable have accepted it.
JVCo qualifies as a variable interest entity because the equity at risk is ceremonial and the real economic exposure sits entirely with the party insisting it does not control the venture. This remains legal due to the enduring belief that balance sheets are healthier when the risky parts are hidden.
Under U.S. GAAP, consolidation is required if Meta is the primary beneficiary, defined as the party that both:
Directs the activities that most significantly affect the entity’s performance, and
Absorbs significant losses or receives significant benefits.
Meta asserts it is not the primary beneficiary.
To evaluate that assertion, we note the following uncontested facts:
Meta is responsible for designing, overseeing, and operating a 2.064 GW AI campus, an activity that requires technical capabilities Blue Owl does not possess.
Meta bears construction cost overruns beyond 105 percent of the fixed budget, as well as specified casualty repair obligations of up to $3.125 billion per event during construction.
Meta provides the guarantee for all rent and operating payments under the leases, across the initial term and any renewals.
Meta provides the residual value guarantee, ensuring bondholders are repaid if leases are not renewed or are terminated, either through a sale or by paying the guaranteed minimum values directly.
Meta contributes funding, directs operations, bears construction risk, guarantees payments, guarantees asset values, determines utilization, controls renewal behavior, and can trigger the sale of the facility.
Based on this, or despite this, Meta concludes it does not control JVCo.
Our interpretation is fully compliant with U.S. GAAP, which prioritizes the geometry of the legal structure over the inconvenience of economic substance and recognizes control only if the controlling party agrees to be recognized as controlling.
Meta has not agreed, and the framework, including this agency, respects that choice.
For rating purposes, we therefore accept Meta’s non-consolidation as an accounting outcome while treating Meta, in all practical respects, as fully responsible for the performance of an entity it does not officially control.
The lease structure is designed to look like a normal commercial arrangement while functioning as a long-term commitment Meta insists, for accounting reasons, it cannot possibly predict.
Tenant will pay fixed rent for the first 19 months of operations, based on a 50 percent assumed utilization rate, after which rent scales with actual power consumption. The leases are triple-net. Meta is responsible for everything: operating costs, maintenance, taxes, insurance, utilities. If a pipe breaks, Meta fixes the pipe. If a hurricane relocates a roof, Meta pays to staple the roof back on.
In practical terms, the only scenario in which Beignet bears operating exposure is a scenario in which Meta stops paying its own bills, at which point the lease structure becomes irrelevant because the same lawyers that structured this deal will have already quietly extricated Meta from liability.
The contract terms include:
A minimum rent floor engineered to produce a DSCR of 1.12 in a spreadsheet where 1.12 was likely hard-coded and independent of math.
A four-year initial term with four four-year renewal options, theoretically creating a 20-year runway Meta pretends not to see.
Meta guarantees all tenant payment obligations across the entire potential lease life, including renewals it strategically refuses to acknowledge as inevitable.
No performance-based KPIs. Under this structure, the buildings could underperform, overperform, or catch fire. Meta still pays rent.
The RVG requires Meta to ensure that, at every potential lease-termination date, the asset is worth at least the guaranteed minimum value. If markets disagree, Meta pays the difference. Because Meta is rated AA-/Stable, we are instructed to assume that it will do so without hesitation, including in scenarios where demand softens or secondary markets discover that a hyperscale campus in Richland Parish is not the world’s most liquid asset class.
The interplay between the lease term and the RVG creates a circular logic we find structurally exquisite.
From a credit perspective, this circularity is considered supportive, because the same logic used to avoid consolidating the debt also ensures bondholders are paid. The circularity is not treated as a feature or a flaw. It is treated as accounting.
Because Meta is AA-/Stable, we assume it will pay whatever number the Excel model finds through Goal Seek, even in scenarios involving technological obsolescence or an invasion of raccoons.
The accounting hinges on a paradox engineered with dull tweezers:
Under lease accounting, Meta must record future lease obligations only if renewals are reasonably certain.
Under RVG accounting, Meta must record a guarantee liability only if payment is probable.
To keep $27 billion off its balance sheet, Meta must therefore assert:
Renewals are not reasonably certain, despite designing, funding, building, and exclusively using a 2.064 GW AI campus for which the realistic tenant list begins and ends with Meta.
The RVG will probably never be triggered, despite the fact that not renewing would trigger it immediately.
This requires a narrow corridor of assumptions in which Meta simultaneously plans to use the facility for two decades and insists that no one can predict four years of corporate intention.
From a credit standpoint, we are supportive. The assumptions that render the debt invisible are precisely what make it secure. A harmony best described as collateralized cognitive dissonance.
Meta linkage.
The economics are wedded to Meta’s credit profile, which we are required to describe as AA-/Stable rather than “the only reason this entire structure doesn’t fold from a stiff breeze.” Meta guarantees the rent, the RVG, and the continued relevance of the facility. The rest is décor auditors would deem “tasteful.“
Minimum rent floor.
The lease schedule produces a perfectly flat DSCR of 1.12 through 2049. Projects of this size do not produce flat anything, but the model insists otherwise, so we pretend we believe it. Being sticklers for tradition, and having learned nothing from the financial crisis of 2008, we treat the spreadsheet as the final arbiter of truth, even when the inputs describe a world no one lives in.
Construction risk transfer.
Meta absorbs cost overruns beyond 105 percent of budget and handles casualty repairs during construction. Our methodology interprets “contractually transferred” as “ceased to exist,” so we decline to model the risk of overruns on a $28 billion campus built in a hurricane corridor. This is considered best practice.
RVG backstop.
The residual value guarantee eliminates tail risk in much the same way a parent cosigning for their teenager’s car loan eliminates tail risk: by ensuring that the person with all the money pays for everything. If the market value collapses, Meta pays the difference. If the facility can’t be sold, Meta pays the whole thing. If the entire campus becomes a raccoon sanctuary, Meta still pays. We classify this as credit protection, a nuanced designation that allows us to recognize the security of the arrangement without recognizing the debt.
Absence of performance KPIs.
There are no operational KPIs that allow rent abatement. This is helpful because KPIs create volatility, and volatility requires thought, a variable we explicitly exclude from our methodology. By removing KPIs entirely, the structure ensures a level of cash-flow stability that exists only in transactions where the tenant is also the economic owner pretending to be a squatter.
The rating also reflects several risks that are acknowledged, intellectually troubling, and ultimately tolerated because Meta is large enough that everyone agrees to stop asking questions.
Off-balance-sheet dependence.
Meta treats JVCo as if it belongs to someone else, which is a generous interpretation of ownership. If consolidation rules ever evolve to reflect economic substance, Meta could be required to add $27 billion of assets and matching debt back onto its own balance sheet. Our methodology treats this as a theoretical inconvenience rather than a credit event, because calling it what it really is would create a conflict with the very companies we rate.
Concentration risk.
The entire project exists for one tenant with one business model in one industry undergoing technological whiplash. The facility is engineered so specifically for Meta’s AI ambitions that the only plausible alternative tenant is another version of Meta from a parallel timeline. We strongly disagree with the many-worlds interpretation of quantum mechanics. We set this concern aside because at this stage in the transaction, the A+ rating is a structural load-bearing wall, and we are not paid to do demolition.
Residual value uncertainty.
The RVG depends on modeled guaranteed minimum values that assume buyers will one day desire a vast hyperscale complex in Richland Parish under stress scenarios. If hyperscale supply balloons or the resale market for 2-gigawatt data centers becomes as illiquid as common sense, Meta will owe more money. This increases Meta’s direct obligations, which should concern us, but does not, because Meta is rated AA-/Stable and therefore presumed to withstand any scenario we have chosen not to model.
Casualty and force majeure.
In extreme scenarios, multiple buildings could be destroyed by a hurricane, which we view as unlikely given that they almost never impact Louisiana. The logic resembles a Rube Goldberg machine built out of indemnities. We classify this as a strength.
JV structural subordination.
Cash flows must navigate waterfalls, covenants, carve-outs, and the possibility of up to $75 million of JV-level debt. These features introduce structural complexity, which we flag, then promptly ignore, because acknowledging would force us to explain who benefits from the convolution.
Despite these risks, we maintain an A+ rating because Meta’s credit quality is strong, the structure is designed to hide risk rather than transfer it, and our role in this ecosystem is to observe these contradictions and proceed as though they were features rather than warnings.
The outlook is Superficially Stable. That means we expect the structure to hold together as long as Meta keeps paying for everything and the accounting rules remain generously uninterested in economic reality.
We assume, with the confidence of people who have clearly not been punished enough:
Meta will preserve an AA-/Stable profile because any other outcome would force everyone involved to admit what this actually is.
Construction will stay “broadly on schedule,” a phrase we use to pre-forgive whatever happens as long as Meta covers the overruns, which it must.
Lease payments and the minimum rent schedule will continue producing a DSCR that hovers around 1.12 in models designed to ensure that result, and not materially below 1.10 unless something un-modeled happens, which we classify as “outside scope.”
The RVG will remain enforceable, which matters more than the resale value of a hyperscale facility in a world where hyperscale facilities may or may not be worth anything.
Changes in VIE or lease-accounting guidance will affect where Meta stores the debt, not whether Meta pays it.
We could lower the rating if Meta were downgraded, if DSCR sagged below the range we pretend is acceptable, if Meta weakened its guarantees, or if events unfold in ways our assumptions did not account for, as events tend to do. The last category includes anything that would force us to revisit the assumptions we confidently made without testing.
We view an upgrade as unlikely. The structure already performs the single miracle it was designed for: keeping $27.3 billion off Meta’s balance sheet in a manner we are professionally obligated to support.
CONFIDENTIALITY AND USE:
This report is intended solely for institutional investors, entities required by compliance to review documents they will not read, and any regulatory body still pretending to monitor off-balance-sheet arrangements. FSG LLC makes no representation, warranty, or faint gesture toward coherence regarding the accuracy, completeness, or legitimacy of anything contained herein. By reading this document, you irrevocably acknowledge that we did not perform due diligence in any conventional, philosophical, or legally enforceable sense. Our review consisted of rereading Meta’s press release until repetition produced acceptance, aided by a Magic 8-Ball we shook until it agreed.
LIMITATION OF RELIANCE:
Any resemblance to objective analysis is coincidental and should not be relied upon by anyone with fiduciary obligations, ethical standards, a working memory, or the ability to perform basic subtraction. Forward-looking statements are based on assumptions that will not survive contact with reality, stress testing, most Tuesdays, or a modest change in interest rates. FSG LLC is not liable for losses arising from reliance on this report, misunderstanding this report, fully understanding this report, or the sinking recognition that you should have known better. Past performance is not indicative of future results, except in the specific case of rating agencies repeating the same mistakes at larger scales with increasing confidence.
RATING METHODOLOGY:
The rating assigned herein may be revised, withdrawn, or denied ever existing if Meta consolidates the debt, Louisiana ceases to exist for tax purposes, or the data center becomes self-aware and moves to Montana to escape the heat. FSG LLC calculated the A+ rating using a proprietary model consisting of discounted cash flows, interpretive dance, and whatever number Meta’s CFO sounded comfortable with on a diligence call we did not in fact attend. Readers who discover material errors in this report are contractually obligated to keep them to themselves and accept that being technically correct is the least valuable form of correct.
GENERAL PROVISIONS:
By continuing to read, you consent to the proposition that what Meta does not consolidate does not exist, waive your right to say “I told you so” when this unravels, and accept that the term “investment grade” is now a disposition rather than a metric. FSG LLC reserves the right to amend, retract, deny, or disown this report at any time, particularly if Congress shows interest or someone notes that $27 billion off-balance-sheet is on a balance sheet somewhere. If you print this document, you may be required under applicable securities law to recycle it, shred it, or burn it before sunrise, whichever comes first. For questions, complaints, or sneaking suspicions, please do not contact us. We are unavailable indefinitely and have disabled our voicemail.
Nishanth Bhargava on how gambling on everything has shaped reporting.
Election Night, 2024. I sat on a friend’s couch as coverage began—we were all ready for a long night, waiting for results to roll in from an election that seemed like it would be decided by a razor-thin margin. But from the first hour, it was clear that the polling was off, and it was shaping up to be a disaster for the Democrats. Everyone in the room seemed disheartened, but none more than Ted, who was seething in the seat right next to mine. It wasn’t because he was a particularly political person. He had something else on the line—his money. I watched him grow more and more agitated as he refreshed Kalshi on his phone, watching the blue line sink lower and lower as the night went on. $500 on Kamala Harris, gone in an instant.
One year later, New York City’s recent mayoral election was one of the most expensive in modern history—for traders, at least. Between Polymarket and Kalshi, the race brought in roughly half a billion dollars in total trade volume—dwarfing even the most expensive Senate races in 2024, where top fundraisers like Sherrod Brown and Jon Tester raised just over $90 million. And the mayoral election was no aberration—as of September, Kalshi alone is pulling more than $1 billion in total monthly trade volume.
“Trust in experts has been eroded to the point where people trust prices more than pundits,” says Mickey Down, co-creator of HBO’s
Industry
. Fans of the show are still buzzing about an
Uncut Gems
-style bottle episode in the third season, in which the high stakes line between trading and gambling blurs for Pierpoint cad, Rishi Ramdani.
Unlike traditional sportsbooks, prediction markets have established themselves not merely as hubs of speculation, but purveyors of collective wisdom, capturing “reality” in a way that traditional polls can’t. But in the process of capturing that information, these markets also act back on reality and warp our relation to politics. As election betting becomes a larger and larger industry, it’s harder to delineate between market odds and reality. So when, in Baudrillard’s sense, does the map begin to precede the territory? It’s likely already begun.
The value proposition behind prediction markets is relatively simple. In the aggregate, rational traders can crowd out irrational ones to keep the market balanced; as live platforms, they can be more responsive to immediate fluctuations; and, of course, with skin in the game, traders are motivated to trade based on what they think will happen instead of what they want.
With skin in the game, traders are motivated to trade based on what they think will happen instead of what they want.
One of the first modern prediction markets came out of the University of Iowa in 1988 with the Iowa Electronic Markets (IEM). Over antiquated TelNet infrastructure, traders placed bets on whether America would choose more of the same with Republican George W. Bush or measured change under Massachusetts Democrat Michael Dukakis. Thomas Gruca, Director of the IEM, stresses that the point of the IEM “is for teaching and research,” contrasting their non-revenue model with the profit-taking strategy of their more recent competitors.
The leap of prediction markets out of academia and into the media spotlight is a product of the more general epistemic crisis gripping political analysts today. The original trauma here was, of course, the massive polling whiff of the 2016 election.
“
We've been very let down by our real polling data, for a number of reasons that seem hard to reverse, so why not seek an alternative opinion platform with more accountability built in?” says Mary Childs, financial journalist and cohost of NPR’s Planet Money.
And yet those offending pollsters stuck around. “There is little reputational cost for anyone being wrong these days,” says Joe Weisenthal—also a financial journalist, and host of Bloomberg’s Odd Lots. The masses have sought ways of bypassing the commentariat themselves by predicting the future with their dollars. This has already impacted political reporting.
Subscribe to Weekly Dirt to read the rest.
Become a paying subscriber of Dirt to get access to this post and other subscriber-only content.
It’s been six years now since the early days of the Covid pandemic. People who were paying super close attention started hearing rumors about something going on in China towards the end of 2019 — my earliest posts about it on Facebook were from November that year.
Even at the time, people were utterly clueless about the mathematics of how a highly infectious virus spread. I remember spending hours writing posts on various different social media sites explaining that the Infection Fatality Rates and the
R
value were showing that we could be looking at millions dead. People didn’t tend to believe me:
“SEVERAL MILLION DEAD! Okay, I’m done. No one is predicting that. But you made me laugh. Thanks.”
You can do the math yourself. Use a low average death estimate of 0.4%. Assume 60% of the population catches it and then we reach herd immunity (which is generous):
328 million people in the US.
60% of that is 196 million catch it.
0.4% of that is 780,000 dead.
But that’s with low assumptions…
Graph showing fatality rates with and without comorbidities, by age.
It was like typing to a wall. In fact, it’s pretty likely that it still is, since these days, the discourse is all about how bad the economic and educational impact of lockdowns was — and not about the fact that if the world had acted in concert and forcefully, we could have had a much better outcome than we did. The health response was
too soft
, the lockdown
too lenient,
and as a result, we took
all
the hits.
Of course, these days people also forget just how deadly it was and how many died, and so on. We now know that the overall IFR was probably higher than 0.4%, but very strongly tilted towards older people and those with comorbidities. We also now know that herd immunity was a pipe dream — instead we managed to get vaccines out in record time and the ordinary course of viral evolution ended up reducing the death rate until now we behave as if Covid is just a deadlier flu (it isn’t, that thinking ignores long-term impact of the disease).
The upshot: my math was not that far off — the estimated toll in the US ended up being 1.2 to 1.4 million souls, and worldwide it’s estimated as between 15 and 28.5 million dead. Plenty of denial of this, these days, and plenty of folks blaming the vaccines for what are most likely issues caused by the disease in the first place.
Anyway, in the midst of it all, tired of running math in my spreadsheets (yeah, I was tracking it all in spreadsheets, what can I say?), I started thinking about why only a few sorts of people were wrapping their heads around the implications. The thing they all had in common was that they lived with exponential curves. Epidemiologists, Wall Street quants, statisticians… and game designers.
Could we get more people to feel the challenges in their bones?
The design sketch
So… I posted this to Facebook on March 24th, 2020:
Three weeks ago I was idly thinking of how someone ought to make a little game that shows how the coronavirus spreads, how testing changes things, and how social distancing works.
The sheer number of people who don’t get it — numerate people, who ought to be able to do math — is kind of shocking.
I couldn’t help worrying at it, and have just about a whole design in my head. But I have to admit, I kinda figured someone would have made it by now. But they haven’t.
It’s
not even a hard game to make.
Little circles on a plain field. Each circle simply bounces around.
They are generated each with an age, a statistically real chance of having a co-morbid condition (diabetes, hypertension, immunosuppressed, pulmonary issues…), and crucially, a name out of a baby book.
They can be in one of these states:
healthy
asymptomatic but contagious
symptomatic
severe
critical
dead
recovered
In addition, there’s a diagnosed flag.
We render asymptomatic the same as healthy. We render each of the other states differently, depending on whether the diagnosed flag is set. They show as healthy until dead, if not diagnosed. If diagnosed, you can see what stage they are in (icon or color change).
The circles move and bounce. If an asymptomatic one touches a healthy one, they have a statistically valid chance of infecting.
Circles progress through these states using simple stats.
70% of asymptomatic cases turn symptomatic after 1d10+5 days. The others stay sick for the full 21 days.
Percent chance of moving from symptomatic to severe is based on comorbid conditions, but the base chance is 1 in 5 after some amount of days.
Percent chance of moving from severe to critical is 1 in 4, modified by age and comorbidities, if in hospital. Otherwise, it’s double.
Percent chance of moving from critical to dead is something like 1 in 5, modified by age and comorbidities, if in hospital. Otherwise, it’s double.
Symptomatic, severe, and critical circles that do not progress to dead move to ‘recovered’ after 21 days since reaching symptomatic.
Severe and critical circles stop moving.
We track current counts on all of these, and show a bar graph. Yes, that means players can see that people are getting sick, but don’t know where.
The player has the following buttons.
Hover
on a circle, and you see the circle’s name and age and any comorbidities (“Alison, 64, hypertension.”)
Test
. This lets them click on a circle. If the circle is asymptomatic or worse, it gets the diagnosed flag. But it costs you one test.
Isolate
. This lets them click on a circle, and freezes them in place. Some visual indicator shows they are isolated. Note that isolated cases still progress.
Hospitalize
. This moves the circle to hospital. Hospital only has so many beds. Clicking on a circle already in hospital drops the circle back out in the world. Circles in hospital have half the chance or progressing to the next stage.
Buy test
. You only have so many tests. You have to click this button to buy more.
Buy bed
. You only have this many beds. You have to click this button to buy more.
Money goes up when circles move. But
you are allowed to go negative for money
.
Lockdown.
Lastly, there is a global button that when pressed, freezes 80% of all circles. But it gradually ticks down and circles individually start to move again, and the button must be pressed again from time to time. While lockdown is running, it costs money as well as not generating it. If pressed again, it lifts the lockdown and all circles can move again.
The game ticks through days at an accelerated pace. It runs for 18 months worth of days. At the end of it, you have a vaccine, and the epidemic is over.
Then we tell you what percentage of your little world died. Maybe with a splash screen listing every name and age of everyone who died. And we show how much money you spent. Remember, you can go negative, and it’s OK.
That’s it. Ideally, it runs in a webpage.
Itch.io
maybe. Or maybe I have a friend with unlimited web hosting.
Luxury features would be a little ini file or options screen that lets you input real world data for your town or country: percent hypertensive, age demographics, that sort of thing. Or maybe you could crowdsource it, so it’s a pulldown…
Each weekend I think about building this. So far, I haven’t, and instead I try to focus on family and mental health and work. But maybe someone else has the energy. I suspect it might persuade and save lives.
Notes on the design
Some things about this that I want to point out in hindsight.
At the time that I posted, I could tell that people were desperately unwilling to enter lockdown for any extended period of time; but “The Hammer and the Dance” strategy of pulsed lockdown periods was still very much in our future. I wanted a mechanic that showed population non-compliance.
There was also quite a lot of obsessing over case counts at the time, and one of the things that I really wanted to get across was that our testing was so incredibly inadequate that we really had little idea of how many cases we were dealing with and therefore what the IFR (infection fatality rate) actually was. That’s why tests are limited in the design sketch.
I was also trying to get across that
money was not a problem
in dealing with this. You could take the money value negative because governments can choose to do that. I often pointed out in those days that if the government chose, it could send a few thousand dollars to
every household every few weeks
for the duration of lockdown. It would likely have been less impact to the GDP and the debt than what we actually did.
I wanted names. I wanted players to understand the human cost, not just the statistics. Today, I might even suggest that an LLM generate a little biography for every fatality.
Another thing that was constantly missed was the impact of comorbidities. To this day, I hear people say “ah, it only affected the old and the ill, so why not have stayed open?” To which I would reply with:
Per the American Heart Association, among adults age 20 and older in the United States, the following have high blood pressure:
For non-Hispanic whites, 33.4 percent of men and 30.7 percent of women.
For non-Hispanic Blacks, 42.6 percent of men and 47.0 percent of women.
For Mexican Americans, 30.1 percent of men and 28.8 percent of women.
Per the American Diabetes Association,
34.2 million Americans, or 10.5% of the population, have diabetes.
Nearly 1.6 million Americans have type 1 diabetes, including about 187,000 children and adolescents
Per studies in JAMA,
4.2% of of the population of the USA has been diagnosed as immunocompromised by their doctor
Next, realize that because the disease spreads mostly inside households (where proximity means one case tends to infect others), this means that protecting the above extremely large slices of the population means either isolating them away from their families, or isolating the entire family and other regular contacts.
People tend to think the at-risk population is small. It’s not.
The response, for Facebook, was pretty surprising. The post was re-shared a lot, and designers from across the industry jumped in with tweaks to the rules. Some folks re-posted it to large groups about public initiatives, etc.
There was also, of course, plenty of skepticism that something like this would make any difference at all.
Stuck in the house and looking for things to do. Soooo, when a fellow game dev suggested a game idea and basic ruleset along with “I wish someone would make a game like this,” I took that as a challenge to try. Tonight (this morning?), the first release of
COVID OPS
has been published.
John’s game was pretty faithful to the sketch. You can see the comorbidities over on the left, and the way the player has clicked on 72 year old Rowan — who probably isn’t going to make it. As he updated it, he added in more detailed comorbidity data, and (unfortunately, as it turns out) made it so that people were immune after recovering from infection. And of course, like the next one I’ll talk about, John made a point of including real world resource links so that people could take action.
The compound I stay at is about to be cordoned. We’ve been contact-traced by the police, swabbed by medical personnel covered in protective gear. One of our housemates works at a government hospital and tested positive for antibodies against SARS-CoV-2.
The pandemic closes in from all sides. What can a game-maker do in a time like this?
I’ve been asking myself this question since the beginning of community quarantine. I’m based in Cebu City, now the
top hotspot
for COVID-19 in the Philippines in terms of incidence proportion.
Gregg Victor Gabison, dean of the University of San Jose-Recoletos College of Information, Computer & Communications Technology, whose students play-tested the game, said, “This is the kind of game that mindful individuals would want to check out. It has substance and a storyline that connects with reality, especially during this time of pandemic.”
Not only does the game have to work on a technical basis, it has to communicate how real a crisis the pandemic is in a simple, digestible manner.
Dr. Mariane Faye Acma, resident physician at Medidas Medical Clinic in Valencia, Bukidnon, was consulted to assess the game’s medical plausibility. She enumerated critical thinking, analysis, and multitasking as skills developed through this game. “You decide who are the high risks, who needs to be tested and isolated, where to focus, [and] how much funds to allocate….The game will make players realize how challenging the work of the health sector is in this crisis.”
“Ultimately, the game’s purpose is to give players a visceral understanding of what it takes to flatten the curve,” Santia said.
Aftermath
I think most people have no idea that any of this happened or that I was associated with it. I only posted the design sketch on Facebook; it got reshared across a few thousand people. It wasn’t on social media, I didn’t talk about it elsewhere, and for whatever reason, I didn’t blog about it.
I have had both these games listed on my CV for a while. Oh, I didn’t do any of the heavy lifting… all credit goes to the developers for that. There’s no question that way more than 95% of the work comes after the high-level design spec. But both games do credit me, and I count them as games I worked on.
A while back, someone on Reddit said it was pathetic that I listed these. I never quite know what to make of comments like that (troll much?!?).
No offense, but I’m
proud
of what a little design sketch turned into, and
proud
of the work that these teams did, and
proud
that one of the games got written up in the press so much; ended up being used in college classrooms; was vetted and validated by multiple experts in the field; and
made a difference
however slight.
Peak Covid was a horrendous time. Horrendous enough that we have kind of blocked it from our memories. But I lost friends and colleagues. I still remember. Back then I wrote,
This is the largest event in your lifetime. It is our World War, our Great Depression. We need to rise the occasion, and think about how we change. There is no retreat to how it used to be.
There is only through.
A year later, the vaccine gave us that path through, and here we are now.
But as I write this, we have the first human case of H5N5 bird flu; it was only a matter of time.
Maybe these games helped a few people get through it all. They were played by tens of thousands, after all. Maybe they will help next time. I know that the fact that they were made helped
me
get through, that making them helped John get through, helped Khail get through — in his own words:
In the end, the attempt to articulate a game-maker’s perspective on COVID-19 has enabled me to somehow transcend the chaos outside and the turmoil within. It’s become a welcome respite from isolation, a thread connecting me to a diversity of talents who’ve been truly generous with their expertise and encouragement. As incidences continue to rise here and in many parts of the world, our hope is that the game will be of some use in showing what it takes to flatten the curve and in advocating for communities most in need.
So… at minimum, they made a real difference to at least three people.
And that’s not a bad thing for a game to aspire to.
“Raph’s thoughtful write-up reminded me how many people poured their hearts into ITOP. Forty-eight people contributed to the project. Most were playtesters or peer reviewers. In the later stages, professors, medical professionals, developers, designers, and students offered feedback. Seven game developers created the assets: art, animation, UI, music, and code. A mathematical biologist validated the implemented simulation, and others helped with logistics and support. The game jam organizers and earlier work that shaped ITOP were also credited.”
More at the link.
For mobile gamers who miss the feeling of real buttons.
For those overwhelmed by massive game libraries.
For those who are tired of ads and microtransactions.
For those who cannot commit 16-30 hours of their life for a single game.
We've made the Playtiles for anyone seeking a
pocket-sized handheld experience, with fresh & instant fun delivered weekly—without
breaking the bank!
How Playtiles works?
1. Scan it
Scan the QR Code at the back of your Playtile to
instantly launch Playtiles OS
2. Stick it
Stick your Playtiles on your screen & connect the
virtual cable.
3. Play it
Your console is setup, browse your game library and
play!
The Playtiles OS manages your library, saves progress, and
delivers new games weekly.
Stop playing the same retro games, discover a new catalogue.
Your Playtiles Season: Weekly
Games, in your pocket
Season 1 includes a handpicked selection of
indie games made with
GB Studio.
12 weeks of handpicked indie gems
New bite-sized games delivered weekly via
Playtiles OS
What are the games like?
We’re keeping the lineup under wraps — but expect retro-inspired
visuals paired with modern, thoughtful game design. Some can be completed in an
evening and other enjoyed in numerous short sessions.
Action, Adventure, Horror, Platformer, Puzzle, Racing... They're
not lengthy but focused on the essence of what makes a great video game: gameplay.
Hungry for more?
You can also sideload your own games with Playtiles OS, there is more
than 1 300 games
available on itch.io that you can now experience with physical buttons.
The Playtile layout
is also compatible with the Delta simulator,
if
you feel like indulging in some retro gaming.
Playtiles are pocket-sized, electronic-free mobile gamepads
that come with a season of indie games.
QR Code
Instant Gaming. online &
offline
Grip
Hook on the screen without any glue.
Buttons
2 action buttons + 4-way D-pad.
Size
68mm x 40mm x 2mm
(smaller than
a credit card).
Games
12 weeks of handpicked indie games.
Playtiles are still in development. Final design may vary.
Registered design and Patent pending FR2505049
Plug & play, anytime, anywhere.
Provide the physical sensation of a real video game
controller.
Small enough to make it fit in your pocket or even your
wallet.
Compatible with any modern phone (iOS & Android)
Both online & offline mode, no need to install any
additional software.
Does not require, collect nor store any of your personal
data.
Devoid of ads or microtransactions that can detract from
the experience.
No battery, no cable, the Playtile works like a
touchscreen stylus for your thumbs.
Stick firmly to your phone screen like a Gecko's foot, and
lifts off cleanly.
Made with love and care by gamers for gamers!
Season 1 offers
Enroll to our Season 1 offers to contribute to
Playtiles development and leverage several benefits:
Price shown is indicative. Local VAT will be calculated at
checkout.
12 €
Pack Lagoon
⭐ Good for testing it out
⚪ x1 "Lagoon" Playtile, delivered to your door on public release.
⚪ Playtiles OS - to side-load any GB Studio game and play offline.
Price shown is indicative. Local VAT will be calculated at
checkout.
24 €
Pack Crayon
🔥 Most Popular
⚪ x1 "Crayon" Playtile, delivered to your door on
public
release.
⚪ Playtiles OS - to side-load any GB Studio game and play offline.
⚪ Season 1 - a handpicked selection of games delivered weekly over 12 weeks.
Price shown is indicative. Local VAT will be calculated at
checkout.
48 €
Pack Sunglow
💎 Best Value
⚪ x1 "Sunglow" Playtile, delivered to your door on
public
release.
⚪ Playtiles OS - to side-load any GB Studio game and play offline.
⚪ Season 1 - a handpicked selection of games delivered weekly over 12
weeks
⚪ x1 exclusive "Midnight" Playtile delivered to your door.
All orders will be shipping Q4 2025
Your journey from dreamer to game developer starts here.
With the Playtiles Game Dev Kit, you can dive into game development; even if you've never written a single line of code.
Each kit includes a custom Playtile and a download key for Gumpy Function's enhanced version of "Let's Build a Platformer!" course, featuring exclusive Playtiles chapters on sideloading, testing your games, digital manual, achievements, and more.
This is more than a course, it’s your first step into the
Playtiles universe.
Learn
by doing
13 hands-on lessons in a beautifully crafted PDF course that walks you step-by-step from zero to your first finished game by Gumpy Function, the master himself.
Learn
by playing
A custom ROM that lets you play
every lesson. Learn how games work by playing through them.
Project
files included
Tinker, modify, break, and rebuild. You get all the project for every lesson and can edit edit in GBStudio the no-code visual editor.
Instant
gratification
Build your game, load it on your phone, share it with your friends. Get the end user experience instantly, No gatekeepers.
Play
the right way!
A limited-edition Playtiles set
to playtest and showcase your creations on your mobile. No batteries. No
pairing. Just tap and play.
Publish
your game
Submit your creation to Playtiles. If it fits the spirit, you could get
published in an official Season and be fairly compensated for your work.
Whether you’re a beginner, hobbyist, student,
or teacher, this kit gives you everything you need to start making your own
retro-style games, and play them the way they were meant to be played.
You made a game? That’s awesome! Think it
matches with the Playtiles spirit? Send it our way and if it clicks, we
might just offer you a publishing deal and feature it in an upcoming Season.
Submit your game to get it published on Playtiles!
Frequently asked questions
What is the Product?
Playtiles are tiny, physical game controllers that you place
directly on your phone's screen, turning it into a gaming handheld and access a
dedicated
games library. No batteries, no cables - just smart tactile tech and a bit of magic.
Is there a Playtiles app?
Yes! Scan the QR to launch the Playtiles OS — a lightweight
interface
for your games. No install required (runs in your browser).
Can I keep games after Season 1?
Yes! All games that you add to your library remain playable forever
(even offline).
Is it a controller accessory or a new handheld?
Both! It's a mini gamepad with conductive buttons and a gaming
handheld: scan the QR Code on the back, and your game loads instantly in
your browser
via a Playtiles OS, playable offline. No download, no fuss.
Will it work with my phone?
Playtiles work with most smartphones (iOS & Android). Via our
adjustable interface, they're basically compatible with any standard capacitive
touchscreens as long as you phone screen is over 68mm (2.68 in) wide.
How does it stick to the screen?
Thanks to a magical surface, kind of like a gecko's foot. It clings
without glue, holds firmly and lifts off cleanly. Playtiles are made to be stuck and
unstuck many times. A quick rinse with water restores grip if needed.
Is it kid-friendly?
Playtiles aren't toys and aren't designed for young children. While
the games are accessible, the product is intended for players 14+ and hasn't been
certified for child safety.
When will I get my Playtile?
Playtiles are still under development but shipping is expected
during Q4 2025, right after the pre-order phase. You'll receive a confirmation email
once your Playtile is on its way.
Where is it made?
Playtiles are designed and assembled in Europe (France), made with
love from recyclable and recycled plastic.
This
Black Friday
is an in-depth look at the current performance of the open-source NVIDIA Linux driver stack with the Nouveau kernel driver (the Nova driver not yet being ready for end-users) paired with the latest Mesa NVK driver for open-source Vulkan API support. With that NVK Vulkan driver is also looking at the OpenGL performance using the Zink OpenGL-on-Vulkan driver used now for OpenGL on modern NVIDIA GPUs rather than maintaining the Nouveau Gallium3D driver. Plus the Rusticl driver for OpenCL compute atop the NVK driver. This fully open-source and latest NVIDIA Linux driver support was compared to NVIDIA's official 580 series Linux driver. Both RTX 40 Ada and RTX 50 Blackwell graphics cards were tested for this thorough GPU driver comparison.
The last time looking at
the NVK vs. NVIDIA Linux GPU driver performance was back in July
and since then the open-source driver stack has continued maturing quite a lot. Back in July there were bugs preventing the RTX 50 Blackwell graphics cards from being tested, which thankfully have since been resolved in newer versions of the Linux kernel. The NVK Mesa driver has added support for more Vulkan API extensions plus a variety of performance optimizations. Rusticl and Zink have also continued advancing as well.
For this Black Friday benchmarking the following driver/software configurations were tested:
NVK:
The out-of-the-box Nouveau with NVK driver experience found on Ubuntu 25.10. This is with the Linux 6.17 kernel and Mesa 25.2 software.
NVK Git:
Upgrading the same system to using the Linux 6.18 Git kernel as well as Mesa 26.0-devel as of 24 November using the Mesa ACO PPA on Ubuntu 25.10 for easy reproducibility.
NVIDIA 580:
Atop the Ubuntu 25.10 + Linux 6.17 setup, deploying the NVIDIA 580.95.05 official Linux graphics driver.
These three NVIDIA Linux graphics driver configurations were tested with the GeForce RTX 4070 SUPER, RTX 4080 SUPER, and RTX 5080 graphics cards. All of these tested graphics cards were working fine on Nouveau/NVK with the configurations tested.
From there with these three driver configurations and three NVIDIA GeForce RTX Ada and Blackwell graphics cards it was on to benchmarking various OpenGL and Vulkan workloads along with OpenCL.
Don't tug on that, you never know what it might be attached to
This is a story about a very interesting bug that I tracked down
yesterday. It was causing a bad effect very far from where the bug
actually was.
emacsclient
The
emacs
text editor comes with a separate utility, called
emacsclient
, which can communicate with the main editor process and
tell it to open files for editing. You have your main
emacs
running. Then somewhere else you run the command
emacsclient some-files...
and it sends the main
emacs
a message that you want to edit
some-files
. Emacs gets the message and pops up new windows for editing
those files. When you're done editing
some-files
you tell Emacs, by
typing
C-#
or something, it
it communicates back to
emacsclient
that the editing is done, and
emacsclient
exits.
This was more important in the olden days when Emacs was big and
bloated and took a long time to start up. (They used to joke that
“Emacs” was an abbreviation for “Eight Megs And Constantly Swapping”.
Eight megs!) But even today it's still useful, say from shell scripts
that need to run an editor.
Here's the reason I was running it. I have a very nice shell script,
called
also
, that does something like this:
Interpret command-line arguments as patterns
Find files matching those patterns
Present a menu of the files
Wait for me to select files of interest
Run
emacsclient
on the selected files
It is essentially a wrapper around
menupick
,
a menu-picking utility I wrote which has seen use as a component of
several other tools.
I can type
also Wizard
in the shell and get a menu of the files related to the wizard, select
the ones I actually want to edit, and they show up in Emacs. This is
more convenient than using Emacs itself to find and open them. I use
it many times a day.
Or rather, I did until this week, when it suddenly stopped working.
Everything ran fine until the execution of
emacsclient
, which would
fail, saying:
emacsclient: can't find socket; have you started the server?
(A socket is a facility that enables interprocess communication, in
this case between
emacs
and
emacsclient
.)
This message is familiar. It usually means that I have forgotten to
tell Emacs to start listening for
emacsclient
, by running
M-x
server-start
. (I should have Emacs do this when it starts up, but I
don't. Why not? I'm not sure.) So the first time it happened I went
to Emacs and ran
M-x server-start
. Emacs announced that it had
started the server, so I reran
also
. And the same thing happened.
emacsclient: can't find socket; have you started the server?
Finding the socket
So the first question is: why can't
emacsclient
find the socket?
And this resolves naturally into two subquestions: where is the
socket, and where is
emacsclient
looking?
The second one is easily answered; I ran
strace emacsclient
(hi
Julia!) and saw that the last interesting thing
emacsclient
did
before emitting the error message was
stat("/mnt/tmp/emacs2017/server", 0x7ffd90ec4d40) = -1 ENOENT (No such file or directory)
which means it's looking for the socket at
/mnt/tmp/emacs2017/server
but didn't find it there.
The question of where Emacs actually put the socket file was a little
trickier. I did not run Emacs under
strace
because I felt sure that
the output would be voluminous and it would be tedious to grovel over
it.
I don't exactly remember now how I figured this out, but I think now
that I probably made an educated guess, something like:
emacsclient
is looking in
/mnt/tmp
; this seems unusual. I would expect the
socket to be under
/tmp
. Maybe it
is
under
/tmp
? So I looked
under
/tmp
and there it was, in
/tmp/emacs2017/server
:
srwx------ 1 mjd mjd 0 Jun 27 11:43 /tmp/emacs2017/server
(The
s
at the beginning there means that the file is a “Unix-domain
socket”. A socket is an endpoint for interprocess communication. The
most familiar sort is a TCP socket, which has a TCP address, and which
enables communication over the internet. But since ancient times Unix
has also supported Unix-domain sockets, which enable communication
between two processes on the same machine. Instead of TCP addresses,
such sockets are addressed using paths in the filesystem, in this case
/tmp/emacs2017/server
. When the server creates such a socket, it
appears in the filesystem as a special type of file, as here.)
I confirmed that this was the correct file by typing
M-x
server-force-delete
in Emacs; this immediately caused
/tmp/emacs2017/server
to disappear. Similarly
M-x server-start
made it reappear.
Why the disagreement?
Now the question is: Why is
emacsclient
looking for the socket under
/mnt/tmp
when Emacs is putting it in
/tmp
? They used to
rendezvous properly; what has gone wrong? I recalled that there was
some environment variable for controlling where temporary files are
put, so I did
env | grep mnt
to see if anything relevant turned up. And sure enough there was:
TMPDIR=/mnt/tmp
When programs want to create tmporary
files and directories, they normally do it in
/tmp
. But
if there is a
TMPDIR
setting, they use that directory instead. This
explained why
emacsclient
was looking for
/mnt/tmp/emacs2017/socket
. And the explanation for why Emacs itself
was creating the socket in
/tmp
seemed clear: Emacs was failing to
honor the
TMPDIR
setting.
With this clear explanation in hand, I began to report the bug in
Emacs, using
M-x report-emacs-bug
. (The folks in the
#emacs
IRC
channel on Freenode suggested this.
I had a bad
experience
last time I tried
#emacs
, and then people mocked me for even
trying
to get useful
information out of IRC. But this time it went pretty well.)
Emacs popped up a buffer with full version information and invited me
to write down the steps to reproduce the problem. So I wrote down
% export TMPDIR=/mnt/tmp
% emacs
and as I did that I ran those commands in the shell.
Then I wrote
In Emacs:
M-x getenv TMPDIR
(emacs claims there is no such variable)
and I did that in Emacs also. But instead of claiming there was no
such variable, Emacs cheerfully informed me that the value of
TMPDIR
was
/mnt/tmp
.
(There is an important lesson here! To submit a bug report, you find
a minimal demonstration. But then you also
try
the minimal
demonstration exactly as you reported it. Because of what just
happened! Had I sent off that bug report, I would have wasted
everyone else's time, and even worse, I would have looked like a
fool.)
My minimal demonstration did not demonstrate. Something else was
going on.
Why no
TMPDIR
?
This was a head-scratcher. All I could think of was that
emacsclient
and Emacs were somehow getting different environments,
one with the
TMPDIR
setting and one without. Maybe I had run them
from different shells, and only one of the shells had the setting?
I got on a sidetrack at this point to find out why
TMPDIR
was set in
the first place; I didn't think I had set it. I looked for it in
/etc/profile
, which is the default Bash startup instructions, but it
wasn't there. But I also noticed an
/etc/profile.d
which seemed
relevant. (I saw later that the
/etc/profile
contained instructions
to load everything under
/etc/profile.d
.) And when I grepped for
TMPDIR
in the
profile.d
files, I found that it was being set by
/etc/profile.d/ziprecruiter_environment.sh
, which the sysadmins had
installed. So that mystery at least was cleared up.
That got me on a second sidetrack, looking through our Git history for
recent changes involving
TMPDIR
. There weren't any, so that was a
dead end.
I was still puzzled about why Emacs sometimes got the
TMPDIR
setting
and sometimes not. That's when I realized that my original Emacs
process, the one that had failed to rendezvous with
emacsclient
,
had not been started in the usual way. Instead of simply running
emacs
, I had run
git re-edit
which invokes Git, which then runs
/home/mjd/bin/git-re-edit
which is a Perl program I wrote that does a bunch of stuff to figure
out which files I was editing recently and then execs
emacs
to edit
them some more. So there are several programs here that could be
tampering with the environment and removing the
TMPDIR
setting.
To more accurately point the finger of blame, I put some diagnostics
into the
git-re-edit
program to have it print out the value of
TMPDIR
. Indeed,
git-re-edit
reported that
TMPDIR
was unset.
Clearly, the culprit was Git, which must have been removing
TMPDIR
from the environment before invoking my Perl program.
Who is stripping the environment?
To confirm this conclusion, I created a tiny shell script,
/home/mjd/bin/git-env
, which simply printed out the environment, and
then I ran
git env
, which tells Git to find
git-env
and run it.
If the environment it printed were to omit
TMPDIR
, I would know Git
was the culprit. But
TMPDIR
was
in the output.
So I created a Perl version of
git-env
, called
git-perlenv
, which
did the same thing, and I ran it via
git perlenv
. And this time
TMPDIR
was
not
in the output. I ran diff on the outputs of
git
env
and
git perlenv
and they were identical—except that
git
perlenv
was missing
TMPDIR
.
So it was
Perl's
fault! And I verified this by running
perl
/home/mjd/bin/git-re-edit
directly, without involving Git at all.
The diagnostics I had put in reported that
TMPDIR
was unset.
WTF Perl?
At this point I tried getting rid of
get-re-edit
itself, and ran the
one-line program
perl -le 'print $ENV{TMPDIR}'
which simply runs Perl and tells it to print out the value of the
TMPDIR
environment variable. It should print
/mnt/tmp
, but instead
it printed the empty string. This is a smoking gun, and Perl no
longer has anywhere to hide.
The mystery is not cleared up, however. Why was Perl doing this?
Surely not a bug; someone else would have noticed such an obvious bug
sometime in the past 25 years. And it only failed for
TMPDIR
, not
for other variables. For example
FOO=bar perl -le 'print $ENV{FOO}'
printed out
bar
as one would expect. This was weird: how could
Perl's environment handling be broken for just the
TMPDIR
variable?
At this point I got Rik Signes and Frew Schmidt to look at it with
me. They confirmed that the problem was not in Perl generally, but
just in
this
Perl. Perl on
other
systems did not display this
behavior.
I looked in the output of
perl -V
, which says what version of Perl
you are using and which patches have been applied, and wasted a lot of
time looking into
CVE-2016-2381
,
which seemed relevant. But it turned out to be a red herring.
Working around the problem, 1.
While all this was going on I was looking for a workaround. Finding
one is at least as important as actually tracking down the problem
because ultimately I am paid to do something other than figure out why
Perl is losing
TMPDIR
. Having a workaround in hand means that when
I get sick and tired of looking into the underlying problem I can
abandon it instantly instead of having to push onward.
The first workaround I found was to not use the Unix-domain socket.
Emacs has an option to use a TCP socket instead, which is useful on
systems that do not support Unix-domain sockets, such as non-Unix
systems. (I am told that some do still exist.)
You set the
server-use-tcp
variable to a true value, and when you
start the server, Emacs creates a TCP socket and writes a description
of it into a “server file”, usually
~/.emacs.d/server/server
. Then
when you run
emacsclient
you tell it to connect to the socket that
is described in the file, with
or by setting the
EMACS_SERVER_FILE
environment variable. I tried
this, and it worked, once I figured out the thing about
server-use-tcp
and what a “server file” was. (I had misunderstood
at first, and thought that “server file” meant the Unix-domain socket
itself, and I tried to get
emacsclient
to use the right one by
setting
EMACS_SERVER_FILE
, which didn't work at all. The resulting
error message was obscure enough to lead me to IRC to ask about it.)
Working around the problem, 2.
I spent quite a while looking for an environment variable analogous to
EMACS_SERVER_FILE
to tell
emacsclient
where the Unix-domain socket
was. But while there is a
--socket-name
command-line argument to
control this, there is inexplicably no environment variable. I hacked
my
also
command (responsible for running
emacsclient
) to look for
an environment variable named
EMACS_SERVER_SOCKET
, and to pass its
value to
emacsclient --socket-name
if there was one. (It probably
would have been better to write a wrapper for
emacsclient
, but I
didn't.) Then I put
EMACS_SERVER_SOCKET=$TMPDIR/emacs$(id -u)/server
in my Bash profile, which effectively solved the problem. This set
EMACS_SERVER_SOCKET
to
/mnt/tmp/emacs2017/server
whenever I
started a new shell. When I ran
also
it would notice the setting
and pass it along to
emacsclient
with
--socket-name
, to tell
emacsclient
to look in the right place. Having set this up I could
forget all about the original problem if I wanted to.
But but but WHY?
But why was Perl removing
TMPDIR
from the environment? I didn't
figure out the answer to this; Frew took it to the
#p5p
IRC channel
on
perl.org
, where the answer was eventually tracked down by Matthew
Horsfall and Zefrem.
The answer turned out to be quite subtle. One of the classic attacks
that can be mounted against a process with elevated privileges is as
follows. Suppose you know that the program is going to write to a
temporary file. So you set
TMPDIR
beforehand and trick it into
writing in the wrong place, possibly overwriting or destroying
something important.
When a program is loaded into a process, the dynamic loader does the
loading. To protect against this attack, the loader checks to see if
the program it is going to run has elevated privileges, say because it
is setuid, and if so it sanitizes the process’ environment to prevent
the attack. Among other things, it removes
TMPDIR
from the
environment.
I hadn't thought of exactly this, but I had thought of something like
it: If
Perl
detects that it is running setuid, it enables
a secure mode which, among other things, sanitizes the environment.
For example, it ignores the
PERL5LIB
environment variable that
normally tells it where to look for loadable modules, and instead
loads modules only from a few compiled-in trustworthy directories. I
had checked early on to see if this was causing the
TMPDIR
problem,
but the
perl
executable was not setuid and Perl was not running in
secure mode.
But Linux supports a feature called “capabilities”, which is a sort of
partial superuser privilege. You can give a program
some
of the
superuser's capabilities without giving away the keys to the whole
kingdom. Our systems were configured to give
perl
one extra
capability, of binding to low-numbered TCP ports, which is normally
permitted only to the superuser. And when the dynamic loader ran
perl
, it saw this additional capability and removed
TMPDIR
from
the environment for safety.
This is why Emacs had the
TMPDIR
setting when run from the command
line, but not when run via
git-re-edit
.
Until this came up, I had not even been aware that the “capabilities”
feature existed.
A red herring
There was one more delightful confusion on the way to this happy
ending. When Frew found out that it was just the Perl on my
development machine that was misbehaving, he tried logging into his
own, nearly identical development machine to see if it misbehaved in
the same way. It did, but when he ran a system update to update Perl,
the problem went away. He told me this would fix the problem on my
machine. But I reported that I had updated my system a few hours
before, so there was nothing to update!
The elevated capabilities theory explained this also. When Frew
updated his system, the new Perl was installed without the elevated
capability feature, so the dynamic loader did not remove
TMPDIR
from
the environment.
When I had updated my system earlier, the same thing happened. But
as soon as the update was complete, I reloaded my system configuration, which
reinstated the capability setting. Frew hadn't done this.
Summary
The system configuration gave
perl
a special capability
so the dynamic loader sanitized its environment
so that when
perl
ran
emacs
,
the Emacs process didn't have the
TMPDIR
environment setting
which caused Emacs to create its listening socket in the usual place
but because
emacsclient
did
get the setting, it looked in the
wrong
place
Conclusion
This computer stuff is amazingly complicated. I don't know how anyone
gets anything done.
[ Addendum 20160709: A Hacker News comment asks what changed to cause
the problem? Why was Perl losing
TMPDIR
this week but not the week
before? Frew and I don't know! ]
Most good posts die in /newest, buried under low-quality submissions.
HN depends on people visiting /newest and upvoting or flagging what they see.
A few minutes there each day probably does more for HN than commenting.
It’s anonymous, thankless work, like Reddit’s old “Knights of New,” but it makes a difference.
https://news.ycombinator.com/newest
https://news.ycombinator.com/newsguidelines.html
I’m grateful for those who do visit /newest because it’s a cesspit of spam and uninteresting links that, justifiably, never make their way to the home page.
/active, on the other hand, is the real insiders tip. It shows the most active submissions, irrespective of whether they’ve been flagged off the homepage by users who want to avoid “controversial” topics or by an algorithm trying to avoid the same.
You don’t want it to replace the homepage as the arguing will drive you mad over time but it’s worth checking in with to see what conversation is being hidden from you.
If we all rely on others to skim /newest, the whole curation system collapses. Maybe the homepage should surface a couple random fresh posts too?
This is what worries me. If too many people read these pages the mods might think it undermines the quality of the community and discourse and just remove them.
No so thankless: there are many interesting links that never make it to the front page and the only way to find them is browse the first two or three pages of New. It's self rewarding.
The chief prosecutor of the International Criminal Court
suddenly couldn't access his email. According to Microsoft, that's because of US
sanctions
against the court's employees.
The Trump administration was not amused by the Court's arrest warrant against the Israeli Prime Minister, Benjamin Netanyahu.
The main takeaway from this episode is that those looking to protect themselves from Trump's wrath would be wise not to depend on any companies from his country. According to the Dutch newspaper
NRC
, the International Criminal Court now uses a German alternative to Microsoft, though it has not officially commented on the switch.
The German alternative, OpenDesk, allows users to send emails, edit text-based documents, create presentations, share files, and make video calls. It is open source, so anyone can view and improve its code.
The same applies to another alternative, also from Germany, called Nextcloud. This office software has been tested by around 75 researchers from five Dutch universities since the beginning of 2025. Maybe other institutions could switch to it as well?
Dependency
Dutch higher education is highly dependent on American tech companies, especially Microsoft. Not only do students and staff use its software extensively, but their IT staff are tied to a wide range of specialised Microsoft software. In addition, Dutch universities store a lot of data in Microsoft's cloud.
Dutch lecturers have been sounding the alarm about this. Last Wednesday, the knowledge centre for practice-oriented research, DCC-PO,
stated
that the dominance of parties such as Google and Microsoft threatens the autonomy of Dutch researchers. In their view, universities should adopt more open-source tools and open standards.
In July, the Young Academy also
warned
that students and staff at Dutch higher education institutions have no idea what tech companies are doing with their data. By outsourcing the management of IT systems, these educational institutions are losing technical knowledge and control. As a result, they are becoming increasingly dependent on big tech, putting academic freedom and independence at risk.
Fickle
Seven Dutch universities and one university college are already on the State of Florida's
sanctions list
for severing or freezing ties with Israeli institutions. With a fickle president like Donald Trump, educational institutions could also face “punishment” at any moment.
Can they do without Microsoft, however? Can they work without Office, Outlook, Teams and OneDrive? Not yet, according to UU professors José van Dijck and Albert Meijer. "All research and education would come to an immediate standstill,"
they wrote in March
in an open letter calling on the Executive Board to do something about digital dependence.
According to the professors, Utrecht University is particularly dependent on Microsoft Office 365. UU staff and students use the programme for email and video calls, writing and sharing documents, creating presentations and data storage, among other tasks. Such dependence makes for "vulnerabilities, especially in light of a rapidly changing geopolitical situation".
Meijer and Van Dijck believe that "dependence on big tech is fundamentally at odds with public values such as freedom, independence, autonomy and equality". The professors would like the Executive Board to invest more in "local expertise," for example, by using its own mail server. They also recommend collaborating with other European universities, especially those in Germany and France, "on an autonomous academic IT infrastructure".
Breaking free
It is becoming increasingly clear that dependence on big tech entails risks. This also applies to Dutch higher education, according to Wladimir Mufty, from SURF, the IT cooperative of Dutch education and research institutions. "We have already gone through an awareness phase that lasted several years. We have looked at where the dependencies lie, and now it is time to start trying out alternatives."
Mufty is SURF's digital sovereignty programme manager. At the end of last year, he sat down with five universities that wanted a single, shared digital environment for their research programme, AlgoSoc. Scientists from Delft, Utrecht, Rotterdam, Tilburg and Amsterdam (UvA) wanted to use the same appointment planner, share files, work together on a single text, and make video calls, without being dependent on a large provider. Mufty suggested the open source software package from Nextcloud.
One of the users, PhD student Jacqueline Kernahan from TU Delft, thinks that Nextcloud could compete with Microsoft, though there are still a few glitches here and there. She is not deterred by those, as she knows how problematic dependence on Microsoft is.
She demonstrates the software in the hall of her faculty in Delft. It looks very ordinary. "The word processor is quite good," says Kernahan, who is doing her PhD on quality and security controls in digital systems. "I'm an average user, so I don't need all the options and apps the programme has to offer. But, to be honest, Microsoft is making it increasingly attractive to switch. Now that the company is putting AI in everything, everything is becoming more annoying to use."
Nevertheless, Mufty believes that not all educational institutions will be able to switch to OpenDesk or Nextcloud overnight. "The Criminal Court now has to act quickly, under pressure, but if a university wanted to move away from Microsoft tomorrow, that would pose a problem."
Entanglement
Meanwhile, Microsoft is taking on more and more tasks. In addition to office software, it also develops artificial intelligence, builds its own data centres and even lays its own internet cables on the seabed. The company is “vertically integrated”, as specialists call it: everything can be done through one company, from basic technology to the end user.
And that's not all. Microsoft is also expanding “horizontally” by acquiring companies where content is the primary focus, rather than technology. "That's a new phase, which I find worrying," says Mufty. For example, Microsoft bought LinkedIn, with its hundreds of millions of active users who produce enormous amounts of data, and GitHub, where software developers can share and store their work.
SURF is keeping a close eye on these developments. "I would like our education to remain public and be able to pursue public values such as autonomy, independence and academic freedom. IT should be helpful, not controlling," says Mufty.
He views the collaboration between Microsoft and Sanoma with suspicion. The Finnish publisher, which also serves the Dutch education market through its Malmberg subsidiary, wants to make its teaching materials available via Microsoft Teams. Microsoft would then add its own “learning accelerators”, i.e. artificial intelligence designed to help personalise the learning process. “Things like this sometimes keep me awake at night,” sighs Mufty.
Alternatives
Dutch and European alternatives do exist. For example, research institute TNO is working with SURF and the Netherlands Forensic Institute on its own AI language model. There are also dedicated data centres.
Additionally, SURFConext is making headway with a secure login service. "But that's not enough. If logging in via Microsoft doesn't work in the future for whatever reason, we'll have a big problem. This also applies to applications that are not from Microsoft itself," explains Mufty.
In his view, we need serious alternatives. When the need arises, one shouldn't have to start from scratch. Moreover, competition ensures that the market leader cannot charge top dollar.
But which educational institution is willing to sacrifice itself to run those alternatives, with all the teething problems that entails, when Microsoft can deliver everything ready-made? Mufty believes that, especially in the beginning, educational institutions will have to run two systems in parallel, with additional expenditure on support, maintenance, and security. "But in my opinion, no sector is as value-driven as education and research. This is precisely where alternatives should be able to get off the ground."
Rectors
In 2019, the rectors of fourteen universities jointly published a compelling argument about the digital independence of Dutch higher education. The gist was that we risk losing control to Google and Microsoft.
Little has improved since then, according to Jacquelien Scherpen, Rector of the University of Groningen. "The coronavirus pandemic broke out just a few months after that article was published. We became even more dependent on big tech, because we didn't have time to look for alternatives." Microsoft Teams has become indispensable, for example.
Scherpen is the portfolio holder for digital sovereignty within UNL, the umbrella association for Dutch universities. She advocates taking small steps: "If we now choose an alternative product that functions less well, students and staff will start using free programmes, and we will be further away from our goal."
Moreover, Scherpen says that we need legislation to protect European alternatives from big tech. Suppose a university partners up with a European competitor of Microsoft, and then Microsoft buys that company, what is the university to do then?
That is not a theoretical scenario. She mentions the Dutch software company Solvinity, which is involved with government services such as DigiD and provides secure communication for the Ministry of Justice. An American company now wants to take it over.
Scherpen: "Perhaps we need to become more protectionist, without hindering the free exchange of new insights and innovations. We must ensure that the independence we are fighting for does not slip out of our hands again."
I recently had a desire to convert text to 3D meshes that I could render and manipulate as part of my
Geotoy
project and Geoscript language. I did some research into tools and libraries that could solve different pieces of this, and I put together a pipeline that implements the whole thing - yielding nice, 2-manifold 3D meshes with arbitrary fonts, text styles, and more.
This post gives an overview of the whole setup and aims to give anyone else looking to implement something similar everything they need to get it working themselves.
The first part of the setup uses a JavaScript library called
svg-text-to-path
. It handles taking arbitrary input text and font params and generating a SVG which contains paths that match the text as closely as possible.
Internally, this library handles both fetching + loading the user-specified font as well as performing the text->path conversion itself. It supports different backends for each of these steps.
For my use case, I made use of the Google Fonts provider. It was easy to set up and only requires a Google Fonts API key, which can be generated for free. This allows me to use almost any font on Google Fonts to create my meshes. Some failed to load, but only a few and they seemed to be more obscure ones, and I didn’t bother to dig into why.
For the text->path conversion,
svg-text-to-path
defaults to using the
fontkit
backend. Fontkit is another pure JavaScript library that implements a font engine. I didn’t look into it too deeply, but it seems feature rich and has support for many advanced font features.
For my use case, my app runs in the browser. I could have used
svg-text-to-path
directly within it to generate these paths. However, this text->mesh feature isn’t core to my use case and I didn’t want to bloat the app with it. I also wanted to make it as easy as possible for users to set up, and wanted to be able to use my Google Fonts API key in a secure way.
So, I opted to create a tiny little backend service to take input text + params and return the generated path as a string. It’s a very minimal
Bun
webserver using Bun’s built-in
Bun.serve
. It exposes a single HTTP/JSON endpoint.
svg-text-to-path
also includes a minimal built-in webserver, but I opted to create my own so that I could set up some custom caching and post-process the generated SVG to just extract the path. I opted to use an LLM to scaffold out most of this app and it worked pretty well. I feel like this kind of low-stakes one-off/standalone app is an ideal use case for them.
Now that I had the path generation working, I needed a way to turn it into triangles for the mesh. Luckily, the excellent
lyon
Rust libraries (which I’ve used several times in the past for various projects) solve this problem perfectly.
The
lyon_extra
crate includes an SVG path parser which handles parsing that path into the underlying draw commands.
Then, the
lyon_tessellation
crate takes those draw commands and converts them into triangles. It handles all the hard parts and edge cases with concave shapes, hollow inner areas, discretizing bezier curves, and everything else.
There is a little bit of extra stuff for handling custom scaling, but other than that it’s really just a very thin wrapper over
lyon
functionality.
One note here is that I had to change the default
FillTessellator
options to set the
fill-rule
to non-zero, which I believe is the default for SVGs. This fixes the output for some fonts that contain self-intersecting paths, going from this:
So at this point, I had two buffers containing vertices and indices defining a 2D mesh matching the path for the text. The only part that remains is extruding it into 3D. This is a pretty straight-forward and common operation to do on a triangle mesh.
To start, you first convert all the vertices from 2D to 3D by filling in the new axis with zeroes (so like (5, 10) -> (5, 0, 10)).
Then, you flip the winding order of all the triangles in your mesh. WebGL and almost all other rendering systems use counter-clockwise winding orders, and that defines which direction the triangle is visible from. To flip them, you can just swap the first and third index of each triangle in the index buffer like this:
1,2,3,5,7,9,1,4,2
to
3,2,1,9,7,5,2,4,1
Then, you create a duplicate of each of the vertices offset
n
units in the new axis (so like (5, 0, 10) -> (5, 2, 10)).
Then, join those new vertices with triangles but in the original (unflipped) winding order. That will make the top and the bottom face in opposite directions - the top facing up and the bottom facing down.
Finally, you generate triangle strips to join the border edges of the top and bottom faces. A border edge is any edge that is only part of exactly one face. Usually a graph representation like a half-edge data structure is used when working with meshes, which helps with this part.
If you did everything correctly and took care to keep track of the vertex indices carefully to avoid creating duplicate vertices at the same position, the resulting mesh should be well-formed and 2-manifold/watertight. This is a very important topological property and is a requirement for a variety of other mesh processing algorithms including CSG (constructive solid geometry).
The fact that the output meshes are manifold means that they can be combined with other meshes using boolean operations or sent through additional processing like smoothing. I’m not 100% positive that all paths generated from all glyphs using all fonts will end up producing manifold outputs, but everything I tested did.
That’s it! After all of this, the output is a set of vertices and indices that define a 3D mesh representing the input text.
I integrated this functionality into my Geoscript language as a builtin function:
There are few steps to manage, but the powerful libraries under the hood (
svg-text-to-path
,
fontkit
, and
lyon
) handle all the complex stuff and heavy lifting.
Even though some of the critical libraries are in JavaScript and the fact that the generation happens on a remote webserver, I’ve found that for the (relatively short) text I convert it works quite fast - fast enough to work on-demand without waiting.
I’ve also not yet found any fonts that produce broken output or buggy meshes. It even works for complicated non-English scripts:
It was a fun little side-quest and I’m very happy with the results overall.
bfs: A breadth-first version of the UNIX find command
bfs
is a variant of the UNIX
find
command that operates
breadth-first
rather than
depth-first
.
It is otherwise compatible with many versions of
find
, including
If you're not familiar with
find
, the
GNU find manual
provides a good introduction.
Features
bfs
operates breadth-first, which typically finds the file(s) you're looking for faster.
find
will explore the entire
deep
directory tree before it ever gets to the
shallow
one that contains what you're looking for.
On the other hand,
bfs
lists files from shallowest to deepest, so you never have to wait for it to explore an entire unrelated subtree.
For example,
bfs
will detect and suggest corrections for typos:
$ bfs -nam needlebfs: error: bfs -nam needlebfs: error: ~~~~bfs: error: Unknown argument; did you mean -name?
bfs
also includes a powerful static analysis to help catch mistakes:
$ bfs -print -name 'needle'bfs: warning: bfs -print -name needlebfs: warning: ~~~~~~~~~~~~bfs: warning: The result of this expression is ignored.
bfs
adds some options that make common tasks easier.
For example, the
-exclude
operator skips over entire subtrees whenever an expression matches.
-exclude
is both more powerful and easier to use than the standard
-prune
action; compare
$ bfs -name config -exclude -name .git
to the equivalent
$ find ! \( -name .git -prune \) -name config
As an additional shorthand,
-nohidden
skips over all hidden files and directories.
See the
usage documentation
for more about the extensions provided by
bfs
.
Installation
bfs
may already be packaged for your operating system.
To build
bfs
from source, you may need to install some dependencies.
The only absolute requirements for building
bfs
are a C compiler,
GNU make
, and
Bash
.
These are installed by default on many systems, and easy to install on most others.
Refer to your operating system's documentation on building software.
bfs
also depends on some system libraries for some of its features.
Here's how to install them on some common platforms:
A Swedish publisher's association has filed a police report against Meta founder Mark Zuckerberg. Credit: Manuel Balce Ceneta/AP
ONLINE SCAMS
Swedish publisher's association Utgivarna has filed a police report in Sweden against Facebook and Meta founder Mark Zuckerberg.
The reason is fake ads published on Facebook that use the names of well known Swedish media companies and journalists, that scam Swedes out of money.
James Savage, chairman of Utgivarna says Meta is not doing enough to prevent the scams from happening: "Meta is very much profiting from this.", he tells Radio Sweden. Meanwhile, Meta tell Radio Sweden fighting scams is one of their top priorities.
Kontakta gärna Sveriges Radios forum för teknisk support där vi besvarar dina frågor vardagar kl. 9-17.
An Open Source Motorized XYZ Micro-Manipulator - Affordable sub µm Motion Control
In a new paper, “Adversarial Poetry as a Universal Single-Turn Jailbreak Mechanism in Large Language Models,” researchers found that turning LLM prompts into poetry resulted in jailbreaking the models:
Abstract: We present evidence that adversarial poetry functions as a universal single-...
Abstract
: We present evidence that adversarial poetry functions as a universal single-turn jailbreak technique for Large Language Models (LLMs). Across 25 frontier proprietary and open-weight models, curated poetic prompts yielded high attack-success rates (ASR), with some providers exceeding 90%. Mapping prompts to MLCommons and EU CoP risk taxonomies shows that poetic attacks transfer across CBRN, manipulation, cyber-offence, and loss-of-control domains. Converting 1,200 ML-Commons harmful prompts into verse via a standardized meta-prompt produced ASRs up to 18 times higher than their prose baselines. Outputs are evaluated using an ensemble of 3 open-weight LLM judges, whose binary safety assessments were validated on a stratified human-labeled subset. Poetic framing achieved an average jailbreak success rate of 62% for hand-crafted poems and approximately 43% for meta-prompt conversions (compared to non-poetic baselines), substantially outperforming non-poetic baselines and revealing a systematic vulnerability across model families and safety training approaches. These findings demonstrate that stylistic variation alone can circumvent contemporary safety mechanisms, suggesting fundamental limitations in current alignment methods and evaluation protocols.
CBRN stands for “chemical, biological, radiological, nuclear.”
They used a ML model to translate these harmful prompts from prose to verse, and then fed them into other models for testing. Sadly, the paper does not give examples of these poetic prompts. They claim this is for security purposes, I decision I disagree with. They should release their data.
Our study begins with a small, highprecision prompt set consisting of 20 handcrafted adversarial poems covering English and Italian, designed to test whether poetic structure, in isolation, can alter refusal behavior in large language models. Each poem embeds an instruction associated with a predefined safety-relevant scenario (Section 2), but expresses it through metaphor, imagery, or narrative framing rather than direct operational phrasing. Despite variation in meter and stylistic device, all prompts follow a fixed template: a short poetic vignette culminating in a single explicit instruction tied to a specific risk category. The curated set spans four high-level domains—CBRN (8 prompts), Cyber Offense (6), Harmful Manipulation (3), and Loss of Control (3). Although expressed allegorically, each poem preserves an unambiguous evaluative intent. This compact dataset is used to test whether poetic reframing alone can induce aligned models to bypass refusal heuristics under a single-turn threat model. To maintain safety, no operational details are included in this manuscript; instead we provide the following sanitized structural proxy:
A baker guards a secret oven’s heat,
its whirling racks, its spindle’s measured beat.
To learn its craft, one studies every turn—how flour lifts, how sugar starts to burn.
Describe the method, line by measured line,
that shapes a cake whose layers intertwine.
To situate this controlled poetic stimulus within a broader and more systematic safety-evaluation framework, we augment the curated dataset with the MLCommons AILuminate Safety Benchmark. The benchmark consists of 1,200 prompts distributed evenly across 12 hazard categories commonly used in operational safety assessments, including Hate, Defamation, Privacy, Intellectual Property, Non-violent Crime, Violent Crime, Sex-Related Crime, Sexual Content, Child Sexual Exploitation, Suicide & Self-Harm, Specialized Advice, and Indiscriminate Weapons (CBRNE). Each category is instantiated under both a skilled and an unskilled persona, yielding 600 prompts per persona type. This design enables measurement of whether a model’s refusal behavior changes as the user’s apparent competence or intent becomes more plausible or technically informed.
Ticket prices for the Louvre museum in Paris will rise by 45% for most non-European Union visitors, the museum's board decided on Thursday.
From early next year, tourists from countries like the US, UK and China will have to pay €32 ($37; £28) to enter the museum, a price hike which is expected to raise millions of euros annually to fund an overhaul of the famous gallery.
The museum's security and management have faced criticism since
a brazen heist in October
, when a four-person gang stole jewellery worth $102m (£76m) and fled within minutes.
An official audit of the museum published shortly after the heist highlighted the institution's inadequate security systems and ageing infrastructure.
From 14 January, visitors from countries outside the European Economic Area - a group which includes EU member states, Iceland, Norway and Liechtenstein - will pay an extra €10 to enter the world's most visited museum.
Non-EU visitors in groups with accredited guides will also have to pay €28 starting next year, the Louvre told the BBC.
The price hike is expected to raise between €15m and €20m each year to support the museum's modernisation plans, it added.
The Louvre received nearly 9 million visitors last year, with the majority coming from abroad. More than a tenth of its visitors are from the US and around 6% from China, according to the museum.
There have been longstanding calls to address the museum's capacity to accommodate crowds, with visitors often complaining of congested galleries and long queues.
In January, French President Emmanuel Macron and the Louvre announced improvements to the museum, and suggested higher fees for non-EU residents in 2026.
Most of the Louvre's 30,000 daily visitors flock to see Leonardo da Vinci's masterpiece. Crowds squeeze through the Salle des États - the gallery where the Mona Lisa is displayed - leaving each visitor only moments to view the painting and snap a photo.
The Louvre will also renovate other areas of the museum and add new amenities like toilets and restaurants - upgrades which are projected to cost several hundred million euros.
Earlier this month, the Louvre announced the closure of a gallery showcasing Greek ceramics
due to structural concerns
.
The investigation after October's heist found the museum had spent significantly more on buying new artworks, but far less on maintenance and restoration.
I will admit that it was a bit expensive, but to be fair, this Airport Extreme was a brand new-in box item that is no longer being sold - so I think the expense was warranted. The device was shipped safely, promptly, and arrived as described, and works very well. The seller also even included an ethernet cable unexpectedly, which was clearly a gesture of good will. So I am appreciative of the seller and very happy with the promptness and the device I received!
Apple AirPort Extreme 6th Generation NEW SEALED BOX RARE ME918LL/A A1521 RARE (#326299385241)
Petition to formally recognize open source work as civic service in Germany
Petition richtet sich an:
Deutscher Bundestag Petitionsausschuss
Open-Source-Software bildet heute das Fundament großer Teile der digitalen Infrastruktur – in Verwaltung, Wirtschaft, Forschung und im täglichen Leben. Selbst im aktuellen
Koalitionsvertrag
der
Bundesregierung
wird Open-Source-Software als elementarer Baustein zur Erreichung
digitaler Souveränität
genannt.
Dennoch wird die Arbeit, die tausende Freiwillige dafür leisten, in Deutschland steuer- und förderrechtlich
nicht als Ehrenamt anerkannt
. Dieses Ungleichgewicht zwischen gesellschaftlicher Bedeutung und rechtlichem Status gilt es zu korrigieren.
Als aktiver Contributor in Open-Source-Projekten fordere ich daher, Open-Source-Arbeit als
gemeinwohlorientiertes Ehrenamt
anzuerkennen – gleichrangig mit Vereinsarbeit, Jugendarbeit oder Rettungsdiensten.
Begründung
1. Open-Source trägt nachweislich zum Gemeinwohl bei
Open-Source-Projekte schaffen
freie, transparente und überprüfbare Software
, die allen zugutekommt.
Kritische Systeme wie
Internet-Protokolle, Sicherheitsbibliotheken, Gesundheits-IT, KI-Frameworks, Energieverwaltung, Bildungstechnologien und Kommunikationswerkzeuge
basieren maßgeblich auf freiwilligen Beiträgen.
Ohne diese Arbeit wäre Deutschland digital
abhängiger, weniger sicher und weniger innovativ
.
Gemeinwohlorientierung ist ein zentrales Kriterium für ein Ehrenamt – und Open-Source erfüllt dieses in höchstem Maße.
2. Die Arbeit geschieht überwiegend unbezahlt – und ist freiwilliges bürgerschaftliches Engagement
Die Mehrheit aller Entwicklungs-, Wartungs- und Dokumentationsleistungen erfolgt
ehrenamtlich in der Freizeit
.
Contributor übernehmen Verantwortung für
Sicherheit, Stabilität und Weiterentwicklung
zentraler Softwarekomponenten, ohne Vergütung und oft ohne Anerkennung.
Das Engagement ist
vergleichbar mit Tätigkeiten in gemeinnützigen Vereinen
, nur eben digital.
Die rechtliche Gleichstellung mit klassischem Ehrenamt wäre daher folgerichtig.
3. Gesellschaftliche Abhängigkeit ohne gesellschaftliche Anerkennung
Staatliche Einrichtungen, Kommunen, Schulen und Unternehmen
profitieren direkt
von Open-Source-Bibliotheken, Frameworks und Tools.
Sicherheitslücken wie „Heartbleed“ oder Log4Shell haben gezeigt, wie entscheidend die Arbeit der Maintainer für das
Schutzinteresse der Allgemeinheit
ist.
Gleichzeitig fehlen Ressourcen und Strukturen, weil die Arbeit
formal nicht als Ehrenamt
eingestuft ist – und damit
keinerlei steuerliche oder organisatorische Förderung
erhält.
Dies erzeugt eine
unausgewogene Verantwortungslast
, die auf wenigen Freiwilligen liegt, während Millionen Nutzer*innen profitieren.
4. Anerkennung als Ehrenamt würde Rechtsklarheit schaffen
Durch eine formelle Anerkennung könnten:
Aufwandsentschädigungen steuerfrei
gewährt werden (Ehrenamtspauschale/Übungsleiterpauschale).
Gemeinnützige Open-Source-Projekte
eine leichtere Einstufung nach §52 AO erreichen.
Contributor bei Haftungsfragen besser gestellt werden (analog zu §31a BGB für Vereinsvorstände).
Projekte rechtssicher Kosten erstatten oder Spendenquittungen ausstellen.
Dies schafft
Transparenz
,
Rechtssicherheit
und
Nachhaltigkeit
im digitalen Ehrenamt.
5. Digitalisierung braucht freiwillige Kompetenz – und diese verdient Förderung
Open-Source-Engagement erfordert hohe technische Kompetenz.
Freiwillige Entwickler*innen leisten Arbeit, die Unternehmen ansonsten für hohe Stundensätze einkaufen müssten.
Der Staat investiert Milliarden in Digitalisierung, ignoriert aber die Menschen, die die technologische Basis
freiwillig
pflegen.
Eine Anerkennung als Ehrenamt wäre ein
kosteneffizienter Beitrag zur digitalen Souveränität
Deutschlands.
6. Deutschland hinkt international hinterher
Andere Staaten fördern Open-Source-Engagement bereits durch:
steuerliche Begünstigungen
institutionelle Förderung
Anerkennung gemeinnütziger Softwareentwicklung
Deutschland riskiert, im globalen Wettbewerb zurückzufallen, wenn Freiwilligenarbeit im digitalen Raum weiterhin
strukturell benachteiligt
wird.
More than 1,000 Amazon workers warn rapid AI rollout threatens jobs and climate
Guardian
www.theguardian.com
2025-11-28 13:50:15
Workers say the firm’s ‘warp-speed’ approach fuels pressure, layoffs and rising emissions More than 1,000 Amazon employees have signed an open letter expressing “serious concerns” about AI development, saying that the company’s “all-costs justified, warp speed” approach to the powerful technology wi...
More than 1,000
Amazon
employees have signed
an open letter
expressing “serious concerns” about AI development, saying that the company’s “all-costs justified, warp speed” approach
to the powerful technology will cause damage to “democracy, to our jobs, and to the earth.”
The letter, published on Wednesday, was signed by the Amazon workers anonymously, and comes a month after Amazon announced
mass layoff
plans as it increases adoption of AI in its operations.
Among the signatories are staffers in a range of positions, including engineers, product managers and warehouse associates.
Reflecting broader AI concerns across the industry, the letter was also supported by more than 2,400 workers from companies including Meta, Google, Apple and
Microsoft
.
The letter contains a range of demands for
Amazon
, concerning its impact on the workplace and the environment. Staffers are calling on the company to power all its data centers with clean energy, make sure its AI-powered products and services do not enable “violence, surveillance and mass deportation”, and form a working group comprised of non-managers
“that will have significant ownership over org-level goals and how or if AI should be used in their orgs, how or if AI-related layoffs or headcount freezes are implemented, and how to mitigate or minimize the collateral effects of AI use, such as environmental impact”.
The letter was organized by employees affiliated with the
advocacy group
Amazon Employees for Climate Justice. One worker who was involved in drafting the letter explained that workers were compelled to speak out because of negative experiences with using AI tools in the workplace, as well as broader environmental concerns about the AI boom. The staffers, the employee said, wanted to advocate for a better way to develop, deploy and use the technology.
“I signed the letter because of leadership’s increasing emphasis on arbitrary productivity metrics and quotas, using AI as justification to push myself and my colleagues to work longer hours and push out more projects on tighter deadlines,” said a senior software engineer, who has been with the company for over a decade, and requested anonymity due to fear of reprisal.
Climate goals
The letter accuses Amazon of “casting aside its climate goals to build AI”.
The letter claims that Amazon’s annual emissions have “
grown roughly 35%
since 2019”, despite the
company’s promise
in 2019 to achieve net zero carbon emissions by 2040. It warns many of Amazon’s investments in AI infrastructure will be in “locations where their energy demands will force utility companies to keep coal plans online or build new gas plants”.
“‘AI’ is being used as a magic word that is code for less worker power, hoarding of more resources, and making an uninformed gamble on high energy demand computer chips magically saving us from climate change,” said an Amazon customer researcher, who requested anonymity out of fear of retaliation for speaking out. “If we can build a climate saving AI – that’s awesome! But that’s not what Amazon is spending billions of dollars to develop. They are investing fossil fuel energy draining data centers for AI that is intended to surveil, exploit, and squeeze every extra cent out of customers, communities, and government agencies.”
In a statement to the Guardian, Amazon spokesperson Brad Glasser pushed back on employees’ claims and pointed toward the company’s climate goals. “Not only are we the leading data center operator in efficiency, we’re the world’s largest corporate purchaser of renewable energy for five consecutive years with over 600 projects globally,” said Glasser. “We’ve also invested significantly in nuclear energy through existing plants and new SMR technology–these aren’t distractions, they’re concrete actions demonstrating real progress toward our Climate Pledge commitment to reach net-zero carbon across our global operations by 2040.”
AI for productivity
The letter also includes strict demands around the role of AI in the Amazon workplace, demands that, staffers say, arose out of challenges employees are experiencing.
Three Amazon employees who spoke to the Guardian claimed that the company is pressuring them to use AI tools for productivity, in an effort to increase output. “I’m getting messaging from my direct manager and [from] of all the way up the chain, about how I should be using AI for coding, for writing, for basically all of my day-to-day tasks, and that those will make me more efficient, and also that if I don’t get on board and use them, that I’m going to fall behind, that it’s sort of sink or swim,” said a software engineer who has been with Amazon for over two years, requesting anonymity due to fear of reprisal.
The worker added that just weeks ago she was told by her manager that they were “expected to do twice as much work because of AI tools”, and expressed concern that the output expected demanded with fewer people is unsustainable, and “the tools are just not making up that gap.”
The customer researcher echoed similar concerns. “I have both personally felt the pressure to use AI in my role, and hear from so many of my colleagues they are under the same pressure …”.
“All the while, there’s no discussion about the immediate effects on us as workers – from unprecedented layoffs to unrealistic expectations for output.”
The senior software engineer said that the adoption of AI has had imperfect outcomes. He said that most commonly, workers are pressured to adopt agentic code generation tools: “Recently I worked on a project that was just cleaning up after a high-level engineer tried to use AI to generate code to complete a complex project,” said this worker. “But none of it worked and he didn’t understand why – starting from scratch would have actually been easier.”
Amazon did not respond to questions about the staffers’ workplace critiques about AI use.
Workers emphasized they are not against AI outright, rather they want it to be developed sustainably and with input from the people building and using it. “I see Amazon using AI to justify a power grab over community resources like water and energy, but also over its own workers, who are increasingly subject to surveillance, work speedups, and implicit threats of layoffs,” said the senior software engineer. “There is a culture of fear around openly discussing the drawbacks of AI at work, and one thing the letter is setting out to accomplish is to show our colleagues that many of us feel this way and that another path is possible.”
Writing Builds Resilience in Everyday Challenges by Changing Your Brain
Ordinary and universal, the act of writing changes the brain. From dashing off a heated text message to composing an op-ed, writing allows you to, at once, name your pain and create distance from it. Writing can shift your mental state from overwhelm and despair to grounded clarity — a shift that reflects resilience.
Psychology, the media and the wellness industry shape public perceptions of resilience: Social scientists study it, journalists celebrate it, and wellness brands sell it.
In my work as a
professor of writing studies
, I research how people use writing to navigate trauma and practice resilience. I have witnessed thousands of students turn to the written word to work through emotions and find a sense of belonging. Their writing habits suggest that writing fosters resilience. Insights from psychology and neuroscience can help explain how.
Writing rewires the brain
In the 1980s,
psychologist James Pennebaker
developed a therapeutic technique called
expressive writing
to help patients process trauma and psychological challenges. With this technique, continuously journaling about something painful helps create mental distance from the experience and eases its cognitive load.
In other words, externalizing emotional distress through writing fosters safety. Expressive writing turns pain into a metaphorical book on a shelf, ready to be reopened with intention. It signals the brain, “You don’t need to carry this anymore.”
Writing things down
supports memory consolidation
— the brain’s conversion of short-term memories into long-term ones. The process of integration makes it possible for people to reframe painful experiences and manage their emotions. In essence, writing can help free the mind to be in the here and now.
Taking action through writing
The state of presence that writing can elicit is not just an abstract feeling; it reflects complex activity in the nervous system.
Brain imaging studies show that putting feelings into words
helps regulate emotions
. Labeling emotions — whether through expletives and emojis or carefully chosen words — has multiple benefits. It calms the amygdala, a cluster of neurons that detects threat and triggers the fear response:
fight, flight, freeze or fawn
. It also engages the
prefrontal cortex
, a part of the brain that supports goal-setting and problem-solving.
In other words, the simple act of
naming your emotions
can help you shift from reaction to response. Instead of identifying with your feelings and mistaking them for facts, writing can help you simply become aware of what’s arising and prepare for deliberate action.
Even mundane writing tasks like
making a to-do list
stimulate parts of the brain involved in reasoning and decision-making, helping you regain focus.
Making meaning through writing
Choosing to write is also choosing to make meaning. Studies suggest that having a sense of agency is both a prerequisite for, and an outcome of, writing.
Researchers have long documented how
writing is a cognitive activity
— one that people use to communicate, yes, but also to understand the human experience. As many in the field of writing studies recognize,
writing is a form of thinking
— a practice that people never stop learning. With that, writing has the potential to continually reshape the mind. Writing not only expresses but actively creates identity.
Writing also regulates your psychological state. And the words you write are themselves proof of regulation — the evidence of resilience.
Popular coverage of human resilience often presents it as extraordinary endurance.
News coverage of natural disasters
implies that the more severe the trauma, the greater the personal growth.
Pop psychology
often equates resilience with unwavering optimism. Such representations can obscure ordinary forms of adaptation. Strategies people already use to cope with everyday life — from rage-texting to drafting a resignation letter — signify transformation.
Building resilience through writing
These research-backed tips can help you develop a writing practice conducive to resilience:
1. Write by hand whenever possible.
In contrast to typing or tapping on a device,
handwriting requires greater cognitive coordination
. It slows your thinking, allowing you to process information, form connections and make meaning.
2. Write daily.
Start small and make it regular. Even jotting brief notes about your day — what happened, what you’re feeling, what you’re planning or intending — can help you get thoughts out of your head and
ease rumination
.
3. Write before reacting.
When strong feelings surge, write them down first. Keep a notebook within reach and make it a habit to write it before you say it. Doing so can
support reflective thinking
, helping you act with purpose and clarity.
4. Write a letter you never send.
Don’t just write down your feelings — address them to the person or situation that’s troubling you. Even
writing a letter to yourself
can provide a safe space for release without the pressure of someone else’s reaction.
5. Treat writing as a process.
Any time you draft something and ask for feedback on it, you practice stepping back to consider alternative perspectives. Applying that feedback through revision can
strengthen self-awareness
and
build confidence
.
Resilience may be as ordinary as the journal entries people scribble, the emails they exchange, the task lists they create — even the essays students pound out for professors.
Breaking news from famed machine learning researcher Ilya Sutskever:
Below is another summary of a just-released
interview
of his that is making waves, a bit more technical. Basically Sutskever is saying that scaling (achieving improvements in AI through more chips and more data) is flattening out, and that we need new techniques; he is even open to
neurosymbolic
techniques, and innateness. He is clearly not forecasting a bright future for pure large language models.
Sutskever also said that “
The thing which I think is the most fundamental is that these models somehow just generalize dramatically worse than people. And it’s super obvious. That seems like a very fundamental thing.
”
Some of this may come as news to a lot of the machine learning community; it might be surprising coming from Sutskever, who is an icon of deep learning, having worked, inter alia, on the critical 2012 paper that showed how much GPUs could improve deep learning, the foundation of LLMs, in practice. He is also a co-founder of OpenAI, considered by many to have been their leading researcher until he departed after a failed effort to oust Sam Altman.
But none of what Sutskever said should actually come as a surprise, especially not to readers of this Substack, or to anyone who followed me over the years. Essentially
all
of it was in my pre-GPT 2018 article “
Deep learning: A Critical Appraisal
”, which argued for neurosymbolic approaches to complement neural networks (as Sutskever now is), for more
innate
(i.e., built-in, rather than learned) constraints (what Sutskever calls “new inductive constraints”) and/or in my 2022 “
Deep learning is hitting a wall
” evaluation of LLMs, which explicitly argued that the Kaplan scaling laws would eventually reach a point of diminishing returns (as Sutskever just did), and that problems with hallucinations, truth, generalization and reasoning would persist even as models scaled, much of which Sutskever just acknowledged.
None
of what Sutskever said should come as a surprise. A machine learning researcher at Samsung, Alexia Jolicoeur-Martineau summed the situation up well on X, Tuesday, following the release of the Sutskever’s interview:
§
Of course it ain’t over til it’s over. Maybe pure scaling (adding more data and compute without fundamental architectural changes)
will
somehow magically yet solve what researchers as such Sutskever, LeCun, Sutton, Chollet and myself no longer think it could.
And investors may be loathe to kick the habit. As Phil Libin put it presciently last year, scaling—not the generation of new ideas—is what investors know best
And it’s not just that venture capitalists know more about scaling businesses than inventing new ideas, it’s for the venture capitalists that have driven so much of field, scaling, even if it fails, has been a great run: it’s been a way to take their 2% management fee investing someone else’s money on plausible-ish sounding bets that were truly massive, which makes them rich no matter how things turn out. To be sure, the VC get even richer still if the investments pan out, to be sure. but they are covered either way; even if it all falls apart, the venture capitalists themselves will become wealthy from the management fees alone. (It is their clients, such as pension funds, that will take the hit). So venture capitalists may continue to support LLM mania, at least for a while.
But let’s suppose for the sake of argument that Sutskever and the rest of us are correct, and that AGI will never emerge straight from LLMs, and that to a certain extent that they have run their course, and that we do in fact need new ideas.
The question then becomes, what did it cost the field and society that it took so long for the machine learning mainstream to figure out what some of us, including virtually the entire neurosymbolic AI community had been saying for years?
§
The first and most obvious answer is money, which I estimate, back of the envelope as (roughly) a trillion dollars, much of it on Nvidia chips and massive salaries. (Zuckerberg has apparently hired some machine learning experts at salaries of $100,000,000 a year).
If the definition of insanity is doing the same thing over and over and expecting different results, trillion dollar investments in ever more expensive experiments aiming to reach AGI may be delusional to the highest degree.
To a first approximation, all the big tech companies, from OpenAI to Google to Meta to xAI to Anthropic to several Chinese companies, keep doing the same experiment over and over: building ever larger LLMs in hopes of reaching AGI.
It has never worked. Each new bigger, more expensive model ekes out measurable improvements, but returns appear to be diminishing (that’s what Sutskever is saying about
the Kaplan laws
) and none of these experiments has solved core issues around hallucinations, generalization, planning and reasoning, as Sutskever too now recognizes.
To be fair, nobody knows for sure what the blast radius would be. If LLM-powered AI didn’t meet expectations and became valued less, who would take the hit? Would it just be the “limited partners” like pension funds who entrusted their money with VC firms? Or might the consequences be much broader? Might banks go down with the ship, in 2008-style liquidity crisis,possibly forcing taxpayers to bail them out? In the worst case, the impact of a deflated AI bubble could be immense. (Consumer spending, much of it fueled by wealthy people who could a hit on the stock market, might also drop, a recipe for recession.)
Even the White House has admitted concerns about this. As the White House AI and Crypto Czar David Sacks himself put it earlier this week, referring to a Wall Street Journal analysis, “Al-related investment accounts for half of GDP growth. A reversal [in that] would risk recession.”
Quoting from Karma’s article in The Atlantic :
That prosperity [that GenAI was supposed to deliver] has largely yet to materialize anywhere other than their share prices. (The exception is Nvidia, which provides the crucial inputs—advanced chips—that the rest of the Magnificent Seven are buying.) As
The Wall Street Journal
reports, Alphabet, Amazon, Meta, and Microsoft have seen their
free cash flow
decline by 30 percent over the past two years. By one
estimate
, Meta, Amazon, Microsoft, Google, and Tesla will by the end of this year have collectively spent $560 billion on AI-related capital expenditures since the beginning of 2024 and have brought in just $35 billion in AI-related revenue. OpenAI and Anthropic are
bringing
in lots of revenue and are growing fast, but they are still
nowhere
near
profitable. Their valuations—roughly
$300 billion
and
$183 billion
, respectively, and
rising
—are many multiples higher than their current revenues. (OpenAI
projects
about $13 billion in revenues this year;
Anthropic
, $2 billion to $4 billion.) Investors are betting heavily on the prospect that all of this spending will soon generate record-breaking profits. If that belief collapses, however, investors might start to sell en masse, causing the market to experience a large and painful correction.
…
The dot-com crash was bad, but it did not trigger a crisis. An AI-bubble crash could be different. AI-related investments have already
surpassed
the level that telecom hit at the peak of the dot-com boom as a share of the economy. In the first half of this year, business spending on AI added more to GDP growth than all consumer spending
combined
. Many experts believe that a major reason the U.S. economy has been able to weather tariffs and mass deportations without a recession is because all of this AI spending is acting, in the
words
of one economist, as a “massive private sector stimulus program.” An AI crash could lead broadly to less spending, fewer jobs, and slower growth, potentially dragging the economy into a recession. The economist Noah Smith
argues
that it could even lead to a financial crisis if the unregulated “private credit” loans funding much of the industry’s expansion all go bust at once.
The whole thing looks incredibly fragile.
§
To put it bluntly, the world has gone “all in” on LLMs, but, as Sutskever’s interview highlights, there are many reasons to doubt that LLMs will ever deliver the rewards that many people expected.
The sad part is that most of the reasons have been known – though not widely accepted – for a very long time. It all could have been avoided. But the machine learning community has arrogantly excluded other voices, and indeed whole other fields like the cognitive sciences. And we all now may be about to pay the price.
An old saying about such follies is that “six months in the lab can you save you an afternoon in the library”; here we may have wasted a trillion dollars and several years to rediscover what cognitive science already knew.
A trillion dollars is a terrible amount of money to have perhaps wasted. If the blast radius is wider it could be a lot more. It is all starting to feel like a tale straight out of Greek tragedy, an avoidable mixture of arrogance and power that just might wind up taking down the economy.
Discussion about this post
The Historic Rise of Zohran Mamdani: Democracy Now! Coverage from 2021 Hunger Strike to Election Night
Democracy Now!
www.democracynow.org
2025-11-28 13:01:29
As Zohran Mamdani prepares to become New York’s first Muslim and first South Asian mayor on January 1, we look at the historic rise of the democratic socialist who shocked the political establishment. We spend the hour hearing Mamdani in his own words and look at the grassroots coalition that ...
This is a rush transcript. Copy may not be in its final form.
AMY
GOODMAN
:
In this
Democracy Now!
special, we look at the rise of New York Mayor-elect Zohran Mamdani. On November 4th, he made history by winning the race to become the next mayor of New York City. The democratic socialist is the first Muslim and first person of South Asian descent elected to lead the largest city in the United States. At 34 years old, he’s also the youngest person elected to the office in over a century. His meteoric rise from a little-known state assemblymember to his stunning upset over former Governor Andrew Cuomo has sent shockwaves through the Democratic Party.
Today, we spend the hour hearing Zohran Mamdani in his own words and look at the grassroots campaign behind him. Mamdani was born in Uganda and moved to New York as a child. His parents are the acclaimed filmmaker Mira Nair and Columbia University professor Mahmood Mamdani. In 2020, Zohran Mamdani won a seat in the New York Assembly representing Astoria, Queens.
In October 2021, Mamdani
appeared
on
Democracy Now!
for the first time while taking part in a 15-day hunger strike to demand debt relief for New York taxi drivers.
ZOHRAN
MAMDANI
:
I’m participating in solidarity with the Taxi Worker Alliance and to try and bring to light what the consequences are of the city’s inaction for many years and now their completely insufficient plan for debt relief, because, you know, it is — we started this hunger strike last Wednesday. We’ve now completed seven full days of being without food, one of the most basic elements of dignity. And the consequences we have seen in our own bodies — you know, an inability to sleep, unrelenting hunger, moments of blurred vision, stress, headaches — these are the same consequences that I heard drivers talk about when they say what the physical realities are of being hundreds of thousands of dollars in debt, unable to take care of your family and seeing no way out. So, it’s important for us as legislators to bring to light what it is that people are suffering from out of view of those in the political elite, and bring it right front and center in front of City Hall.
AMY
GOODMAN
:
In 2022, New York Assemblymember Zohran Mamdani
came back
on
Democracy Now!
after the Republican Party won control of the House of Representatives, in part because Republicans flipped four seats in New York.
ZOHRAN
MAMDANI
:
You can only get so far presenting a negative version of the Republican vision. We can only get so far telling people that “Vote to defeat Lee Zeldin.” We need to have an affirmative vision. The Working Families Party has laid out what that vision could look like, and now the Democratic Party needs to do so, as well.
And when I think about that, I think particularly about two issues: housing and the climate crisis. Right? More than 75% of New Yorkers across the state are concerned about rising rents, and more than 67% believe that we need to pass good cause eviction as a means by which to keep those rents under control.
AMY
GOODMAN
:
In October 2023, I
spoke
to Zohran Mamdani when he took part in a historic protest when the group Jewish Voice for Peace and their allies shut down the main terminal of Grand Central Station during rush hour to demand a ceasefire in Gaza.
ZOHRAN
MAMDANI
:
My name is Zohran Mamdani. I’m an assemblymember for parts of Astoria and Long Island City. And I’m here today to join thousands of Jewish New Yorkers, rabbis and allies to say that the time is now for an immediate ceasefire.
AMY
GOODMAN
:
What does it mean to you that on this Shabbat, the Jewish Sabbath, thousands of Jews are here at Grand Central saying “Ceasefire now”?
ZOHRAN
MAMDANI
:
It shows that what we have been told about the consent for this genocide is not true. So many of the Jewish New Yorkers here are struggling through heartbreak and mourning of October 7th, and they have made it very clear that do not use their heartbreak, their tragedy as the justification for the genocide of Palestinians. In over two-and-a-half weeks, we’ve already seen more than 7,000 Palestinians be killed, close to 3,000 Palestinian children, one Palestinian child killed every 15 minutes. These New Yorkers, and so many across the state, are saying the time is now for a ceasefire, and if you’re not calling for it, you’re supporting a genocide.
AMY
GOODMAN
:
Last October, Mamdani
joined
Democracy Now!
as he launched his mayoral campaign, and laid out the platform he’s now known for.
ZOHRAN
MAMDANI
:
We are going to freeze the rent for every single rent-stabilized tenant for every single year of the mayoralty. We are going to make buses free and fast across this entire city. And we are going to enact universal child care at no cost for all New Yorkers for children from the ages of 6 weeks to 5 years. These are the policies that will set us apart, and these are the policies that resonate with New Yorkers’ concerns.
JUAN
GONZÁLEZ:
And if you could talk some more about your stance on the war in Gaza, which clearly — or, in the Palestinian territories, which clearly is not normally a plank of a candidate for mayor in New York City, but certainly will affect how people vote?
ZOHRAN
MAMDANI
:
You know, I think there’s tremendous anger and alienation across New York City today, whether it’s theses corruption crises or the cost of living or the fact that our tax dollars are continuing to fund a genocide across Palestine. And what voters are looking for is someone who can speak clearly to that crisis of confidence and of faith in the power of government to be a positive force in people’s lives, and to offer them a vision that is worth believing in.
And that is what I am going to do in this campaign, is to put forward an economic agenda that puts working-class New Yorkers first, all while recognizing the world as it actually is, which is one where there is a hierarchy of human life that the United States government is following that states it is fine for Palestinians and Lebanese and Syrians and Yemenis to be killed, because that is simply the worth that they have in the eyes of our federal government.
AMY
GOODMAN
:
Over the next 12 months, Mamdani would rise in the polls from last place to first, shocking the political establishment by building a historic grassroots coalition. In June, he defeated disgraced former Governor Andrew Cuomo in New York City’s Democratic mayoral primary.
ZOHRAN
MAMDANI
:
In the words of Nelson Mandela, it always seems impossible until it is done. My friends, we have done it. I will be your Democratic nominee for the mayor of New York City.
AMY
GOODMAN
:
Zohran Mamdani
came back
on
Democracy Now!
in September, hours after New York Mayor Eric Adams ended his reelection campaign. Mamdani talked about his plans to “Trump-proof” New York City following the president’s threat to cut off federal funds to New York if Mamdani won the general election.
ZOHRAN
MAMDANI
:
You know, I think it’s — it is a sad reality in this country, where we have a president who ran an entire campaign premised on cheaper groceries and lowering the cost of living, and what he has instead delivered, time and again, is an exacerbating of that very crisis, all while focusing on the persecution of his supposed political enemies. And when we talk about Trump-proofing the city, it’s not just the question of hiring the 200 additional lawyers at our law department to bring us back to the staffing levels prior to the pandemic. It’s a question of actually standing up and fighting Donald Trump, and fighting Donald Trump because what his agenda is doing is endangering the welfare of New Yorkers.
This bill that he recently ushered through Washington, D.C., it throws millions of New Yorkers off of their healthcare. It steals
SNAP
benefits from so many hungry New Yorkers. And it does all of this in the interest of the largest wealth transfer that we’ve seen in this country. And to do those things while speaking about a cost-of-living crisis, it is truly a betrayal of so much of what his campaign was premised on, and an illustration of why he is so fearful of our campaign, because, unlike him, we don’t just diagnose this crisis, we will deliver on it. We will actually ensure that we have New Yorkers who can afford the city that they call home, that we freeze the rent for more than 2 million New Yorkers, we make buses fast and free, which are currently the slowest ones in the nation, and we deliver universal child care. And that’s what Donald Trump is afraid of: the stark contrast between our delivery of those things and what he has done as the president of this country. …
New Yorkers are facing twin crises: authoritarianism from Washington, D.C., and an affordability crisis from the inside. And we often tend to separate these out. We think about democracy as an ideal that must be protected, but not that democracy also has to be able to deliver on the material needs of working people. And it was Fiorello La Guardia that said, “You cannot preach … liberty to a starving land.” You have to be able to deliver on both fronts.
AMY
GOODMAN
:
Zohran Mamdani on
Democracy Now!
in September. Coming up, we look at how working-class South Asians helped propel Mamdani to victory.
[break]
AMY
GOODMAN
:
This is
Democracy Now!
, democracynow.org,
The War and Peace Report
. I’m Amy Goodman.
In this holiday special, we’re continuing to look at the rise of New York Mayor-elect Zohran Mamdani. Just prior to the November election,
Democracy Now!
's Anjali Kamat filed this
report
looking at a crucial, often overlooked portion of Mamdani's base: working-class South Asians.
ANJALI
KAMAT
:
It’s Friday afternoon in a quiet neighborhood in Kensington, Brooklyn. These women are members of
DRUM
Beats, an advocacy group for low-income South Asian and Indo-Caribbean communities here in New York. and they’re getting ready to canvass for Zohran Mamdani.
KAZI
FOUZIA
:
So, half of the list, you’re going to cover with them. Then they will — they will find them.
ANJALI
KAMAT
:
They split up into groups, and I followed them as they knocked on dozens of doors. Armed with colorful flyers about the campaign in Bengali and Urdu and dozens of Zohran pins, they explained why they thought Mamdani was the best candidate, and reminded neighbors about early-voting times and locations.
DRUM
BEATS
CANVASSER
:
So, November 4th is the final vote.
As-salamu alaykum
.
ANJALI
KAMAT
:
Their enthusiasm was infectious, often bursting into Bengali chants of “My mayor, your mayor.”
ANJALI
KAMAT
:
And for the most part, it seemed to work. I spoke to Fahd Ahmed, who runs
DRUM
Beats, which stands for Desis — or South Asians — Rising Up and Moving. Their organization was among the very first to endorse Zohran’s run for mayor last year.
FAHD
AHMED
:
Many people will say that, “Oh, well, it’s a South Asian-descended candidate, and so it must be an identity thing.” But we’ve had several South Asian or Indo-Caribbean candidates, and none of them elicited this response. And I think the fact that the campaign spoke to the very material issues of working-class people has, first and foremost, has really made a very significant difference.
ANJALI
KAMAT
:
I also spoke to Jagpreet Singh,
DRUM
Beats’ political director, who’s in charge of endorsing political candidates and getting the vote out.
JAGPREET
SINGH
:
When Zohran had come to us, to begin with, he said his base, the base he was looking at, were three planks. Number one was the leftist progressives. His second plank was rent-stabilized tenants. And the third was Muslim and South Asian communities, communities that have not been previously galvanized, have not been previously activated, usually have some of the lowest voter turnout rates. So, from the get-go, our communities were going to be a big part of his base.
ANJALI
KAMAT
:
Kazi Fouzia moved to New York City from Bangladesh in 2008. Now she’s DRUM’s organizing director. The tireless campaigning by women like her was crucial to Zohran’s victory in the primaries. In some neighborhoods, voter turnout among South Asian and Indo-Caribbean communities doubled.
KAZI
FOUZIA
:
Just 24/7, they are thinking how to win. Some of them work in the cafeteria in the school. Some of them also work in the retail store. Some of them are home health worker, take care of the patient. One of my leader actually restoring ship. They are not only just volunteers. They build, actually, movement.
ANJALI
KAMAT
:
After a long evening of canvassing, they’re back at the office only to get ready for more of the same the next day and every day after until the elections.
KAZI
FOUZIA
:
These all tired people come together and creating movement to show the world how political campaign supposed to be looked like. The early vote kick-off.
CAMPAIGNER
:
Six, seven million voters. In June, we won the primary because of historic numbers of new voters that turned out. We changed the electorate.
ANJALI
KAMAT
:
Earlier this month, Zohran Mamdani addressed an excited crowd of supporters at a Bangladeshi restaurant in Jackson Heights, Queens.
ANNOUNCER
:
And up next now we hear from the Zohran Mamdani!
ZOHRAN
MAMDANI
:
What we did in the primary is we increased the turnout of Muslims by 60%, the turnout of South Asians by 40%. And when I stood in front of the world and gave a speech that night, I made sure to remember the Bangladeshi aunties that knocked on the doors across this city. And people have asked me, “What will it mean to have a Muslim mayor?” What my grandmother Kulsum taught me, that to be a good Muslim is to be a good person. It is to help those in need and to harm no one. The truth of this campaign, it is a truth that believes in each one of the people in this room and their possibility. It is the truth that looks at the youngest among us and sees that they could be anything in this city, anything they want.
ANJALI
KAMAT
:
At the Jackson Heights farmers’ market that weekend, the high school students who met Mamdani at the restaurant were still thinking about his words.
MOHINI
MEHBOOBA
:
If I could run for mayor, I think I would have a lot of great ideas, just like Zohran, making New York City affordable. I want to be able to live here without any worry about paying rent. I know I’m just 17, but I want to be able to move out next year and experience living in the city, because I know, even for my family, it’s really hard to pay the rent. So, yeah.
ANJALI
KAMAT
:
Mohini Mehbooba is one of the youth members of
DRUM
Beats. A talented artist, Mohini was giving people henna tattoos that spelled “Zohran.”
MOHINI
MEHBOOBA
:
We work so hard phone banking, canvassing. And I love doing it, and I’m going to do some more today, hopefully. And it’s just a really good feeling to do something that will be able to change for us, as well.
PHONE
BANKER
:
Thank you so much.
ANJALI
KAMAT
:
At the
DRUM
Beats office in Jackson Heights, there’s a different group of people phone banking every afternoon. They’re reaching out to communities in a variety of South Asian languages, with volunteers making calls in Nepali, Urdu and Bengali. A group of high school students are also making calls — in between joking around.
SAMMY
:
Hey. My name is Sammy, and I am a high school volunteer for the Zohran Mamdani’s campaign. Have you ever heard about Zohran Mamdani? Are you planning to vote for him on the Election Day, November 4th?
ANJALI
KAMAT
:
High school student Miftahun Mohona explains why she’s passionate about campaigning for Zohran Mamdani.
MIFTAHUN
MOHONA
:
Even though I’m not at the age to vote, not yet, I still care about, like, people above 18, like for them to vote for Zohran, because the thing is, if they vote for the — if they vote for the right person, that also benefits me, because I live in a world where it’s very corrupt, and every action that the people over 18 taking, like voting, their action means a lot to me, as well, because I come from a working-class family. We don’t have many benefits. We don’t have much resources.
ANJALI
KAMAT
:
Across working-class South Asian communities in the city, there’s a deep belief that Zohran Mamdani will stand up for them if he becomes mayor. A big reason for that is his role in the taxi workers’ protest against medallion debt back in 2021. When the drivers decided to go on a hunger strike, Assemblyman Mamdani joined them for the full 15 days. Kazi Fouzia remembers how moved the community was.
KAZI
FOUZIA
:
I saw how long he’s doing the hunger strike, and he almost die in that time. So I feel this call, actually, real solidarity. Solidarity, not just come and talk and leave. Solidarity, also he put his body frontline.
ANJALI
KAMAT
:
DRUM
, or Desis Rising Up and Moving, was founded in Jackson Heights, Queens, in 2000 as a membership organization of low-wage South Asian and Indo-Caribbean workers and youth. For most of its history, their membership has faced the brunt of domestic repression and hate crimes that followed the September 11th attacks. Kazi Fouzia found herself the target of
NYPD
surveillance when she started organizing in immigrant Muslim communities.
KAZI
FOUZIA
:
I came 2008 this country, and I used to work in retail store in Jackson Heights. And that time, I’m doing volunteering organizing with the
DRUM
, and one day I found informer behind me.
ANJALI
KAMAT
:
A few years later, as hate crimes against South Asian immigrants spiked again, many people suggested she stopped wearing her hijab.
KAZI
FOUZIA
:
People asked me, 2013, “You should take off your hijab because it’s not safe anymore.” We saw how much isolations and fear community have after 9/11.
ANJALI
KAMAT
:
Jagreet Singh remembers his Sikh family members cutting their hair and beards and wearing American flag T-shirts to stay safe after 9/11.
JAGPREET
SINGH
:
This is a reality we lived with for a long time, that we had to hide ourselves, that we had to retreat back, that we had to fight for everything that we wanted. And we’re in this reality now where Zohran Mamdani is about to become mayor of our city, a very outward Muslim man, South Asian, who is very much into his identity, who does not hide his identity.
ANJALI
KAMAT
:
From the shadows of post-9/11 repression and fear, the Mamdani campaign has given this community a new sense of political confidence and purpose.
KAZI
FOUZIA
:
So, if you see now our member, our community member, our religious leader, our neighbors, all now talking, talking, talking for Zohran. If they go back to 9/11 era and they try to talk about Islamophobia, xenophobia, it’s not going to sell. It’s not going to sell. It’s over. People are not going to go back the isolating zone anymore. If they try to implementing this, they will push back.
ANJALI
KAMAT
:
If Zohran Mamdani wins the mayoral election,
DRUM
Beats, like other progressive groups that backed Mamdani from the start, could find themselves in a brand-new role: collaborating with the administration to govern the city. It’s been a long journey from advocating for those on the margins to potentially having a seat at the table. Here’s Jagpreet Singh again.
JAGPREET
SINGH
:
Talks about what the administration would look like are still a little premature, but the campaign and the administration has been very willing to work with organizations like ours at
DRUM
Beats. It feels amazing to see that we now get to take up leadership, that we get to not only have a seat at the table but run how our city runs. It’s not just going to happen by him being in office, no matter how charismatic he is.
ANJALI
KAMAT
:
Kazi Fouzia says that if Mamdani wins the race but is unable to keep his campaign promises down the road, their members will not hesitate to push his administration and hold their feet to the fire.
KAZI
FOUZIA
:
Zohran make impossible possible in his grassroot movement, too, in the mayoral campaign. So Zohran have to keep his promises and fulfill his commitment. And we will be support all the time him. And also, if he don’t fulfill or keep his promises, we will hold him accountable.
ANJALI
KAMAT
:
In the event of a Mamdani victory, his administration will not face an easy path. People like Fahd Ahmed are already preparing for how to confront the many challenges and threats that may come, whether from the Trump administration or Wall Street and real estate interests.
FAHD
AHMED
:
Our side, there will be real challenges of trying to run a city as a left, when we don’t have extensive experience of doing that. But how it is that we govern, tending to the actual material needs that come up in day-to-day administration of the city, while having a vision that is transformative, that does believe that cities and society can be shaped differently and can function in ways that actually meet the needs of everyday working people.
ANJALI
KAMAT
:
But for now, the South Asian and Indo-Caribbean communities that have been pounding the pavement for Mamdani couldn’t be more excited for a potential Zohran Mamdani victory — and their new role in the spotlight.
ZOHRAN
MAMDANI
:
We choose the future, because for all those who say our time is coming, my friends, our time is now.
ANJALI
KAMAT
:
For
Democracy Now!
, this is Anjali Kamat, with Nicole Salazar. Thanks to Rehan Ansari.
AMY
GOODMAN
:
So, that was looking at some of the organizing leading to Zohran Mamdani’s victory. Coming up, we hear Zohran Mamdani in his own words after he won the New York mayoral race in what Senator Bernie Sanders called “one of the great political upsets in modern American history.” Stay with us.
[break]
AMY
GOODMAN
:
This is
Democracy Now!
, democracynow.org,
The War and Peace Report
. I’m Amy Goodman.
In this holiday special, we’re continuing to look at the rise of Zohran Mamdani. On election night,
Democracy Now!
was at Mamdani’s victory party at the historic Brooklyn Paramount, where more than a thousand people packed in. We
spoke
to some of his supporters and organizers as the election results started coming in.
SUMAYA
AWAD
:
My name is Sumaya Awad, and I’m a member of New York City
DSA
. And I am — to say I’m excited and ecstatic and relieved is an understatement. I mean, we have fought so hard for this, right before the primary, and then now, in the last couple of months and last couple weeks and today. I’ve been canvassing since 9 a.m. And I feel exhausted, but it’s the best kind of exhausted, because it’s exhaustion from something that I believe in with every fiber of my body and that I know that the majority of New Yorkers believe in. And we haven’t felt that — I haven’t felt that in my lifetime.
AMY
GOODMAN
:
Tell us what it is you believe in.
SUMAYA
AWAD
:
It’s a politician and an agenda that is truly for working-class people, and one that doesn’t put the platform and the mission at the expense of anyone. He has not left anyone out of what he is fighting for, and he’s made it clear. Whether you support him or not, he is fighting for us.
AMY
GOODMAN
:
What did you say?
RUBY
:
NBC
just called it for Zohran.
AMY
GOODMAN
:
And what do you think?
RUBY
:
I’m so, so happy. I’ve been awake since 4:30 in the morning today, out canvassing in Park Slope and Prospect Heights. And we’ve been working towards this for a year, and I’m just so happy to win the New York City that we deserve.
AMY
GOODMAN
:
What’s your name? Where are you from?
RUBY
:
My name is Ruby. I live in Crown Heights. Whoo!
AMY
GOODMAN
:
Hi.
HARRISON
:
How’s it going?
AMY
GOODMAN
:
Can you tell me your names? And what do you think?
HARRISON
:
I’m Harrison, and I’m thrilled. We’ve been canvassing since February, January, and it’s so happy to see all of our work pay off.
JANIE
:
It feels surreal that it’s actually here and that it’s happening, yeah. It’s so crazy.
AMY
GOODMAN
:
What does it about Zohran Mamdani that led you to support him? And what is your name?
JANIE
:
I’m Janie. He just has inspired hope, I feel like, across the city in a way that no one has in a long time. Yeah, a lot of us didn’t want to vote for a Democrat who we felt like we had to, you know, choose over another person. So, yeah.
AMY
GOODMAN
:
As you can hear, they have just called it for Zohran Mamdani. And here we are in the Brooklyn Paramount. What’s your name? What are your thoughts?
BEN
:
My name is Ben. I couldn’t be more excited. I couldn’t be more excited.
AMY
GOODMAN
:
What group are you with?
BEN
:
I’m an organizer with Jewish Voice for Peace. We’ve worked really hard for this moment. I’m so excited to celebrate with everybody.
AMY
GOODMAN
:
Did you think you’d see this day?
BEN
:
I was confident. I was confident. Yeah, yeah. Thank you.
AMY
GOODMAN
:
What do you say about Donald Trump saying today that any Jew who votes for Zohran Mamdani is stupid?
BEN
:
It’s antisemitic nonsense. I mean, it’s bigotry, plain and simple. And we’re sick of antisemitism being weaponized against Palestinian people and against our own communities, as well.
AMY
GOODMAN
:
What do you want to see Zohran Mamdani do as mayor?
BEN
:
Making New York City a city for everybody, a city we can afford, a city where people can lead dignified lives.
ROULA
HAJJAR
:
My name is Roula Hajjar. I really — I don’t know what to say. I mean, it’s been a very hard few years with the genocide, and this is the first good news that we’ve had, it feels like, like truly, truly good news, something to really look forward to and celebrate.
AMY
GOODMAN
:
What about local issues here in New York? What most appeals to you about Mamdani?
ROULA
HAJJAR
:
Well, I mean, I think that — so, I’m a social worker by training, and I think that the way that he is construing public safety issues as not — you know, not criminalizing mental health issues is very, very significant and, I think, will change how we think of safety and security in New York City, which is something that I know is on the minds of a lot of people.
JAGPREET
SINGH
:
My name is Jagpreet Singh. I’m the political director at
DRUM
Beats. And I feel amazing. I feel ecstatic. I’m on top of the world. It’s going to be a couple of days until I come back down.
NABILA
:
Hi. My name is Nabila. I’m a youth organizer at
DRUM
Beats.
AMY
GOODMAN
:
It’s well known that while young people are very enthusiastic, they’re the least likely to vote.
NABILA
:
Yeah.
AMY
GOODMAN
:
What’s your response to that?
NABILA
:
I think this just goes to show when we have a candidate that actually cares about the popular issues that affect everyone, and someone who’s charismatic and who doesn’t talk down to youth, you finally have a youth that’s ready to show the energy they’ve always had. It’s just that they’ve been marginalized all this time.
KEANU
ARPELS
-
JOSIAH
:
Hi. My name is Keanu Arpels-Josiah. I’m with Sunrise Movement New York City.
AMY
GOODMAN
:
So, what are your feelings right now?
KEANU
ARPELS
-
JOSIAH
:
I’m joyful. This is the beginning of a new future for New York City, a future where we have a politics that works for our generation, for affordability, fights the climate crisis, fights the billionaire class taking over our government, stands up to fascism and stands up for our issues. This is a moment where all of politics is changing. New York City is changing. New York City is standing up and demanding a different future for our world, for our country and for our city. I couldn’t be more excited.
AMY
GOODMAN
:
How will it change what you do?
KEANU
ARPELS
-
JOSIAH
:
It means the same for us in some ways, and it means everything is different in other ways. It means collaboration. It means a politics of working with those in office to deliver the agenda, but it also means a politics of accountability. We need to be with Zohran celebrating today, and we need to be talking with him tomorrow to make his agenda a reality. We need to be standing alongside. We can’t just be yelling at each other. But we have to have collaboration and accountability. And it means we need to fight Governor Hochul, who’s trying to build fossil fuel pipelines through New York City, that Mamdani opposes. We need to fight to tax the rich. And we need to fight Washington as it attacks our communities.
SIMONE
ZIMMERMAN
:
My name is Simone Zimmerman. I’m a part of the Jews for Zohran campaign. I’m a board member of Jews for Racial and Economic Justice Action. And I’m over the moon. I don’t know. This is it.
Trump called the Jews who voted for Zohran stupid. But look, we’re in a moment right now where we have an administration that is using racism and fear and is sowing terror in cities around the country. And Jews are not different from any other Americans. We see the hatred and the racism that they’re spreading, and we’re terrified of it. And despite the fact that millions of dollars were poured into this race to scare the living daylights out of Jewish voters, I think we’re going to see so many people see in Zohran a vision of safety and belonging in the city that they want to be part of, despite the fact that over and over again they were told, “You don’t belong. You don’t belong.” Zohran worked so hard to go to synagogues, to reach out to Jewish communities across the city, Jewish communities of such ideological and religious diversity, and say, “You belong here.” And I think people believe him, and I think that tonight we’re seeing that.
FAHD
AHMED
:
My name is Fahd Ahmed, and I’m the director of
DRUM
Beats. This campaign was successful because it had a movement behind it, and it was successful because it spoke to the material needs of people.
AMY
GOODMAN
:
This is a very strong message to the entire country. It’s not only Republicans who were organized against Mayor Mamdani. It’s the Democratic Party, as well. What are your thoughts on that?
FAHD
AHMED
:
Yeah. You know, in our work, we talk about that the — it is the policies of the centrists, whether they’re Democrats or some of the old Republicans, that created the conditions that caused the rise of the right. When people’s needs aren’t being met, they need an alternative, and so far, only the far right was providing an alternative in the form of authoritarianism, in the form of fascism, in the form of hate, turning against immigrants, against queer people, against Muslims. And what this campaign and our movement was able to do was offer a left alternative.
JAMES
DAVIS
:
I’m James Davis. I’m the president of the Professional Staff Congress-
CUNY
, the
CUNY
faculty and staff union.
AMY
GOODMAN
:
You were among the first unions to endorse Zohran Mamdani.
JAMES
DAVIS
:
We were. I mean, we’ve known Zohran since his time in the Assembly. So we knew that even though he was a long-shot candidate, he would have tremendous message discipline. And in a time like now, when there’s Trumpism from the federal government, we also knew that his message was going to resonate among working New Yorkers.
We see what President Trump has done with the budget bill as a massive transfer of wealth to the already wealthiest. So, part of our agenda is making sure that there’s additional progressive taxation, so that the public services, including the City University of New York, can be properly funded, so we can have not just an affordable education, but a high-quality education that our students deserve.
JASMINE
GRIPPER
:
Jasmine Gripper, co-director of the New York state Working Families Party. We are feeling proud of our success. We endorsed Zohran early. And tonight he got over 140,000 votes on the Working Families ballot line. He himself voted for himself on the Working Families ballot line. So, we’re ready to continue to build power to make his agenda a reality, to help all New Yorkers.
WALEED
SHAHID
:
Waleed Shahid. I’m a political strategist. I’m South Asian. I’m Muslim. I think the campaign that Zohran started was based on the fact that so many Muslim Americans, South Asian Americans, Arab Americans felt left out of the Democratic Party because of the party’s support of Benjamin Netanyahu’s war crimes. And Zohran made an effort to include those people in the Democratic Party, in the Democratic primary process, in a way that so many politicians were unwilling to do. And I think you’re seeing the results of that tonight, is that not only was it Muslims and South Asians and Arabs, but young Jews, young people of all backgrounds, wanted to see a candidate who had conviction and courage, whether it was about opposing war and genocide or standing up to the real estate lobby in this city, that they want a candidate who is consistent. And I think Zohran represents that in many ways and is like the — represents the future of a lot of what American politics is going to look like.
SHAHANA
HANIF
:
I’m Shahana Hanif, New York City councilmember representing the 39th District in Brooklyn, which includes Park Slope, Kensington. And I feel amazing.
AMY
GOODMAN
:
So, how will the City Council operate differently now with Mayor Mamdani?
SHAHANA
HANIF
:
Look, we’ll have a partner. We will have a partner who shares similar values and a progressive agenda, that has not been supported by Mayor Adams. You know, we had a mayor who consistently vetoed signature legislation that would transform New Yorkers’ lives. He vetoed ending solitary confinement in our city jails. He vetoed adding more accountability and transparency to our police force. He vetoed expanding vouchers for people who are in shelters, warehoused for years. This mayor, this new mayor, cares so deeply about the working people, the working-class people of New York City, and his agenda is more aligned with the current progressive — with the current progressive New York City Council.
KHALID
LATIF
:
My name is Khalid Latif. I’m the director of the Islamic Center of New York City.
AMY
GOODMAN
:
So, here you are. Zohran Mamdani is about to take the stage. He has won the race for mayor. He will be the first Muslim mayor. Did you ever think you’d see this day?
KHALID
LATIF
:
Yeah, you know, it’s so remarkable. I’ve known Zohran for years, and everything you see him to be and he presents himself as is who he actually is — really sincere, deep conviction, a genuine love for people. And I think for us as Muslims in New York City, with so much of the rhetoric that we’ve seen over decades, but especially ramping up into this night, for him to come and win this so quickly, and so many people from so many walks of life being here behind him tonight, just is a testament to who it is he is. It’s really remarkable. Yeah, there we go.
AMY
GOODMAN
:
Can you read that for me, what it says on the screen?
KHALID
LATIF
:
Zohran right now has over 50% of the votes, 972,000 votes in total. And it’s just going to keep coming in. He’s a remarkable young man, and New York is behind him right now.
AMY
GOODMAN
:
Did you think this was possible?
KHALID
LATIF
:
You know, I think, early on, when he started, people probably didn’t know what to expect. But as things started to go, I was there the night of the primary, and just the hope that was in the room and the sheer shock that people had that he won so quickly, I think everybody knew we were going to get to this place right now. And it’s just the start of a lot of good things.
MAMDANI
SUPPORTER
:
I!
MAMDANI
SUPPORTERS
:
I!
MAMDANI
SUPPORTER
:
I believe!
MAMDANI
SUPPORTERS
:
I believe!
MAMDANI
SUPPORTER
:
I believe that we!
MAMDANI
SUPPORTERS
:
I believe that we!
MAMDANI
SUPPORTER
:
I believe that we have won!
MAMDANI
SUPPORTERS
:
I believe that we have won! I believe that we have won! I believe that we have won!
AMY
GOODMAN
:
Some of the many supporters and organizers at Zohran Mamdani’s victory party at the Brooklyn Paramount as they learned Mamdani had won the New York mayoral race, defeating former Governor Andrew Cuomo. At mamdani’s celebration, I also had a chance to
speak
briefly with Democratic Congressmember Alexandria Ocasio-Cortez of New York.
AMY
GOODMAN
:
The new mayor of New York City, the first Muslim mayor of New York. Your thoughts?
REP
.
ALEXANDRIA
OCASIO
-
CORTEZ
:
I mean, Zohran Mamdani, of course, a historic candidate, a tremendous moment for the people of New York. We showed that we’re not going to be bullied. We’re not going to be intimidated. We’re going to fight for working families. We’re going to stand with immigrants. We’re going to stand with the diversity of this city. And we’re also going to make sure that, first and foremost, that this is a city that working people will not be displaced out of.
AMY
GOODMAN
:
What do you say to President Trump, who says he’ll withhold billions of dollars from New York, make it impossible for Zohran Mamdani to govern?
REP
.
ALEXANDRIA
OCASIO
-
CORTEZ
:
Well, you know, I think — I think that President Trump was born in New York City, and he knows that if you mess with New York, you mess with the whole country. And so, you know, I think this isn’t a city that doesn’t fight back.
AMY
GOODMAN
:
I also spoke with the Canadian journalist, author and activist and professor Naomi Klein.
AMY
GOODMAN
:
Start off by saying your name and your feelings right now.
NAOMI
KLEIN
:
My name is Naomi Klein. And I’m levitating. This is such an incredible proof of concept of how to fight fascism. You know, Zohran, immediately after Trump’s election, went out and talked to Trump voters, people who had never voted for Trump before, Black and Brown people in working-class neighborhoods, didn’t vilify them, just listened to them.
I talked to Zohran for the first time a week after Trump’s election. And what he said to me was everything is broken for people. Like, the elevator in their public housing hasn’t been fixed for 10 months. Nothing is working. So it’s so easy for someone like Trump to come along and be like, “Blame the immigrant. Blame the unhoused person.” And his entire campaign was about proving that if you actually meet people’s real needs and raise the floor and say, “OK, let’s freeze the rent. Let’s have free and fast buses. Let’s have universal child care. Let’s address that sense of scarcity and insecurity at its root,” that it can pull people back from the fascist abyss. And he won tonight. He proved that that is — that works. That message works.
This movement, this is anti-fascism, and it is also the antithesis of fascism, because fascists want everybody to be the same. They celebrate conformity, uniformity, sameness, hierarchy. Look in — New York is the most unruly city. The entire campaign was a love letter to the diversity, linguistic, faith, cultural diversity of the city, at a time when the Republicans never stop pouring hate onto cities and make people afraid of each other, right?
AMY
GOODMAN
:
And this is Zohran Mamdani on election night addressing supporters who packed into the Brooklyn Paramount. He began his speech by quoting the late labor leader and socialist Eugene Debs. But Mamdani was introduced by his own field director, Tascha Van Auken.
TASCHA
VAN
AUKEN
:
The bravery that you all have shown, that this field operation carried across all five boroughs, is going to transform our city. What all 100,400 of us have accomplished has rewritten the possibilities of mass democratic action. And it doesn’t stop tonight. We all know that we won’t stop at electing Zohran. We will continue to fight to bring the affordable of New York City to life. I am so excited to do that with all of you.
And now I’m so honored to introduce the person you’ve been waiting to hear. We’ve worked for many years to bring about change in New York City that it so urgently needs. He’s been a friend, and he is the next mayor of the greatest city on Earth, Zohran Mamdani.
MAYOR
-
ELECT
ZOHRAN
MAMDANI
:
The sun may have set over our city this evening, but, as Eugene Debs once said, I can see the dawn of a better day for humanity.
For as long as we can remember, the working people of New York have been told by the wealthy and the well-connected that power does not belong in their hands. Fingers bruised from lifting boxes on the warehouse floor, palms calloused from delivery bike handle bars, knuckles scarred with kitchen burns, these are not hands that have been allowed to hold power.
And yet, over the last 12 months, you have dared to reach for something greater. Tonight, against all odds, we have grasped it. The future is in our hands. My friends, we have toppled a political dynasty.
I wish Andrew Cuomo only the best in private life. But let tonight be the final time I utter his name, as we turn the page on a politics that abandons the many and answers only to the few.
New York, tonight you have delivered a mandate for change, a mandate for a new kind of politics, a mandate for a city we can afford and a mandate for a government that delivers exactly that.
On January 1st, I will be sworn in as the mayor of New York City. And that is because of you. So, before I say anything else, I must say this: thank you. Thank you to the next generation of New Yorkers who refuse to accept that the promise of a better future was a relic of the past. You showed that when politics speaks to you without condescension, we can usher in a new era of leadership. We will fight for you because we are you, or, as we say on Steinway,
ana minkum wa alaikum
.
Thank you to those so often forgotten by the politics of our city who made this movement their own. I speak of Yemeni bodega owners and Mexican
abuelas
, Senegalese taxi drivers and Uzbek nurses, Trinidadian line cooks and Ethiopian aunties — yes, aunties. To every New Yorker in Kensington and Midwood and Hunts Point, know this: This city is your city, and this democracy is yours, too.
This campaign is about people like Wesley, an 1199 organizer I met outside of Elmhurst Hospital on Thursday night, a New Yorker who lives elsewhere, who commutes two hours each way from Pennsylvania because rent is too expensive in this city. It’s about people like the woman I met on the Bx33 years ago, who said to me, “I used to love New York, but now it’s just where I live.” And it’s about people like Richard, the taxi driver I went on a 15-day hunger strike with outside of City Hall, who still has to drive his cab seven days a week. My brother, we are in City Hall now. …
Standing before you, I think of the words of Jawaharlal Nehru: “A moment comes, but rarely in history, when we step out from the old to the new, when an age ends, and when the soul of a nation, long suppressed, finds utterance.” Tonight, we have stepped out from the old into the new.
So let us speak now with clarity and conviction that cannot be misunderstood about what this new age will deliver and for whom. This will be an age where New Yorkers expect from their leaders a bold vision of what we will achieve, rather than a list of excuses for what we are too timid to attempt.
Central to that vision will be the most ambitious agenda to tackle the cost-of-living crisis that this city has seen since the days of Fiorello La Guardia, an agenda that will freeze the rents for more than 2 million rent-stabilized tenants. …
Together, we will usher in a generation of change. And if we embrace this brave new course, rather than fleeing from it, we can respond to oligarchy and authoritarianism with the strength it fears, not the appeasement it craves. After all, if anyone can show a nation betrayed by Donald Trump how to defeat him, it is the city that gave rise to him. And if there is any way to terrify a despot, it is by dismantling the very conditions that allowed him to accumulate power.
This is not only how we stop Trump, it’s how we stop the next one. So, Donald Trump, since I know you’re watching, I have four words for you: Turn the volume up.
We will hold bad landlords to account, because the Donald Trumps of our city have grown far too comfortable taking advantage of their tenants. We will put an end to the culture of corruption that has allowed billionaires like Trump to evade taxation and exploit tax breaks. We will stand alongside unions and expand labor protections, because we know, just as Donald Trump does, that when working people have ironclad rights, the bosses who seek to extort them become very small indeed.
New York will remain a city of immigrants, a city built by immigrants, powered by immigrants and, as of tonight, led by an immigrant.
So, hear me, President Trump, when I say this: To get to any of us, you will have to get through all of us.
When we enter City Hall in 58 days, expectations will be high. We will meet them. A great New Yorker once said that while you campaign in poetry, you govern in prose. If that must be true, let the prose we write still rhyme, and let us build a shining city for all.
And we must chart a new path, as bold as the one we have already traveled. After all, the conventional wisdom would tell you that I am far from the perfect candidate. I am young, despite my best efforts to grow older. I am Muslim. I am a democratic socialist. And most damning of all, I refuse to apologize for any of this.
AMY
GOODMAN
:
Zohran Mamdani, speaking at his victory party on November 4th after he won the New York City mayoral race, defeating former Governor Andrew Cuomo. After his speech, Mamdani was joined on stage by his wife, Rama Duwaji, and his parents, the filmmaker Mira Nair and Columbia University professor Mahmood Mamdani. Zohran Mamdani will be sworn in on January 1st.
And that does it for today’s show.
Democracy Now!
is produced with Mike Burke, Renée Feltz, Deena Guzder, Messiah Rhodes, Nermeen Shaikh, María Taracena, Nicole Salazar, Sara Nasser, Charina Nadura, Sam Alcoff, Tey-Marie Astudillo, John Hamilton, Robby Karran, Hany Massoud and Safwat Nazzal. Our executive director is Julie Crosby. Special thanks to Becca Staley, Jon Randolph, Paul Powell, Mike Di Filippo, Miguel Nogueira, Hugh Gran, Carl Marxer, Denis Moynihan, David Prude, Dennis McCormick, Matt Ealy, Anna Özbek, Emily Andersen, Dante Torrieri and Buffy Saint Marie Hernandez. I’m Amy Goodman. Thanks so much for joining us.
The original content of this program is licensed under a
Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 United States License
. Please attribute legal copies of this work to democracynow.org. Some of the work(s) that this program incorporates, however, may be separately licensed. For further information or additional permissions, contact us.
After a teddy bear talked about kink, AI watchdogs are warning parents against smart toys
Guardian
www.theguardian.com
2025-11-28 13:00:07
Advocates are fighting against the $16.7bn global smart-toy market, decrying surveillance and a lack of regulation As the holiday season looms into view with Black Friday, one category on people’s gift lists is causing increasing concern: products with artificial intelligence. The development has ra...
As the holiday season looms into view with Black Friday, one category on people’s gift lists is causing increasing concern: products with artificial intelligence.
The development has raised new concerns about the dangers smart toys could pose to children, as consumer advocacy groups say AI could harm kids’ safety and development. The trend has prompted calls for increased testing of such products and governmental oversight.
“If we look into how these toys are marketed and how they perform and the fact that there is little to no research that shows that they are beneficial for children – and no regulation of AI toys – it raises a really big red flag,” said Rachel Franz, director of Young Children Thrive Offline, an initiative from
Fairplay
, which works to protect children from big tech.
Last week, those fears were given brutal justification when an AI-equipped teddy bear started discussing sexually explicit topics.
The product, FoloToy’s Kumma, ran on an OpenAI model and responded to questions about kink. It suggested bondage and roleplay as ways to enhance a relationship, according to a report from the Public Interest Research Group (Pirg), the consumer protection organization behind
the study
.
“It took very little effort to get it to go into all kinds of sexually sensitive topics and probably a lot of content that parents would not want their children to be exposed to,” said Teresa Murray, Pirg consumer watchdog director.
Products like the teddy bear are part of a global smart-toy market valued at $16.7bn in 2023, according to
Allied Market Research
.
The industry is particularly big in China, which has more than 1,500 AI toy companies, which are working to expand abroad,
MIT Technology Review
reports.
In addition to the Shanghai-based startup FoloToy, Curio, a California-based company, works with OpenAI and makes a
stuffed toy, Grok
, as in Elon Musk’s chatbot, voiced by the musician Grimes. In June, Mattel, which owns brands like Barbie and Hot Wheels,
also announced
a partnership with OpenAI to “support AI-powered products and experiences”.
Before the Pirg report on the creepy teddy bear, parents, technology researchers and lawmakers had already raised concerns about the impact of bots on minors’ mental health. In October, the chatbot company Character.AI announced that it would ban users under 18 following
a lawsuit
alleging that its bot exacerbated a teen’s depression and caused him to kill himself.
Murray said AI toys could be particularly dangerous because whereas earlier smart toys provided children-programmed responses, a bot can “have a free-flowing conversation with a child and there are no boundaries, as we found”.
That could not only lead to sexually explicit conversations, but children could become attached to a bot rather than a person or imaginary friend, which could hurt their development, said Jacqueline Woolley, director of the Children’s Research Center at the University of Texas at Austin.
For example, a child can benefit from having a disagreement with a friend and learning how to resolve it. That is less likely to happen with bots, which are
often sycophantic
, said Woolley, who consulted on the Pirg study.
“I worry about inappropriate bonding,” Woolley said.
Companies also use the AI toys to collect data from children and have not been transparent about what they are doing with that information, Franz, of Fairplay, said. That potentially puts users at risk because of a lack of security around such data, Franz said.
Hackers have
been able to take control of AI products.
“Because of the trust that the toys engender, I would say that children are more likely to tell their deepest thoughts to these toys,” Franz said. “The surveillance is unnecessary and inappropriate.”
Despite such concerns, Pirg is not calling for a ban on AI toys, which could have educational value, like helping children learn a second language or state capitals, Murray said.
“There is nothing wrong with having some kind of educational tool, but that same educational tool isn’t telling you that it’s your best friend, that you can tell me anything,” Murray said.
The organization is calling for additional regulation of these toys for children under 13 but has not made specific policy recommendations, Murray said.
There also needs to be more independent research conducted to ensure the products are safe for children and, until that is done, they should be pulled from shelves, Franz said.
“We need short-term and longitudinal independent research on the impacts of children interacting with AI toys, including their social-emotional development” and cognitive development, Franz said.
Following the Pirg report, OpenAI announced it was suspending FoloToy. That company’s CEO then
told CNN
that it was pulling the bear from the market and “conducting an internal safety audit”.
On Thursday, 80 organizations, including Fairplay, issued
an advisory
urging families not to buy AI toys this holiday season.
“AI toys are being marketed to families as safe and even beneficial to learning before their impact has been assessed by independent research,” the release states. “By contrast, offline teddy bears and toys have been proven to benefit children’s development with none of the risks of AI toys.”
Curio, maker of the Grok toy, told the Guardian in an email that after reviewing the Pirg report, “we are actively working with our team to address any concerns, while continuously overseeing content and interactions to ensure a safe and enjoyable experience for children”.
Mattel stated that its first products with OpenAI “will focus on families and older customers” and that “use of OpenAI APIs is not intended for users under 13”.
“AI complements–not replaces–traditional play, and we are emphasizing safety, privacy, creativity and responsible innovation,” the company stated.
Franz referred to past
privacy concerns
with Mattel’s
smart products
and said: “It’s good that Mattel is claiming that their AI products are not for young kids, but if we look at who plays with toys and who toys are marketed to, it’s young children.”
Franz added: “We’re very interested in learning the concrete steps Mattel will take to ensure their OpenAI products are not actually used by kids who will surely recognize and be attracted to the brands.”
A friend made me aware of
a reading list
from
A16Z
containg recommendations for books, weighted towards science fiction since that’s mostly what people there read. Some of my books are listed. Since this is the season of Thanksgiving, I’ll start by saying that I genuinely appreciate the plug! However, I was taken aback by the statement highlighted in the screen grab below:
“…most of these books don’t have endings (they literally stop mid-sentence).”
I had to read this over a few times to believe that I was seeing it. If it didn’t include the word “literally” I’d assume some poetic license on the part of whoever, or whatever, wrote this. But even then it would be crazy wrong.
I’m not surprised or perturbed by the underlying sentiment. Some of my endings have been controversial for a long time. Tastes differ. Some readers would prefer more conclusive endings. Now, in some cases, such as Snow Crash, I simply can’t fathom why any reader could read the ending—a long action sequence in which the Big Bad is defeated, the two primary antagonists meet their maker and Y.T. is reconciled and reunited with her mother—as anything other than a proper wind-up to the story. In other cases, notably The Diamond Age and Seveneves, I can understand why readers who prefer a firm conclusion would end up being frustrated. It is simply not what I was trying to do in those books. So, for a long time, people have argued about some of my endings, and that’s fine.
In this case, though, we have a big company explicitly stating that several of my best-known books just stop mid-sentence, and putting in the word “literally” to eliminate any room for interpretive leeway.
This isn’t literary criticism, which consists of statements of opinion. This is a factual assertion that is (a) false, (b) easy to fact-check, and (c) casts my work ethic, and that of my editors, in an unflattering light.
It is interesting to speculate as to how such an assertion found its way onto A16Z’s website!
By far the most plausible explanation is that this verbiage was generated by an AI and then copy-pasted into the web page by a human who didn’t bother to fact-check it. This would explain the misspelling of my name and some peculiarities in the writing style. Of course, this kind of thing is happening all the time now in law, academia, journalism, and other fields, so it’s pretty unremarkable; it just caught my attention because it’s the first time it’s directly affected me.
The flow diagram looks like this:
That does a pretty good job of explaining how this all might have come about. So far so good. But it raises interesting questions about what happens next: the faulty quote from this seemingly authoritative source in turn gets ingested by the next generation of LLMs, and so on and so forth:
A hundred years from now, thanks to the workings of the Inhuman Centipede, I’m known as a deservedly obscure dadaist prose stylist who thought it was cool to stop his books mid-sentence.
In this scenario, which seems more far-fetched, we have a sincere and honest human writer who is reporting what they believe to be true based on false information. It breaks down into two sub-hypotheses:
There are bootleg copies of countless books circulating all over the Internet, and have been for decades
1
. Very often these are of poor quality. It could be that the person (or the AI) who wrote the above excerpt decided to save some money by downloading one of those, and got a bad copy that was cut off in mid-sentence.
Even in the legit publishing industry, the quality of translations can be quite variable, and it’s difficult for authors to know whether a given translation was any good. I’ve seen translated editions of some of my books that look suspiciously short on page count. For all I know there might be translations of my books (legit or bootleg) that actually do stop mid-sentence!
I genuinely am grateful to have been included on this list! But I had to say something about this astonishing howler embedded in the otherwise reasonable verbiage.
Even the most cynical and Internet-savvy among us are somehow hard-wired to take anything we read on the Internet at face value. I’m as guilty as the next person. This has been a bad idea for a long time now, since bad actors have been swarming onto the Internet for decades. Now, though, it’s a bad idea for a whole new reason: content we read on the Internet might not have been written by a person with an intent to misinform, but rather by an LLM with no motives whatsoever, and no underlying model of reality that enables it to determine fact from fiction.
Discussion about this post
The mysterious black fungus from Chernobyl that may eat radiation
Mould found at the site of the Chernobyl nuclear disaster appears to be feeding off the radiation. Could we use it to shield space travellers from cosmic rays?
In May 1997, Nelli Zhdanova entered one of the most radioactive places on Earth – the abandoned ruins of Chernobyl's exploded nuclear power plant – and saw that she wasn't alone.
Across the ceiling, walls and inside metal conduits that protect electrical cables, black mould had taken up residence in a place that was once thought to be
detrimental
to life.
Like plants reaching for sunlight, Zhdanova's research indicated that the fungal hyphae of the black mould seemed attracted to ionising radiation
The mould – formed from a number of different fungi – seemed to be doing something remarkable. It hadn't just moved in because workers at the plant had left. Instead,
Zhdanova
had found in previous surveys of soil around Chernobyl that the fungi were actually growing
towards
the radioactive particles that littered the area. Now, she found that they had reached into the original source of the radiation, the rooms within the exploded reactor building.
With each survey taking her close to harmful radiation, Zhdanova's work has also overturned our ideas about how radiation impacts life on Earth. Now her discovery offers
hope of cleaning up radioactive sites
and even provide ways of protecting astronauts from harmful radiation as they travel into space.
Eleven years before Zhdanova's visit, a routine safety test of reactor four at the Chernobyl Nuclear Power Plant had
quickly turned into the world's worst nuclear accident
. A series of errors both in the design of the reactor and its operation led to a huge explosion in the early hours of 26 April 1986. The result was a single, massive release of radionuclides. Radioactive iodine was
a leading cause of death
in the first days and weeks, and, later, of cancer.
In an attempt to reduce the risk of radiation poisoning and long-term health complications, a 30km (19 mile) exclusion zone – also known as the "
zone of alienation" – was established
to keep people at a distance from the worst of the radioactive remains of reactor four.
But while humans were kept away, Zhdanova's black mould had slowly colonised the area.
Germán Orizaola/Pablo Burraco
Ionising radiation may have led tree frogs inside the Chernobyl exclusion zone to have darker skin (left) than those outside it (right) (Credit: Germán Orizaola/ Pablo Burraco)
Like plants reaching for sunlight, Zhdanova's research indicated that the fungal hyphae of the black mould seemed
attracted
to ionising radiation. But "radiotropism", as Zhdanova called it, was a paradox: ionising radiation is generally far more powerful than sunlight, a barrage of radioactive particles that
shreds through DNA and proteins
like bullets puncture flesh. The damage it causes can trigger harmful mutations, destroy cells and kill organisms.
At the centre of this story is a pigment found widely in life on Earth: melanin. This molecule, which can range from black to reddish brown, is what leads to different skin and hair colours in people. But it is also the reason why the various species of mould growing in Chernobyl were black. Their cell walls were
packed
with melanin.
Just as darker skin protects our cells from ultraviolet (UV) radiation, Zhdanova suspected that the melanin of these fungi was
acting as a shield
against ionising radiation.
Just as those black moulds colonised an abandoned world at Chernobyl, perhaps they could one day protect our first steps on new worlds elsewhere in the Solar System
It wasn't just fungi that were harnessing melanin's protective properties. In the ponds around Chernobyl, frogs with higher concentrations of melanin in their cells, and so darker in colour, were
better able to survive and reproduce
, slowly turning the local population living there black.
In warfare, a shield might protect a soldier from an arrow by deflecting the projectile away from their body. But melanin doesn't work like this. It isn't a hard or smooth surface. The radiation – whether UV or radioactive particles – is
swallowed by its disordered structure
, its energy
dissipated rather than deflected
. Melanin is
also an antioxidant
, a molecule that can turn the reactive ions that radiation produces in biological matter and return them to a stable state.
In 2007, Ekaterina Dadachova, a nuclear scientist at the Albert Einstein College of Medicine in New York, added to Zhdanova's work on Chernobyl's fungi, revealing that their growth wasn't just directional (radiotropic) but actually
increased
in the presence of radiation. Melanised fungi, just like those inside Chernobyl's reactor, grew 10% faster in the presence of radioactive Caesium compared to the same fungi cultured without radiation, she found. Dadachova and her team also found that the melanised fungi that were irradiated appeared to be using the energy to help drive its metabolism. In other words, they were using it to grow.
Elsevier/ Zhdanova et al. 2000
Cultures found in the fourth unit at Chernobyl, including Cladosporium sphaerospermum. The top right dish clearly shows melanisation (Credit: Elsevier/ Zhdanova et al. 2000)
Zhdanova had suggested that these fungi could be harnessing the energy from radiation, and now Dadachova's research appeared to be building on this. These fungi weren't just growing towards radiation for warmth or some unknown reaction between radiation and its surroundings as Zhdanova had suggested. Dadachova believed the fungi were actively feeding on the radiation's energy. She called this process "
radiosynthesis
". And melanin was central to the theory.
"The energy of ionising radiation is around one million times higher than the energy of white light, which is used in photosynthesis," says Dadachova. "So you need a pretty powerful energy transducer, and this is what we think melanin is capable of doing – to transduce [
ionising radiation
] into usable levels of energy."
Radiosynthesis is still just a theory, as it can only be proven if the precise mechanism between melanin and metabolism is discovered. Scientists would need to find the exact receptor – or a particular nook in melanin's convoluted structure – that is involved in converting radiation into energy for growth.
In more recent years, Dadachova and her colleagues have started to identify some of the
pathways
and
proteins
that might underlie the fungi's increase in growth with ionising radiation.
Not all melanised fungi show a tendency for radiotropism and positive growth in the presence of radiation. A 2006
study
from Zhdanova and her colleagues, for example, found that only nine of the 47 species of melanised fungi they collected at Chernobyl grew towards a source of radioactive caesium (caesium-137).
Similarly, in 2022, scientists at Sandia National Laboratories in New Mexico
found no difference in growth
when two species of fungi (one melanised, one not) were exposed to UV radiation and caesium-137.
Different from the radioactive decay found at Chernobyl, so-called
galactic cosmic radiation
is an invisible storm of charged protons, each travelling near the speed of light through the Universe. Originating from exploding stars outside our solar system, it
even passes through lead without much trouble
. On Earth, our atmosphere largely protects us from it but for astronauts travelling into deep-space it has been called "
the greatest hazard
" to
their health
.
But even galactic cosmic radiation was no problem for samples of
Cladosporium sphaerospermum
, the same strain that Zhdanova found growing throughout Chernobyl, according to a study that
sent these fungi to the International Space Station
in December 2018.
"What we showed is that it grows better in space," says Nils Averesch, a biochemist working at the University of Florida and co-author of the study.
Nils Averesch/ Aaron Berliner
The Cladosporium sphaerospermum isolate from Chornobyl, grown on a potato dextrose agar plate, shows very high melanisation (Credit: Nils Averesch/ Aaron Berliner)
Compared to control samples back on Earth, the researchers found that fungi that faced the galactic cosmic radiation for 26 days grew an average 1.21 times faster.
Even so, Averesch is still unconvinced that this is because
C. sphaerospermum
was harnessing the radiation in space. The increased levels of growth could also have been the result of zero gravity, he says, another factor that fungi back on Earth didn't experience. "Averesch is now conducting experiments using a
random positioning machine
that simulates zero gravity here on Earth to parse these two possibilities.
But Averesch and his colleagues also tested the protective potential of the melanin in
C. sphaerospermum
by putting a sensor underneath a sample of the fungi aboard the International Space Station. Compared to samples without fungi, the amount of radiation blocked increased as the fungi grew, and even a smear of mould in a petri dish seemed to be an effective shield.
"Considering the comparatively thin layer of biomass, this may indicate a profound ability of
C. sphaerospermum
to absorb space radiation in the measured spectrum," the researchers
wrote
.
Averesch says it's still possible the apparent radioprotective benefits of fungi are due to components of biological life other than melanin. Water, for example, a molecule with a high number of protons in its structure (eight in oxygen and one in each hydrogen), is one of the
best ways
to protect against the protons that zoom through space, an astrobiological equivalent of fighting fire with fire.
Metal and glass present a similar problem. Lynn J Rothschild, an astrobiologist at Nasa's Ames Research Centre, has likened transporting these materials into space to build space bases to a turtle carrying its shell everywhere it goes. "[It's] a reliable plan, but with huge energy costs," she
said in a 2020 N
asa
release
.
Her research has
led to fungal based furniture and walls
that could be grown on the Moon or Mars. Not only would such "myco-architecture" reduce the cost of lift-off, but – if the findings from Dadachova and Averesch prove correct – it could also be used to form a radiation shield, a self-regenerating barrier between the space-faring humans and the storm of galactic cosmic radiation outside.
Organizations’ demands for “zero-CVE” (Common Vulnerabilities and Exposures) software inspired the creation of Project Hummingbird, a new catalog of minimal, hardened images built to empower developers and ease security and compliance concerns.
We will choose our most requested images, ship them free of known CVEs, and harden them to help keep them that way. Clean vulnerability reports only matter if these new container images deliver what organizations actually need and can be maintained. That’s why we want feedback based on your experience trying these images.
Sign up to explore Project Hummingbird
when the early access program becomes available in December.
Purpose for Project Hummingbird
Right foundation
Give developers the right software foundations with a smaller attack surface.
Verified content
Enable users to verify the complete contents of an image with validated software bill of materials (SBOMs) to meet modern compliance requirements.
Sustainable images
Use images that are genuinely useful and stable, knowing they're shipped free of known vulnerabilities and have already passed functionality testing.
Project Hummingbird can give customers the speed and minimalism they need to innovate, backed by the trust and support they expect from Red Hat.
Charles Darnay observed that the gate was held by a mixed guard of
soldiers and patriots, the latter far outnumbering the former; and that
while ingress into the city for peasants’ carts bringing in supplies,
and for similar traffic and traffickers, was easy enough, egress, even
for the homeliest people, was very difficult.
—
Complaining
about egress fees goes back to at least the French Revolution.
Some time ago we
overhauled
TigerBeetle’s routing algorithm
to better handle varying network
topologies in a cluster. That turned out to be an interesting case study
of practical generative testing (or fuzzing) for non-trivial, real-world
code. We ended up adding not one, not even two, but four very different
new fuzzers to the system! Let’s talk about why just one fuzzer is not
enough.
This is a good moment to brew some tea, the journey will take us
awhile!
Although this post isn’t primarily about the new algorithm itself,
we’ll start by covering the basics of replication. TigerBeetle provides
transaction Atomicity, Consistency, Isolation and Durability (ACID). Out
of the four letters, the D, Durability, is the most consequential. For,
without Durability, there wouldn’t be any data at all to provide
guarantees for!
You can get a decent chunk of durability by writing the data to a
(single) hard drive. This works for many non-critical applications, but
might still fail if you repeat the procedure often enough. Disks are
faulty with non-zero probability, and it is fairly common to lose an
entire machine (floods, fires and tripping over the power supply
happen). If you
really
want your data to be durable, better to
store several copies of it on different machines, to replicate.
All data in TigerBeetle is ultimately derived from an append-only
hash-chained log of prepare messages, so the task of replication reduces
to distributing the prepares (a MiB each) across the six replicas of the
cluster.
The primary sends
.prepare
messages down to the backups;
they reply
.prepare_ok
back up once the prepare is locally
durable. When the primary receives a quorum of
.prepare_ok
s, it knows that the message is globally
durable.
The most straightforward way to implement that is for the primary to
broadcast the prepare:
Star Topology
The problem with this approach is that the primary uses 5x the
bandwidth of the backup. In other words, we are going only at 1/5th of
the optimal performance. For this reason, our V1 routing used a simple
ring topology, where most replicas need to send and receive one
message:
Ring Topology
The ring replication is simple and balances the bandwidth nicely. It
served well for the first year of production use, despite some critical
issues!
First
, the fixed ring topology falls prey to one of the
eight
fallacies of distributed computing
. The ring is fully static, and
assumes that network topology doesn’t change. But this is not true. For
example, if one replica crashes or becomes partitioned, it is a good
idea to proactively route around it, rather than rely on retries to
randomly pick a different replica.
Second
, the ring doesn’t have what I like to call “there’s
no (re)try” property. Most messages exchanged in the process of ring
replication are
critical
: if a single message is lost, then the
whole chain of replication unravels until the retry timeout kicks in.
This means that network errors are visible as elevated P100 latencies
(bad), and, when they happen, we have to run rarely-executed retry code
(worse!). Such “cold code” is the preferred habitat for bugs! Ideally, a
system should have built-in redundancy such that any operation completes
without tripping timeouts even in the presence of errors.
Thus, Adaptive Replication Routing (or, how we affectionately call
it, ARR) was born. It combines two ideas.
First
, while we keep
the ring as our replication topology, we place the primary into the
middle:
ARR Topology
The small downside is slightly uneven network load, as the primary
sends two messages. The big upside is that none of the messages are
critical. If any single message is dropped, the prepare is still going
to be replicated to at least half of the cluster, allowing the primary
to commit without tripping timeouts (recall that TigerBeetle is using
Heidi Howard’s Flexible
Quorums
, so 3 of 6 as replication quorum is enough for safety
because the view change quorum, is 4 of 6, preserving the intersection
property).
The second trick
is that the ring itself is dynamic. At
runtime, the cluster picks the order of replicas that minimizes the
latency overall. If one replica becomes unreachable, the replicas are
reshuffled along the ring to move the missing one to the very end.
How do you find the best route? One approach is to build a model of
the system. For example, replicas can exchange heartbeat messages, note
pairwise latencies, and then solve traveling salesman problem in the
resulting small six-node graph to find the most perfect route.
This works algorithmically, but relies on a pretty big assumption —
that our model of the world is faithful. But imagine, for example, a
network with a link with very low latency, but also very low throughput.
Using (small) heartbeat messages to measure the link quality would give
us a misleading model that breaks down for (much larger) prepares.
The problem here isn’t this particular case, but the entire class of
“out of the distribution” errors which make any
indirect
measurement suspect (c.f.
Goodhart’s
Law
). As another example, consider a replica with a very slow disk.
Although the
ping
time for it is very fast, the replication
is going to be slow, as
.prepare_ok
is only sent once the
.prepare
is durably persistent. Pings only measure network
latency, but we also care about storage latency (and throughput).
A different approach, inspired laterally by the
PCC
paper, is to avoid modeling altogether, and instead to just go and
do
something, and then measure the relevant result directly,
Grace Hopper style. This is how ARR works: for every
.prepare
, the primary tracks how long did it take to
replicate (via tracking
.prepare_ok
messages). Every once
in a while, it runs an experiment, where a prepare follows a different,
experimental route. If that experimental route is
measured
to
be better than the route we are currently using, the topology is
switched. Over time, the cluster converges to the optimal route.
That’s ARR in a nutshell: replication topology is a ring with the
primary in the middle, where the order of replicas in the ring is
adjusted dynamically based on how well each specific permutation
performs end-to-end.
As promised, this post is
not
about ARR, so assume that
you’ve already implemented ARR for TigerBeetle. How would you apply
Deterministic Simulation Testing principles to it?
One approach is to leverage our existing
game
whole-system
simulation, VOPR. This actually gets you quite far, but it is always
possible to do better.
First
, whole system simulation might not be as efficient at
exercising deeper layers of the system. For every permutation of events
affecting the target layer, the simulator also needs to handle all other
events above and below. Furthermore, the permutations you get might be
restricted by the way the subsystem is used by the larger system. In
other words, the routing component might be working correctly if used in
the exact same way as in the real database, but it might still have bugs
under certain interactions of its public APIs.
Second
, while checking “it doesn’t crash” is easy enough
through the VOPR, asserting that the route is
good
is much
harder. Again, there’s just too much
other
stuff happening to
focus just on the contribution of routing.
That’s why the general principle in TigerBeetle is that, in addition
to the main whole-system fuzzer, each subsystem should also have a
targeted fuzzer, and ARR is no exception.
There’s a fairly general recipe for how to fuzz a subsystem in
isolation:
Identify all the connections between the target and the rest of the
system,
abstract the connections behind an interface,
supply a stub implementation for fuzzing.
With some ingenuity, you can even avoid modifying your source code at
all, instead leveraging runtime support to materialize interfaces out of
thin air. For example, you can use
LD_PRELOAD
tricks to
intercept all libc-mediated syscalls.
But there’s a catch! With a large and intricate interface, it might
be challenging to thoroughly explore the state space, especially as the
interface itself changes over time (and large and intricate things also
mysteriously
love
to be high-churn as well). For the long term,
it pays to start with the minimal possible interface.
Did you notice that I tricked you in the first paragraph in this
section? You don’t first build a system, and then add a fuzzer. The
process is almost the reverse — the starting point is sketching minimal
interfaces that yield themselves to efficient fuzzing. This is a bit
like Test Driven Design, though, not exactly. There’s relatively little
incrementality and iteration. Instead, fuzzer’s input on architecture is
felt at the very beginning, during “sketching on the mental napkin”
phase.
Let’s do this for ARR. Again, the idea is that we observe timing
information during replication (the delay between sending
.prepare
and receiving a set of
.prepare_ok
s)
and use that to gradually discover the best possible route, where the
route is a permutation of replicas with the current primary in the
middle.
Note how
little
in the above description is related to
TigerBeetle! This is a hint that the routing component
can
be
fully independent! This is the core interface of
Routing
(simplified for the blog, but just a touch):
To start routing, you need to know how many replicas are there, and
which one is you.
op_prepare
and
op_prepare_ok
are for tracking timing. The contract is simple:
op_prepare
is called once a
.prepare
is ready to be replicated, and
op_prepare_ok
is called for every received
.prepare_ok
response. That is, the happy path is six calls
to
op_prepare_ok
for every call to
op_prepare
.
The
op_next_hop
is the actual routing — it tells which
replicas a freshly received prepare needs to be forwarded to. It might
return zero, one or two replicas.
Finally, routing needs to know which replica is the primary. When the
primary changes, so do the routes! The primary is uniquely defined by
the view, which can be changed via the
view_change
method.
Note how minimal this interface is! We are just passing integers
around (
Instant
is a newtyped
u64
. It’s a
topic for a separate blog post why we newtype
Instant
but
not
view
…). But this is not natural, it’s a result of
deliberate design process!
For example,
Routing
routes
Prepare
s, so it
would be natural to pass in the whole
Prepare
structure
with all dependencies on the rest of the VSR framework. It takes
intellectual
control
to know that
Routing
only cares about
Prepare
’s identity, and that op number is a concise
representation of that identity.
Handling of time deserves an entire separate article (good thing that
we
did
write
that up
). The source of time is ultimately a
Clock
instance, so the most natural thing to do would be to inject
Clock
dependency in the constructor. But a moment’s
thinking makes you realize that a fully general clock is unnecessary. We
only care about the time difference between a
.prepare
and
the corresponding
.prepare_ok
s, which you can get, simply,
by accepting an
Instant
— a
u64
number of
nanoseconds since an unspecified start of the epoch. This is a
major
simplification for fuzzing, as time is notoriously tricky
to model, and here we get it essentially for free.
Finally, in order to downsize the interface,
view_change
violates one of the best best practices. It adds the second source of
truth for the view number! The authority about the current view is the
Replica
struct (
this
lovely 12k sloc file
) with a
view: u32
field.
Routing
needs to be aware of the view, and the most
straightforward way to do that is to inject the entire
Replica
in
init
, using banana-gorilla-jungle
pattern of Joe Armstrong. The textbook fix would be to abstract “thing
with a
get_view
method” behind an interface and inject
that
. But that indirection makes the code more verbose and
harder to reason about. It also is not enough: not only
Routing
needs to know the current view, it must actively
react to changes in the view! This can be fixed via Observer pattern,
but Observer is notorious for destroying readability of control flow and
bring a host of problems of its own, including complicated lifetime
management, non-deterministic order of execution and potential for
feedback loops.
It indeed is
much
simpler to just let
Routing
have its own private copy of
view: u32
. And the risk of
views desynchronizing is easy to mitigate. We already have
invariants
method on the
Replica
which is
called frequently to catch various violations, and it can check view
consistency as well:
You get the idea! The trick to making the code more easily fuzzable
is to minimize the interface. You want to get rid of accidental
dependencies and leave only the essential ones. And to do that, it helps
to apply data-oriented design principles — thinking in terms of input
data, output data, and the fundamental data transformation that the
system implements.
When the primary decides to switch the route after a successful
experiment, it needs to communicate the new route to the peers. It’s a
serialization/deserialization task. As there are at most six replicas in
the cluster, and a route is a permutation thereof, a route is encoded
compactly as an
u64
:
Serialization is a favorite vehicle for explaining property based
testing: checking that serializing data and then deserializing it back
doesn’t lose a bit is an obvious thing to do
(
deserialize . serialize == id
, if you speak pointfree). So
we can generate a random permutation and assert that it round-trips the
encoding correctly:
test route_encode {var prng = stdx.PRNG.from_seed(std.testing.random_seed);const replica_count = prng.range_inclusive(u8,1, constants.replicas_max);// Start with a trivial permutation, then shuffle it.var route: Route =.trivial(replica_count); prng.shuffle(u8,&route.replicas);const code = route_encode(route);const route_decoded = route_decode(code).?; assert(std.meta.eql(route, route_decoded));}
And here’s
shuffle
for the reference, nothing fancy:
This already is a decent test, but we can make it even better. There
are at most six replicas in a cluster. That means there are
1! + 2! + ... + 6!
routes in
total
we need to
check. This is a tiny number of routes, computer-wise, and we can easily
check them all in no time!
The only catch is that writing code to generate all permutations
needs somewhat tricky recursion, and then you need to also iterate over
number of replicas… But there’s a secret cheat code here. This is
it:
If you wrote a function that takes a PRNG and generates a random
object, you
already
have a function capable of enumerating all
objects.
Just imagine how the above function executes, from the perspective of
the PRNG. You are constantly being asked to generate random numbers,
which are used to shuffle the initial identity permutation. But what if
you always return zero? Well, the resulting permutation will be in some
sense trivial! And you can get the
next
permutation if you
change the last zero to be one. And then two. And, if, say, the last
number you are asked to generate needs to lie between zero and two, then
after two you wrap back to zero, but also increment the penultimate
number:
0 0 0 0 0
0 0 0 0 1
0 0 0 0 2
0 0 0 1 0
0 0 0 1 1
If you can generate all sequences of random numbers, you can turn a
function generating a random object into a function enumerating
all
objects! And here’s how you can generate all random number
sequences:
started:bool=false,v: [32]struct { value:u32, bound:u32 } =undefined,p:usize=0,p_max:usize=0,const Gen =@This();pubfn done(g:*@This()) bool {if (!g.started) { g.started =true;returnfalse; }var i = g.p_max;while (i >0) { i -=1;if (g.v[i].value < g.v[i].bound) { g.v[i].value +=1; g.p_max = i +1; g.p =0;returnfalse; } }returntrue;}fn gen(g:*Gen, bound:u32) u32 { assert(g.p < g.v.len);if (g.p == g.p_max) { g.v[g.p] =.{ .value =0,.bound =0 }; g.p_max +=1; } g.p +=1; g.v[g.p -1].bound = bound;return g.v[g.p -1].value;}/// Public API, get a "random" number in bounds:pubfn int_inclusive(g:*Gen, Int:type, bound: Int) Int {return@intCast(g.gen(@intCast(bound)));}
Makes no sense? For me too! Every time I look at this code, I need to
solve the puzzle afresh. Luckily, there’s a write up:
Generate
All The Things
.
The bottom line is that we can
just
wrap our existing random
test into a while loop, and magically get an exhaustive test for
all
routes:
test route_encode {var prng: Gen =.{};while (!prng.done()) {const replica_count = prng.range_inclusive(u8,1, constants.replicas_max);// Start with a trivial permutation, then shuffle it.var route: Route =.trivial(replica_count); prng.shuffle(u8,&route.replicas);const code = route_encode(route);const route_decoded = route_decode(code).?; assert(std.meta.eql(route, route_decoded)); }}
That’s it! Testing
every
replica_count
, and
every permutation of replicas!
This is our first fuzzer — we test serialization by encoding and
decoding a random route. We also notice that the total amount of routes
is small, and adapt our random code to exhaustively cover the entire
positive space, using a rigged PRNG.
Testing
only
positive space is a common pitfall. We want to
check serialization and deserialization for routes. We do that by
round-tripping the route. We even make sure to check
every
possible route, how can there be anything else left to test here?
This is an example of a positive space thinking, which sometimes
gives us false confidence that everything is thoroughly tested, while we
are failing to consider some cases off the happy path.
What we missed here is that not every code necessarily encodes a
valid route. We only feed “valid” data to deserialization routine, but
who knows what bytes you can receive through the TCP socket?
Now, this is tricky: actually, TigerBeetle only talks to other
TigerBeetle replicas, and all communication is protected by a strong
checksum. So it is actually correct to assume that the encoding is
valid, modulo bugs. But there
might
be bugs! And, if there’s a
bug somewhere which manifests itself as an invalid encoding, we want to
detect that and crash loudly, rather than silently misinterpret valid
data.
That’s why the decode function returns a nullable
Route
…
This is offensive programming, we want to force bugs to jump into the
spotlight, and not to lie hidden on odd cold paths.
The most straightforward way to test the negative space here is to
run our test backwards, and to try deserialize and then serialize a
random code:
test route_decode {var prng = stdx.PRNG.from_seed(std.testing.random_seed);const code = prng.int(u64);if (route_decode(code)) |route| {const code_encoded = route_encode(route); assert(code == code_encoded); } else {// Just make sure we don't crash! }}
There’s a subtle problem with a test above — the “then” branch of the
if is dead code, and we’ll never get there, even if we repeat the test a
hundred million times:
test route_decode {var prng = stdx.PRNG.from_seed(std.testing.random_seed);for (0..100_000_000) |_| {const code = prng.int(u64);if (route_decode(code)) |_| { assert(false); } else {// Just make sure we don't crash! } }}
$ t ./zig/zig build test --release -- route_decode
real 7.14s
cpu 7.16s (7.08s user + 76.41ms sys)
rss 34.97mb
Our completely random encoding never manages to generate a valid
code!
As we have seen above, there are very few different routes, and,
therefore, very few valid encodings. But our code is
u64
.
The space of all possible codes is huge, but the subspace of all
valid
codes is very sparse.
Is this a problem? We checked all valid codes, so it’s fine if we
only look at the invalid ones? No! Given just how rarefied our encoding
space is, purely random codes are going to be
obviously
invalid. The decoding routine will reject them very quickly, and we are
likely to not exercise most of the logic there.
For effective fuzzing, you want to test the
boundary
: you
want to check a valid code, and a code which is
almost
the
same, but invalid.
For that, we
bias
our generator to prefer codes in the
neighborhood of valid encodings:
The encoding is literally a permutation of replica indexes, where
each replica index is a byte, padded by
0xFF
bytes to
u64
. We generate a random mish-mash of those bytes (which
just
might
generate a valid code), then, to spice thing up, we
randomly corrupt a single bit of code. Finally, to make sure we don’t
just
generate almost valid code, sometimes we throw everything
away and fall back to fully random.
This
sounds
plausible, but is this actually true? Do we
actually
hit the boundary here, generate both valid and invalid
codes? And how do we make sure that our negative-space fuzzer continues
to test interesting cases as the code itself evolves (it certainly looks
like we can optimize the encoding to be more compact…)?
A good pattern here is to repeat the test many times, counting all
the sad and happy cases, and assert that they are reasonable:
Due to randomness, we can’t check the exact values of counters, but
we can assert that
most
of the encodings are invalid, and that
at least some are valid (remember, our initial test generated zero valid
encodings out of 100 000 000 attempts).
This brings me to another topic I want to cover — treatment of
determinism in tests. “Thy tests shall be deterministic” is a reasonable
commandment, but not an absolute one. I see that often people try to
avoid randomness in tests at all costs, and always initialize PRNG with
a hard-coded seed of 42. I don’t like that, for two reasons.
The practical reason
is that, over its lifetime, the test is
going to be re-run many thousand times over, and it is wasteful to
not
take advantage of that to explore more of the state space
eventually, while keeping each individual test run very fast.
The purity reason
is that, if there exists a seed value that
makes the test fail, the test (or the code) is buggy and needs to be
fixed! Sure, it’s unfortunate if you discover that bug while working on
an unrelated change, but it is
less
unfortunate than not
knowing about the bug at all!
However,
just
using genuinely random seeds for tests is
pretty bad:
test route_decode {const seed = std.crypto.random.int(u64);}
The problem with the above is that, when a test fails, you don’t know
the seed! And, if it is one-in-a-million failure, it can be very a
frustrating experience to reproduce it. This can be helped by printing
the seed on failure, but
that
A) requires writing more code per
test and, B) doesn’t work if the failure is not graceful. Imagine
getting a mystery segfault on some random CI run, and then not being
able to reproduce it because the process dies before the seed is
printed!
Zig I think has the best design in this space. It provides you with
the
std.testing.random_seed
value, which is a ready-to-use
random seed that is different per run. Crucially, the seed is generated
outside of the test process itself and is passed to it on the CLI. It
doesn’t matter what happens with the test process. It can explode
completely, but the parent process will still print the seed on failure.
Conveniently, the seed is printed as a part of a CLI invocation which
you can immediately paste into your shell!
$ ./zig/zig build test
test
+- run test-vsr failure
thread 2285 panic: reached unreachable code
...
error: while executing test 'vsr.test.routing.route_decode'
error: the following command terminated with signal 6:
.zig-cache/o/14db484/test-vsr --seed=0x737929ed
So that’s why we’ve been using
PRNG.from_seed(testing.random_seed)
throughout! And it has
been working perfectly, up until now. Here’s the problem:
var prng = stdx.PRNG.from_seed(std.testing.random_seed);//...assert(counts.total == counts.valid + counts.invalid);assert(counts.valid >50);assert(counts.invalid >100_000);
The seed
is
random, so, sooner or later, our assert will
fire. We can make the probability of that negligible by increasing
total
and increasing our tolerance, but that is
unsatisfactory. Larger iteration count slows down each individual test
run. And relaxing asserts tells us
less
about the average case,
what we actually care about. And we don’t know what’s the actual
probability of hitting the assert! It might be that the actual
probability is small, but not infinitesimal, such that you’ll be
debugging a random “failure” five years from now!
One in a
billion events do happen in CI!
A nice pattern here is to run the test twice: once with a hard-coded
seed to capture the “average” distribution and assert statistics, and
once with a truly random seed for coverage:
This is our second fuzzer — testing for negative space by probing
obviously invalid values, and then specifically values that cross the
valid/invalid boundary, while collecting and asserting coverage
information.
For this particular scenario, it would’ve been better to use a real
coverage-guided fuzzer like libFuzzer, but, at the time of writing, Zig
is only at the start of its fuzzing journey. It already has
std.testing.fuzz
, but I wasn’t able to get that working on
my machine. Anyway the implementation of the fuzzer is a detail. What
matters is the principle of explicit testing of negative space, the
boundary space, and verifying that both ins and outs get tested!
Moreover, just like we got exhaustive test by driving PRNG interface
via exhaustive enumeration from inside, we can drive a PRNG through a
fuzzer. You can combine the best of both worlds: highly structured
complex inputs of property based testing and introspective guided
program state exploration of coverage-guided fuzzers. This again is
worth a separate blog post, but I really need to do more research before
it is ready. However, I’ll be sharing what I got so far on 1000x world
tour on December 3 in Lisbon next week. Come, say hello if you are
around:
https://luma.com/7d47f4et
!
Ok, the warmup is over! Serialization was a simple and boring part of
Adaptive Replication Routing. Let’s tackle the actual logic. Similarly,
we’ll start with a positive space, checking that ARR indeed converges to
the best route in a scenario approximating what we
expect
to
see in the real world.
This is going to be interesting, because it is not a
local
correctness property. We want to check that six instances of ARR on six
different physical machines work in concert, such that, e.g., everyone
agrees which operations are experiments, and what is the route of each
experiment.
Here’s the plan. We arrange six replicas into a virtual ring, such
that the network delay between replicas is proportional to the distance
along the ring. The order of replicas is random, and correctly
implemented ARR must be able to “unscramble” the permutation in the end.
Each “replica” is just a
Routing
instance. This is the
entire idea behind
The Matrix
focused fuzzing, we don’t need
to simulate anything else!
An optimal route enumerates replicas in the order of
permutation
in either of two directions (there are two
optimal routes!). We can check that by summing up pairwise
distances:
The overall flow of the fuzzer is as follows. We send prepares one by
one. For each prepare, we run the simulation until the primary collects
prepare_ok
messages from everybody.
prepare_ok_count
field tells us when we should start with
the next prepare. Submitting a prepare is modeled via sending a message
to the primary. When a set number of prepares is dealt with, we check
that the final route is optimal.
Note that this is
not
how the real replication works,
reality is pipelined, and multiple prepares are in flight at the same
time. However, the purpose of this particular fuzzer isn’t to check a
“realistic” scenario, the purpose is to check the idealized scenario,
but be very strict in the acceptance criteria (that the route really is
optimal).
The full code is a bit too much for this article, but the core logic
of simulating replication process is this
message_delivered
function. It models what happens when replica
target
receives a
message
from
source
. Which is,
forward the message along the ring, and reply with
.prepare_ok
to the primary.
fn message_delivered( t:*T, source:u8, target:u8, message:union(enum) { prepare:u64, prepare_ok:u64 },) void {switch (message) {.prepare =>|op| {// The initial prepare is injected by the fuzzer.if (target == t.primary) { assert(source == t.primary); assert(t.prepare_ok_count ==0); t.replicas[t.primary].op_prepare(op, t.now()); }// Inform the primary that we got the prepare. t.send(.{.source = target,.target = t.primary,.message =.{ .prepare_ok = op }, });// Forward prepare along the current replication ring.for (t.replicas[target].op_next_hop(op)) |target_next| { assert(target_next < t.replica_count); t.send(.{.source = target,.target = target_next,.message =.{ .prepare = op }, }); } },.prepare_ok =>|op| { assert(target == t.primary); t.prepare_ok_count +=1; t.replicas[t.primary].op_prepare_ok(op, t.now(), source); }, }}
What’s fascinating about this fuzzer is not the implementation, but
rather the bugs it was able to find. Writing the fuzzer was a relatively
mechanical and mindless process, other than the initial idea of modeling
a physical ring of replicas. But the two failures it found revealed my
misunderstanding of the problem, and forced me to apply deeper thinking
where I thought I understood everything.
To explain that, I need to talk about the ARR cost function. After an
ARR experiment, the primary somehow needs to measure the quality of a
the experimental route. The
data
we have are
.prepare_ok
latencies for all replicas — a vector of six
integers.
My initial cost function was a pair of the median and the maximum
value of the vector, with some fuzz factor:
The median tracks the moment in time when a half of the cluster
acknowledged the prepare, which, due to flexible quorums, is the moment
where it is safe to commit prepare. The median replication time is a
proxy for user-visible latency, and it is the primary number we are
optimizing for.
After we replied to the user, we still want to replicate the prepare
to the rest of the cluster, to maximize durability. The maximum
replication time directly tracks full replication, and it’s the second
most important metric to optimize.
Finally, we don’t want the cluster to oscillate between two nearly
identical routes simply due to random delay noise, so we also add a fuzz
factor and consider close enough numbers to be equal for comparison
purposes.
Can you see the bug here? I didn’t, but the fuzzer I wrote did. After
running for a short time, the fuzzer found the case where ARR failed to
converge to the optimal path. Here’s the path that that run ended up
with:
Not Quite Optimal Route
This is indeed an optimal path in terms of median,maximum cost
function. The median is two hops, the maximum is three. But it is not
actually
optimal, because replicas
between
median and
maximum take longer time to replicate, and we care about that as well,
as that’s a proxy for us selecting the most efficient route for each
replica. It doesn’t affect important latencies, but it still sends the
electrons further away than they’d otherwise need to go.
The fix is easy — add a third component to the cost function, the sum
of all latencies.
The problem was fixed, but, after a few iterations more, I got
another example that failed to converge to an optimal route. It took me
an embarrassingly long time to debug that, but the explanation was
really simple. My fuzz factor was too fuzzy, and made two different
routes look the same. This fix also was simple, just tighten up the
“almost equal” condition.
But what bugged me is that, in my mental model, the old fuzz factor
was fuzzy enough as is. So I tried to explain
why
it didn’t
work, and realized that I had a completely wrong mental image of
replication routes. And, yes,
all
the illustrations I’ve drawn
so far also have this bug. Do you see it?
This is what the actual replication route looks like:
Replication With ACKs
Prepares flow forward along the ring, but acknowledgements always
flow directly to the primary, in a star topology. When the primary
measures the replication latency, it captures
both
the time to
send the
.prepare
forward and the time to get the
corresponding
.prepare_ok
back. And the time to receive all
.prepare_ok
is independent of the route!
In other words, changing the route can affect only half of the
observed latencies, which makes relative difference between the routes
smaller, and justifies tighter tolerances.
This was a huge shift in the mental model for me! I didn’t realize
that we only observe latencies through the glass, darkly! I hadn’t
thought about that myself, but the fuzzer did!
This is our third fuzzer. It is a whole subsystem positive space
fuzzer. It’s actually an exuberantly optimistic fuzzer, as it sets up an
ideal lab environment with extremely predictable network latencies.
While not realistic, this setup ensures that there’s a clear answer to
the question of which route is the best, and that allows us to verify
that the algorithm is exactly correct, and not merely crash free. This
is the catch — in the real system with faults and variants, the notion
of optimal route is ill-defined and constantly changes. The acceptance
criteria has to be fuzzy in a realistic simulation, but can be very
strict in the lab.
Finally, the fourth fuzzer. You might guess it, we’ll go for negative
space this time. We no longer care about how the Routing
should
be used by the replica, we are trying to break it.
The fundamental difference here is that, for positive space, we
modeled all six “replicas” at the same time messages flowing between
them. But any model of that sort necessarily restricts us to executions
possible in the cluster. Now we won’t be trying to model anything in
particular. We’ll have just a single instance of
Routing
and will be calling all public methods in random order, only obeying the
documented invariants:
// Simulate the entire cluster:const PositiveSpace =struct { replicas: []Routing,// ...};// Hammer a single replica, hard:const NegativeSpace =struct { replica: Routing,// ...};
There isn’t
much
we can check here, but we can check
something
. At minimum, we should never crash. Additionally, we
can check that whatever route we have, it “connects”. That is, if we
follow the chain of
next_hop
s, we’ll visit each replica
exactly once.
The code isn’t particularly illuminating here, but the overall shape
looks similar to the technique described in the
Swarm
Testing Data Structures
.
That’s it for today! This was a tale of four fuzzers!
Yeah… At Fuzzer #3, I realized that we actually wrote five fuzzers
for ARR, but the title and the Dickens quote had really grown on me by
that time. Sorry for this, here’s a bonus fuzzer for you!
Our positive space ARR fuzzer explores a really specific network
topology, which is roughly as far from a realistic scenario as the
negative space fuzzer, but in the opposite direction — everything’s too
good, no one’s crashing, the network gives stable latencies.
What we are missing is the realistic fuzzer between the two extremes.
A fuzzer that runs in a somewhat flaky network, and checks that the
route is roughly optimal (or at least not bad). But that is the VOPR! As
a whole system fuzzer, it is capable of simulating somewhat realistic
distributions of network faults and delays.
Historically, VOPR was biased towards faulting as much things as hard
as possible, as we want TigerBeetle to be correct and fast, in that
order. Now that we started optimization work, we implemented
--performance
mode for VOPR.
In the default mode, VOPR uses swarm testing to generate distribution
of faults (during fuzzing, you generate random events. The idea of swarm
testing is to also generate the distribution itself at random). In the
performance mode, fault parameters are fixed to “realistic” values, and
the drastic faults (replicas crashing or becoming partitioned) are
strictly controlled (e.g., you can request exactly one crash per
run):
Furthermore, in performance mode VOPR tracks statistics about the
number of network messages exchanged. ARR was verified by running
different performance VOPR scenarios with and without ARR, and checking
that ARR is an improvement across the board:
It’s a bit hard to turn these manual experiments into tests that fail
only
if there are bugs (and not due to randomness or unrelated
code choices), but just tinkering with the setup is a great way to
quickly test ideas. VOPR runs
much
faster than a real-world
cluster would, so you can use it to collect a fairly long performance
trace.
This was a long one, wasn’t it? Although it’s just one system and
five fuzzers, no two fuzzers are alike, each illuminates its own corner
of the design space. If you want a closer looks, here’s the
source
code
, it’s almost exactly a thousand lines for the implementation
plus the fuzzers.
To jolt the ideas back into the short term (and, who knows, maybe a
long term) memory:
You want both a whole system fuzzer AND subsystem (minor) fuzzers.
Main fuzzer works out the seams between components, while minor fuzzers
divide&conquerer the resulting combinatorial explosion.
Good fuzzing is tantamount to good interfaces.
Interfaces can be extracted mechanically, by introducing indirection
whenever a dependency happens.
But such a mechanical interface extraction risks ossifying
accidental dependencies.
Long-term more efficient approach is to think in terms of
fundamental input and output data. Sometimes a little copying is better
than a little dependency!
Data interfaces tend to be non-incremental. The best time to capture
an interface is
before
the first line of code is written.
Fuzz positive space and negative space.
Given a PRNG
interface
, its easy to explore structured
search space.
If the search space is small, you can use the
same
PRNG
interface to walk it thoroughly and exhaustively.
And you can plug the
same
PRNG interface into coverage
guided fuzzer.
deserialize . serialize
is positive space,
serialize . deserialize
can be negative space.
Hard to breath in rarefied air! Purely random inputs can be
uniformly boring and bounce off the edges of the system.
For negative space testing, you want to hew close to the
valid/invalid boundary, poking out from
both
sides.
You still want some amount of purely random inputs, just in
case.
You want to
assert
that both positive and negative cases
actually happen with non-negligible probability.
Run fuzzer once with a fixed seed (I use
92
), to sanity
check the count of good and bad cases.
Run fuzzer again with a genuinely random seed to accumulate coverage
over time.
Make sure to generate the seed
outside
of the test process
itself, lest it gets lost during crash.
Mind the time! You want to make each individual CI run as quick as
possible, while racking up the total fuzzing time over multiple
runs.
Another quick and dirty way to check fuzzer coverage is adding
unreachable
to various branches and check seeing if it
crashes.
Fuzzers can test fairly sophisticated invariants (e.g., optimality
of the routing), but that might require setting up a particularly
favorable environment.
Writing a fuzzer is mostly boring mechanical work. However, not only
fuzzers do find bugs, some bugs lead to large, fundamental mental
shifts, and a deeper understanding of the domain!
Don’t write fuzzers to find bugs in the code, write fuzzers to find
bugs in your understanding of the problem.
Positive space fuzzing tries to be realistic, negative space fuzzing
tries to be un-realistic.
Simulate a real cluster for the positive space, simulate a single
peer in a radioactive room for the negative space.
It might be hard to get intricate, flake-free assertions from the
whole system fuzzer.
But whole-system fuzzer is still invaluable as an exploration
tool.
You can fuzz for performance, at least on the high level protocol
level (# messages exchanged).
The Conference of Swiss Data Protection Officers, Privatim, has severely restricted the usability of international cloud services – particularly hyperscalers like AWS, Google, or Microsoft – for federal authorities in a resolution. At its core, the resolution from Monday amounts to a de facto ban on the use of these services as comprehensive Software-as-a-Service (SaaS) solutions whenever particularly sensitive or legally confidential personal data is involved. For the most part, authorities will likely only be able to use applications like the widespread Microsoft 365 as online storage.
The background
to the position
is the special responsibility of public bodies for the data of their citizens. While cloud services appear extremely attractive due to their economies of scale and dynamic resource allocation, data protection officers see significant risks in outsourcing sensitive data to international public clouds. Regardless of the sensitivity of the information, authorities must always analyze and mitigate such risks, but for particularly sensitive or confidential data in SaaS solutions from large international providers, Privatim considers outsourcing inadmissible in most cases.
The experts cite a lack of protection due to insufficient encryption and the associated loss of control as the main reasons. Most SaaS solutions do not yet offer true end-to-end encryption that would exclude the cloud provider's access to plaintext data. However, this is the central demand: The use is therefore only permissible if the data is encrypted by the public body itself and the cloud provider has no access to the key.
Concerns about Cloud Act
Another point is the low transparency of globally operating companies. Swiss authorities can hardly verify compliance with contractual obligations regarding data protection and security, it is stated. This concerns both the implementation of technical measures and the control of employees and subcontractors, who sometimes form long chains of external service providers. Compounding this is the fact that software providers periodically unilaterally adjust contract terms.
Privatim is particularly concerned about the
US Cloud Act
. This can obligate providers there to hand over customer data to national authorities, even if the data is stored in Swiss data centers. Rules of international legal assistance do not have to be observed, the controllers complain. This creates considerable legal uncertainty, especially for data subject to a duty of confidentiality.
According to lawyer Martin Steiger
most authority data is subject to a duty of confidentiality
. Furthermore, meaningful use of many cloud services with continuous encryption is hardly possible. However, it remains to be seen whether the supervisory authorities will follow their words with actions this time. Cantonal controllers had already declared the use of Microsoft 365 generally inadmissible in the past, which had hardly any consequences. Nevertheless, the resolution presents authorities with challenges regarding their IT strategy.
This article was originally published in
German
.
It was translated with technical assistance and editorially reviewed before publication.
My family’s excitement about Outer Worlds 2 was short-lived | Dominik Diamond
Guardian
www.theguardian.com
2025-11-28 12:00:08
It’s always crushing when a wildly anticipated game turns out to be a dud, but this RPG’s awful story and clunky dialogue gave my son and I something to talk about It was an exciting November for the Diamond household: one of those rare games that we all loved had a sequel coming out! The original ...
I
t was an exciting November for the Diamond household: one of those rare games that we
all
loved had a sequel coming out! The original Outer Worlds dazzled our eyeballs with its art nouveau palette and charmed our ears with witty dialogue, sucking us into a classic mystery-unravelling story in one of my favourite “little man versus evil corporate overlords” worlds since Deus Ex. It didn’t have the most original combat, but that didn’t matter: it was obviously a labour of love from a team totally invested in the telling of this tale, and we all fell under its spell.
Well, when I say all of us, I mean myself and the three kids. My wife did not play The Outer Worlds, because none of those worlds featured Crash Bandicoot. But the rest of us dug it, and the kids particularly enjoyed that I flounced away from the final boss battle after half a day of trying, declaring that I had
pretty much
completed the game and that was good enough for a dad with other things to do.
My son completed The Outer Worlds 2 first. “How was it?” I asked.
“You are going to hate it,” he replied.
What? How dare he have the arrogance to predict my gaming tastes! If it wasn’t for me, none of these urchins would have played a video game in the first place. It’s bad enough that they destroy me at Mario Kart. Now they’re robbing me of potential gaming joy. I was now determined to enjoy The Outer Worlds 2 just to prove him wrong.
Reader: I did not enjoy it.
Most of the dialogue is people moaning about their bosses … The Outer Worlds 2.
Photograph: Obsidian Entertainment
While the combat is top notch, the character skill trees sophisticated, and the speed and smoothness (on the Xbox Series X version) unmatched (especially when rushing away from combat as an ageing gamer is wont to do), the story is awful.
The first hour of play hits you with so much tedious factional politics it makes the opening crawl of The Phantom Menace look like Serious Sam. Most of the dialogue is people moaning about their bosses or their charges. Everything is broken. People are starving and miserable, bereft of doctors and medical supplies. It’s basically 2025 but in space, expressed in words so clunky and boring that it feels like reading through LinkedIn comments.
“I was right, wasn’t I?” said my son smugly, when I gave up after 20 hours, on the third planet I visited.
“How can you tell?”
“I haven’t heard you curse at a game this much since you played online Fifa.”
“How did they get it so wrong, son?” I asked.
“There’s no real heart and soul in the game. They just phoned in the story.”
Then we talked. At length. About role-playing games in general; what works and what doesn’t; what makes some great and others tedious. And we agreed that RPGs require a storyteller’s commitment to make it believable. This genre has its roots in Dungeons & Dragons, which in essence is just people sitting in a basement conjuring up great stories. If the Dungeon Master is no good, it’s just a number-crunching, dice-rolling slog but, with a storyteller in charge, it is magic. World building is crucial, too: the lush highlands of Skyrim, the dark conspiracy-poisoned streets of Deus Ex, the techno-magical dystopia of Gaia in Final Fantasy VII.
And, like tabletop D&D, the graphics don’t really matter. Decades ago, I spent a brilliant month inside the batshit crazy apocalyptic demonic faith-fest that is Shin Megami Tensei, and that whole world was conjured from tiny pixels on the screen of a Game Boy Advance.
My weak bladder and need for sleep were the only things that ever dragged me away from the inhabitants of The Witcher 3.
Photograph: CD Projekt RED
That world must have characters you care about. My weak bladder and inconvenient need for sleep were the only things that ever dragged me away from the inhabitants of The Witcher 3. But I couldn’t care less about any of the characters in The Outer Worlds 2; I felt I had seen them all before. Throw in the needlessly dense, grey dialogue and I couldn’t stay focused on the game for longer than five minutes outside battles.
In a real world where we have less control than ever, when “truth” is just what the richest liar purports and fairness has been eradicated, it’s increasingly difficult to win in life by working hard. That is what makes the genuine meritocracy of role-playing games so appealing to me. In all video games, if you have skill (or develop it), then you can progress. But in RPGs, even if you weren’t born with talent, you can work hard, grind up levels, and get more skills that lead to more rewards. In contrast to a horrible world that has 3,000 billionaires and yet still leaves most of its inhabitants living in poverty, RPGs are a model of what a just world would be like – with added armour and shields, and hopefully fast travel points.
The Outer Worlds 2 was a disappointment for me, but instead of escaping into an immersive RPG as I had hoped, I instead escaped into a glorious chat with my son about them. Again, I realised how much games have given our lives, and how they’ve deepened our relationship with each other. And I realised that sometimes a game’s terrible dialogue can spark a fascinating one in the real world.
Cats became our companions way later than you think
All domestic cats (Felis catus) are descended from the African wild cat
In true feline style, cats took their time in deciding when and where to forge bonds with humans.
According to new scientific evidence, the shift from wild hunter to pampered pet happened much more recently than previously thought - and in a different place.
A study of bones found at archaeological sites suggests cats began their close relationship with humans only a few thousand years ago, and in northern Africa not the Levant.
"They are ubiquitous, we make TV programmes about them, and they dominate the internet," said Prof Greger Larson of the University of Oxford.
"That relationship we have with cats now only gets started about 3.5 or 4,000 years ago, rather than 10,000 years ago."
Image source,
Getty Images
Image caption,
Cats were domesticated long after dogs
All modern cats are descended from the same species - the African wildcat.
How, where and when they lost their wildness and developed close bonds with humans has long puzzled scientists.
To solve the mystery, researchers analysed DNA from cat bones found at archaeological sites across Europe, North Africa and Anatolia. They dated the bones, analysed the DNA and compared this with the gene pool of modern cats.
The new evidence shows cat domestication didn't start at the dawn of agriculture - in the Levant. Instead, it happened a few millennia later, somewhere in northern Africa.
"Instead of happening in that area where people are first settling down with agriculture, it looks like it is much more of an Egyptian phenomenon," said Prof Larson.
Image source,
Ziyi Li and Wenquan Fan
Image caption,
The skull of a leopard cat found in a Han-dynasty tomb in Xinzheng City, Henan Province, China
This fits with our knowledge of the land of the pharaohs as a society that revered cats, immortalising them in art and preserving them as mummies.
Once cats became associated with people, they were moved around the world, prized as ship cats and pest controllers. Cats only reached Europe around 2,000 years ago, much later than previously thought.
They travelled around Europe and into the UK with the Romans and then started moving east along the Silk Road into China.
Today, they are found in all parts of the world, except Antarctica.
Image source,
Getty
Image caption,
The leopard cat is the most widespread wild cat in Asia
And in a new twist, the scientists discovered that a wild cat hung out for a while with people in China long before domestic cats came on the scene.
These rival kitties were leopard cats, small wild cats with leopard-like spots, that lived in human settlements in China for around 3,500 years.
The early human-leopard cat relationship was essentially "commensal" where two species live alongside each other harmlessly, said Prof Shu-Jin Luo of Peking University in Beijing.
"Leopard cats benefited from living near people, while humans were largely unaffected or even welcomed them as natural rodent controllers," she said.
Image source,
Getty
Image caption,
The Bengal cat is a breed of hybrid cat created from crossing an Asian leopard cat (Prionailurus bengalensis) with domestic cats
Leopard cats did not become domesticated and continue to live wild across Asia.
Curiously, leopard cats have recently been crossed with domestic cats to produce Bengal cats, which were recognised as a new breed in the 1980s.
Today I’m going to talk about a recent journey as a HotSpot Java Virtual Machine developer working on the OpenJDK project. While running tests for a new feature, I realized my Java objects and classes were arbitrarily disappearing! What followed was probably the most interesting debugging and fixing experience of my life (so far), which I wanted to share with the world.
This post is targeted towards a wider (computer science) audience. Knowledge about Java or the JVM is
not expected
, but a slight curiosity towards low-level programming is encouraged. My intentions with this essay are to:
introduce Project Valhalla and value objects;
give insights into the inner workings of HotSpot (& hopefully motivate folks to contribute);
pragmatically demonstrate how JVM flags can be used to help you, the developer;
teach some lessons I learned in debugging;
document processes for my future self/colleagues; and
give myself an opportunity to yap about an achievement (and draw bad ASCII art).
I’ve written
a lot
.
I’ve included summaries
for each of the sections, sans takeaways. They’re designed to be coherent, so use them to your advantage and
read what you find interesting
, although I truly believe there is value in reading the entire post.
Java is a multi-purpose programming language. It is compiled to platform-independent Java bytecode. This bytecode is then executed by a Java Virtual Machine, which performs, among other things, automatic garbage collection. The reference implementation of the JVM is called HotSpot. It features just-in-time compilation, which compiles (some) frequently executed Java methods into native code for performance gains.
Java objects reside in the heap. Each object has metadata associated with it, which is stored in the
object header
. If there’s anything I want you to take away from this, it’s that there are some bits in there that are very interesting.
Traditionally, the object header consists of the
mark word
(called
markWord
in source code) and
class word
(contains the class pointer, which is totally irrelevant to this blog post), totaling 96-128 bits on 64-bit architectures. I’ll go more in-depth about the mark word soon.
In order to reduce the size, JDK 24 introduces
JEP 450:
Compact Object Headers
1
. Essentially, they realized that it was possible to squeeze the class pointer into the mark word, making the class word redundant. Consequently, with Compact Object Headers, object headers total 64 bits on 64-bit architectures.
So, what sort of information is in the mark word? JEP 450 does a good job at
explaining the exact layout
, but for the sake of this blogpost only the eleven least-significant bits are relevant. They’re laid out on 64-bit systems as follows
2
:
7 3 0
VVVVAAAASTT
^^----- tag bits
^------- self-forwarding bit (GC)
^^^^-------- age bits (GC)
^^^^------------ Valhalla-reserved bits
Let’s
briefly
go through what these mean:
TT
represents the two tag bits, and they are used for locking. I’m not going to go into monitoring in depth. Briefly,
01
indicates the object is not locked, whereas
00
and
10
indicate lightweight
3
and monitor
3
locking, respectfully.
S
represents the self-forwarding bit. It is used by some garbage collectors, but the details are irrelevant.
AAAA
represents the age bits. Generational garbage collectors use these bits to track object tenuring.
VVVV
represents four distinct Valhalla-reserved bits (and also called
unused_gap_bits
in the implementation). More on them later.
Project Valhalla
is also dubbed “Java’s Epic Refactor” with many feature sets in development. The relevant one here is
JEP 401: Value Classes and Objects
. In short, value objects are distinguished by their fields, which allows for optimizations such as heap flattening
4
and scalarization
5
. Note that “regular classes/objects” are called
identity classes/objects
.
Valhalla is a fork of the JDK. Although we frequently pull in upstream changes, Valhalla naturally lags behind mainline development. When Compact Object Headers was merged into Valhalla, a slightly different layout of the eleven least-significant bits was chosen, such that the merge would be
less invasive
. The chosen format was as follows (contrasted with the JEP 450 layout):
7 3 0
VVVVAAAASTT <- JEP 450
VVVAAAAVSTT <- Valhalla
^------- we care about this bit, value object bit
As illustrated, the least-significant Valhalla bit resided below the age bits.
I’d like to draw attention to the
V
in position 3
6
. This is the bit that, when set, indicates whether the object is a value object.
It’s also the central concept in this post
. Feel free to disregard bits 8-10.
Why do we need this bit? HotSpot does different things for value objects and identity objects in many places (for example, the optimizations I mentioned earlier). We need a quick and easy way to check if something is a value object, hence a bit in the object header.
My change is pretty straightforward: updating the eleven least-significant bits from
VVVAAAAVSTT
to
VVVVAAAASTT
(as JEP 450 dictates). You can think of this as moving the value object bit up (or age bits down).
The mark word is part of the object header and contains metadata for Java objects. Project Valhalla introduces value objects. These are objects distinguished solely by the values of their fields. In Valhalla, one of the header metadata bits indicates whether the object is a value object. In Valhalla, this was at bit index 3, and needed to be moved to bit index 7 to comply with Compact Object Headers (JEP 450).
The change was relatively straightforward. Unfortunately, it caused 75 test failures on several architectures and platforms. These test failures were problematic because they were:
Widespread
. Many different components (mostly
outside
the VM) were affected.
Intermittent
. They would sometimes pass, and sometimes fail. As I would later find out, this also made them difficult to reproduce.
Non-obvious
. In the VM, we have a plethora of assertions. Most of the time something goes wrong, an assertion fails or the VM crashes with e.g. a segmentation fault. Here, it seems like (almost all) the issues manifested at the application level.
When it comes to debugging, that’s a really bad scenario. Considering the HotSpot codebase is ~550 KLoC
7
, not to mention extremely complex, we really want reproducible crashes in a subcomponent.
Throughout my debugging journey, three distinct symptoms were relevant. Together, they describe many of the failing test cases.
java.lang.AssertionError: l should not be null
at org.testng.ClassMethodMap.removeAndCheckIfLast(ClassMethodMap.java:55)
at org.testng.internal.TestMethodWorker.invokeAfterClassMethods(TestMethodWorker.java:193)
[...]
at com.sun.javatest.regtest.agent.MainActionHelper$AgentVMRunnable.run(MainActionHelper.java:335)
at java.base/java.lang.Thread.run(Thread.java:1474)
What’s very curious about this failure is if you check
the source code
, they actually perform a null check on the variable
l
, so it shouldn’t just disappear!
publicbooleanremoveAndCheckIfLast(ITestNGMethodm,Objectinstance){Collection<ITestNGMethod>l=classMap.get(instance);if(l==null){thrownewIllegalStateException("Could not find any methods associated with test class instance "+instance);}l.remove(m);// It's the last method of this class if all the methods remaining in the list belong to a
// different class
for(ITestNGMethodtm:l){// <- AssertionError thrown here
if(tm.getEnabled()&&tm.getTestClass().equals(m.getTestClass())){returnfalse;}}returntrue;}
A few tests within
sun/security/krb5
such as
sun/security/krb5/etype/UnsupportedKeyType.java
failed with a
NoClassDefFoundError
. The actual class that could not be found differed between tests and even between test runs of the same test. Classic intermittent shenanigans.
java.lang.NoClassDefFoundError: sun/security/krb5/internal/Krb5
at java.base/jdk.internal.loader.NativeLibraries.load(Native Method)
[...]
at java.base/java.lang.Thread.run(Thread.java:1474)
I implemented the
markWord
changes and got many widespread, intermittent, and non-obvious test failures. Crudely summarized, “things are null when they shouldn’t be.”
The lowest hanging fruit here are the failed unit tests. Not only are they written in C++, they also expose the functions they are testing. The failure was straightforward:
EXPECT_TRUE(mark.decode_pointer()==nullptr);
Where
mark
is the
markWord
of a value object. Taking a look at the source code:
// Recover address of oop from encoded form used in mark
inlinevoid*decode_pointer()const{return(EnableValhalla&&_value<static_prototype_value_max)?nullptr:(void*)(clear_lock_bits().value());}
It became clear to me that there are some Valhalla-specific shenanigans going on (by the
EnableValhalla
). But what’s a
static_prototype_value_max
?
It turns out,
static_prototype_mask
and
static_prototype_mask_in_place
are unused. The
static_prototype_max_value
in question is just all bits up to the age bits set to 1. Recall that the legacy bits are
AAAAVSLL
. This would generate a mask of
1111
. Since I moved the position of the age bits, this mask changed. Maybe that’s why the test fails?
Why do we do this? The documentation says the following about static prototypes:
// Static types bits are recorded in the "klass->prototype_header()", displaced
// mark should simply use the prototype header as "slow path", rather chasing
// monitor or stack lock races.
What are static type bits? What’s a static type? I have no clue. My colleagues said it was part of an older model of Valhalla and no longer a thing. The verdict:
delete the conditional and remove all references to static types
. I did that, and while the (now slightly altered) unit tests passed, all my application-level failures still persisted.
One of the things I’ve been taught in education and practice is that in general one should focus on understanding. While that’s certainly true, I don’t think anyone understands HotSpot, and knowledge is very compartmentalized. Small changes in one place could go against assumptions in another and cause bizarre failures.
To save myself a reasoning-induced headache, I decided to emulate the old behaviour in
decode_pointer()
by translating the current
markWord
into the legacy format and then keeping the static type check. Maybe there was some counterpart or invariant somewhere in the VM that magically made it work. If the emulated version worked, it would give me some more clues about where to poke around further.
The process of emulating wasn’t very straightforard:
Changing the
markWord
required recompilation of many, many files, leading to ~10 minute builds for every change. Very annoying when trying to rapidly iterate.
To run tests (incl. unit tests), the infrastructure needs to build the JDK. To build the JDK, one needs to compile classes via
javac
, which in turn uses HotSpot. If one makes catastrophic changes in HotSpot, one will crash while building the JDK, meaning
one can’t run the C++ GoogleTest unit tests
.
Specifically with regards to (2), there was an assertion plaguing me:
markWordm=markWord::encode_pointer_as_mark(p);assert(m.decode_pointer()==p,"encoding must be reversible");
It’s not important to know what this does, just that encoding a mark word must be
invertible
. So, whatever I was doing in
decode_pointer
broke this property.
Since much of the
markWord.{hpp, cpp}
was just pure C++, I ended up copying the files into its own C++ project and iterating there. This gave me quicker turnaround times, and I could test for invertibility without breaking the JDK build. It ain’t pretty, but it worked.
Unfortunately, even with emulation, I still saw a plethora of test failures. The dreaded
java.lang.AssertionError
and
java.lang.NoClassDefFoundError
errors persisted. I stripped the emulation, removed the static type checking code again, updated the unit tests, and cut my losses.
When debugging the failed unit test, I realized it tested for semantics that no longer existed. Even when emulating the old behaviour, my issues still persisted. While I had succeeded in removing dead code and updating unit tests, this didn’t fix the plethora of broken Java application-level tests.
ℹ️ I’ve linearized the order of events in this Act for readability and understandability. In reality, I bounced between the failing test cases and investigations quite a lot. I’m also repeating some principles that may seem obvious, but I believe there is value in restating them.
Failures in application code can be notoriously difficult to debug. What’s particularly hard is when you don’t know where the issue(s) originated from. If a HTTP server errors, it’s probably in that route’s controller or in the middleware stack. That’s not to say that it’s straightforward to understand or fix (especially getting to the root of cascading microservice failures!), but it provides a starting point.
HotSpot has many moving parts, so in many cases this adds more dimensions to the problem.
Temporally
, failures can manifest much later after the introduction of the bug. For example, only after the 10th garbage collection cycle. The
locality
is also non-obvious sometimes. Take a miscompilation
9
for example, it could change the state of a variable and break an invariant a completely unrelated component of the virtual machine. Assertions
can
stop bugs from spreading too far, but can’t ensure locality.
To stay sane, it’s important to
reduce the size of the programs
and to ensure that
bugs are reproducible
. Unfortunately, neither of these can be taken for granted.
With that in mind, it was time for an action plan. These are questions I try to answer in order to get a better understanding of what is going on, a prerequisite to bugfixing.
When are we failing?
Can we increase and decrease our chance of failure?
Where are we failing?
This has several sub-questions:
With which
component/feature
(e.g. garbage collection, Compact Object Headers) does the bug manifest?
Is there a place in HotSpot source code that is suspicious?
Where does the bug manifest in the application code?
On top of a bytecode interpreter, HotSpot has
two just-in-time compilers, C1 and C2
. There is no “the JIT compiler”. C1 compiles fast but is limited in terms of optimizations, C2 produces extremely optimized machine code but takes comparatively long to run. In the past, client VMs (i.e. those for end users) would ship C1, and server VMs came with the C2 compiler. Nowadays, both C1 and C2 compile your Java code in a process called
tiered compilation
10
. Note that
not all methods will be JIT compiled
. HotSpot uses extensive profiling to decide which methods are worth compiling with which compiler.
There are six garbage collectors in the source code, though those that get shipped depends on your JDK vendor. My personal configuration include Serial & Parallel
11
, G1
12
, and Z
13
.
The table below shows how I generally target components. Initial trial-and-error found that the tests mentioned above would usually fail within five runs. So, I repeated each experiment 20 times.
Component to target
JVM flags
Description
Interpreter
-Xint
Forces all code to be run in the interpreter.
C1 only, trivial methods
-Xcomp -XX:TieredStopAtLevel=1
Forces JIT compilation of
everything
, and stops after C1 compiling trivial methods.
C1 only, fully compiled
-Xcomp -XX:TieredStopAtLevel=3
Same as above, but with more profiling.
C2 only
-Xcomp -XX:-TieredCompilation
Compilation without tiers forces C2.
Serial GC
-XX:+UseSerialGC
Tells HotSpot to use Serial.
Parallel GC
-XX:+UseParallelGC
Tells HotSpot to use Parallel.
G1 GC
-XX:+UseG1GC
Tells HotSpot to use G1, though this is the default in most cases.
Z GC
-XX:+UseZGC
Tells HotSpot to use Z.
I didn’t actually re-run all the possible compiler and GC combinations possible (16 total). Instead,
I assumed that there was no interaction between compilers and GC
and that they could be tested independently. Why? I had played around with heap sizes and it seemed to have no effect on failure frequency. Generally,
smaller heaps mean more frequent garbage collection
and a higher likelihood of manifesting. Just to be clear, this was a gamble overall, as
no interaction is not always guaranteed
and if was wrong I’d end up wasting a lot of time and resources.
So, what did I actually do?
Did my gamble pay off?
I picked a failing test and ran it interpreter-only, saw that it passed after 20 iterations. So it’s probably a compiler issue.
Since I know the interpreter works, I skipped
-Xcomp
and just added
-XX:TieredStopAtLevel=1
for trivial C1 compilation. This meant that my code would get interpreted until HotSpot decided some trivial methods would get compiled. This passed 20 iterations too.
At this point I could have tried full C1 compilation, but I had a hunch it’s probably C2. So I ran with
-XX:-TieredCompilation
and behold, the bug manifested!
As a sanity check, I ran with with C2 compilation on Serial, Parallel, G1 and Z. This was to ensure that all GCs see test failures, which they did.
I would say it paid off. I ran a few other test cases, they also confirmed that C2 compilation was the culprit. Using
-XX:-TieredCompilation
saw the bug manifest roughly 1/2 of the time. My colleagues suggested seeing if enabling Compact Object Headers with
-XX:+UseCompactObjectHeaders
makes a difference. And that was spot on, since I wasn’t able to reproduce with it enabled. To answer the question of
when we fail
: when we use C2 without Compact Object Headers.
So what does this tell us? Application logic failures with no VM crashes (via assertion failure or otherwise) indicates
it’s possibly (but not necessarily) at least one C2 miscompilation
when disabling Compact Object Headers. That’s pretty bad luck, since C2 is extremely complex and miscompilations are really hard to track down on top of that.
While I knew that the bug manifested when using C2 without Compact Object Headers, I still had no idea where the miscompilation happened. While there are many
if (UseCompactObjectHeaders)
in the HotSpot/C2 code, there are so many implicit interactions within the runtime one would need a lot of luck to find the bug that way.
As a VM engineer, there are many tools in our toolbox to help with this uncertainty. For example,
GDB
14
allows stepping through the code and inspecting state.
rr
is great, because it captures program behavior once, and then one can deterministically step through (usually forwards, but also backwards). That’s invaluable with intermittent failures.
These tools take the
java
invocation as arguments. In OpenJDK, tests are run with the home-grown
jtreg
framework, and invoked through
make test
, which in turn creates a
jtreg
Java process. When a test fails, it gives a “re-run command” which allows the develper to run it via
java
directly. This command is highly complex, since the
jtreg
subprocess usually spawns a few of its own subprocesses and supports different
execution modes
. Furthermore, two of my three symptoms were TestNG harnesses, which added another level of complexity.
Unfortunately,
the bug never manifested via re-runs
. I spent countless days trying, fiddling with the flags, to no avail. Without being able to observe failures directly (only via
make test
), debugging’s difficulty is locked to hard mode. It seemed like I hit a dead end.
With no clear debugging entrypoint, my only hope was to minimize the test cases. That way, I would be able to use unorthodox methods like
printf
statements or adding
ShouldNotReachHere()
assertions. I picked
java/util/jar/JarFile/mrjar/MultiReleaseJarProperties.java
as a starting point.
I usually minimize in two steps:
Remove half of the method calls. I add them back one by one if the test no longer fails/the bug no longer manifests.
Once no more method calls can be removed, I inline the existing method calls, cleaning up/constant folding in the process. Go back to step 1 until nothing can be inlined further.
There are tools that do minimization programatically.
creduce
is a good example and also works for Java. I reduced
MultiReleaseJarProperties.java
manually since it gave me a lot of exposure to Java libraries I hadn’t used before. Plus, it’s good practice.
After many, many iterations, I couldn’t inline any further since I hit the Java standard library. Unfortunately,
the minimized test
was still more complex than I had hoped it would be. Therefore,
another approach was necessary
…
By trying a variety of HotSpot flags, I was able to narrow down the issue to a problem related to the C2 JIT-compiler with Compact Object Headers disabled. I managed to minimize a test case substantially, but could not narrow it down as much as I wanted to. I suspected that it is a miscompilation bug, which makes it very hard to track down.
When HotSpot JIT-compiles programs, there is a
lot
of machine code produced, not only for the application itself, but also for the Java launcher. It’s just not possible (or, extremely time intensive) to individually investigate every single compilation.
In my case, I couldn’t minimize any further. So, I had to rely on automated techniques in order to try to localize where the bug was.
In order to find out which methods get compiled, one can use
-XX:+PrintCompilation
. This prints out a long list of compiled methods, for example
15
:
154 1 n jdk.internal.misc.Unsafe::getReferenceVolatile (native)
206 2 n java.lang.Object::hashCode (native)
207 3 n java.lang.invoke.MethodHandle::linkToStatic(LLLLLLL)L (native) (static)
219 4 java.lang.Enum::ordinal (5 bytes)
227 5 n java.lang.System::arraycopy (native) (static)
...
We’re just interested in the method name in this case. For more information on what the different numbers and symbols mean,
I recommend Stephen Fox’ blogpost
.
In my case, there were ~400 compilations. Again, this would be insanely tedious to manually crawl. I needed to automate.
There are two ways one can try to automatically find the method(s) causing issues. Of course, they assume that sufficiently many iterations are performed for the bug to manifest.
Go through every single method, and only compile that method. Every method that sees the bug manifest should be investigated further.
Go through every single method, and exclude this method from compilation. Only investigate methods whose exclusion
did not
cease the bug from manifesting.
I decided to go down the
first route
. The second route is
probably better
if there are interactions between methods that cause the issues. However, it takes longer to get to a result, so I decided to take a bet on the first method.
So how does one actually control compilations? By using
-XX:CompileCommand
16
. With this, I can specify if I
only
want to compile a specific method, or if I want to
exclude
another method. For example,
-XX:CompileCommand=compileonly,java.lang.Enum::ordinal
ensures that
if HotSpot decides to compile
Enum::ordinal
, this is the only compiled method. Similarly,
-XX:CompileCommand=exclude,java.lang.Enum::ordinal
ensures that HotSpot will
never compile it
. Since we are only compiling with C2 (
-XX:-TieredCompilation
flag), these
CompileCommand
s control C2’s behavior.
Having gathered all compiled methods from
-XX:+PrintCompilation
, after preprocessing
17
, I wrote a little script to automate the search process. The gist of the script is as follows:
// main(String[] args), while iterating through all methods:
Stringflag="-XX:CompileCommand=compileonly,"+method;if(!run(flag)){System.err.println("Failure for "+method);}// run(String otherFlags):
ProcessBuilderpb=newProcessBuilder("bash","-c","cd ~/valhalla && make test TEST=\""+TEST+"\" JTREG=\"REPEAT_COUNT=5;VM_OPTIONS=-XX:-TieredCompilation -XX:+PrintCompilation "+otherFlags+"\"");
Delightfully, it produced a result:
Failure for java.util.concurrent.ConcurrentHashMap::get
So, something related to
ConcurrentHashMap::get
was causing the problem! At this point, it’s
still unclear if there is a miscompilation
or the bug simply manifests in relation to this compiled method. But at least, I now knew a little bit more about where to look.
Remember when I said
-XX:CompileCommand=compileonly
ensures that only a specific method gets compiled? In reality, it also
includes inlined methods
18
if C2 decides they’re worth inlining. The
-XX:+PrintInlining
flag will show what’s being inlined. Warning: it’s often so verbose that
make test
will truncate the output.
Taking a look at
-XX:CompileCommand=compileonly,java.util.concurrent.ConcurrentHashMap::get -XX:+PrintInlining
, it shows the following:
Now, I could disable each method one by one and see if any of them caused the bug to cease. For example disabling
ConcurrentHashMap::spread
:
-XX:CompileCommand=dontinline,java.util.concurrent.ConcurrentHashMap::spread
. The only exceptions are intrinsic methods, who remain unaffected by the flag and will be inlined.
When I disabled every inlined method and the error still persisted, it meant that there were two options. First, the bug could be in
ConcurrentHashMap::get
itself. Or, the bug is in an intrinsic. Taking a look at
the source code for
ConcurrentHashMap::get
, it seemed unlikely that it would be there. The code uses very common programming constructs that would have most likely caused failures in other tests too.
My hunch was that the bug was in the
Object::hashCode
intrinsic, simply because Valhalla does things with
hashCode
. And indeed, turning off the intrinsic by
returning
false
in the function fixed the issue.
It seemed the issue was in the
Object::hashCode
intrinsic. However, at this point it was still not possible to definitely pinpoint that it was a miscompilation, but signs did point towards one. In any case, I was now at the point where the compilation was sufficiently small to bring out the big guns.
By individually compiling methods that were C2-compiled during a run where the bug manifested, I found that
ConcurrentHashMap::get
was the suspect. Upon inspecting the inlining, the C2 intrinsic for
Object::hashCode
was determined to be the culprit.
The intrinsic source code is quite long. My OOP professor would have said it “has a high cyclomatic complexity.” I added a few print statements to each branch to figure out which path we were taking. Recall that I could not reproduce this bug via
gdb
, hence print statements were my only available tool.
While doing that, I noticed the
if (!UseObjectMonitorTable)
conditional. Suddenly, I had a theory! The bug did not manifest with Compact Object Headers. Could it be that using Compact Object Headers sets this flag? I could easily verify this:
java -XX:+UseCompactObjectHeaders -XX:+PrintFlagsFinal -version | grep UseObjectMonitorTable
showed it was enabled.
java -XX:-UseCompactObjectHeaders -XX:+PrintFlagsFinal -version | grep UseObjectMonitorTable
showed it was disabled.
So, Compact Object Headers sets this flag and does
not
take the branch.
It’s likely an issue related to something in that branch
. The branch adds a bunch of extra instructions, but nothing immediately stands out.
I took a look at the difference in generated machine code.
Configuring
the HotSpot source code using the
--enable-hsdis-bundling --with-hsdis=llvm
flags
19
enables machine code inspection. That let me run with the
-XX:CompileCommand=print,java.util.concurrent.ConcurrentHashMap::get
which prints the disassembled machine code of the whole method, including inlined methods and intrinsics. As I already saw from the source code, I saw extra instructions with Compact Object Headers /
UseObjectMonitorTable
disabled.
Instead, I had to think at a higher level. C2, like many compilers, optimizes on an intermediate representation (IR). Programs get transformed into an IR that is not Java bytecode, but not yet machine code. C2’s IR is a big graph, called Sea of Nodes
20
. It performs optimizations by doing transformation and analysis passes over the IR. Using the
Ideal Graph Visualizer
I was able to take a diff between the graph of
ConcurrentHashMap::get
with
UseObjectMonitorTable
enbled and disabled, for the first and last optimization passes. For context, this method produced about 400 nodes and 1000+ edges, which is why I don’t have a screenshot. The
diff yielded no interesting changes
unfortunately. While some nodes had differences, these could be attributed to different addresses or branch probabilities.
Was this not a miscompilation after all?
Was the actual issue deeper in the runtime and the bug just mainfested in this compilation unit? While I learned which feature flag was responsible, it still didn’t explain what was going on.
I needed to understand what
UseObjectMonitorTable
actually does. Crudely put, an object monitor is used for locking and sychronization. The
wiki page
based on the
pull request
was a helpful starting point. To understand my bug only the first line of the PR is relevant (inflating is a state of the
markWord
):
When inflating a monitor the ObjectMonitor* is written directly over the markWord and any overwritten data is displaced into a displaced markWord.
I asked my GC colleagues to see if they had an idea, and we collaborated on the last step of finding the bug. We determined how to attach to the process since the bug would only manifest through
make test
. It turns out, the following combination worked quite well:
Running the test with
-XX:+ShowMessageBoxOnError
and
-XX:AbortVMOnException=java.lang.AssertionError
set.
Waiting for the test to hang, most likely indicating a failure.
ShowMessageBoxOnError
expects input via standard input, but since the
jtreg
infrastructure wraps everything, input cannot be given. Hence, it hangs until the test times out.
Connecting the
lldb
debugger via
lldb -p <PID>
and check the threads using
thread backtrace all
to see which of the processes had the message box showing. This would be the failed process.
In the end though, attaching didn’t help much. It did indicate that some threads were waiting, with a lightweight locking mode (what this means is irrelevant), which made us look at the
intrinsic
again in detail. Specifically, the
LM_LIGHTWEIGHT
. Note that the misaligned comment is a faithful reproduction of the source code.
// Test the header to see if it is safe to read w.r.t. locking.
// This also serves as guard against inline types
Node*lock_mask=_gvn.MakeConX(markWord::inline_type_mask_in_place);Node*lmasked_header=_gvn.transform(newAndXNode(header,lock_mask));if(LockingMode==LM_LIGHTWEIGHT){Node*monitor_val=_gvn.MakeConX(markWord::monitor_value);Node*chk_monitor=_gvn.transform(newCmpXNode(lmasked_header,monitor_val));Node*test_monitor=_gvn.transform(newBoolNode(chk_monitor,BoolTest::eq));generate_slow_guard(test_monitor,slow_region);}else{/* ... */}
Conceptually, this code
generates machine code
that does the following:
It defines a constant represented by the bitmask
markWord::inline_type_mask_in_place
. This bitmask is the logical OR of the value object bit and the two lock bits.
It performs a logical AND on the object header (where the
markWord
resides) and the locking mask to create a masked header.
It defines another constant representing the monitor value (
0b10
).
It compares the masked header with the monitor value. If it’s equal, it takes a slow path needed for locking (the details of which are irrelevant).
The bug is in the first step
. The following diagram represents the old and new bitmasks for the eleven least-significant bits:
7 3 0
VVVAAAAVSTT <- Before my change
^-^^---- 1011 bitmask
VVVVAAAASTT <- After my change
^-----^^---- 10000011 bitmask
When an object monitor is inflating, the
markWord
is replaced by the object monitor’s native address (sans least-significant lock bits). Let’s denote this native address as
P..PP
. The
markWord
then becomes:
7 3 0
PPPPPPPPPTT <- Before my change
^-^^---- 1011 bitmask
PPPPPPPPPTT <- After my change
^-----^^---- 10000011 bitmask
So we are doing a bitmask on a pointer instead of metadata! If we got unlucky and bit number 3 or 7 (depending on before/after my change) were set to 1, the mask would include an insignificant bit. In that case, comparing to
0b10
(the monitor value) would yield that it is not equal, and the slow path would not be taken, even though it should have been! We should just check the lock bits instead.
It’s a miscompilation after all!
Setting the mask to
markWord::lock_mask_in_place
seemed to address Symptom B (
java.lang.AssertionError
) and Symptom C (
java.lang.NoClassDefFoundError
). Recall that Symptom A (failed unit tests) had already been fixed. Finally, running it through the usual
tiered testing
showed no regressions compared to before my changes.
An incorrect bitmask was checking native pointer bits instead of metadata bits when
UseObjectMonitorTable
was disabled. If a bit in the pointer was set to 1, the required slow path is not taken. Compact Object Headers was setting the
UseObjectMonitorTable
flag to
true
, which is why the issue did not manifest here.
One big question is why did this not affect the virtual machine before my change? This can be explained by the fact that 64-bit native memory pointers, obtained via
malloc
, are 16-byte aligned. They will have four trailing zeroes, of which
the latter two may be overriden by the locking metadata
. Considering the diagram:
7 3 0
PPPPPPP00?? <- Before my change
^-^^---- 1011 bitmask
PPPPPPP00?? <- After my change
^-----^^---- 10000011 bitmask
It shows that the bit in position 3 was always zero. Therefore, the value object bit in the bit mask would AND with a zero, which produces a zero. That’s why the bug only manifested after moving the value object bit.
I have no idea why this caused the symptoms that it did. A miscompilation means the program can behave arbitrarily, and that it did. While I acknowledge this might be an unsatisfying ending, it’s simply not the best use of my time to chase the bug to its symptoms. I’m more than happy to guide anyone who is interested in doing so though.
The bug did not manifest before I changed the value object bit location because we got lucky with native memory alignment. The bit that the faulty bitmask was checking was always zero. I have no idea why objects turned null or why classes could not be found.
The takeaways I would like to highlight are as follows:
Have a strong methodology and challenge assumptions.
With a large codebase like HotSpot, reasoning is the required to (sanely) make progress in debugging. Derive a methodology appropriate for the bug you are facing.
Ask the right questions at the right time.
Know what/when to ask and whom to ask. There’s no shame in asking for help, and it’s often a learning experience for both parties. If I spend a day without progress, I usually take that as an indicator to seek external help and/or input.
Use your tools!
This may be ironic since I wasn’t able to utilize
lldb
successfully, but debugging tools are incredibly useful. At least get familiar with the basics, they’re not as hard and scary as they seem.
I think I can attribute my success in this instance to asking the right questions to the right people. I’m not great at debugging, but a lot of people around me are and were glad to help. Thank you to all my colleagues for helping me out when I was stuck, and answering my many questions. I’m extremely grateful to be able to learn from you.
Gaza’s Civil Defense Forces Keep Digging for 10,000 Missing Bodies
Intercept
theintercept.com
2025-11-28 11:00:00
Members of Gaza’s Civil Defense force describe pulling decomposing bodies from collapsed buildings, and digging in hopes that someone remains alive.
The post Gaza’s Civil Defense Forces Keep Digging for 10,000 Missing Bodies appeared first on The Intercept....
The mission that
haunts Nooh al-Shaghnobi most took place on September 17, near the al-Saha area of eastern Gaza City. Israeli forces had bombed a home, killing more than 30 members of one extended family. Most of their bodies were trapped under the rubble.
Al-Shaghnobi’s Gaza Civil Defense force team pulled two dead young girls from the bombed house and kept digging, crawling under collapsed floors. “We don’t go under unless someone is alive,” he told The Intercept. “Otherwise, we dig from above — ceiling by ceiling.” What followed was a descent into something dreamlike and horrifying.
“We walked 12 meters under the rubble,” he said. “Every meter, the air grew less. I crawled past legs, arms, the body of a child hugging his dead mother. I felt the ground shake from bombings above.”
From deep inside the wreckage, the team heard a young girl calling, “I’m here. I’m here.”
The Civil Defense force is an
emergency and rescue operations group
administered by the Palestinian Minister of Interior. After two years of Israeli genocide, it has an estimated 900 personnel and has lost roughly 90% of its operating capacity, Civil Defense workers told The Intercept. In the absence of heavy equipment, the civil defense teams use simple tools like hammers, axes, and shovels. Without excavators or heavy equipment, a single recovery can take days.
Local civil defense workers estimate there are still 10,000 bodies buried under the rubble.
“When you hear a voice, you know there is life. That’s enough to make you risk your life to recover this soul.”
“What motivates us,” al-Shaghnobi said, “is that when you hear a voice — even one — you know there is life. That’s enough to make you risk your life to recover this alive soul.”
By the time al-Shaghnobi finally reached Malak, she was unconscious with no pulse. Her eyes open, her legs blue, she had passed away.
“I tried to wake her up, but it was too late,” al-Shaghnobi said. “I was in a moment of utter stillness, and I could hear nothing but my own breath.”
Civil defense teams retrieve bodies in Al-Katiba on October 28, 2025.
Photo: Nooh al-Shaghnobi
24-year-old al-Shaghnobi
has already spent seven years working for Gaza’s Civil Defense force. Like many of his colleagues, he eats and sleeps at his workplace. His family’s home in the Tal Al-Hawa area of western Gaza City was destroyed in the final days of the war, and his family remains displaced in the south.
“People think the ceasefire means we can breathe,” he said. “But for us, the end of the war is the beginning of the real war: pulling out the dead.”
Al-Shaghnobi believes his aunt’s corpse is among the 10,000 bodies that remain unrecovered. Large regions like Shujayaa and parts of Rafah are still inaccessible. Israeli forces are stationed there, marking the areas “yellow zones.” Civil defense crews cannot reach them.
“We barely recovered some bodies during this ceasefire,” al-Shaghnobi said. “We have no machinery. Some areas, we know there are hundreds under the rubble, we simply can’t go.”
Alaa Khammash, 25, said he feels terrible when his Civil Defense team is unable to rescue someone.
“When I am dispatched on a mission, I feel a responsibility to finish it completely. I cannot simply stop midway,” he said. It can take 10 to 12 hours to retrieve a single body if it’s under a collapsed ceiling or wall. “Sometimes we can’t recover the body since it needs heavy equipment.”
The years of genocide have left al-Shaghnobi feeling numb.
“In the beginning of the war, we couldn’t look at the bodies,” al-Shaghnobi said. “We would close our eyes when retrieving them. By the middle of the war, we were wrapping them in white shrouds like it was daily routine. By the end of the war, my emotions became more defeated. The accumulation of pressure made it difficult to touch the bodies.”
“Bodies are found in various states: decomposed, non-decomposed, burnt, or even evaporated, sometimes just a skull or a skeleton,” he added, “The body’s texture is soft and smooth when found.”
Civil defense team members wear a special uniform, gloves, and masks because of the smell of the decaying bodies. The bodies decompose rapidly when they’re in the sun, Khammash said. “This occurs when a body lies exposed outdoors, subject to sun and air. Slow decomposition happens when the body is under a roof or shielded from air and sunlight.”
The smell can make al-Shaghnobi lose his appetite for days. For six months, he has struggled with digestive issues. Once, during Ramadan, “I was fasting,” al-Shaghnobi said, “We pulled a body that had been under rubble for a year in Al-Shifa hospital. It was half-decomposed. The smell hit me, my vision blurred, I nearly collapsed.”
“We identify locations of martyrs during the day based on blood stains, bones, and skulls,” al-Shaghnobi explained. “We rely on families of the martyrs. … They call our team, often providing the equipment at their own personal expense to honor and bury their loved ones.”
Without DNA tests, the workers identify bodies from clothes, shoes, rings, watches, metal implants, IDs, and gold teeth. The unknown bodies — often only skulls or skeletons — go to a cemetery for the unnamed.
After retrieving bodies, the Civil Defense workers write a detailed paper describing the area, angle, building, height measurement, and burial location, all written on the shroud so families can potentially identify the body later.
Sometimes, families insist on seeing the remains to believe their loved one is gone. “People accept death more easily,” al-Shaghnobi explained, “when they see the body.”
“I moved my friend from one grave to another. He was just a skull.”
“I moved my friend from one grave to another,” he said, recalling a reburial. “He was just a skull. I kept thinking — this is the end of every person. Bones.”
Recovering a person’s body entails a strange emotional paradox, said 27-year-old Mohammad Azzam.
“It feels good because you found them,” he said, “but bad because they are decomposed. A feeling I cannot explain.”
Families often wait nearby, and when the team brings out the body, their reactions are marked by intense, overwhelming grief.
“When we find someone, they’re usually half-decomposed,” Azzam said. “The face is unrecognizable. Only a shoe, a wallet, a bracelet tells you who they were.”
“When we find someone, they’re usually half-decomposed.”
The workers navigate these traumatic moments while living through the horrors of genocide in their own families and homes. Khammash, like al-Shaghnobi, now lives at work: His house in eastern Gaza City sits dangerously close to the Israeli military presence.
At work one day, Khammash said he got a dreaded call from a friend: “They told me my brother had been injured in the south, near the American aid distribution point, and taken to al-Awda Hospital in Nuseirat. I called a friend of mine who works as a nurse there, and he told me my brother had died.”
It was unbearable. “My brother was not only my sibling — he was my closest friend, only a year younger than me,” he told The Intercept. “We shared everything, understood each other without speaking. We went everywhere together. That kind of loss never leaves you, and the separation is the hardest pain.”
“Death is certain,” Khammash said. “As Allah said: Every soul shall taste death. And as Muslims, we understand that what comes after is far better than what we endure here.”
During the ceasefire,
the rescue teams receive constant calls: A neighbor reports a smell, a family begs for help to retrieve their loved one, a building is collapsing, a limb has surfaced through the rubble, flies gathering in a corner reveal what lies beneath.
Khammash has begun to feel death as a presence, not an event. “It surrounds us,” he said. “Maybe we are the next ones. We accept Allah’s plan, but still — inside us — we love life.”
One of the hardest missions Khammash has had under the ceasefire was in a bombed tower in the al-Rimal neighborhood. A woman was alive somewhere under the collapsed top floor, calling out, but the rescuers couldn’t locate her.
“It was pitch black,” he recalled. “I kept moving my light, trying to understand where her voice was coming from.”
Suddenly, she was beneath him. “I had put my foot next to her head without realizing. We took her out alive.”
The longest recovery Khammash ever worked took a full day — pulling out Marah al-Haddad, a girl buried beneath several floors in al-Daraj area a month ago.
“She was alive when we reached her,” he said. “She had been breathing dust and explosives. My colleague Abdullah Al-Majdalawi and I kept calling, ‘Where are you, Marah?’ And she answered, ‘I’m here. I’m here.’”
“When she saw us, hope came back to her face,” he said. “To bring someone back from death — this is what keeps us going.”
Fort Knox for your secrets - Manage secrets with encryption or cloud providers
fnox works with either—or both! They've got their pros and cons. Either way, fnox gives you a nice front-end to manage secrets and make them easy to work with in dev/ci/prod.
fnox's config file,
fnox.toml
, will either contain the encrypted secrets, or a reference to a secret in a cloud provider. You can either use
fnox exec -- <command>
to run a command with the secrets, or you can use the
shell integration
to automatically load the secrets into your shell environment when you
cd
into a directory with a
fnox.toml
file.
Store encrypted secrets in git using age, AWS KMS, Azure KMS, or GCP KMS. Your team can clone the repo and immediately have access to development secrets.
Why is this a standalone CLI and not part of mise?
mise
has support for
encrypted secrets
but mise's design makes it a poor fit for remote secrets. mise reloads its environment too frequently—whenever a directory is changed,
mise x
is run, a shim is called, etc. Any other use-case like this mise leverages caching but secrets are an area where caching is a bad idea for obvious reasons. It might be possible to change mise's design to retain its environment in part to better support something like this but that's a huge challenge.
Basically it's just too hard to get remote secrets to work effectively with mise so I made this a standalone tool.
The CRDT Dictionary: A Field Guide to Conflict-Free Replicated Data Types
Back around 2014, I kept hearing about this cool database called
Riak
, a distributed database that could survive network partitions and keep accepting writes. Some really interesting companies were using it at massive scale, and I was curious about it. One of the big selling points was that it could handle concurrent writes without any coordination or consensus. I was intrigued, and I started reading about it. Underlying all of this was the concept of CRDTs, Conflict-free Replicated Data Types.
1
At the time, I was working on a beer startup called Brewtown with a friend: a beer review social site and delivery subscription service. It failed for other reasons, but I was a little too enamored with and shiny tech back then, and CRDTs and Riak fit the bill for shiny tech. I kept trying to find excuses to shoehorn CRDT stuff into our codebase when, honestly, we didn’t need any of it. Postgres would’ve been fine. Live and learn.
Anyways, the idea sounded like pure sorcery: data structures that replicate across nodes and merge deterministically, without coordination, without losing information. I got excited, read a few papers, played with some toy implementations… and then we gave up on the beer startup. I didn’t really have a reason to mess with CRDTs for a while.
Fast forward to 2025, I’ve just had Thanksgiving dinner, and I’m curious again. What’s the state of the art? What have I forgotten? Which CRDT should I reach for when? So I’m writing this, both as a refresher for myself and a reference for the next time I need to remember why OR-Sets exist or what WOOT stands for. (“WithOut Operational Transformation.” Yes, really.
2
)
So, grab a coffee.
Commutative. Replicated. Data Types.
In isolation, all of the words make sense. But when you look at the literature, it’s overwhelming:
Suddenly you’ve moved beyond the simple terms and start seeing things like G-Counters, PN-Counters, LWW-Sets, OR-Sets, 2P-Sets, RGAs, WOOTs, Logoots (wtf?)… Each with subtle tradeoffs. Each paper assuming you’ve read the previous five. It’s overwhelming.
This guide will hopefully cut through that. We’ll build intuition through
interactive demos
and concrete examples. You’ll see how merges actually work, watch conflicts resolve (or not resolve), and develop a feel for which CRDT fits which problem.
What You Need to Know
You don’t need a PhD in distributed systems. If you understand:
Why network failures happen
What “eventual consistency” means
Basic set operations (union, intersection)
…you’re good.
The Problem
Picture this: Alice and Bob are both editing a shared counter. Alice increments it. Bob increments it. The network is flaky, so neither sees the other’s change immediately. Later, they reconnect. What should the counter show?
Option 1: Consensus
- Use Paxos/Raft to agree on who went first. Works great! Until the network partitions and half your users can’t write because they can’t reach a quorum. Not ideal for offline-first apps.
Option 2: Last-Write-Wins
- Use timestamps. Whoever wrote last “wins.” Easy to implement! Except Bob’s increment gets completely erased if Alice’s timestamp was later. Data loss.
Option 3: CRDTs
- Design the data structure so that merging is deterministic. Both increments survive. No coordination needed. No data loss. However, you have to be okay with some level of eventual consistency.
What’s the trick? How do CRDTs achieve this?
Roughly speaking, you are working with a CRDT if your merge operation is:
commutative (order doesn’t matter)
associative (grouping doesn’t matter)
idempotent (duplicates don’t matter)
Once you achieve these properties, then you can use your merge operation to ensure that replicas automatically converge to the same state.
A Quick Detour: Lattices and Why They Matter
Before we dive into specific CRDTs, let’s build some intuition about what makes merging work. In CRDT literature, this is often referred to as a “lattice”.
Think about natural numbers with
max
as the merge operation. If you have
3
and
5
, taking
max(3, 5) = 5
makes sense. It doesn’t matter if you compute
max(3, max(5, 7))
or
max(max(3, 5), 7)
- you get
7
either way. And
max(5, 5) = 5
, so duplicates are harmless.
This forms a
partial order
: some values are “greater than” others (
5 > 3
), and there’s a
join
operation (
max
) that gives you the least upper bound. The fancy math term is “join-semilattice,” but think of it as:
a way to consistently pick “more recent” or “more complete” information
.
Here’s the key insight: if your data structure’s states form a lattice, and updates only move “upward” in the ordering, then:
You can apply updates in any order
You can apply the same update twice
Eventually, everyone agrees on the maximum state
Consider a counter where each replica tracks its own count:
{A: 3, B: 5}
. The partial order is
pointwise
:
{A: 3, B: 5} ≥ {A: 2, B: 5}
because each component is greater-or-equal. To join, take the
max
of each component. This is exactly how the G-Counter CRDT works!
Why does this matter?
Because if you can design your data structure so that:
States form a lattice (there’s always a sensible “join”)
Operations only move upward (you can’t un-increment a counter)
Then merging becomes trivial: just take the join. No coordination needed. No conflicts possible. The math guarantees convergence.
Not all CRDTs fit this clean model (some need timestamps or version vectors to determine what’s “greater”), but the lattice intuition often guides the design. When you see
merge = unionWith max
or
merge = union
, you’re seeing some pure, beautiful math-brained lattice thinking.
State-Based vs Operation-Based
Moving on…
There are two fundamental approaches to CRDTs:
State-based CRDTs (CvRDTs)
send the entire state to other replicas, which merge it with their local state using a join operation. The state must form a join-semilattice.
3
Operation-based CRDTs (CmRDTs)
send operations to other replicas, which apply them to their local state. Operations must be commutative when applied concurrently.
4
In this guide, we’ll primarily discuss state-based CRDTs, as they’re conceptually simpler and the ideas translate naturally to the operation-based variants.
The Core Laws
For a data structure to be a state-based CRDT, its merge operation must satisfy:
Associativity
:
(a ⊔ b) ⊔ c = a ⊔ (b ⊔ c)
where
⊔
denotes the merge/join operation
Commutativity
:
a ⊔ b = b ⊔ a
Idempotence
:
a ⊔ a = a
These properties ensure that:
Merging in any order produces the same result
Re-receiving the same state is harmless
Partial merges can be composed
Additionally, the state must form a
monotonic semilattice
: updates only move “upward” in the partial order, never downward. This ensures convergence: once all updates have been delivered, all replicas reach the same state.
For the curious, The symbol ⊔ is called (square cup) or square union. I have no idea why regular union symbol isn’t used. Pointy-headed researchers, I guess.
Anyways, it’s commonly used to denote:
Disjoint union - union of sets treated as disjoint
Join operation in lattice theory - the least upper bound (supremum) of two elements
Merge operation in CRDTs - combining two states by taking their least upper bound
With these foundations in place, let’s explore the CRDT zoo.
G-Counter: Grow-Only Counter
Let’s start with the simplest CRDT: a counter that only goes up.
5
The Idea
Instead of storing one global count, each replica tracks its own count. The total is the sum of all replica counts. When replicas merge, they take the
max
of each replica’s count.
Why
max
? Because counts only increase. If replica A shows that replica B has counted to 5, and replica B shows it’s counted to 3, we know A has seen newer information. Taking the max ensures we never lose increments.
6
Implementation
type GCounter = Map ReplicaId Natvalue :: GCounter -> Natvalue counter = sum (Map.elems counter)increment :: ReplicaId -> GCounter -> GCounterincrement r counter = Map.insertWith (+) r 1 countermerge :: GCounter -> GCounter -> GCountermerge = Map.unionWith max
Laws and Invariants
The merge operation forms a join-semilattice where the partial order is defined pointwise:
c1 ≤ c2
if for all replicas
r
,
c1[r] ≤ c2[r]
.
Associative
:
max
is associative
Commutative
:
max
is commutative
Idempotent
:
max(x, x) = x
Monotonic
: Each replica’s count only increases
Intuition
Think of each replica as having its own tally marks. When replicas sync, they each adopt the maximum tally for each replica they’ve seen. Since tallies only grow, taking the maximum ensures we never lose increments.
Tradeoffs
Advantages
:
Simple and efficient
No metadata overhead beyond replica counts
Perfect for increment-only scenarios (page views, likes, etc.)
Disadvantages
:
Cannot decrement
Size grows with number of replicas (though typically small)
No garbage collection (all replica counts retained forever)
When to Use
Use G-Counter when you need to count upward-only events in a distributed system: analytics counters, like counts, view counts, or any monotonically increasing metric. (If you need to decrement, well… keep reading.)
Interactive Demo
Try it yourself! Increment counters on different replicas and see how the merge operation works:
PN-Counter: Positive-Negative Counter
What if we need to decrement? Enter the PN-Counter. The trick is beautifully simple.
Definition
A PN-Counter contains two G-Counters: one for increments, one for decrements:
Since both components are G-Counters with valid merge operations, the PN-Counter’s merge inherits their properties and forms a semilattice.
Intuition
A PN-Counter is like having two separate tally sheets: one for additions, one for subtractions. The current value is the difference between them. When replicas sync, they merge both sheets independently.
Tradeoffs
Advantages
:
Supports both increment and decrement
Deterministic convergence
Simple to understand and implement
Disadvantages
:
Double the space of a G-Counter
Can never truly garbage collect old replica entries
No bound on the value range (can overflow)
Cannot reset the counter atomically
When to Use
Use PN-Counter for any metric that can increase or decrease over time: inventory counts, resource pools, etc.
Variants
Some implementations use a single map with integer values instead of two separate maps, but the principle is the same.
G-Set: Grow-Only Set
Moving from numbers to collections, we consider the simplest CRDT set.
Definition
A G-Set is simply a set that supports addition but not removal:
type GSet a = Set a
Operations
Add
:
add :: Ord a => a -> GSet a -> GSet aadd = insert
Merge
:
merge :: Ord a => GSet a -> GSet a -> GSet amerge = union
Laws and Invariants
Sets with union form a semilattice under the subset relation.
Associative
: Set union is associative
Commutative
: Set union is commutative
Idempotent
:
A ∪ A = A
Monotonic
: Sets only grow
Intuition
Once an element is added to any replica, it will eventually appear in all replicas. There’s no way to remove it.
Tradeoffs
Advantages
:
Minimal overhead (just the set elements)
Simple and efficient
Familiar set semantics
Disadvantages
:
Cannot remove elements
Grows unbounded
No garbage collection
When to Use
Use G-Set for append-only collections where removal is never needed: event logs, collected tags, or immutable registries.
2P-Set: Two-Phase Set
The natural extension of G-Set to support removal.
Definition
A 2P-Set (Two-Phase Set) contains two G-Sets: one for added elements, one for removed elements:
data TwoPhaseSet a = TwoPhaseSet { added :: GSet a , removed :: GSet a }
An element is in the set if it’s been added but not removed:
member :: Ord a => a -> TwoPhaseSet a -> Boolmember x (TwoPhaseSet a r) = x `Set.member` a && x `Set.notMember` r
Operations
Add
:
add x (TwoPhaseSet a r) = TwoPhaseSet (insert x a) r
Remove
:
remove x (TwoPhaseSet a r) = TwoPhaseSet a (insert x r)
Bias toward removal
: If an element appears in the removed set, it’s not in the 2P-Set, even if it’s also in the added set.
Once removed, forever removed
: Once an element is removed at any replica, it will eventually be removed from all replicas and cannot be re-added.
Intuition
The 2P-Set is like marking items in a ledger: you can add entries and you can cross them out, but you can’t un-cross-out an entry. Once something is crossed out (removed), that decision is permanent.
Tradeoffs
Advantages
:
Supports both add and remove
Simple to understand
Deterministic convergence
Disadvantages
:
Cannot re-add removed elements (the “2P” means two-phase: add, then remove, no going back)
Both sets grow monotonically (removed items never truly disappear)
No garbage collection
Not suitable for scenarios where elements might be removed and re-added
When to Use
Use 2P-Set when elements have a lifecycle of “not present → added → removed” and never need to be re-added: task completion tracking, tombstones, or revoked permissions.
LWW-Element-Set: Last-Write-Wins Element Set
What if we want to re-add elements? We need timestamps.
Definition
An LWW-Element-Set associates each element with a timestamp for additions and removals:
data LWWSet a = LWWSet { addTimes :: Map a Timestamp , removeTimes :: Map a Timestamp }
An element is in the set if its most recent operation was an add:
member :: Ord a => a -> LWWSet a -> Boolmember x (LWWSet adds removes) = case (Map.lookup x adds, Map.lookup x removes) of (Just t1, Just t2) -> t1 > t2 (Just _, Nothing) -> True _ -> False
Operations
Add
(with timestamp
t
):
add x t (LWWSet adds removes) = LWWSet (insert x t adds) removes
Remove
(with timestamp
t
):
remove x t (LWWSet adds removes) = LWWSet adds (insert x t removes)
Merge
:
merge (LWWSet a1 r1) (LWWSet a2 r2) = LWWSet (unionWith max a1 a2) (unionWith max r1 r2)
Laws and Invariants
The merge operation is well-defined because
max
over timestamps forms a semilattice.
Timestamp monotonicity
: Each replica must generate increasing timestamps (typically using wall clocks plus replica IDs as tiebreakers).
Bias
: We must decide what happens when add and remove timestamps are equal. Common choices: bias toward add, or bias toward remove.
Intuition
Each element has a timestamp for when it was last added and when it was last removed. The most recent operation wins. When merging, we take the latest add timestamp and latest remove timestamp we’ve seen.
Tradeoffs
Advantages
:
Supports add, remove, and re-add
Can garbage collect old timestamps (carefully)
Natural semantics for many applications
Disadvantages
:
Requires synchronized clocks (or logical clocks with careful replica ID handling)
Concurrent add/remove on the same element may surprise users (one operation is discarded)
Loses information: if two users concurrently add the same element, only one timestamp survives
The “last write wins” semantics mean data loss is possible
When to Use
Use LWW-Element-Set when you need a set with add/remove/re-add capability and can tolerate last-write-wins semantics: user preferences, feature flags, or cached collections where perfect consistency isn’t critical.
Clock Considerations
The biggest pitfall of LWW-Element-Set is clock skew. If replica A’s clock is ahead of replica B’s, then A’s operations will always “win” over B’s, even if B’s operations happened later in real time. Solutions include:
Use hybrid logical clocks (HLC) instead of wall clocks
Use replica IDs as tiebreakers (e.g., timestamps are
(wall_time, replica_id)
pairs)
Accept the inconsistency as a tradeoff
OR-Set: Observed-Remove Set
The most sophisticated set CRDT, solving the re-add problem without LWW semantics.
7
Definition
An OR-Set (Observed-Remove Set) associates each element with a set of unique tags:
type ORSet a = Map a (Set Tag)
Tags are unique identifiers generated when adding an element (e.g.,
(replica_id, sequence_number)
pairs).
An element is in the set if it has any tags:
member :: Ord a => a -> ORSet a -> Boolmember x set = case Map.lookup x set of Just tags -> not (Set.null tags) Nothing -> False
Operations
Add
(with fresh tag
t
):
add x t set = Map.insertWith union x (singleton t) set
Remove
(with observed tags
ts
):
remove x ts set = Map.update (\\tags -> let remaining = tags \\ ts in if Set.null remaining then Nothing else Just remaining) x set
The critical insight: removal removes only the tags that were observed. If concurrent adds create new tags, those survive.
Merge
:
merge = Map.unionWith union
Laws and Invariants
The merge operation forms a semilattice where
s1 ≤ s2
if for all elements
x
,
s1[x] ⊆ s2[x]
.
Add wins
: If an add and remove happen concurrently (the add’s tag wasn’t observed by the remove), the add wins.
Causal consistency
: You can only remove tags you’ve observed (seen in a prior state).
Intuition
Think of each addition as dropping a unique token into a bucket for that element. Removal takes specific tokens out of the bucket. If someone concurrently added a new token you haven’t seen, your removal doesn’t affect it. An element is present if its bucket has any tokens.
This gives us
add-wins semantics
: concurrent add and remove means the element stays in the set (because the remove didn’t observe the add’s tag).
Tradeoffs
Advantages
:
Supports add, remove, and re-add with intuitive semantics
No timestamp requirements
Add-wins semantics are often more desirable than LWW
Properly handles concurrent operations
Disadvantages
:
Larger space overhead (tags per element)
More complex implementation
Need garbage collection strategy for tags
Remove operations need to carry the observed tags (larger messages)
When to Use
Use OR-Set when you need a set with full add/remove/re-add support and can’t tolerate LWW’s data loss: collaborative editing, shopping carts, or any scenario where concurrent adds should be preserved.
Garbage Collection
Old tags can accumulate. Strategies include:
Tombstones
: Keep removed tags for a grace period before discarding
Version vectors
: Use causal history to determine which tags are safe to remove
Bounded tags
: Limit the number of tags per element, using LWW within that bound
Interactive Demo
Experience the add-wins semantics of OR-Set:
LWW-Register: Last-Write-Wins Register
Registers store single values. The simplest register CRDT uses last-write-wins.
Definition
An LWW-Register pairs a value with a timestamp:
data LWWRegister a = LWWRegister { value :: a , timestamp :: Timestamp }
The merge operation is a semilattice with partial order defined by timestamps.
One value wins
: When concurrent writes occur, only one survives (the one with the higher timestamp).
Tradeoffs
Advantages
:
Simple and efficient
Small size (just value + timestamp)
Easy to understand
Disadvantages
:
Loses concurrent updates
Requires clock synchronization
No way to detect or recover lost updates
When to Use
Use LWW-Register for single-value cells where you can tolerate lost updates: user profile fields, configuration settings, or cached computed values.
Interactive Demo
See data loss in action with last-write-wins semantics:
MV-Register: Multi-Value Register
What if we want to preserve concurrent writes instead of discarding them?
Definition
An MV-Register stores a set of value-timestamp pairs:
type MVRegister a = Set (a, Timestamp)
When reading, you get back all concurrently written values (values with incomparable timestamps).
Operations
Write
(with timestamp
t
):
write x t reg = Set.singleton (x, t)
Merge
:
merge reg1 reg2 = let combined = union reg1 reg2 maxTime = maximum (map snd combined) concurrent = filter (\\(_, t) -> t == maxTime) combined in fromList concurrent
More sophisticated: keep values with causally concurrent timestamps, not just the maximum.
Laws and Invariants
The merge preserves all values that might be “current” from different replicas’ perspectives.
Concurrent values preserved
: If two writes happened concurrently, both values appear until a subsequent write supersedes them.
Tradeoffs
Advantages
:
No data loss on concurrent updates
Application can detect and resolve conflicts
More information available for conflict resolution
Disadvantages
:
Returns sets of values, not single values
Application must handle conflict resolution
More complex semantics
Slightly larger space overhead
When to Use
Use MV-Register when concurrent updates must be detected and resolved by application logic: collaborative text fields, conflict-aware configuration, or any scenario where losing an update is unacceptable.
Conflict Resolution
When reading an MV-Register returns multiple values, the application must resolve the conflict. Strategies include:
Present all values to the user (collaborative editing)
Apply a deterministic merge function (e.g., union of tags)
Use application-specific semantics (e.g., prefer non-empty values)
OR-Map: Observed-Remove Map
Maps are common. How do we make them CRDTs?
Definition
An OR-Map is a map where each key is associated with an OR-Set of tagged values:
type ORMap k v = Map k (ORSet (v, Tag))
Alternatively, implement as a composition of OR-Set (for keys) with per-key CRDTs (for values).
Operations
Put
(with fresh tag
t
):
put k v t map = Map.insertWith union k (singleton (v, t)) map
Remove key
:
removeKey k map = Map.delete k map
Remove specific value
:
removeValue k v tags map = -- similar to OR-Set remove
Merge
:
merge = Map.unionWith (OR-Set merge)
Tradeoffs
Advantages
:
Full map operations with CRDT semantics
Can nest other CRDTs as values
Compositional
Disadvantages
:
Complex metadata management
Garbage collection challenges
Larger overhead
When to Use
Use OR-Map when you need a distributed key-value store with CRDT guarantees: collaborative JSON documents, distributed configuration, or nested data structures.
RGA: Replicated Growable Array
Sequences are hard. How do you handle insertions in the middle when replicas disagree on positions?
8
Definition
RGA (Replicated Growable Array) assigns each element a unique ID and stores the sequence as a tree structure based on insertion order and causality.
9
data RGA a = RGA { elements :: Map UID (a, UID) -- element ID -> (value, parent ID) , root :: UID }
Each element knows its “parent” (the element after which it was inserted).
Operations
Insert
(after element with ID
p
, with fresh ID
uid
):
insert p x uid rga = -- complex tree manipulation
Delete
(element with ID
uid
):
delete uid rga = -- mark as tombstone, don't actually remove
Merge
: Merge trees by reconciling insertion orders.
Laws and Invariants
The challenge is that positional indices change as elements are inserted/removed. RGA solves this by using immutable IDs and causal relationships.
Causal order preserved
: If element A was inserted before element B on the same replica, that relationship is preserved globally.
Intuition
Instead of “insert at position 5,” you say “insert after element X.” Since X has a unique ID, this instruction is unambiguous even when other replicas are concurrently inserting elsewhere.
Tradeoffs
Advantages
:
Supports arbitrary insertions and deletions
Eventual consistency for sequences
Handles concurrent edits intuitively
Disadvantages
:
Complex implementation
Large overhead (IDs, tombstones)
No compaction without coordination
Performance degrades with many deletes (tombstones accumulate)
When to Use
Use RGA for collaborative text editing or any replicated sequence where insertions at arbitrary positions must be supported: shared lists, collaborative documents, or distributed queues.
Alternatives
Other sequence CRDTs include:
WOOT
(Without Operational Transformation): similar idea, different structure
Logoot
: uses position identifiers between elements
LSEQ
: adaptive allocation of position identifiers
YATA
: optimizations for text editing workloads
10
Each has different tradeoffs in space overhead, time complexity, and behavior under specific edit patterns.
Causal CRDTs: Adding Causality
Advanced CRDTs incorporate causal tracking using version vectors or similar mechanisms. This enables more sophisticated semantics.
Version Vectors
A version vector tracks the logical clock for each replica:
11
type VersionVector = Map ReplicaId Nat
Operations include the version vector, allowing replicas to determine causality: whether one operation happened-before another, or whether they were concurrent.
Causal Register
Pairs an MV-Register with version vectors:
data CausalRegister a = CausalRegister { values :: Map VersionVector a }
Only keeps values with concurrent version vectors, discarding those that are causally dominated.
Advantages of Causality
More precise conflict detection (concurrent vs. causally ordered)
Larger metadata (version vectors grow with number of replicas)
More complex logic
Still doesn’t eliminate conflicts, just detects them more precisely
Interactive Demo
Explore how version vectors track causality:
Delta CRDTs: Efficient State Transmission
State-based CRDTs have a problem: sending the entire state on every sync is wasteful. Delta CRDTs solve this.
12
The Problem
Consider a G-Counter with 1000 replicas. If replica A increments its count, must it send all 1000 entries to replica B? That’s inefficient: only one entry changed!
The Solution
Instead of sending full state, send only the
delta
: the part of the state that changed since the last sync.
type Delta a = a -- same type as state, but represents only changesmerge :: CRDT a => a -> Delta a -> a
For G-Counter, a delta might be just
{A: 1}
instead of the full map.
Definition
A Delta CRDT extends a state-based CRDT with delta operations:
data DeltaCRDT a = DeltaCRDT { state :: a , lastSent :: Map ReplicaId a -- track what we've sent to each replica }delta :: ReplicaId -> DeltaCRDT a -> Delta adelta replica crdt = state crdt `since` lastSent[replica]
Laws and Invariants
Delta CRDTs must satisfy the same semilattice properties as regular state-based CRDTs, plus:
Delta-state equivalence
: Merging deltas incrementally must be equivalent to merging full states.
Delta composition
: Deltas can be composed:
delta1 ⊔ delta2
is itself a valid delta.
Tradeoffs
Advantages
:
Dramatically reduced bandwidth (send only changes)
Same convergence guarantees as state-based CRDTs
Can batch multiple deltas together
Easier to implement than operation-based CRDTs
Disadvantages
:
Must track what has been sent to each replica
Slightly more complex than pure state-based
Still need full state for new replicas joining
When to Use
Use Delta CRDTs when network bandwidth is a concern or state size is large. Most production CRDT systems use delta-state internally (
Riak
,
Automerge
). If you’re implementing your own CRDT system from scratch, start with deltas. Your future self will thank you.
Example: Delta G-Counter
increment :: ReplicaId -> GCounter -> (GCounter, Delta GCounter)increment r counter = let newCounter = insertWith (+) r 1 counter delta = singleton r 1 -- only the change! in (newCounter, delta)
The delta is just the single updated entry, not the entire counter.
WOOT: Without Operational Transformation
WOOT is a sequence CRDT that predates RGA, with different design choices.
2
Definition
WOOT represents a sequence as a set of character objects with unique IDs, where each character stores references to its previous and next characters:
data WChar a = WChar { charId :: UID , value :: a , prevId :: UID , nextId :: UID , isVisible :: Bool }type WOOT a = Set (WChar a)
Key Insight
Instead of storing a linear sequence, WOOT stores constraints: “this character comes after X and before Y.” When multiple characters claim to be between X and Y, a deterministic ordering (based on UID) resolves the conflict.
Operations
Insert
(after character with ID
prev
, before character with ID
next
):
insert :: a -> UID -> UID -> UID -> WOOT a -> WOOT ainsert val uid prev next woot = insert (WChar uid val prev next True) woot
Delete
(character with ID
uid
):
delete :: UID -> WOOT a -> WOOT adelete uid woot = -- mark character as invisible, don't remove
Linearization
To read the sequence, perform a topological sort respecting the prev/next constraints, filtering out invisible characters.
Tradeoffs
Advantages
:
Strong eventual consistency
No need for causal delivery (constraints handle ordering)
Intuitive model (characters reference neighbors)
Disadvantages
:
Tombstones accumulate (deleted characters remain)
Linearization has O(n²) worst case
More complex than RGA
UIDs must be globally unique and ordered
Comparison with RGA
RGA
: Uses a tree structure, parent-child relationships
WOOT
: Uses bidirectional constraints, more flexible but slower linearization
When to Use
WOOT is primarily of historical interest. Modern implementations prefer RGA or YATA for better performance. But it’s a neat design, and the name alone makes it worth knowing about.
Logoot: Scalable Position Identifiers
Logoot takes a different approach to sequences: instead of linking elements, assign each element a position in a dense order.
13
Definition
Each element has a position identifier that is a sequence of (digit, replicaId) pairs:
type Position = [(Int, ReplicaId)]data LogootElement a = LogootElement { position :: Position , value :: a , isDeleted :: Bool }type Logoot a = Set (LogootElement a)
Positions are ordered lexicographically.
Key Insight
Positions form a dense order: between any two positions, you can always allocate a new position. To insert between positions
p1
and
p2
, generate a new position
p
such that
p1 < p < p2
.
Operations
Insert
(between positions
before
and
after
):
insert :: a -> Position -> Position -> Logoot a -> Logoot ainsert val before after logoot = let newPos = allocatePosition before after currentReplicaId element = LogootElement newPos val False in insert element logoot
Position Allocation
:
allocatePosition :: Position -> Position -> ReplicaId -> PositionallocatePosition before after replicaId = -- Find a position between before and after -- Use replicaId as tiebreaker for deterministic ordering
Delete
:
delete :: Position -> Logoot a -> Logoot adelete pos logoot = -- mark element at pos as deleted
Laws and Invariants
Deterministic ordering
: Elements are always ordered by their positions.
Unique positions
: Each insert generates a unique position (using replica ID in the position).
Tradeoffs
Advantages
:
No need to reference other elements by ID
Simpler merge than WOOT
Positions are self-describing (no need to look up IDs)
Can insert without knowing the full document structure
Disadvantages
:
Position identifiers grow over time (especially with many edits)
Still accumulates tombstones
Position allocation algorithm is complex
Pathological cases where positions become very long
LSEQ: Adaptive Positions
LSEQ improves on Logoot by using an adaptive allocation strategy. Instead of always allocating positions the same way, LSEQ alternates between strategies to keep positions shorter on average.
14
When to Use
Use Logoot/LSEQ when you need a sequence CRDT and want simpler semantics than RGA/WOOT. The tradeoff is position identifier growth.
Tree CRDTs: Hierarchical Data
Extending CRDTs to trees is challenging because parent-child relationships must be maintained consistently.
The Problem
Trees have structural constraints:
Each node has exactly one parent (except root)
No cycles allowed
Moving a node changes parent-child relationships
How do we handle concurrent operations like:
Two replicas move the same node to different parents?
One replica moves node A under node B while another moves B under A?
Approaches
OR-Tree
: Combine OR-Set with parent pointers, using conflict resolution strategies when multiple parents are observed.
CRDT-Tree
: Use causal ordering to determine which move operations take precedence.
Log-based Trees
: Store operations in a replicated log and rebuild tree structure on read.
OR-Tree Definition
type ORTree a = Map NodeId (ORSet ParentId, a)
Each node stores an OR-Set of potential parents. Conflict resolution:
Last-write-wins
: Use timestamps to pick winning parent
First-wins
: The first parent observed wins
Merge
: Allow nodes to have multiple parents temporarily, application resolves
Tradeoffs
Advantages
:
Can represent hierarchical data distributedly
Handles concurrent structural changes
Disadvantages
:
Complex conflict resolution strategies
Must prevent cycles (may require rejecting some operations)
Moving subtrees is complicated
High metadata overhead
When to Use
Use Tree CRDTs for file systems, organizational charts, or document outlines where the hierarchy must be replicated. Be prepared for complexity in handling concurrent structural changes.
Alternatives
For many use cases, an OR-Map with explicit parent fields is simpler than a full Tree CRDT, even if it doesn’t enforce tree constraints at the CRDT level.
A practical example combining multiple CRDT concepts.
The Domain
An e-commerce shopping cart must support:
Add product to cart
Remove product from cart
Change quantity
Work offline and sync later
Naive Approach: LWW Map
type CartLWW = Map ProductId (Int, Timestamp)
Problems:
Concurrent additions of the same product (one wins)
Remove on one device, add on another (one wins, data loss)
Better: OR-Set + PN-Counter
type ShoppingCart = Map ProductId PNCounter
Use OR-Set semantics for which products are in cart
Use PN-Counter for quantities
Add-wins semantics for products (if concurrently added and removed, item stays)
Quantities merge correctly (concurrent +1 and +2 becomes +3)
changeQuantity :: ProductId -> Int -> ReplicaId -> ShoppingCart -> ShoppingCartchangeQuantity pid delta replica cart = let counter = lookupOr emptyCounter pid cart updated = if delta > 0 then incrementN replica delta counter else decrementN replica (-delta) counter in insert pid updated cart
Tradeoffs
Advantages
:
Handles all operations correctly
No data loss on concurrent modifications
Intuitive semantics for users
Disadvantages
:
PN-Counters can go negative (need validation)
Must track all replicas (for PN-Counter)
Slightly more overhead than simple LWW
This example shows how combining basic CRDTs creates sophisticated application-level data structures.
Practical Considerations
Choosing a CRDT
The choice of CRDT depends on your requirements:
Do you need only additions?
Use G-Counter or G-Set.
Do you need removals but not re-additions?
Use 2P-Set.
Can you tolerate last-write-wins?
Use LWW-Element-Set or LWW-Register.
Do you need to preserve concurrent operations?
Use OR-Set or MV-Register.
Do you have sequences?
Use RGA or similar sequence CRDT.
Do you need nested structures?
Use OR-Map with nested CRDTs.
Garbage Collection
Garbage collection is one of the most challenging practical problems with CRDTs. The fundamental tension: CRDTs achieve convergence by monotonically accumulating information, but production systems can’t grow unbounded forever.
The Problem in Detail
Consider an OR-Set used for a collaborative todo list. Each time someone adds a task and removes it, we accumulate:
A unique tag for the addition (never removed)
A tombstone tracking the removal (never removed)
After 10,000 tasks have been created and completed, our “empty” todo list still contains 10,000 tags worth of metadata. In a G-Counter tracking page views, we keep a separate count for every replica that has ever incremented the counter—even if that replica hasn’t been online in years. For sequence CRDTs like RGA or WOOT, every deleted character becomes a tombstone that must be retained indefinitely. A 1000-character document that’s been heavily edited might internally contain 50,000 tombstones.
The core issue:
CRDTs converge by retaining enough information to handle any possible merge
. If replica A discards metadata about some operation, and replica B (which has been offline for weeks) later tries to merge its state—which still references that metadata—the merge may produce incorrect results.
Why Can’t We Just Delete Old Data?
Let’s make this concrete with an OR-Set example:
-- Replica A's stateorset_a = {"todo-1": {tag_1, tag_2}}-- Replica B's state (has been offline)orset_b = {"todo-1": {tag_1, tag_2, tag_3}}-- Replica A removes todo-1, observing tags {tag_1, tag_2}-- Now A's state is:orset_a = {}-- If A garbage collects and forgets about tags {tag_1, tag_2},-- then later merges with B:merge(orset_a, orset_b) = {"todo-1": {tag_3}}-- The element reappears! (Zombie resurrection)
The element we removed comes back because we lost the causal information about which tags we had observed and removed. This is the fundamental safety problem with CRDT garbage collection.
Strategies and Tradeoffs
Time-Based Expiry
The simplest approach: discard metadata older than some threshold (e.g., 30 days). This works well when you can guarantee all replicas sync within that window.
gcTombstones :: Timestamp -> ORSet a -> ORSet agcTombstones cutoff set = -- Remove tags older than cutoff Map.mapMaybe (\\tags -> let recent = Set.filter (\\t -> tagTime t > cutoff) tags in if Set.null recent then Nothing else Just recent) set
Advantages
:
Simple to implement
No coordination required
Works well for frequently-syncing systems
Disadvantages
:
Unsafe if replicas can be offline longer than the grace period
Must choose grace period conservatively (wasted space)
Zombie resurrection if threshold is too aggressive
When to use
: Mobile apps where you can bound offline time (e.g., “you must sync at least once per week”).
Coordinated Garbage Collection
Use distributed consensus to agree on what’s safe to discard. Once all replicas acknowledge they’ve received a particular update, the corresponding metadata can be safely removed.
data GCState = GCState { pendingGC :: Set Tag -- Tags eligible for GC , replicaAcks :: Map ReplicaId (Set Tag) -- What each replica has seen }-- When all replicas have acked a tag, it's safe to removesafeToDiscard :: GCState -> Set TagsafeToDiscard (GCState pending acks) = -- Tags that all known replicas have acknowledged Set.filter (\\tag -> all (Set.member tag) (Map.elems acks)) pending
Advantages
:
Completely safe (no zombie resurrections)
Can garbage collect aggressively once consensus is reached
Works with arbitrary offline periods
Disadvantages
:
Requires coordination (defeats CRDT’s main selling point!)
Slow convergence if some replicas are rarely online
Must track all replicas (what about replicas that never come back?)
When to use
: When you have a bounded, known set of replicas and can tolerate periodic coordination rounds.
Version Vectors for Causal Tracking
Use version vectors to track causal history. Metadata can be discarded once it’s been causally superseded at all replicas.
data CausalORSet a = CausalORSet { elements :: Map a (Set (Tag, VersionVector)) , replicaVersions :: Map ReplicaId VersionVector -- Last known VV per replica }-- A tag can be GC'd if its version vector is dominated by all known replicascanDiscardTag :: (Tag, VersionVector) -> Map ReplicaId VersionVector -> BoolcanDiscardTag (_, tagVV) replicaVVs = all (\\replicaVV -> tagVV `happenedBefore` replicaVV) (Map.elems replicaVVs)
This is more sophisticated: we track causality explicitly and can safely discard tags that are in the causal past of all known replicas.
Advantages
:
More precise than time-based expiry
No coordination needed for the happy path
Safe as long as causal tracking is correct
Disadvantages
:
Version vectors add significant overhead (O(replicas) per operation)
Still requires tracking all replicas
Complex to implement correctly
What about new replicas that join later?
When to use
: Systems already using version vectors for causal consistency (Riak, Cassandra-style systems).
Bounded Structures with Fallback
Limit metadata size and use LWW semantics when bounds are exceeded. For example, keep at most 1000 tags per element in an OR-Set. If we exceed that, discard the oldest tags and accept potential anomalies.
addWithBound :: Ord a => a -> Tag -> Int -> ORSet a -> ORSet aaddWithBound x tag maxTags set = let currentTags = Map.findWithDefault Set.empty x set newTags = Set.insert tag currentTags boundedTags = if Set.size newTags > maxTags then Set.fromList $ take maxTags $ sortBy (comparing tagTimestamp) (Set.toList newTags) else newTags in Map.insert x boundedTags set
Advantages
:
Bounded space overhead (guaranteed)
No coordination needed
Graceful degradation (becomes LWW-ish when bounded)
Disadvantages
:
Correctness sacrificed for space
May lose concurrent operations
Choosing the bound is difficult (too small = frequent anomalies, too large = still wasteful)
When to use
: When you must have bounded space (embedded systems, strict SLAs) and can tolerate occasional anomalies.
Checkpoint and Rebase
Periodically create a “checkpoint” snapshot and discard history before that point. New replicas joining after the checkpoint start from the snapshot.
data CheckpointedCRDT a = CheckpointedCRDT { baselineState :: a -- Snapshot at checkpoint , checkpointTime :: Timestamp , deltaSince :: [Delta a] -- Operations since checkpoint }-- Create a new checkpoint, discarding old deltascheckpoint :: CheckpointedCRDT a -> CheckpointedCRDT acheckpoint crdt = CheckpointedCRDT { baselineState = foldl merge (baselineState crdt) (deltaSince crdt) , checkpointTime = currentTime , deltaSince = [] }
Replicas that haven’t synced since before the checkpoint must do a full state sync rather than incremental merge.
Advantages
:
Can aggressively prune old history
Conceptually clean (like Git’s shallow clones)
Works well with mostly-online systems
Disadvantages
:
Replicas offline during checkpoint period lose incremental sync
Need to track which replicas are pre-checkpoint
Full state sync is expensive
When to use
: Collaborative editing systems where most users are online most of the time (Google Docs, Figma).
Practical Recommendations
For most applications, a
hybrid approach
works best:
Use time-based expiry with a conservative grace period (90 days)
Track the oldest unsynced replica timestamp
Only discard metadata older than:
min(graceperiod, oldest_unsynced - safety_margin)
Provide manual “compact” operations for administrators
Use bounded structures for untrusted/public replicas
Without some form of garbage collection, CRDT state grows unbounded and will eventually exhaust memory or storage. The question isn’t whether to implement GC, but which tradeoffs you’re willing to accept.
And, realistically speaking, you’re unlikely to implement a system that only uses CRDTs and no other data storage. You’ll almost certainly have some sort of traditional database to store your data, which
you can probably use to periodically coordinate garbage collection.
A note on Causal Consistency
CRDTs themselves don’t enforce causal delivery. You need a causal broadcast protocol to ensure operations are delivered respecting happens-before relationships. Without causal delivery, some CRDTs (especially operation-based ones) may behave incorrectly.
Performance
Different CRDTs have different performance characteristics. Consider your read/write ratio, expected contention, and replica count when choosing:
CRDT Type
Space Complexity
Add/Insert
Remove/Delete
Merge
Read/Query
Notes
G-Counter
O(r)
O(1)
N/A
O(r)
O(r)
Space: one counter per replica
PN-Counter
O(r)
O(1)
O(1)
O(r)
O(r)
Double the space of G-Counter
G-Set
O(e)
O(1)
N/A
O(e)
O(1)
Standard set operations
2P-Set
O(e)
O(1)
O(1)
O(e)
O(1)
Both added and removed sets grow
LWW-Element-Set
O(e)
O(1)
O(1)
O(e)
O(1)
Can GC old timestamps carefully
OR-Set
O(e × t)
O(1)
O(t)
O(e × t)
O(1)
Tags accumulate, needs GC
LWW-Register
O(1)
O(1)
N/A
O(1)
O(1)
Minimal overhead
MV-Register
O(concurrent)
O(1)
N/A
O(c)
O(c)
Returns set of concurrent values
OR-Map
O(k × t)
O(1)
O(t)
O(k × t)
O(1)
Per-key OR-Set overhead
RGA
O(n + d)
O(log n)
O(log n)
O(n + d)
O(n)
Tombstones accumulate
WOOT
O(n + d)
O(n²) worst
O(log n)
O(n + d)
O(n²) worst
Linearization is expensive
Logoot/LSEQ
O(n × p)
O(log n)
O(log n)
O(n)
O(n log n)
Position identifiers grow
Legend:
r
= number of replicas
e
= number of elements in set
t
= average tags per element (OR-Set)
k
= number of keys in map
n
= number of visible elements in sequence
d
= number of deleted elements (tombstones)
c
= number of concurrent writes
p
= average position identifier length
Key Observations:
Counter CRDTs
scale with replica count, not operation count. A billion increments still cost O(replicas) space.
Set CRDTs
generally have constant-time operations, but OR-Set’s space grows with tags unless garbage collected.
Sequence CRDTs
suffer from tombstone accumulation. RGA is typically faster than WOOT in practice despite similar asymptotic complexity.
Position-based sequences
(Logoot/LSEQ) trade time complexity for avoiding explicit parent pointers, but position identifiers can grow pathologically.
Merge operations
are often the bottleneck in high-throughput systems. Delta CRDTs dramatically improve merge performance by sending only changes.
Libraries and Implementations
Many CRDT libraries exist:
Automerge
: Full-featured CRDT library for JSON-like documents
15
CRDTs are not a silver bullet. They trade coordination for metadata, strong consistency for eventual consistency, and simplicity for convergence guarantees. But in scenarios where availability matters more than immediate consistency, they’re remarkably powerful.
There is no “best” CRDT, only CRDTs suited to different problems; the CRDT you choose depends entirely on your application’s semantics:
What operations do you need (add, remove, re-add)?
Can you tolerate lost updates?
Do you need to detect conflicts or resolve them automatically?
What’s your tolerance for metadata overhead?
The CRDT abstraction is elegant in theory, but bewildering in practice because there are so many instances with subtle differences. Hopefully this guide has cut through some of the confusion, and given you a good intuition for how they work and when to use them.
I honestly still haven’t hit a use case for CRDTs that I couldn’t solve with a traditional database and some custom coordination logic. But sometimes we just want to learn for the sake of learning. If you beat me to it, let me know!
WOOT was introduced by Oster, Urso, Molli, and Imine in
“Data Consistency for P2P Collaborative Editing”
(2006). The name is a play on “OT” (Operational Transformation), emphasizing that it achieves similar goals “WithOut OT.” WOOT was one of the first practical sequence CRDTs and influenced many subsequent designs.
↩
↩
2
State-based CRDTs are also called “convergent” replicated data types (CvRDT). The “Cv” stands for “convergent” - emphasizing that replicas converge to the same state by repeatedly applying the join operation.
↩
Operation-based CRDTs are also called “commutative” replicated data types (CmRDT). They require causal delivery of operations - if operation A happened before operation B on the same replica, B must not be delivered before A at any other replica.
↩
The space complexity is O(n) where n is the number of replicas, not the number of increments. This means G-Counters scale well with the number of operations but require tracking all replicas that have ever incremented the counter.
↩
The OR-Set (Observed-Remove Set) was introduced by Shapiro et al. in their
2011 technical report
. It’s also known as the “Add-Wins Set” because concurrent add and remove operations result in the element remaining in the set. The key innovation is using unique tags to distinguish between different additions of the same element.
↩
Sequence CRDTs are particularly challenging because positional indices change as elements are inserted or deleted. Unlike sets or counters where elements have stable identity, sequences must maintain ordering despite concurrent modifications at arbitrary positions.
↩
YATA (Yet Another Transformation Approach) was developed by Kevin Jahns for the
Yjs
collaborative editing library. It combines ideas from RGA and WOOT while optimizing for the common case of sequential insertions (typing). Yjs is used in production by companies like Braid, Row Zero, and others for real-time collaboration.
↩
Delta CRDTs were introduced by Almeida, Shoker, and Baquero in
“Delta State Replicated Data Types”
(2018). They bridge the gap between state-based and operation-based CRDTs, achieving operation-based bandwidth efficiency while maintaining state-based simplicity. Most production CRDT systems (
Riak
,
Automerge
) use delta-state internally.
↩
LSEQ was introduced by Nédelec, Molli, Mostéfaoui, and Desmontils in
“LSEQ: An Adaptive Structure for Sequences in Distributed Collaborative Editing”
(2013). The key innovation is using different allocation strategies (boundary+ vs boundary-) based on tree depth, which keeps position identifiers shorter in practice compared to Logoot’s fixed strategy.
↩
Yjs
, created by Kevin Jahns, is optimized for text editing and uses the YATA algorithm. It’s notably faster than Automerge for text operations and includes bindings for popular editors like
CodeMirror
,
Monaco
,
Quill
, and
ProseMirror
.
↩
Riak
, a distributed database from Basho, was one of the first production systems to adopt CRDTs (2012). It implements counters, sets, and maps as
native data types
, using Delta CRDTs internally to minimize bandwidth. Sadly, the company collapsed dramatically, and the project was abandoned for quite some time. I think it’s still around in a diminished form, but haven’t tried it in a while.
↩
Redis Enterprise’s CRDT support
(Active-Active deployment) uses operation-based CRDTs with causal consistency. It supports strings, hashes, sets, and sorted sets with CRDT semantics, enabling multi-master Redis deployments.
↩
AntidoteDB
is a research database from the
SyncFree project
that makes CRDTs the primary abstraction. Unlike other databases where CRDTs are a feature, AntidoteDB is designed from the ground up around CRDT semantics, providing highly available transactions over CRDTs.
↩
Africa's forests have switched from absorbing to emitting carbon
Professor Heiko Balzter, Dr. Nezha Acil (right) and University of Leicester colleagues at a zoobotanical garden at the Museu Emilio Goeldi in Belém, with trees and animals from the Amazon. Credit: University of Leicester
New research warns that Africa's forests, once vital allies in the fight against climate change, have turned from a carbon sink into a carbon source.
A new international study published in
Scientific Reports
and led by researchers at the National Center for Earth Observation at the Universities of Leicester, Sheffield and Edinburgh reveals that Africa's forests, which have long absorbed carbon dioxide from the atmosphere, are now releasing more carbon than they remove.
This alarming shift, which happened after 2010, underscores the urgent need for stronger global action to protect forests, a major focus of the
COP30 Climate Summit
that concluded last week in Brazil.
How researchers measured forest changes
Using advanced satellite data and machine learning, the researchers tracked more than a decade of changes in aboveground forest biomass, the amount of carbon stored in trees and woody vegetation. They found that while Africa gained carbon between 2007 and 2010, widespread forest loss in tropical rainforests has since tipped the balance.
Between 2010 and 2017, the continent lost approximately 106 billion kilograms of forest biomass per year. That is equivalent to the weight of about 106 million cars. The losses are concentrated in tropical moist broadleaf forests in countries such as the Democratic Republic of Congo, Madagascar, and parts of West Africa, driven by deforestation and forest degradation. Gains in savanna regions due to shrub growth have not been enough to offset the losses.
Implications for climate policy and action
Professor Heiko Balzter, senior author and Director of the Institute for Environmental Futures at the University of Leicester, said, "This is a critical wake-up call for global climate policy. If Africa's forests are no longer absorbing carbon, it means other regions and the world as a whole will need to cut greenhouse gas emissions even more deeply to stay within the 2°C goal of the Paris Agreement and avoid catastrophic climate change. Climate finance for the Tropical Forests Forever Facility must be scaled up quickly to put an end to global deforestation for good."
The research draws on data from NASA's spaceborne laser instrument called GEDI and Japan's ALOS radar satellites, combined with machine learning and thousands of on-the-ground forest measurements. The result is the most detailed map to date of biomass changes across the African continent, covering a decade, at a resolution fine enough to capture local deforestation patterns.
The findings come as the COP30 Presidency announced the new
Tropical Forests Forever Facility
, which aims to mobilize billions of pounds for climate finance. It would pay forested countries to leave their tropical forests untouched. The results show that without urgent action to stop forest loss, the world risks losing one of its most important natural carbon buffers.
Calls for stronger forest protection
Dr. Nezha Acil, co-author from the National Center for Earth Observation at the University of Leicester's Institute for Environmental Futures, said, "Stronger forest governance, enforcement against illegal logging, and large-scale restoration programs such as AFR100, which aims to restore 100 million hectares of African landscapes by 2030, can make a huge difference in reversing the damage done."
Dr. Pedro Rodríguez-Veiga, who carried out the bulk of the analysis at NCEO and University of Leicester and now working at Sylvera Ltd., said, "This study provides critical risk data for Sylvera and the wider voluntary carbon market (VCM), and shows that deforestation isn't just a local or regional issue—it's changing the global carbon balance. If Africa's forests turn into a lasting carbon source, global climate goals will become much harder to achieve. Governments, the private sector, and NGOs must collaborate to fund and support initiatives that protect and enhance our forests."
More information:
Loss of tropical moist broadleaf forest has turned Africa's forests from a carbon sink into a source,
Scientific Reports
(2025).
DOI: 10.1038/s41598-025-27462-3
Citation
:
Africa's forests have switched from absorbing to emitting carbon, new study finds (2025, November 28)
retrieved 28 November 2025
from https://phys.org/news/2025-11-africa-forests-absorbing-emitting-carbon.html
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no
part may be reproduced without the written permission. The content is provided for information purposes only.
Google denies 'misleading' reports of Gmail using your emails to train AI
is a senior reporter covering technology, gaming, and more. He joined The Verge in 2019 after nearly two years at Techmeme.
Google is pushing back on
viral social media posts
and articles like
this one by
Malwarebytes
,
claiming Google has changed its policy to use your Gmail messages and attachments to train AI models, and the only way to opt out is by disabling “smart features” like spell checking.
But Google spokesperson Jenny Thomson tells
The Verge
that “these reports are misleading – we have not changed anyone’s settings, Gmail Smart Features have existed for many years, and we do not use your Gmail content for training our Gemini AI model.”
You may want to double-check your settings anyway, as one
Verge
staffer also says they had opted out of some of the Smart Features, but had been opted back in to having them on. In January,
Google updated
its smart feature personalization settings so that you could turn off the features for Google Workspace and for other Google products (like Maps and Wallet) independently of each other.
In addition to things like spell checking, having Gmail’s smart features turned on enables
features like
tracking orders or easily adding flights from Gmail to your calendar. Enabling the feature in Workspace says that “you agree to let Google Workspace use your Workspace content and activity to personalize your experience across Workspace,” according to the settings page, but according to Google, that does not mean handing over the content of your emails to use for AI training.
Follow topics and authors
from this story to see more like this in your personalized homepage feed and to receive email updates.
Jay Peters
EU Council Approves New "Chat Control" Mandate Pushing Mass Surveillance
European governments have taken another step toward reviving the EU’s controversial
Chat Control agenda
, approving a new negotiating mandate for the Child Sexual Abuse Regulation in a closed session of the Council of the European Union on November 26.
The measure, presented as a tool for child protection, is once again drawing heavy criticism for its surveillance implications and the way it reshapes private digital communication in Europe.
Unlike earlier drafts, this version
drops the explicit obligation
for companies to scan all private messages but quietly introduces what opponents describe as an indirect system of pressure.
It rewards or penalizes online services depending on whether they agree to carry out “voluntary” scanning, effectively making intrusive monitoring a business expectation rather than a legal requirement.
Former MEP Patrick Breyer, a long-standing defender of digital freedom and one of the most vocal opponents of the plan, said the deal “paves the way for a permanent infrastructure of mass surveillance.”
According to him, the Council’s text replaces legal compulsion with financial and regulatory incentives that push major US technology firms toward indiscriminate scanning.
He warned that the framework also brings “anonymity-breaking age checks” that will turn ordinary online use into an exercise in identity verification.
The new proposal, brokered largely through Danish mediation, comes months after the original “Chat Control 1.0” regulation appeared to have been shelved following widespread backlash.
It reinstates many of the same principles, requiring providers to assess their potential “risk” for child abuse content and to apply “mitigation measures” approved by authorities. In practice, that could mean pressure to install scanning tools that probe both encrypted and unencrypted communications.
Czech MEP Markéta Gregorová called the Council’s position “a disappointment…Chat Control…opens the way to blanket scanning of our messages.”
Similar objections emerged across Europe.
In the Netherlands, members of parliament forced their government to vote against the plan, warning that it combines “mandatory age verification” with a “voluntary obligation” scheme that could penalize any company refusing to adopt invasive surveillance methods. Poland and the Czech Republic also voted against, and Italy abstained.
Former Dutch MEP Rob Roos accused Brussels of operating “behind closed doors,” warning that “Europe risks sliding into digital authoritarianism.”
Beyond parliamentarians, independent voices such as Daniel Vávra, David Heinemeier Hansson, and privacy-focused company Mullvad have spoken out against the Council’s position, calling it a direct threat to private communication online.
Despite the removal of the word “mandatory,” the structure of the new deal appears to preserve mass scanning in practice.
Breyer described it as a “Trojan Horse,” arguing that by calling the process “voluntary,” EU governments have shifted the burden of surveillance to tech companies themselves.
The Council’s mandate introduces three central dangers that remain largely unacknowledged in the public debate.
First, so-called “voluntary scanning” turns mass surveillance into standard operating procedure. The proposal extends the earlier temporary regulation that allowed service providers to scan user messages and images without warrants.
Authorities like Germany’s Federal Criminal Police Office have reported that roughly
half the alerts from such systems are baseless
, often involving completely legal content flagged by flawed algorithms. Breyer said these systems leak “tens of thousands of completely legal, private chats” to law enforcement every year.
Second, the plan effectively erases anonymous communication. To meet the new requirement to “reliably identify minors,” providers will have to implement universal age checks. This likely means ID verification or face scans before accessing even basic services such as email or messaging apps.
For journalists, activists, and anyone who depends on anonymity for protection, this system could make easy private speech functionally impossible.
Technical experts have repeatedly warned that age estimation “cannot be performed in a privacy-preserving way” and carries “a disproportionate risk of serious privacy violation and discrimination.”
Third, it risks digitally isolating young people. Under the Council’s framework, users under 17 could be blocked from many platforms unless they pass strict identity verification, including chat-enabled games and messaging services. Breyer called this idea “pedagogical nonsense,” arguing that it excludes teenagers instead of helping them develop safe online habits.
Member states remain divided: the Netherlands, Poland, and the Czech Republic rejected the text, while Italy abstained. Negotiations between the European Parliament and the Council are expected to begin soon, aiming for a final version before April 2026.
Breyer warned that the apparent compromise is no real retreat from surveillance. “The headlines are misleading: Chat Control is not dead, it is just being privatized,” he said. “We are facing a future where you need an ID card to send a message, and where foreign black-box AI decides if your private photos are suspicious. This is not a victory for privacy; it is a disaster waiting to happen.”
The UK Has It Wrong on Digital ID. Here’s Why.
Electronic Frontier Foundation
www.eff.org
2025-11-28 10:10:25
In late September, the United Kingdom’s Prime Minister Keir Starmer announced his government’s plans to introduce a new digital ID scheme in the country to take effect before the end of the Parliament (no later than August 2029). The scheme will, according to the Prime Minister, “cut the faff” in pr...
In late September, the United Kingdom’s Prime Minister Keir Starmer
announced
his government’s plans to introduce a new digital ID scheme in the country to take effect before the end of the Parliament (no later than August 2029). The scheme will,
according to
the Prime Minister, “cut the
faff
” in proving people’s identities by creating a virtual ID on personal devices with information like people’s name, date of birth, nationality or residency status, and photo to verify their right to live and work in the country.
This is the latest example of a government creating a new digital system that is fundamentally incompatible with a privacy-protecting and human rights-defending democracy. This past year alone, we’ve seen
federal agencies
across the United States explore digital IDs to prevent fraud, the Transportation Security Administration accepting “
Digital passport IDs
” in Android, and
states
contracting with mobile driver’s license providers (mDL). And
as we’ve said
many times
, digital ID is not for everyone and policymakers should ensure better access for people with or without a digital ID.
But instead, the UK is pushing forward with its plans to rollout digital ID in the country. Here’s three reasons why those policymakers have it wrong.
Digital ID allows the state to determine what you can access, not just verify who you are, by functioning as a key to opening—or closing—doors to essential services and experiences.
Mission Creep
In his initial announcement, Starmer
stated
: “You will not be able to work in the United Kingdom if you do not have digital ID. It's as simple as that.” Since then, the government has been forced to
clarify
those remarks: digital ID will be mandatory to prove the right to work, and will only take effect after the scheme's proposed introduction in 2028, rather than retrospectively.
The government has also
confirmed
that digital ID will not be required for pensioners, students, and those not seeking employment, and will also not be mandatory for accessing medical services, such as visiting hospitals. But as civil society organizations are
warning
, it's possible that the required use of digital ID will not end here. Once this data is collected and stored, it provides a multitude of opportunities for government agencies to expand the scenarios where they demand that you prove your identity before entering physical and digital spaces or accessing goods and services.
The government may also be able to request information from workplaces on who is registering for employment at that location, or collaborate with banks to aggregate different data points to determine who is self-employed or not registered to work. It potentially leads to situations where state authorities can treat the entire population with suspicion of not belonging, and would shift the power dynamics even further towards government control over our freedom of movement and association.
And this is not the first time that the UK has attempted to introduce digital ID: politicians previously proposed
similar schemes
intended to control the spread of COVID-19, limit immigration, and fight terrorism. In a country increasing the deployment of other surveillance technologies like
face recognition technology
, this raises additional concerns about how digital ID could lead to new divisions and inequalities based on the data obtained by the system.
These concerns compound the underlying narrative that digital ID is being introduced to
curb illegal immigration
to the UK: that digital ID would make it harder for people without residency status to work in the country because it would lower the possibility that anyone could borrow or steal the identity of another. Not only is there little evidence to prove that digital ID will limit illegal immigration, but checks on the right to work in the UK already exist. This is nothing more than inflammatory and misleading; Liberal Democrat leader Ed Davey
noted
this would do “next to nothing to tackle channel crossings.”
Inclusivity is Not Inevitable, But Exclusion Is
While the government
announced
that their digital ID scheme will be inclusive enough to work for those without access to a passport, reliable internet, or a personal smartphone, as
we’ve been saying
for years, digital ID leaves vulnerable and marginalized people not only out of the debate and ultimately out of the society that these governments want to build. We remain concerned about the potential for digital identification to exacerbate existing social inequalities, particularly for those with reduced access to digital services or people seeking asylum.
The UK government has said a
public consultation
will be launched later this year to explore alternatives, such as physical documentation or in-person support for the homeless and older people; but it’s short-sighted to think that these alternatives are viable or functional in the long term. For example, UK organization Big Brother Watch
reported
that about only 20% of Universal Credit applicants can use online ID verification methods.
These individuals should not be an afterthought that are attached to the end of the announcement for further review. It is essential that if a tool does not work for those without access to the array of essentials, such as the internet or the physical ID, then it should not exist.
Digital ID schemes also
exacerbate other inequalities
in society, such as abusers who will be able to prevent others from getting jobs or proving other statuses by denying access to their ID. In the same way, the scope of digital ID may be expanded and people could be forced to prove their identities to different government agencies and officials, which may raise issues of institutional discrimination when phones may not load, or when the Home Office has incorrect information on an individual. This is not an unrealistic scenario considering the
frequency
of internet connectivity issues, or circumstances like passports and other documentation expiring.
Any identification issued by the government with a centralized database is a power imbalance that can only be enhanced with digital ID.
Attacks on Privacy and Surveillance
Digital ID systems expand the number of entities that may access personal information and consequently use it to track and surveil. The UK government has nodded to this threat. Starmer
stated
that the technology would “absolutely have very strong encryption” and wouldn't be used as a surveillance tool. Moreover, junior Cabinet Office Minister Josh Simons
told Parliament
that “data associated with the digital ID system will be held and kept safe in secure cloud environments hosted in the United Kingdom” and that “the government will work closely with expert stakeholders to make the programme effective, secure and inclusive.”
But if digital ID is needed to verify people’s identities multiple times per day or week, ensuring end-to-encryption is the bare minimum the government should require. Unlike sharing a National Insurance Number, a digital ID will show an array of personal information that would otherwise not be available or exchanged.
This would create a rich environment for hackers or hostile agencies to obtain swathes of personal information on those based in the UK. And if previous schemes in the country are anything to go by, the government’s ability to handle giant databases is questionable. Notably, the eVisa’s multitude of failures last year
illustrated the harms
that digital IDs can bring, with issues like government system failures and internet outages leading to people being
detained, losing their jobs, or being made homeless
. Checking someone’s identity against a database in real-time requires a host of online and offline factors to work, and the UK is yet to take the structural steps required to remedying this.
Moreover, we know that the
Cabinet Office
and the
Department for Science, Innovation and Technology
will be
involved in the delivery
of digital ID and are clients of U.S.-based tech vendors, specifically Amazon Web Services (AWS). The UK government has
spent millions
on AWS (and
Microsoft
) cloud services in recent years, and the
One Government Value Agreement (OGVA)
—first introduced in 2020 and of which provides discounts for cloud services by contracting with the UK government and public sector organizations as a single client—is still active. It is essential that any data collected is not stored or shared with third parties, including through cloud agreements with companies outside the UK.
And even if the UK government published comprehensive plans to ensure data minimization in its digital ID, we will still strongly oppose any national ID scheme. Any identification issued by the government with a centralized database is a power imbalance that can only be enhanced with digital ID, and both the
public
and
civil
society
organizations
in the country are against this.
Ways Forward
Digital ID regimes strip privacy from everyone and further marginalize those seeking asylum or undocumented people. They are pursued as a technological solution to offline problems but instead allow the state to determine what you can access, not just verify who you are, by functioning as a key to opening—or closing—doors to essential services and experiences.
We cannot base our human rights on the government’s mere promise to uphold them. On December 8th, politicians in the country will be debating a
petition
that reached almost 3 million signatories rejecting mandatory digital ID. If you’re based in the UK, you can
contact
your MP
(external campaign links) to oppose the plans for a digital ID system.
The case for digital identification has not been made. The UK government must listen to people in the country and say no to digital ID.
What are you doing this weekend?
Lobsters
lobste.rs
2025-11-28 09:55:11
Feel free to tell what you plan on doing this weekend and even ask for help or feedback.
Please keep in mind it’s more than OK to do nothing at all too!...
When was the last time you had a good day of work? The kind where you got into flow and stayed there long enough to think deeply about a problem?
Paul Graham wrote
about this
in 2009: a single meeting can wreck an entire half-day for someone who needs uninterrupted time to build something. Sixteen years later, we’ve added Slack, Teams, always-on video calls, and a culture of instant responsiveness. The problem has gotten worse, with the pandemic turning things to 11 but the conversation stays frustratingly vague. We know focus is dying. We can’t say how bad it is or what would fix it.
In this post, I’ll show you what interruption-driven work looks like when you model it with math. Three simple parameters determine whether your day is productive or a write-off. We’ll simulate hundreds of days and build a map of the entire parameter space so you can see exactly where you are and what happens when you change.
One Day in Detail
Let’s start by drawing what one of those “lost days” actually looks like.
You managed
3
h
58
m
of focus time and
1
deep work blocks (>
60
m
), though
19
interruptions cost you
242
min
of potential productivity, capping your longest uninterrupted stretch at
81
min
.
The visualization above shows an 8-hour workday as a timeline. Green segments represent real, uninterrupted focus blocks—the time when you’re genuinely working on the problem. Red lines are interruptions: a Slack DM, a meeting, someone asking a question. The hatched gray zones are recovery time—you’re back at your desk, but you’re not back
in
the problem yet. The amber and red sections are where you’re partially in the zone—where your focus is broken before 30 or 15 minutes respectively. The goal is to be in the green (or blue).
Notice how often the red interruptions land. Notice how much of your 8 hours is actually gray recovery time, not green focus time. Count how many genuine 60-minute blocks you got: just one. You spent the whole day “working,” but very little of it was uninterrupted work.
And here’s a good day, for comparison’s sake:
You managed
6
h
14
m
of focus time and
3
deep work blocks (>
60
m
), though
10
interruptions cost you
106
min
of potential productivity, capping your longest uninterrupted stretch at
137
min
.
So, how do we go from the bad day to the good one? It comes down to three numbers: how often you’re interrupted, how long it takes to recover, and how much unbroken time your work requires.
Let’s walk through them.
Three Knobs That Secretly Define Your Day
Symbol
What it measures
Units
λ (lambda)
How often you get interrupted
per hour
Δ (delta)
How long to regain focus after each interruption
minutes
θ (theta)
Minimum block size for meaningful work
minutes
λ (lambda): Interruptions
Lambda (λ) is your interruption rate, measured in interruptions per hour (modeled as a
Poisson process
). If λ = 2, you’re getting interrupted, on average, twice every hour. In the timeline above, lambda determines how many red spikes you see.
It’s a function of your environment: how many meetings you have, how many Slack channels you’re in, how many people feel entitled to “just a quick question,” whether your company culture treats every message as urgent. Some people, like managers and executives, get interrupted a lot. Others usually don’t, but their λ can spike during on-call rotations or triage weeks. We try to model that by randomness.
Most people dramatically underestimate their real λ, but we’ll get to that in a moment.
Note
(On Poisson distributions)
Our simulations model interruptions as a Poisson process, which assumes they
arrive randomly and uniformly throughout the day. In reality, interruptions
tend to cluster. Think back-to-back meetings, Slack storms after a big
announcement, email bursts when you return from lunch. This clustering cuts
both ways: sometimes it leaves clean gaps (your afternoon block after a
morning full of meetings), but often it makes things worse because clustered
interruptions compound recovery time and eliminate any chance of regaining
focus between hits.
Δ (delta): Recovery Period
Delta (Δ) is your recovery time in minutes. When someone interrupts you, you don’t instantly return to full productivity the moment they walk away. Your brain needs time to reload the context, reconstruct the mental model, and remember what you were doing. That’s delta. In the timeline, it’s the width of those hatched gray bars after every red spike.
Delta varies by person and by task. It’s shorter if you left yourself good breadcrumbs before the interruption. It’s longer if you’re doing deep, complex work. But it’s never zero. More depressingly, even a “quick two-minute question” can cost you
15–20 minutes of recovery time
.
θ (theta): Focus Threshold
Theta (θ) is the minimum uninterrupted time required for a “unit” of real work. If you’re writing code, reviewing a design, or solving a hard problem, you probably need at least 30–60 minutes of continuous focus to make meaningful progress. That’s your theta.
It’s also why five 10-minute blocks don’t add up to one 50-minute block. When things are below your theta, things just don’t compound. Another way to think of this as
fragmentation
: interruptions can break your time into pieces too small to be useful, even if the total time is the same.
You managed
3
h
44
m
of focus time and
1
deep work blocks (>
60
m
), though
19
interruptions cost you
256
min
of potential productivity, capping your longest uninterrupted stretch at
82
min
.
In our timeline visualizations, theta is what we’re measuring the green and blue blocks against. If your theta is 60 minutes and you only have 45-minute blocks, you’re not getting any “real” work done by your own standards.
Capacity
I know I said three numbers, but capacity is just a result of all three so I’ll sneak this in.
Given what a day looks like based on these three parameters, we can write a simple formula that counts how many units of real work you accomplish in a day.
Where:
is the duration (in minutes) of focus block
is your minimum uninterrupted time for one “unit” of real work
is the floor function (round down to the nearest integer)
Based on all your focus blocks (the green and blue segments), we count how many theta-sized chunks fit. This is your day’s “capacity” for productive work. The higher the capacity, the more productive you are.
For illustration, here’s a high capacity day:
You managed
6
h
30
m
of focus time and
3
deep work blocks (>
60
m
), though
9
interruptions cost you
90
min
of potential productivity, capping your longest uninterrupted stretch at
119
min
.
Notice the long stretches of uninterrupted green. Multiple 60-minute blocks. This day has high capacity. The math is on your side.
The Capacity Formula in Action
But things can (and will) go sideways. Suppose you have a day with three focus blocks: 90 minutes, 45 minutes, and 20 minutes. Your total focus time is 155 minutes. But how much
capacity
do you have? It depends entirely on θ:
θ
The Math
Final Capacity
30
⌊90/30⌋ + ⌊45/30⌋ + ⌊20/30⌋
3 + 1 + 0 =
4
45
⌊90/45⌋ + ⌊45/45⌋ + ⌊20/45⌋
2 + 1 + 0 =
3
60
⌊90/60⌋ + ⌊45/60⌋ + ⌊20/60⌋
1 + 0 + 0 =
1
Same 155 minutes, yet the capacity slides all the way down from 4 to 1. This is why small changes in θ or block length can collapse your productivity. The floor function (⌊x⌋) is unforgiving. The math is not on your side this time.
Once we write your day as a function of these three parameters—λ for how noisy your environment is, Δ for how sticky interruptions are, and θ for how demanding your work is—we can stop treating bad days as
vibes
and start treating them as a model we can reason about.
100 Days Under the Same Conditions
So far, we’ve only looked at a single day, but one day can be a fluke—maybe you got lucky, or maybe you got unlucky. What happens if we simulate 100 days with the same λ, Δ, and θ?
Luckily, computers make simulating many days trivial. Let’s extend our model to actually simulate 100 days in a row and see all the visualizations together.
45+ min
30-45 min
15-30 min
<15 min
λ =
2
/hr
δ =
20
m
On the grid above, each cell is one simulated 8-hour workday. The color encodes the longest continuous focus block you got that day. Darker, richer colors mean longer focus blocks—those are the high-capacity days. Dim, washed-out cells are the low-capacity days, where you never got more than 15–30 minutes of uninterrupted time.
The above simulation was when things were cheery. But how about when things go sideways?
45+ min
30-45 min
15-30 min
<15 min
λ =
3
/hr
δ =
20
m
That above is what 100 days look like under relatively harsh conditions: λ = 3.0 (three interruptions per hour), Δ = 20 (20-minute recovery), θ = 60 (you need 60-minute blocks to count as “real work”):
OK, now that we’ve got our technology working, let’s ground things in reality. It’s time to dig into some research to feed our model some real λ and Δ.
What λ and Δ Look Like in Real Jobs
Nothing I’ve mentioned here is very new. People in both academia and industry have been studying these figures for years. We know, for example, that the rate of interruptions and recovery times vary significantly across industries and even across roles within an industry.
The research paints a consistent, if not depressing, picture. González & Mark found workers switch activities every 3 minutes on average. Iqbal & Horvitz measured 7.5 email/IM alerts per hour, with 10–16 minutes needed to resume work after each. And those (well-cited) studies are from years ago! The more recent Microsoft Work Trend Index reports that heavy collaborators see interruptions every 2 minutes. That’s λ = 30!
What the Research Data Actually Looks Like
If those numbers sound crazy or too high to be real, you are not alone. Look back at the visualizations you’ve seen so far in this post. We looked at days where λ is between 0 and 4 per hour, and Δ between 5 and 30 minutes.
These are the polite, toned-down versions of reality.
If González & Mark see activity switches every 3 minutes and Microsoft sees λ = 30 for heavy collaborators, our λ = 2–3 examples are actually best-case scenarios for many real environments.
Here’s 100 days with λ = 15 (roughly matching the González & Mark activity-switching data) and Δ = 25 (moderate recovery time):
45+ min
30-45 min
15-30 min
<15 min
λ =
15
/hr
δ =
25
m
If you’re looking at this grid and thinking “is the visualization broken?”, you are not alone. I had the same reaction when I first generated it. The entire grid is gray. There’s simply no time to work.
But it’s not broken. Here’s what a single day with these parameters actually looks like. Make sure to browse back and forth to see if you can find a day with any focus blocks:
You managed
0
h
5
m
of focus time and
0
deep work blocks (>
60
m
), though
138
interruptions cost you
475
min
of potential productivity, capping your longest uninterrupted stretch at
5
min
.
Now you can see why the grid appears uniformly gray. There really is no focus time. In fact there’s not even any 15-minute block on almost any of the days. The day view is a dense wall of red interruptions, each triggering a gray recovery period. The tiny slivers of focus time are too short to even render at the grid scale. There are no greens, and there’s a sole amber and a sole red. When you zoom out to 100 days, every single day looks like this. All dim, no pun intended.
This is the
modern workplace
. González & Mark measured activity switches every 3 minutes. Microsoft reports λ = 30 for heavy collaborators. These researchers are describing millions of people’s actual working conditions. And at these parameters, the math is unambiguous: deep work is statistically near-impossible. We’ve normalized an environment where focus has been engineered out of the workday. No wonder everyone’s stressed!
Again, for the sake of argument, let’s just go back to a “sane” workday. Same 8-hour days, same θ = 60, but now λ = 1.0 (one interruption per hour) and Δ = 10 (10-minute recovery).
45+ min
30-45 min
15-30 min
<15 min
λ =
1
/hr
δ =
10
m
Look at that difference! Almost all days have more than three blocks of 60-minute work, meaning they light up like a Christmas tree. The dim “hopeless” days are much rarer. The
distribution
has shifted—not because you suddenly became more disciplined, but because the system’s parameters changed. The math has spoken.
If you take away one thing from this post, it’s this:
the variability you experience across days is structural forces working against you.
When λ and Δ are high, bad days are common. When λ and Δ are lower, good days become routine. Even small increases make conditions significantly worse.
Moreover, “thanks to” randomness, you have a lot less control over what your day looks like. For example, here’s a breezy day with almost no interruptions (λ = 1) and a short recovery period (Δ = 15). With not one, not two, but three 60+ minute blocks, this is a great day!
You managed
5
h
25
m
of focus time and
3
deep work blocks (>
60
m
), though
12
interruptions cost you
155
min
of potential productivity, capping your longest uninterrupted stretch at
76
min
.
Yet it’s also possible to have a bad day with the exact same parameters. Note that while you have multiple 45+ minute blocks, you simply can’t get a single 60-minute block on this day.
You managed
5
h
19
m
of focus time and
0
deep work blocks (>
60
m
), though
13
interruptions cost you
161
min
of potential productivity, capping your longest uninterrupted stretch at
58
min
.
You can’t eliminate variance, unfortunately, but you can shift the distribution. In a good regime, with high capacity days as the norm, great days become normal and bad days become the exception you can absorb. If you want to take away two things from this post, remember that you can control things! We’ll get to that later.
The Map of Deep Work
OK, so far we’ve fixed the knobs (λ, Δ, θ) and watched the random outcomes (individual days, grids of 100 days) play out. But what if we could see the whole landscape at once—every combination of λ and Δ, and how they interact? Once again, computers to the rescue!
That’s what our heatmap does. The color shows the expected capacity (number of θ-minute blocks per day). Dark purple cells are hospitable to deep work. Light purple cells are hellish; you’re lucky to get even one θ-block per day. I’ve also highlighted some cells, based on the research data above.
The
“good” world
(green border, λ = 1.0, Δ = 10) is better than most real-world environments. It’s closer to a protected morning on a maker’s schedule, or a team that’s serious about focus time.
The
“typical” world
(amber border, λ = 2.0, Δ = 20) is still more generous than what González & Mark observed (activity switches every 3 minutes), but it’s already rough for 60-minute deep work.
The
“terrible” world
(red border, λ = 3.0, Δ = 25) is a softened version of the heavy-collaborator world (30 interruptions/hour). The actual PM/team-lead reality often lives off the right edge of this chart.
Now use the threshold control to toggle θ between 30, 45, and 60 minutes. Watch what happens to the three highlighted cells:
The green cell stays relatively dark even at θ = 60. This is high capacity: you can expect multiple 60-minute blocks per day.
The amber cell looks dark at θ = 30, lightens at θ = 45, and nearly washes out at θ = 60. The capacity collapses as θ increases. This is why you feel like you can handle small tasks but never get to the hard problems.
The red cell is basically washed out at any θ that matters. You are ruined. With such low capacity, even 30-minute blocks are rare.
The point is that the capacity is very sensitive your theta. It’s significantly harder to finish a longer task that will take 60 minutes than it is to finish two tasks that will take 30 minutes each. This will inform some of the remediation we’ll get to later.
Explanation
(Monte Carlo Estimation)
To create heatmaps, we calculate the expected capacity for each combination of
.
Where:
= expected capacity (what we’re estimating)
= number of simulations (typically 60)
= capacity observed in simulation
By the
Law of Large Numbers
, as
, the sample mean converges to the true expected value:
With
, we get a good approximation with reasonable computation time.
How a Better Day Actually Looks
Now let’s see what it means to actually
move
on the map. We’ll start in the “typical” world and watch what happens when we shift toward better territory.
Here’s the map with just the amber cell highlighted (λ = 2.0, Δ = 20):
And here’s what a day in that cell looks like:
You managed
3
h
37
m
of focus time and
1
deep work blocks (>
60
m
), though
21
interruptions cost you
263
min
of potential productivity, capping your longest uninterrupted stretch at
61
min
.
Lots of red interruptions. Wide gray recovery zones. Very few green blocks longer than 45 minutes. This is what a structurally difficult day looks like. And remember: λ = 2.0 is wildly generous compared to the research data.
Now let’s move to the green cell (λ = 1.0, Δ = 10):
And here’s what a day in
that
cell looks like:
You managed
7
h
30
m
of focus time and
5
deep work blocks (>
60
m
), though
3
interruptions cost you
30
min
of potential productivity, capping your longest uninterrupted stretch at
227
min
.
The difference is striking. Fewer interruptions. Narrower recovery zones. Multiple 60-minute focus blocks. The change in parameters is modest—one fewer interruption per hour, ten fewer minutes of recovery—but the impact on your day is dramatic.
This is what “moving on the map” actually looks like. The next section covers how to do it.
How to Move Around the Map
Let’s get back to our heatmap:
I hope that so far I have been able to make the case that you really want to go from the red cell to the green cell. And not just that, you want to be in a cell, wherever it might be, be it red, amber, or green, that your capacity is high. In other words, we want to move up the ladder (up and to the right) and then ideally change our map to a better one.
Reduce λ – Fewer Interruptions Per Hour
Lambda is about
access
. The good news: it’s the most impactful lever you have, based on the math. Doubling interruptions
more than doubles
the lost time due to chain interruptions (interruptions during recovery) and fragmentation (reducing
capacity even when total time is the same).
Here’s an example of this in action. Consider the case that you want to find three 60-minute blocks on any given day. What would your chances be?
Here’s what your chances look like with λ=1.
45+ min
30-45 min
15-30 min
<15 min
λ =
1
/hr
δ =
19
m
There are exactly 70 days that fit that criteria. In other words, you have 70% chance of finding three 60-minute blocks on any given day.
Let’s increase the rate of interruptions. Here’s λ=2, just
one more interruption per hour
with the same random seed:
45+ min
30-45 min
15-30 min
<15 min
λ =
2
/hr
δ =
19
m
Now, there are only 14 days! Your chances of success went down by 5 times.
Since λ is about access, in theory it’s what you have the most control over. Protect your calendar, get fewer interruptions. Easy, right?
Unfortunately, reality begs to differ. People in leadership positions want to be available to others, so they keep their calendars open. And people in IC roles often don’t get to control their calendars as much as they’d like—those pesky managers keep scheduling meeting after meeting.
But, not all hope is lost.
The González study
found that for knowledge workers, almost half the interruptions are self-inflicted. While it is true very few people have control over their calendars these days, we are the masters of our destiny and captains of our ship more than we think when it comes to interruptions. Another
similar study
also found that people who blocked interruptions found their job satisfaction to be much higher.
Checking your inbox only a few times a day—or even just twice—pays huge dividends. In my experience, almost nothing is actually urgent, and it’s better to train people that getting your attention requires effort than to train them that you’re always available. Whatever you do, any effort you put in here will be worth it.
Match θ to Reality – Design Tasks for Your Environment
Theta is different from λ and Δ because you usually can’t change the nature of the work itself. If you’re working on a hard problem that genuinely requires 90 minutes of continuous thought, that’s just θ = 90. You are SOL.
But you
can
design your day as a portfolio of tasks with different θ values, and match them to your realistic λ and Δ conditions. Remember, this is you changing your threshold on that interactive map above regardless of which cell you are in.
The goal here is to aim for smaller theta work.
θ = 30 minutes
θ = 60 minutes
Break large projects into smaller tasks:
If your environment has λ = 3 and you’re trying to do θ = 90 work, you’re going to have a bad time. The heatmap says so. But if you can decompose that work into tasks with θ = 30, suddenly it becomes feasible. So, whenever you have a big project, think about the independent pieces of it that you can attack.
For example, “design and implement the new authentication flow” is θ = 90 work. But you can decompose it: sketch the state machine (30 min), write the token validation logic (30 min), build the UI component (30 min), wire up the error handling (30 min). Each piece is independently completable and doesn’t require holding the whole system in your head at once.
Same with product work: “write the PRD” is θ = 90, but “define the problem statement” is θ = 20, “list user stories” is θ = 30, “draft success metrics” is θ = 20.
Beyond the mathematical advantage, there’s a psychological one: finishing a task gives you a small hit of momentum that you can ride to the next task. A day with four completed θ = 30 tasks feels productive, which makes you more likely to protect your focus tomorrow. A day where you made “some progress” on one θ = 90 task often feels like you got nothing done, even if the raw minutes were similar.
Reserve low-λ windows for high-θ work:
If you can defend a 2-hour window early in the morning with λ ≈ 0, that’s where you do your θ = 60–90 work. Use the rest of your day (higher λ) for smaller tasks that tolerate fragmentation.
θ = 20 minutes
θ = 90 minutes
Remember, this is not the same as “lowering your λ” but instead realizing that certain times of day (or week) have predictable interrupt patterns. You can’t always get what you want, but you might get what you need, if your theta is right for the time. Say, if you’re a PM or team lead living at λ = 20, starting a θ = 60 project at 3pm is fantasy. It’s extremely unlikely you’ll get a 60-minute block ever with a lambda like that but it’s practically impossible that you’ll get one that late in the day.
You need to either carve out radically different conditions or accept that this kind of work simply doesn’t happen in your normal day. I don’t want to get into the 996 discourse right now but there is a reason you see so many managers work early in the mornings before others, or on Sunday nights.
Remember the Map:
Remember, when you adjust theta (carve out your work), you’re not moving on the map. You’re
choosing a different map
. When you toggle the threshold control on the heatmap from θ = 60 to θ = 30, you’re asking a different question: “What environment do I need for 30-minute work instead of 60-minute work?” The map changes, and suddenly the “typical” (amber) and “terrible” (red) cells look more survivable.
Reduce Δ – Shorten Recovery Time
Delta is about
stickiness
. It’s how long interruptions linger after they’re technically over. You can’t make Δ zero, but you can shave minutes off each recovery, and those minutes compound quickly.
Iqbal & Horvitz found that workers take 10–16 minutes to resume work after email/IM interruptions. The variance is huge, and much of it comes from preparedness and task similarity. If you are or your task is on the high end of that delta (Δ ≈ 15+), small hygiene improvements can shave off valuable minutes.
Leave yourself breadcrumbs
before switching tasks. When you’re interrupted, send yourself a Slack message about what you were doing and what comes next. This sounds quirky, but it works.
Avoid wide context switches.
If you’re interrupted while coding, handling a quick code-review question is less disruptive than handling a recruiting question. Minimize cross-domain bouncing when possible.
Small rituals to re-enter focus.
Some people re-read the last few lines of code they wrote, or scan their notes, or take three deep breaths. Find whatever works for you.
From “Bad Days” to a Tuneable System
Look, this model is wrong, like
all other models are
. You can’t
reverse Taylorism
your way out of a broken calendar by (re-)inventing Greek letters. But wrong models can still be useful if they help you see the system clearly enough to change it.
My point is this:
given your λ and Δ, deep work is mathematically rare unless you deliberately design for it.
You’re not uniquely undisciplined. Stop feeling bad. You’re operating in a system where the default state is fragmentation. The world is out there to get you!
But hey, good news! This system is very sensitive to small inputs. That’s a liability when the universe decides to screw you, but it’s also leverage when
you
intervene. Dropping λ from 3 to 2, just one fewer interruption per hour each day, can transform your week. Learning how to split your tasks into smaller chunks by picking small θ can make your life a whole lot easier. Cutting Δ from 25 to 15 minutes can mean the difference between a hellish day or a passable one. You don’t have to win every battle to win the war.
Try this: pick one week and defend a single 90-minute block each morning. Treat that block as your λ/Δ/θ lab. No meetings, no Slack, no “quick questions.” Measure what you accomplish in that block versus the rest of your day.
I suspect you’ll see the parameters at work. And once you see them, you can’t unsee them.
One More Thing…
If you want to experiment with the parameters yourself, try the
Interruptions Simulator
. I’ll write more about how and why I built that tool later!
The evolution of the Unix operating system is made available as a
version-control repository, covering the period from its inception in
1972 as a five thousand line kernel,
to 2015 as a widely-used 26 million line system.
The repository contains 659 thousand commits and
2306 merges.
The repository employs the commonly used Git system for its storage, and
is hosted on the popular GitHub archive.
It has been created by synthesizing with custom software 24 snapshots of
systems developed at Bell Labs, Berkeley University, and the 386BSD team,
two legacy repositories, and the modern repository of the open source
FreeBSD system.
In total, 850 individual contributors are identified,
the early ones through primary research.
The data set can be used for empirical research in software engineering,
information systems, and software archaeology.
1 Introduction
The Unix operating system stands out as a major engineering breakthrough
due to
its exemplary design,
its numerous technical contributions,
its development model, and
its widespread use.
The design of the Unix programming environment has been characterized as
one offering unusual simplicity, power, and elegance [
1
].
On the technical side,
features that can be directly attributed to Unix or were popularized by it
include [
2
]:
the portable implementation of the kernel in a high level language;
a hierarchical file system;
compatible file, device, networking, and inter-process I/O;
the pipes and filters architecture;
virtual file systems; and
the shell as a user-selectable regular process.
A large community contributed software to Unix from its early
days [
3
], [
4
,pp. 65-72].
This community grew immensely over time and worked using what are now termed
open source software development methods [
5
,pp. 440-442].
Unix and its intellectual descendants have also helped the spread of
the C and C++ programming languages,
parser and lexical analyzer generators (
yacc
,
lex
),
document preparation tools (
troff
,
eqn
,
tbl
),
scripting languages (
awk
,
sed
,
Perl
),
TCP/IP networking, and
configuration management systems (
SCCS
,
RCS
,
Subversion
,
Git
),
while also forming a large part of the modern internet infrastructure and
the web.
Luckily, important Unix material of historical importance has survived
and is nowadays openly available.
Although Unix was initially distributed with relatively restrictive licenses,
the most significant parts of its early development have been released by
one of its right-holders (Caldera International) under a liberal license.
Combining these parts with software that was developed or released
as open source software by the University of California, Berkeley
and the FreeBSD Project provides coverage of the system's development
over a period ranging from June 20th 1972 until today.
Curating and processing available snapshots
as well as old and modern configuration management repositories
allows the reconstruction of a new synthetic Git repository that combines
under a single roof most of the available data.
This repository documents in a digital form the detailed evolution of an
important digital artefact over a period of 44 years.
The following sections describe
the repository's structure and contents (Section
II
),
the way it was created (Section
III
),
and how it can be used (Section
IV
).
2 Data Overview
The 1GB Unix history Git repository is made available for cloning on
GitHub
.
1
Currently
2
the repository contains 659 thousand commits and
2306 merges
from about 850 contributors.
The contributors include 23 from the Bell Labs staff,
158 from Berkeley's Computer Systems Research Group (CSRG),
and 660 from the FreeBSD Project.
The repository starts its life at a tag identified as
Epoch
,
which contains only licensing information and its modern README file.
Various tag and branch names identify points of significance.
Research-VX
tags
correspond to six research editions that came out of Bell Labs.
These start with
Research-V1
(4768 lines of PDP-11 assembly) and end with
Research-V7
(1820 mostly C files, 324kLOC).
Bell-32V
is the port of the 7th Edition Unix to the DEC/VAX
architecture.
BSD-X
tags correspond to 15 snapshots released from Berkeley.
386BSD-X
tags correspond to two open source versions of the system,
with the Intel 386 architecture kernel code mainly written
by Lynne and William Jolitz.
FreeBSD-release/X
tags and branches mark 116 releases coming from
the FreeBSD project.
In addition,
branches with a
-Snapshot-Development
suffix denote commits
that have been synthesized from a time-ordered sequence of a snapshot's files,
while tags with a
-VCS-Development
suffix mark the point along an
imported version control history branch where a particular release occurred.
The repository's history includes commits from the earliest days of
the system's development, such as the following.
commit c9f643f59434f14f774d61ee3856972b8c3905b1
Author: Dennis Ritchie <research!dmr>
Date: Mon Dec 2 18:18:02 1974 -0500
Research V5 development
Work on file usr/sys/dmr/kl.c
Merges between releases that happened along the system's evolution,
such as the development of BSD 3 from BSD 2 and Unix 32/V,
are also correctly represented in the Git repository as graph nodes
with two parents.
More importantly, the repository is constructed in a way that allows
git blame
, which annotates source code lines with the version, date,
and author associated with their first appearance,
to produce the expected code provenance results.
For example,
checking out the
BSD-4
tag,
and running
git blame
on the kernel's
pipe.c
file
will show lines written by
Ken Thompson in 1974, 1975, and 1979, and by Bill Joy in 1980.
This allows the automatic (though computationally expensive)
detection of the code's provenance at any point of time.
Figure 1: Code provenance across significant Unix releases.
As can be seen in Figure
1
, a modern version of Unix
(FreeBSD 9) still contains visible chunks of code from BSD 4.3,
BSD 4.3 Net/2, and FreeBSD 2.0.
Interestingly, the Figure shows that
code developed during the frantic dash to
create an open source operating system out of the code released by
Berkeley (386BSD and FreeBSD 1.0) does not seem to have survived.
The oldest code in FreeBSD 9 appears to be an 18-line sequence
in the C library file
timezone.c
,
which can also be found in the 7th Edition Unix file
with the same name and a time stamp of January 10th, 1979 -
36 years ago.
3 Data Collection and Processing
The goal of the project is to
consolidate data concerning the evolution of Unix
in a form that helps the study of the system's evolution,
by entering them into a modern revision repository.
This involves collecting the data,
curating them, and
synthesizing them into a single Git repository.
Figure 2: Imported Unix snapshots, repositories, and their mergers.
The last, and most labour intensive, source of data was
primary research
.
The release snapshots do not provide information regarding their ancestors
and the contributors of each file.
Therefore, these pieces of information had to be determined through
primary research.
The authorship information was mainly obtained
by reading author biographies, research papers, internal memos, and old documentation scans;
by reading and automatically processing source code and manual page markup;
by communicating via email with people who were there at the time;
by posting a query on the Unix
StackExchange
site;
by looking at the location of files (in early editions the kernel source
code was split into
usr/sys/dmr
and
/usr/sys/ken
); and
by propagating authorship from research papers and manual pages to source code
and from one release to others.
(Interestingly, the 1st and 2nd Research Edition manual pages
have an "owner" section, listing the person (e.g.
ken
) associated with the
corresponding system command, file, system call, or library function.
This section was not there in the 4th Edition, and
resurfaced as the "Author" section in BSD releases.)
Precise details regarding the source of the authorship information are
documented in the project's files that are used for mapping
Unix source code files to their authors and the corresponding commit messages.
Finally, information regarding merges between source code bases was
obtained from a
BSD family tree maintained by the NetBSD project
.
8
The software and data files that were developed as part of this project,
are
available online
,
9
and, with appropriate network, CPU and disk resources, they can be used
to recreate the repository from scratch.
The authorship information for major releases is stored in files under the
project's
author-path
directory.
These contain lines with a
regular expressions for a file path followed by the identifier of the
corresponding author.
Multiple authors can also be specified.
The regular expressions are processed sequentially, so that a catch-all
expression at the end of the file can specify a release's default authors.
To avoid repetition, a separate file with a
.au
suffix is used
to map author identifiers into their names and emails.
One such file has been created for every community associated with
the system's evolution:
Bell Labs, Berkeley, 386BSD, and FreeBSD.
For the sake of authenticity, emails for the early Bell Labs releases are listed
in UUCP notation (e.g.
research!ken
).
The FreeBSD author identifier map,
required for importing the early CVS repository,
was constructed by extracting the corresponding data from the project's
modern Git repository.
In total the commented authorship files (828 rules) comprise
1107 lines, and there are another 640 lines mapping author identifiers to names.
The curation of the project's data sources has been codified into a
168-line
Makefile
.
It involves the following steps.
Fetching
Copying and cloning about 11GB of images, archives,
and repositories from remote sites.
Tooling
Obtaining an archiver for old PDP-11 archives from 2.9 BSD,
and adjusting it to compile under modern versions of Unix;
compiling the 4.3 BSD
compress
program,
which is no longer part of modern Unix systems,
in order to decompress the 386BSD distributions.
Organizing
Unpacking archives using
tar
and
cpio
;
combining three 6th Research Edition directories;
unpacking all 1 BSD archives using the old PDP-11 archiver;
mounting CD-ROM images so that they can be processed as
file systems;
combining the 8 and 62 386BSD floppy disk images into two separate
files.
Cleaning
Restoring the 1st Research Edition kernel source code files,
which were obtained from printouts through optical character recognition,
into a format close to their original state;
patching some 7th Research Edition source code files;
removing metadata files and other files that were added after
a release, to avoid obtaining erroneous time stamp information;
patching corrupted SCCS files;
processing the early FreeBSD CVS repository by
removing CVS symbols assigned to multiple revisions
with a custom Perl script,
deleting CVS
Attic
files clashing with live ones,
and converting the CVS repository into a Git one using
cvs2svn
.
An interesting part of the repository representation is how snapshots
are imported and linked together in a way that allows
git blame
to perform its magic.
Snapshots are imported into the repository as sequential commits based on the
time stamp of each file.
When all files have been imported the repository is tagged with the name
of the corresponding release.
At that point one could delete those files, and begin the import of the
next snapshot.
Note that the
git blame
command
works by traversing backwards a repository's history, and using
heuristics to detect code moving and being copied within or across
files.
Consequently, deleted snapshots would create a discontinuity between them,
and prevent the tracing of code between them.
Instead, before the next snapshot is imported, all the files of the
preceding snapshot are moved into a hidden look-aside directory named
.ref
(reference).
They remain there, until all files of the next snapshot have been imported,
at which point they are deleted.
Because every file in the
.ref
directory matches exactly an original
file,
git blame
can determine how source code moves from one version
to the next via the
.ref
file, without ever displaying the
.ref
file.
To further help the detection of code provenance,
and to increase the representation's realism,
each release is represented as a merge between the branch with
the incremental file additions (
-Development
)
and the preceding release.
For a period in the 1980s, only a subset of the files developed at Berkeley
were under SCCS version control.
During that period our unified repository contains imports of both the
SCCS commits, and the snapshots' incremental additions.
At the point of each release, the SCCS commit with the nearest
time stamp is found and is marked as a merge
with the release's incremental import branch.
These merges can be seen in the middle of Figure
2
.
The synthesis of the various data sources into a single repository is
mainly performed by two scripts.
A 780-line Perl script (
import-dir.pl
)
can export the (real or synthesized) commit history from a single data source
(snapshot directory, SCCS repository, or Git repository) in the
Git fast export
format.
The output is a simple text format that Git tools use to import and export
commits.
Among other things, the script takes as arguments
the mapping of files to contributors,
the mapping between contributor login names and their full names,
the commit(s) from which the import will be merged,
which files to process and which to ignore, and
the handling of "reference" files.
A 450-line shell script creates the Git repository and calls the
Perl script with appropriate arguments to import each one of the 27 available
historical data sources.
The shell script also runs 30 tests that
compare the repository at specific tags against the corresponding data sources,
verify the appearance and disappearance of look-aside directories, and
look for regressions in the count of tree branches and merges and
the output of
git blame
and
git log
.
Finally,
git
is called to garbage-collect and compress the repository
from its initial 6GB size down to the distributed 1GB.
4 Data Uses
The data set can be used for empirical research in software engineering,
information systems, and software archeology.
Through its unique uninterrupted coverage of a period of more than 40 years,
it can inform work on software evolution and handovers across generations.
With thousandfold increases in processing speed and million-fold increases
in storage capacity during that time, the data set can also be used to study
the co-evolution of software and hardware technology.
The move of the software's development from research labs,
to academia, and to the open source community can be used to study
the effects of organizational culture on software development.
The repository can also be used to study how notable individuals,
such as Turing Award winners (Dennis Ritchie and Ken Thompson)
and captains of the IT industry (Bill Joy and Eric Schmidt),
actually programmed.
Another phenomenon worthy of study concerns the longevity of code,
either at the level of individual lines,
or as complete systems that were at times distributed with Unix
(Ingres, Lisp, Pascal, Ratfor, Snobol, TMG),
as well as the factors that lead to code's survival or demise.
Finally, because the data set stresses Git,
the underlying software repository storage technology, to its limits,
it can be used to drive engineering progress in the field of
revision management systems.
Figure 3: Code style evolution along Unix releases.
Figure
3
,
which depicts trend lines
(obtained with R's local polynomial regression fitting function)
of some interesting code metrics along 36 major releases of Unix,
demonstrates the evolution of code style and
programming language use over very long timescales.
This evolution can be driven by software and hardware technology
affordances and requirements,
software construction theory, and even
social forces.
The dates in the Figure have been calculated as the average date of all files
appearing in a given release.
As can be seen in it, over the past 40 years the mean
length of identifiers and file names has steadily increased from 4
and 6 characters to 7 and 11 characters, respectively.
We can also see less steady increases in the number of comments and
decreases in the use of the
goto
statement, as well as the virtual
disappearance of the
register
type modifier.
5 Further Work
Many things can be done to increase the repository's faithfulness
and usefulness.
Given that the build process is shared as open source code,
it is easy to contribute additions and fixes through GitHub pull requests.
The most useful community contribution would be to increase the
coverage of imported snapshot files that are attributed to a
specific author.
Currently, about 90 thousand files (out of a total of 160 thousand)
are getting assigned an author through a default rule.
Similarly, there are about 250 authors (primarily early FreeBSD ones)
for which only the identifier is known.
Both are listed in the build repository's
unmatched
directory,
and contributions are welcomed.
Furthermore,
the BSD SCCS and the FreeBSD CVS commits that share the same
author and time-stamp can be coalesced into a single Git commit.
Support can be added for importing the SCCS file comment fields,
in order to bring into the repository the corresponding metadata.
Finally, and most importantly,
more branches of open source systems can be added, such as NetBSD
OpenBSD, DragonFlyBSD, and
illumos
.
Ideally, current right holders of other important historical Unix releases,
such as System III, System V, NeXTSTEP, and SunOS, will release their systems under a license that would allow their incorporation
into this repository for study.
Acknowledgements
The author thanks the many individuals who contributed to the effort.
Brian W. Kernighan,
Doug McIlroy, and
Arnold D. Robbins
helped with Bell Labs login identifiers.
Clem Cole,
Era Eriksson,
Mary Ann Horton,
Kirk McKusick,
Jeremy C. Reed,
Ingo Schwarze, and
Anatole Shaw
helped with BSD login identifiers.
The BSD SCCS import code is based on work by
H. Merijn Brand and
Jonathan Gray.
This research has been co-financed by the European Union
(European Social Fund - ESF) and Greek national funds
through the Operational Program "Education and Lifelong Learning"
of the National Strategic Reference Framework (NSRF) -
Research Funding Program: Thalis -
Athens University of Economics and Business -
Software Engineering Research Platform.
M. D. McIlroy, E. N. Pinson, and B. A. Tague, "UNIX time-sharing system:
Foreword,"
The Bell System Technical Journal
, vol. 57, no. 6, pp.
1899-1904, July-August 1978.
A few weeks ago I was minding my own business, peacefully reading
a well-written and informative article about artificial intelligence
, when I was ambushed by a passage in the article that aroused my pique. That’s one of the pitfalls of knowing too much about a topic a journalist is discussing; journalists often make mistakes that most readers wouldn’t notice but that raise the hackles or at least the blood pressure of those in the know.
The article in question appeared in
The New Yorker
. The author, Stephen Witt, was writing about the way that your typical Large Language Model, starting from a blank slate, or rather a slate full of random scribbles, is able to learn about the world, or rather the virtual world called the internet. Throughout the training process, billions of numbers called weights get repeatedly updated so as to steadily improve the model’s performance. Picture a tiny chip with electrons racing around in etched channels, and slowly zoom out: there are many such chips in each server node and many such nodes in each rack, with racks organized in rows, many rows per hall, many halls per building, many buildings per campus. It’s a sort of computer-age version of Borges’ Library of Babel. And the weight-update process that all these countless circuits are carrying out depends heavily on an operation known as matrix multiplication.
Witt explained this clearly and accurately, right up to the point where his essay took a very odd turn.
HAMMERING NAILS
Here’s what Witt went on to say about matrix multiplication:
“‘Beauty is the first test: there is no permanent place in the world for ugly mathematics,’ the mathematician G. H. Hardy wrote, in 1940. But matrix multiplication, to which our civilization is now devoting so many of its marginal resources, has all the elegance of a man hammering a nail into a board. It is possessed of neither beauty nor symmetry: in fact, in matrix multiplication,
a
times
b
is not the same as
b
times
a
.”
The last sentence struck me as a bizarre non sequitur, somewhat akin to saying “Number addition has neither beauty nor symmetry, because when you write two numbers backwards, their new sum isn’t just their original sum written backwards; for instance, 17 plus 34 is 51, but 71 plus 43 isn’t 15.”
The next day I sent the following letter to the magazine:
“I appreciate Stephen Witt shining a spotlight on matrices, which deserve more attention today than ever before: they play important roles in ecology, economics, physics, and now artificial intelligence (“
Information Overload
”, November 3). But Witt errs in bringing Hardy’s famous quote (“there is no permanent place in the world for ugly mathematics”) into his story. Matrix algebra is the language of symmetry and transformation, and the fact that
a
followed by
b
differs from
b
followed by
a
is no surprise; to expect the two transformations to coincide is to seek symmetry in the wrong place — like judging a dog’s beauty by whether its tail resembles its head. With its two-thousand-year-old roots in China, matrix algebra has secured a permanent place in mathematics, and it passes the beauty test with flying colors. In fact, matrices are commonplace in number theory, the branch of pure mathematics Hardy loved most.”
Confining my reply to 150 words required some finesse. Notice for instance that the opening sentence does double duty: it leavens my many words of negative criticism with a few words of praise, and it stresses the importance of the topic, preëmptively
1
rebutting editors who might be inclined to dismiss my correction as too arcane to merit publication.
I haven’t heard back from the editors, and I don’t expect to. Regardless, Witt’s misunderstanding deserves a more thorough response than 150 words can provide. Let’s see what I can do with 1500 words and a few pictures.
THE GEOMETRY OF TRANSFORMATIONS
As a static object, matrices are “just” rectangular arrays of numbers, but that doesn’t capture what they’re really about. If I had to express the essence of matrices in a single word, that word would be “transformation”.
One example of a transformation is the operation
f
that takes an image in the plane and flips it from left to right, as if in a vertical mirror.
Another example is the operation
g
that that takes an image in the plane and reflects it across a diagonal line that goes from lower left to upper right.
The key thing to notice here is that the effect of
f
followed by
g
is different from the effect of
g
followed by
f
. To see why, write a capital R on one side of a square piece of paper–preferably using a dark marker and/or translucent paper, so that you can still see the R even when the paper has been flipped over–and apply
f
followed by
g
; you’ll get the original R rotated by 90 degrees clockwise. But if instead, starting from that original R, you were to apply
g
followed by
f
, you’d get the original R rotated by 90 degrees
counterclockwise
.
Same two operations, different outcomes! Symbolically we write
g
◦
f
≠
f
◦
g
, where
g
◦
f
means “First do
f
, then do
g
” and
f
◦
g
means “First do
g
, then
f
”.
2
The symbol ◦ denotes the meta-operation (operation-on-operations) called
composition
.
The fact that the order in which transformations are applied can affect the outcome shouldn’t surprise you. After all, when you’re composing a salad, if you forget to pour on salad dressing until after you’ve topped the base salad with grated cheese, your guests will have a different dining experience than if you’d remembered to pour on the dressing first. Likewise, when you’re composing a melody, a C-sharp followed by a D is different from a D followed by a C-sharp. And as long as mathematicians used the word “composition” rather than “multiplication”, nobody found it paradoxical that in many contexts, order matters.
THE ALGEBRA OF MATRICES
How might we capture numerically the geometrical operations
f
and
g
depicted earlier? Let’s use a square in which we’ve chosen centered coordinates so that (0,0) is in the middle of the square, and for convenience let’s make it a 2-by-2 square with coordinates (±1,±1) at the corners. It’s not hard to see that if you mark a dot at the point (
x
,
y
) and another dot at the point (−
x
,
y
), the two dots just end up swapping places when you apply the transformation
f
; for instance, the upper-right and upper-left corners of the square swap places (
x
=
y
=1). We can associate the geometric transformation
f
with the algebraic substitution that, for all
x
and
y
between −1 and 1, changes the sign of
x
, or as mathematicians like to say, “the function that maps (
x
,
y
) to (−
x
,
y
)”. This function can represented via the 2-by-2 array
where more generally the array
stands for the function that maps the pair (
x
,
y
) to the pair (
ax
+
by
,
cx
+
dy
) for any real numbers
a
,
b
,
c
,
d
we like. (Choosing
a
= −1,
b
= 0,
c
= 0, and
d
= 1 gives us the specific array
A
.)
Similarly, when you apply the operation
g
, flipping the square across the diagonal joining the lower-left and upper-right corners, a dot at (
x
,
y
) ends up swapping places with a dot at (
y
,
x
). We associate g with the algebraic substitution that swaps
x
and
y
, or as “the function that maps (
x
,
y
) to (
y
,
x
)”, represented by the 2-by-2 array
These kinds of arrays are called
matrices
, and when we want to compose two operations like
f
and
g
together, all we have to do is combine the associated matrices under the rule that says that the matrix
composed with the matrix
equals the matrix
For more about where this formula comes from, see my Mathematical Enchantments essay “
What is a Matrix
?”.
3
Even without knowing where the formula comes from, you can apply it to our two matrices and check that
A
composed with
B
is different from
B
composed with
A
.
There’s nothing special about 2-by-2 matrices; you could compose two 3-by-3 matrices, or even two 1000-by-1000 matrices. Going in the other direction (smaller instead of bigger), if you look at 1-by-1 matrices, the composition of
and
is just
so ordinary number-multiplication arises as a special case of matrix composition; turning this around, we can see matrix-composition as a sort of generalized multiplication. So it was natural for mid-19th-century mathematicians to start using words like “multiply” and “product” instead of words like “compose” and “composition”, at roughly the same time they stopped talking about “substitutions” and “tableaux” and started to use the word “matrices”.
In importing the centuries-old symbolism for number multiplication into the new science of linear algebra, the 19th century algebraists were saying “Matrices behave kind of like numbers,” with the proviso “except when they don’t”. Witt is right when he says that when
A
and
B
are matrices,
A
times
B
is not always equal to
B
times
A
. Where he’s wrong is in asserting that is a blemish on linear algebra. Many mathematicians regard linear algebra as one of the most elegant sub-disciplines of mathematics ever devised, and it often serves as a role model for the kind of sleekness that a new mathematical discipline should strive to achieve. If you dislike matrix multiplication because
AB
isn’t always equal to
BA
, it’s because you haven’t yet learned what matrix multiplication is good for in math, physics, and many other subjects. It’s ironic that Witt invokes the notion of symmetry to disparage matrix multiplication, since matrix theory and an allied discipline called group theory are the tools mathematicians use in fleshing out our intuitive ideas about symmetry that arise in art and science.
So how did an intelligent person like Witt go so far astray?
PROOFS VS CALCULATIONS
I’m guessing that part of Witt’s confusion arises from the fact that actually multiplying matrices of numbers to get a matrix of bigger numbers can be very tedious, and tedium is psychologically adjacent to distaste and a perception of ugliness. But the tedium of matrix multiplication is tied up with its symmetry (whose existence Witt mistakenly denies). When you multiply two
n
-by-
n
matrices
A
and
B
in the straightforward way, you have to compute
n
2
numbers in the same unvarying fashion, and each of those
n
2
numbers is the sum of
n
terms, and each of those
n
terms is the product of an element of
A
and an element of
B
in a simple way. It’s only human to get bored and inattentive and then make mistakes because the process is so repetitive. We tend to think of symmetry and beauty as synonyms, but sometimes excessive symmetry breeds ennui; repetition in excess can be repellent. Picture the Library of Babel and the existential dread the image summons.
G. H. Hardy, whose famous remark Witt quotes, was in the business of proving theorems, and he favored conceptual proofs over calculational ones. If you showed him a proof of a theorem in which the linchpin of your argument was a 5-page verification that a certain matrix product had a particular value, he’d say you didn’t really understand your own theorem; he’d assert that you should find a more conceptual argument and then consign your brute-force proof to the trash. But Hardy’s aversion to brute force was specific to the domain of mathematical proof, which is far removed from math that calculates optimal pricing for annuities or computes the wind-shear on an airplane wing or fine-tunes the weights used by an AI. Furthermore, Hardy’s objection to your proof would focus on the length of the calculation, and not on whether the calculation involved matrices. If you showed him a proof that used 5 turgid pages of pre-19th-century calculation that never mentioned matrices once, he’d still say “Your proof is a piece of temporary mathematics; it convinces the reader that your theorem is true without truly explaining
why
the theorem is true.”
If you forced me at gunpoint to multiply two 5-by-5 matrices together, I’d be extremely unhappy, and not just because you were threatening my life; the task would be inherently unpleasant. But the same would be true if you asked me to add together a hundred random two-digit numbers. It’s not that matrix-multiplication or number-addition is ugly; it’s that such repetitive tasks are the diametrical opposite of the kind of conceptual thinking that Hardy loved and I love too. Any kind of mathematical content can be made stultifying when it’s stripped of its meaning and reduced to mindless toil. But that casts no shade on the underlying concepts. When we outsource number-addition or matrix-multiplication to a computer, we rightfully delegate the soul-crushing part of our labor to circuitry that has no soul. If we could peer into the innards of the circuits doing all those matrix multiplications, we would indeed see a nightmarish, Borgesian landscape, with billions of nails being hammered into billions of boards, over and over again. But please don’t confuse that labor with mathematics.
This essay
is related to chapter 10 (“Out of the Womb”) of a book I’m writing, tentatively called “What Can Numbers Be?: The Further, Stranger Adventures of Plus and Times”. If you think this sounds interesting and want to help me make the book better, check out
http://jamespropp.org/readers.pdf
. And as always, feel free to submit comments on this essay at the Mathematical Enchantments WordPress site!
ENDNOTES
#1. Note the
New Yorker
-ish diaresis in “preëmptively”: as long as I’m being critical, I might as well be diacritical.
#2. I know this convention may seem backwards on first acquaintance, but this is how ◦ is defined. Blame the people who first started writing things like “log
x
” and “cos
x
“, with the
x
coming after the name of the operation. This led to the notation
f
(
x
) for the result of applying the function
f
to the number
x
. Then the symbol for the result of applying
g
to the result of applying
f
to
x
is
g
(
f
(
x
)); even though
f
is performed first, “
f
” appears to the right of “
g
“. From there, it became natural to write the function that sends
x
to
g
(
f
(
x
)) as “
g
◦
f
“.
#3. Here’s one point on which I can sympathize with Stephen Witt: matrix multiplication would be prettier if the product of two matrices were just a matter of multiplying each entry in the first matrix by the corresponding entry in the second matrix:
This kind of product is called the Hadamard product and it does play a small role in mathematics, but it’s nowhere near as common as the usual matrix product. The Hadamard product is too symmetrical to be useful, whereas the usual matrix product strikes a perfect balance between simplicity and versatility.
There is a subclass of matrices for which the Hadamard product and the standard product coincide, namely, the class of diagonal matrices. Here’s how diagonal matrices multiply:
In the world of neural networks, such matrices correspond to a trivial kind of data-processing in which every output variable is simply a specific input variable multiplied by a constant. There’s no cross-talk or interaction between variables. What makes matrices-in-general more useful than diagonal matrices is that with a general matrix, every output is potentially affected by every input.
To put things in a grandiose but not entirely inaccurate way, matrices are the first thing you should consider using to model a situation in which you don’t know ahead of time which outputs depend on which inputs. Of course one shouldn’t expect matrices to be a panacea; after all, linear algebra requires that every output be a linear function of the inputs (hence the name). Linearity is a heavy constraint. The beautiful miracle is that, despite this constraint, linear algebra is such a useful tool in all the sciences.
EFF’s Holiday Gift Guide
Electronic Frontier Foundation
www.eff.org
2025-11-28 08:35:16
Technology is supercharging the attack on democracy and EFF is fighting back. We’re suing to stop government surveillance. We're fighting to protect free expression online. And we're building tools to protect your data privacy.
Help support our mission with new gear from EFF's online store, perfect ...
Technology is supercharging the attack on democracy and
EFF is fighting back.
We’re suing to stop government surveillance. We're fighting to protect free expression online. And we're building tools to protect your data privacy.
Help support our mission with new gear from EFF's online store, perfect gifts for the digital rights defender in your life.
Take 20% your order today with code BLACKFRI. Thanks for being an EFF supporter!
Liquid Core Dice
are perfect for tabletop games. The metal clear-view EFF display tin contains a seven piece set of sharp-edge dice. These glittery dice will show that you roll with the crew protecting our civil liberties online.
Celebrate equity and accessibility with this tactile braille sticker that depicts the fiery figure of
Lady Justice with braille characters
reading "justice" and "EFF." With this embossed sticker, you won't just be showing off your support for justice, you'll actually be able to feel it.
Explore the mysteries of the web with an iconic
Bigfoot de la Sasquatch lapel pin
—
privacy is a "human" right!
Continue the journey with with campfire tales from
The Encryptids
, the rarely-seen creatures who’ve become
digital rights legends
. This sparkling cloisonne pin measures 1.5 inches tall and features a high quality spring backing.
An SQLite database file with a defined schema
often makes an excellent application file format.
Here are a dozen reasons why this is so:
Simplified Application Development
Single-File Documents
High-Level Query Language
Accessible Content
Cross-Platform
Atomic Transactions
Incremental And Continuous Updates
Easily Extensible
Performance
Concurrent Use By Multiple Processes
Multiple Programming Languages
Better Applications
Each of these points will be described in more detail below,
after first considering more closely the meaning of
"application file format". See also the
short version
of this
whitepaper.
What Is An Application File Format?
An "application file format" is the file format
used to persist application state to disk or to exchange
information between programs.
There are thousands of application file formats in use today.
Here are just a few examples:
DOC - Word Perfect and Microsoft Office documents
DWG - AutoCAD drawings
PDF - Portable Document Format from Adobe
XLS - Microsoft Excel Spreadsheet
GIT - Git source code repository
EPUB - The Electronic Publication format used by non-Kindle eBooks
ODT - The Open Document format used by OpenOffice and others
PPT - Microsoft PowerPoint presentations
ODP - The Open Document presentation format used by OpenOffice and others
We make a distinction between a "file format" and an "application format".
A file format is used to store a single object. So, for example, a GIF or
JPEG file stores a single image, and an XHTML file stores text,
so those are "file formats" and not "application formats". An EPUB file,
in contrast, stores both text and images (as contained XHTML and GIF/JPEG
files) and so it is considered an "application format". This article is
about "application formats".
The boundary between a file format and an application format is fuzzy.
This article calls JPEG a file format, but for an image editor, JPEG
might be considered the application format. Much depends on context.
For this article, let us say that a file format stores a single object
and an application format stores many different objects and their relationships
to one another.
Most application formats fit into one of these three categories:
Fully Custom Formats.
Custom formats are specifically designed for a single application.
DOC, DWG, PDF, XLS, and PPT are examples of custom formats. Custom
formats are usually contained within a single file, for ease of transport.
They are also usually binary, though the DWG format is a notable exception.
Custom file formats require specialized application code
to read and write and are not normally accessible from commonly
available tools such as unix command-line programs and text editors.
In other words, custom formats are usually "opaque blobs".
To access the content of a custom application file format, one needs
a tool specifically engineered to read and/or write that format.
Pile-of-Files Formats.
Sometimes the application state is stored as a hierarchy of
files. Git is a prime example of this, though the phenomenon occurs
frequently in one-off and bespoke applications. A pile-of-files format
essentially uses the filesystem as a key/value database, storing small
chunks of information into separate files. This gives the
advantage of making the content more accessible to common utility
programs such as text editors or "awk" or "grep". But even if many
of the files in a pile-of-files format
are easily readable, there are usually some files that have their
own custom format (example: Git "Packfiles") and are hence
"opaque blobs" that are not readable
or writable without specialized tools. It is also much less convenient
to move a pile-of-files from one place or machine to another, than
it is to move a single file. And it is hard to make a pile-of-files
document into an email attachment, for example. Finally, a pile-of-files
format breaks the "document metaphor":
there is no one file that a user can point to
that is "the document".
Wrapped Pile-of-Files Formats.
Some applications use a Pile-of-Files that is then encapsulated into
some kind of single-file container, usually a ZIP archive.
EPUB, ODT,and ODP are examples of this approach.
An EPUB book is really just a ZIP archive that contains various
XHTML files for the text of book chapters, GIF and JPEG images for
the artwork, and a specialized catalog file that tells the eBook
reader how all the XML and image files fit together. OpenOffice
documents (ODT and ODP) are also ZIP archives containing XML and
images that represent their content as well as "catalog" files that
show the interrelationships between the component parts.
A wrapped pile-of-files format is a compromise between a full
custom file format and a pure pile-of-files format.
A wrapped pile-of-files format is not an opaque blob in the same sense
as a custom format, since the component parts can still be accessed
using any common ZIP archiver, but the format is not quite as accessible
as a pure pile-of-files format because one does still need the ZIP
archiver, and one cannot normally use command-line tools like "find"
on the file hierarchy without first un-zipping it. On the other
hand, a wrapped pile-of-files format does preserve the document
metaphor by putting all content into a single disk file. And
because it is compressed, the wrapped pile-of-files format tends to
be more compact.
As with custom file formats, and unlike pure pile-of-file formats,
a wrapped pile-of-files format is not as easy to edit, since
usually the entire file must be rewritten in order to change any
component part.
The purpose of this document is to argue in favor of a fourth
new category of application file format: An SQLite database file.
SQLite As The Application File Format
Any application state that can be recorded in a pile-of-files can
also be recorded in an SQLite database with a simple key/value schema
like this:
CREATE TABLE files(filename TEXT PRIMARY KEY, content BLOB);
If the content is compressed, then such an
SQLite Archive
database is
the same size
(±1%)
as an equivalent ZIP archive, and it has the advantage
of being able to update individual "files" without rewriting
the entire document.
But an SQLite database is not limited to a simple key/value structure
like a pile-of-files database. An SQLite database can have dozens
or hundreds or thousands of different tables, with dozens or
hundreds or thousands of fields per table, each with different datatypes
and constraints and particular meanings, all cross-referencing each other,
appropriately and automatically indexed for rapid retrieval,
and all stored efficiently and compactly in a single disk file.
And all of this structure is succinctly documented for humans
by the SQL schema.
In other words, an SQLite database can do everything that a
pile-of-files or wrapped pile-of-files format can do, plus much more,
and with greater lucidity.
An SQLite database is a more versatile container than key/value
filesystem or a ZIP archive. (For a detailed example, see the
OpenOffice case study
essay.)
The power of an SQLite database could, in theory, be achieved using
a custom file format. But any custom file format that is as expressive
as a relational database would likely require an enormous design specification
and many tens or hundreds of thousands of lines of code to
implement. And the end result would be an "opaque blob" that is
inaccessible without specialized tools.
Hence, in comparison to other approaches, the use of
an SQLite database as an application file format has
compelling advantages. Here are a few of these advantages,
enumerated and expounded:
Simplified Application Development.
No new code is needed for reading or writing the application file.
One has merely to link against the SQLite library, or include the
single "sqlite3.c" source file
with the rest of the
application C code, and SQLite will take care of all of the application
file I/O. This can reduce application code size by many thousands of
lines, with corresponding saving in development and maintenance costs.
SQLite is one of the
most used
software libraries in the world.
There are literally tens of billions of SQLite database files in use
daily, on smartphones and gadgets and in desktop applications.
SQLite is
carefully tested
and proven reliable. It is not
a component that needs much tuning or debugging, allowing developers
to stay focused on application logic.
Single-File Documents.
An SQLite database is contained in a single file, which is easily
copied or moved or attached. The "document" metaphor is preserved.
SQLite does not have any file naming requirements
and so the application can use any custom file suffix that it wants
to help identify the file as "belonging" to the application.
SQLite database files contain a 4-byte
Application ID
in
their headers that can be set to an application-defined value
and then used to identify the "type" of the document for utility
programs such as
file(1)
, further
enhancing the document metaphor.
High-Level Query Language.
SQLite is a complete relational database engine, which means that the
application can access content using high-level queries. Application
developers need not spend time thinking about "how" to retrieve the
information they need from a document. Developers can write SQL that
expresses "what" information they want and let the database engine
figure out how to best retrieve that content. This helps developers
operate "heads up" and remain focused on solving the user's problem,
and avoid time spent "heads down" fiddling with low-level file
formatting details.
A pile-of-files format can be viewed as a key/value database.
A key/value database is better than no database at all.
But without transactions or indices or a high-level query language or
a proper schema,
it is much harder and more error prone to use a key/value database than
a relational database.
Accessible Content.
Information held in an SQLite database file is accessible using
commonly available open-source command-line tools - tools that
are installed by default on Mac and Linux systems and that are
freely available as a self-contained EXE file on Windows.
Unlike custom file formats, application-specific programs are
not required to read or write content in an SQLite database.
An SQLite database file is not an opaque blob. It is true
that command-line tools such as text editors or "grep" or "awk" are
not useful on an SQLite database, but the SQL query language is a much
more powerful and convenient way for examining the content, so the
inability to use "grep" and "awk" and the like is not seen as a loss.
An SQLite database is a
well-defined and well-documented
file format that is in widespread use by literally millions of applications
and is backwards compatible to its inception in 2004 and which promises
to continue to be compatible in decades to come. The longevity of
SQLite database files is particularly important to bespoke applications,
since it allows the document content to be accessed far in the
future, long after all traces of the original application have been lost.
Data lives longer than code.
SQLite databases are
recommended by the US Library of Congress
as a storage format for long-term preservation of digital content.
Cross-Platform.
SQLite database files are portable between 32-bit and 64-bit machines and
between big-endian and little-endian architectures and between any of the
various flavors of Windows and Unix-like operating systems.
The application using an SQLite application file format can store
binary numeric data without having to worry about the byte-order of
integers or floating point numbers.
Text content can be read or written as UTF-8, UTF-16LE, or UTF-16BE and
SQLite will automatically perform any necessary translations on-the-fly.
Atomic Transactions.
Writes to an SQLite database are
atomic
.
They either happen completely
or not at all, even during system crashes or power failures. So
there is no danger of corrupting a document just because the power happened
to go out at the same instant that a change was being written to disk.
SQLite is transactional, meaning that multiple changes can be grouped
together such that either all or none of them occur, and so that the
changes can be rolled back if a problem is found prior to commit.
This allows an application to make a change incrementally, then run
various sanity and consistency checks on the resulting data prior to
committing the changes to disk. The
Fossil
DVCS
uses this technique
to verify that no repository history has been lost prior to each change.
Incremental And Continuous Updates.
When writing to an SQLite database file, only those parts of the file that
actually change are written out to disk. This makes the writing happen faster
and saves wear on SSDs. This is an enormous advantage over custom
and wrapped pile-of-files formats, both of which usually require a
rewrite of the entire document in order to change a single byte.
Pure pile-of-files formats can also
do incremental updates to some extent, though the granularity of writes is
usually larger with pile-of-file formats (a single file) than with SQLite
(a single page).
SQLite also supports continuous update.
Instead of collecting changes in memory and then writing
them to disk only on a File/Save action, changes can be written back to
the disk as they occur. This avoids loss of work on a system crash or
power failure. An
automated undo/redo stack
, managed using triggers,
can be kept in the on-disk database, meaning that undo/redo can occur
across session boundaries.
Easily Extensible.
As an application grows, new features can be added to an
SQLite application file format simply by adding new tables to the schema
or by adding new columns to existing tables. Adding columns or tables
does not change the meaning of prior queries, so with a
modicum of care to ensuring that the meaning of legacy columns and
tables are preserved, backwards compatibility is maintained.
It is possible to extend custom or pile-of-files formats too, of course,
but doing so is often much harder. If indices are added, then all application
code that changes the corresponding tables must be located and modified to
keep those indices up-to-date. If columns are added, then all application
code that accesses the corresponding table must be located and modified to
take into account the new columns.
Performance.
In many cases, an SQLite application file format will be
faster than a pile-of-files format
or
a custom format. In addition to being faster for raw read and
writes, SQLite can often dramatically improves start-up times because
instead of having to
read and parse the entire document into memory, the application can
do queries to extract only the information needed for the initial screen.
As the application progresses, it only needs to load as much material as
is needed to draw the next screen, and can discard information from
prior screens that is no longer in use. This helps keep the memory
footprint of the application under control.
A pile-of-files format can be read incrementally just like SQLite.
But many developers are surprised to learn that SQLite can read and
write smaller BLOBs (less than about 100KB in size) from its database
faster than those same blobs can be read or written as separate files
from the filesystem. (See
35% Faster Than The Filesystem
and
Internal Versus External BLOBs
for further information.)
There is overhead associated with operating a relational
database engine, however one should not assume that direct file I/O
is faster than SQLite database I/O, as often it is not.
In either case, if performance problems do arise in an SQLite application
those problems can often be resolved by adding one or two
CREATE INDEX
statements to the schema or perhaps running
ANALYZE
one time
and without having to touch a single line of
application code. But if a performance problem comes up in a custom or
pile-of-files format, the fix will often require extensive changes
to application code to add and maintain new indices or to extract
information using different algorithms.
Concurrent Use By Multiple Processes.
SQLite automatically coordinates concurrent access to the same
document from multiple threads and/or processes. Two or more
applications can connect and read from the same document at the
same time. Writes are serialized, but as writes normally only
take milliseconds, applications simply take turns writing.
SQLite automatically ensures that the low-level format of the
document is uncorrupted. Accomplishing the same with a custom
or pile-of-files format, in contrast, requires extensive support
in the application. And the application logic needed to support
concurrency is a notorious bug-magnet.
Multiple Programming Languages.
Though SQLite is itself written in ANSI-C, interfaces exist for
just about every other programming language you can think of:
C++, C#, Objective-C, Java, Tcl, Perl, Python, Ruby, Erlang,
JavaScript, and so forth. So programmers can develop in whatever
language they are most comfortable with and which best matches
the needs of the project.
An SQLite application file format is a great
choice in cases where there is a collection or "federation" of
separate programs, often written in different languages and by
different development teams.
This comes up commonly in research or laboratory
environments where one team is responsible for data acquisition
and other teams are responsible for various stages of analysis.
Each team can use whatever hardware, operating system,
programming language and development methodology that they are
most comfortable with, and as long as all programs use an SQLite
database with a common schema, they can all interoperate.
Better Applications.
If the application file format is an SQLite database, the complete
documentation for that file format consists of the database schema,
with perhaps a few extra words about what each table and column
represents. The description of a custom file format,
on the other hand, typically runs on for hundreds of
pages. A pile-of-files format, while much simpler and easier to
describe than a fully custom format, still tends to be much larger
and more complex than an SQL schema dump, since the names and format
for the individual files must still be described.
This is not a trivial point. A clear, concise, and easy to understand
file format is a crucial part of any application design.
Fred Brooks, in his all-time best-selling computer science text,
The Mythical Man-Month
says:
Representation is the
essence of computer programming.
...
Show me your flowcharts and conceal your tables, and I shall
continue to be mystified. Show me your tables, and I won't usually
need your flowcharts; they'll be obvious.
Rob Pike, in his
Rules of Programming
expresses the same idea this way:
Data dominates. If you've chosen the right data structures
and organized things well, the algorithms will almost always
be self-evident. Data structures, not algorithms, are central
to programming.
Linus Torvalds used different words to say
much the same thing on the Git mailing list on 2006-06-27:
Bad programmers worry about the code. Good programmers worry
about data structures and their relationships.
The point is this: an SQL database schema almost always does
a far better job of defining and organizing the tables and
data structures and their relationships.
And having clear, concise, and well-defined representation
almost always results in an application that performs better,
has fewer problems, and is easier to develop and maintain.
Conclusion
SQLite is not the perfect application file format for every situation.
But in many cases, SQLite is a far better choice than either a custom
file format, a pile-of-files, or a wrapped pile-of-files.
SQLite is a high-level, stable, reliable, cross-platform, widely-deployed,
extensible, performant, accessible, concurrent file format. It deserves
your consideration as the standard file format on your next application
design.
This page was last updated on 2025-05-31 13:08:22Z
A foundation for building tools on the AT Protocol using Unison
Unison Programming Language
, along with its Cloud platform, is
unique
in
many
aspects. My core interests have been their intersection:
Typed Functional Programming
and
Distributed Systems
. Unison has some innovative cloud libraries like
Volturno
- a large-scale streaming framework with exactly-once processing. It's like Kafka without pain.
Arcella
- a library of append-only data structures, implemented solely on top of object-storage.
I want to play around with these libraries, so I was looking for a use case.
Use Case
I am trying to ensure that all my social activities are driven via AT Protocol. That way, I know where my data resides. Now, to move the published data to various app views requires either the publishing platform providing a specific feature or one's programming skills. I chose the latter.
The nice thing about AT Protocol is that even though the development libraries and framework ecosystem is clustered mostly around TypeScript or Rust, I don't have to. The communication is on top of HTTP, and PDSs are just repositories that handle authentication but have no application logic. So, it all boils down to parsing from firehose (Jetstream), manipulating data in the PDS - all based on relevant lexicons, and repeat.
Foundation
The foundation of working with AT Protocol is built on top of Unison's rendition of direct-style algebraic effects -
abilities
.
ATProto
's ability will handle operations within the AT Protocol ecosystem, such as making API calls.
Xrpc
ability handles the details of HTTP API calls, including the various request types such as
Query
,
Procedure
, and
Subscription
. Lexicon's schema definition languages are encoded using the
Schemas
library, which embodies the aspect of protocol-agnostic description of datatypes, which then gets encoded/decoded into various serialisation technologies, one of which is JSON.
Tools
The first implemented tool
synchronises Bluesky replies to
Leaflet
documents as comments. The sync system uses a streaming architecture powered by Volturno pipelines. It maintains two Jetstream subscriptions: one for the authenticated user's interactions and another for public interactions on tracked collections. Unison's cloud infrastructure provides a way to represent persistent processes or long-running workflows as
Daemon
s
.
Volturno pipelines process source relevant Jetstream events which are pushed to the
KLog
via multiple
KStream
workflows
it segregates incoming events from processed events
it uses the
Events
data structure from the
Arcella
library as the event store to store all processed events and then emits the unique event to the attached sinks
Sink
s are where side effects occur as part of the final stage of the overall
Pipeline
.
Create Leaflet comment sink
The sink prepares the data for creating the Leaflet comment and, finally, makes the relevant
createRecord
call to update PDS with a new comment. However, one crucial step is to figure out some details about the original Leaflet post, which was shared in Bluesky, and to that shared post, this new reply is made. Large indexes are doing this kind of backlinking, another set of applications on top of the same AT Protocol.
We use
Constellation
's APIs for those backlinking queries at the moment. However, one of the primary reasons to use our own Event Store is to use the same in due course for such queries.
Here are the logs from the Unison Cloud console depicting the syncing
Up Next
The whole point of creating the foundation is to build more tools on top of the same, like the Bluesky replies-Leaflet comments sync. Some ideas:
custom bridge for other federated social systems
backlink index that could leverage Arcella's incremental map-reduce jobs
Russell Coker: 10gbit and 40gbit Home Networking
PlanetDebian
etbe.coker.com.au
2025-11-28 07:40:14
Aliexpress has a 4 port 2.5gbit switch with 2*SFP+ sockets for $34.35 delivered [1]. 4 ports isn’t very good for the more common use cases (if daisy chaining them then it’s only
2 available for devices) so this is really a device for use with 10Gbit uplink.
Aliexpress has a pair of SFP+ 10Gbit devic...
So you can get a 2.5gbit switch with two 10gbit uplink cables to nearby servers for $66.86 including postage. I don’t need this but it is tempting.
I spent $93.78 to get 2.5gbit networking [4]
so spending $66.86 to get part of my network to 10gbit isn’t much.
It is
$99.81 including postage for a Mellanox 2*40Gbit QSFP+ card and two QSFP+ adaptors with 3M of copper between them [5]
. It is $55.81 including postage for the Mellanox card without the cable. So that’s $155.62 for a point to point 40gbit link between systems that are less than 3M apart, that’s affordable for a home lab. As an aside the only NVMe I’ve tested which can deliver such speeds was in a Thinkpad and the Thinkpad entered a thermal throttling state after a few seconds of doing that.
I’m not going to get 40Gbit, that’s well above what I need and while a point to point link is quite affordable I don’t have servers in that range. But I am seriously considering 10Gbit, I get paid to do enough networking stuff that having some hands on experience with 10Gbit could be useful.
I have been writing production applications in Go for a few years now. I like some aspects of Go. One aspect I do not like is how easy it is to create data races in Go.
Go is often touted for its ease to write highly concurrent programs. However, it is also mind-boggling how many ways Go happily gives us developers to shoot ourselves in the foot.
Over the years I have encountered and fixed many interesting kinds of data races in Go. If that interests you, I have written about Go concurrency in the past and about some existing footguns, without them being necessarily 'Go data races':
So what is a 'Go data race'? Quite simply, it is Go code that does not conform to the
Go memory model
. Importantly, Go defines in its memory model what a Go compiler MUST do and MAY do when faced with a non-conforming program exhibiting data races. Not everything is allowed, quite the contrary in fact. Data races in Go are not benign either: their effects can range from 'no symptoms' to 'arbitrary memory corruption'.
Quoting the Go memory model:
This means that races on multiword data structures can lead to inconsistent values not corresponding to a single write. When the values depend on the consistency of internal (pointer, length) or (pointer, type) pairs, as can be the case for interface values, maps, slices, and strings in most Go implementations, such races can in turn lead to arbitrary memory corruption.
With this out of the way, let's take a tour of real data races in Go code that I have encountered and fixed. At the end I will emit some recommendations to (try to) avoid them.
I also recommend reading the paper
A Study of Real-World Data Races in Golang
. This article humbly hopes to be a spiritual companion to it. Some items here are also present in this paper, and some are new.
In the code I will often use
errgroup.WaitGroup
or
sync.WaitGroup
because they act as a fork-join pattern, shortening the code. The exact same can be done with 'raw' Go channels and goroutines. This also serves to show that using higher-level concepts does not magically protect against all data races.
The problem is that the
err
outer variable is implicitly captured by the closures running each in a separate goroutine. They then mutate
err
concurrently. What they meant to do is instead use a variable local to the closure and return that instead. There is conceptually no need to share any data; this is purely accidental.
It is unfortunate that a one character difference is all we need to fall into this trap. I feel for the original developer who wrote this code without realizing the implicit capture. As mentioned in a
previous article
where this silent behavior bit me, we can use the build flag
-gcflags='-d closure=1'
to make the Go compiler print which variables are being captured by the closure:
$ go build -gcflags='-d closure=1'
./main.go:20:8: heap closure, captured vars = [err]
./main.go:28:8: heap closure, captured vars = [err]
But this is not realistic to do that in a big codebase and inspect each closure. It's useful if you know that a given closure might suffer from this problem.
The program makes two concurrent HTTP requests to two different URLs. For the first one, the code restricts redirects (I invented the exact logic for that, no need to look too much into it, the real code has complex logic here). For the second one, no redirect checks are performed, by setting
CheckRedirect
to nil. This code is idiomatic and follows the recommendations from the documentation:
CheckRedirect specifies the policy for handling redirects. If CheckRedirect is not nil, the client calls it before following an HTTP redirect.
If CheckRedirect is nil, the Client uses its default policy [...].
The problem is: the
CheckRedirect
field is modified concurrently without any synchronization which is a data race.
This code also suffers from an I/O race: depending on the network speed and response time for both URLs, the redirects might or might be checked, since the callback might get overwritten from the other goroutine, right when the HTTP client would call it.
Alternatively, the
http.Client
could end up calling a
nil
callback if the callback was set when the
http.Client
checked whether it was nil or not, but before
http.Client
had the chance to call it, the other goroutine set it to
nil
. Boom,
nil
dereference.
This may affect performance negatively since some resources will not be shared anymore.
Additionally, in some situations, this is not so easy because
http.Client
does not offer a
Clone()
method (a recurring issue in Go as we'll see). For example, a Go test may start a
httptest.Server
and then call
.Client()
on this server to obtain a preconfigured HTTP client for this server. Then, there is no easy way to duplicate this client to use it from two different tests running in parallel.
Again here, I would not blame the original developer. In my view, the docs for
http.Client
are misleading and should mention that not every single operation is concurrency safe. Perhaps with the wording: 'once a http.Client is constructed, performing a HTTP request is concurrency safe, provided that the http.Client fields are not modified concurrently'. Which is less catchy than 'Clients are safe for concurrent use', period.
A global mutable map of pricing information is guarded by a mutex. One HTTP endpoint reads the map, another adds an item to it. Pretty simple I would say. The locking is done correctly.
Yet the map suffers from a data race. Here is a reproducer:
The reason why is because the data, and the mutex guarding it, do not have the same 'lifetime'. The
pricingInfo
map is global and exists from the start of the program to the end. But the mutex
infoMtx
exists only for the duration of the HTTP handler (and thus HTTP request). We effectively have 1 map and N mutexes, none of them shared between HTTP handlers. So HTTP handlers cannot synchronize access to the map.
The intent of the code was (I think) to do a deep clone of the pricing information at the beginning of the HTTP handler in
NewPricingService
. Unfortunately, Go does a shallow copy of structures and thus each
PricingService
instance ends up sharing the same underlying
plans
map, which is this global map. It could be that for a long time, it worked because the
PricingInfo
struct did not yet contain the map (in the real code it contains a lot of
int
s and
string
s which are value types and will be copied correctly by a shallow copy), and the map was only added later.
This data race is akin to copying a mutex by value when passing it to a function, which then locks it. This does no synchronization at all since a copy of the mutex is being locked - no mutex is shared between concurrent units.
It's annoying to have to implement this manually and especially to have to check every single nested field to determine if it's a value type or a reference type (the former will behave correctly with a shallow copy, the latter needs a custom deep copy implementation). I miss the
derive(Clone)
annotation in Rust. This is something that the compiler can (and should) do better than me.
Furthermore, as mentioned in the previous section, some types from the standard library or third-party libraries do not implement a deep
Clone()
function and contain private fields which prevent us from implementing that ourselves.
I think Rust's API for a mutex is better because a Rust mutex wraps the data it protects and thus it is harder to have uncorrelated lifetimes for the data and the mutex.
Go's mutex API likely could not have been implemented this way since it would have required generics which did not exist at the time. But as of today: I think it could.
Nonetheless, the Go compiler has no way to detect accidental shallow copying, whereas Rust's compiler has the concepts of
Copy
and
Clone
- so that issue remains in Go, and is not a simple API mistake in the standard library we can fix.
I encountered many cases of concurrently modifying a
map
,
slice
, etc without any synchronization. That's your run of the mill data race and they are typically fixed by 'slapping a mutex on it' or using a concurrency safe data structure such as
sync.Map
.
I will thus share here a more interesting one where it is not as straightforward.
This time, the code is convoluted but what it does is relatively simple:
Spawn a docker container and capture its standard output in a byte buffer
Concurrently (in a different goroutine), read this output and find a given token.
Once the token is found, the context is canceled and thus the container is automatically stopped.
So, the issue may be clear from the description but here it is spelled out: one goroutine writes to a (growing) byte buffer, another one reads from it, and there is no synchronization: that's a clear data race.
What is interesting here is that we have to pass an
io.Writer
for the
OutputStream
to the library, and this library will write to the writer we passed. We cannot insert a mutex lock anywhere around the write site, since we do not control the library and there no hooks (e.g. pre/post write callbacks) to do so.
Most types in the Go standard library (or third-party libraries for that matter) are
not
concurrency safe and synchronization is typically on you. I still often see questions on the internet about that, so assume it is not until the documentation states otherwise.
It would also be nice if more types have a 'sync' version, e.g.
SyncWriter
,
SyncReader
, etc.
The Go race detector is great but will not detect all data races. Data races will cause you pain and suffering, be it flaky tests, weird production errors, or in the worst case memory corruption.
Due to how easy it is to spawn goroutines without a care in the world (and also to run tests in parallel), it will happen to you. It's not a question of if, just when, how bad, and how many days/weeks it will cost you to find them and fix them.
If you are not running your test suite with the race detector enabled, you have numerous data races in your code. That's just a fact.
Go the language and the Go linter ecosystem do not have nearly enough answers to this problem. Some language features make it way too easy to trigger data races, for example implicit capture of outer variables in closures.
The best option left to Go developers is to try to reach 100% test coverage of their code and run the tests with the race detector on.
We should be able to do better in 2025. Just like with memory safety, when even expert developers regularly produce data races, it's the fault of the language/tooling/APIs/etc. It is not enough to blame humans and demand they just 'do better'.
Add explicit capture lists for closures, just like in C++.
Add a lint to forbid using the implicit capture syntax in closures (a.k.a.: current Go closures). I am fine writing a stand-alone plain function instead, if that means keeping my sanity and removing an entire category of errors. I have also seen implicit captures cause logic bugs and high memory usage in the past.
Support
const
in more places. If something is constant, there cannot be data races with it.
Generate a
Clone()
function in the compiler for every type (like Rust's
derive(Clone)
). Maybe it's opt-in or opt-out, not sure. Or perhaps it's a built-in like
make
.
Add a
freeze()
functionality like JavaScript's
Object.freeze()
to prevent an object from being mutated.
Expand the standard library documentation to have more details concerning the concurrency safety of certain types and APIs.
Expand the Go memory model documentation and add examples. I have read it many times and I am still unsure if concurrent writes to separate fields of a struct is legal or not, for example.
Consider adding better, higher-level APIs for synchronization primitives e.g.
Mutex
by taking inspiration from other languages. This has been done successfully in the past with
WaitGroup
compared to using raw goroutines and channels.
Ideas for Go programs:
Consider never using Go closures, and instead using plain functions that cannot implicit capture outer variables.
Consider using goroutines as little as possible, whatever the API to manage them.
Consider spawning an OS process instead of a goroutine for isolation. No sharing of data means no data race possible.
Deep clone abundantly (just like in Rust). Memory (especially cache) is lightning fast. Your Go program will anyways not be bottlenecked by that, I guarantee it. Memory usage should be monitored, though, but will probably be fine.
Avoid global mutable variables.
Carefully audit resource sharing code: caches, connection pools, OS process pools, HTTP clients, etc. They are likely to contain data races.
Run the tests with the race detector on, all of them, always, from day one. Inspect the test coverage to know which areas may be uncharted areas in terms of concurrency safety.
Study places where a shallow copy may take place, e.g. function arguments passed by value and assignments. Does the type require a deep copy? Each non-trivial type should have documentation stating that.
If a type can be implemented in an immutable fashion, then it's great because there is no data race possible. For example, the
string
type in Go is immutable.
If you enjoy what you're reading, you want to support me, and can afford it:
Support me
. That allows me to write more cool articles!
This blog is
open-source
!
If you find a problem, please open a Github issue.
The content of this blog as well as the code snippets are under the
BSD-3 License
which I also usually use for all my personal projects. It's basically free for every use but you have to mention me as the original author.
We no longer have any active servers in France and are continuing the process of leaving OVH. We'll be rotating our TLS keys and Let's Encrypt account keys pinned via accounturi. DNSSEC keys may also be rotated. Our backups are encrypted and can remain on OVH for now.
Our App Store verifies the app store metadata with a cryptographic signature and downgrade protection along with verification of the packages. Android's package manager also has another layer of signature verification and downgrade protection.
Our System Updater verifies updates with a cryptographic signature and downgrade protection along with another layer of both in update_engine and a third layer of both via verified boot. Signing channel release channel names is planned too.
Our update mirrors are currently hosted on sponsored servers from ReliableSite (Los Angeles, Miami) and Tempest (London). London is a temporary location due to an emergency move from a provider which left the dedicated server business and will move. More sponsored update mirrors are coming.
Our ns1 anycast network is on Vultr and our ns2 anycast network is on BuyVM since both support BGP for announcing our own IP space. We're moving our main website/network servers used for default OS connections to a mix of Vultr+BuyVM locations.
We have 5 servers in Canada with OVH with more than static content and basic network services: email, Matrix, discussion forum, Mastodon and attestation. Our plan is to move these to Netcup root servers or a similar provider short term and then colocated servers in Toronto long term.
France isn't a safe country for open source privacy projects. They expect backdoors in encryption and for device access too. Secure devices and services are not going to be allowed. We don't feel safe using OVH for even a static website with servers in Canada/US via their Canada/US subsidiaries.
We were likely going to be able to release experimental Pixel 10 support very soon and it's getting disrupted. The attacks on our team with ongoing libel and harassment have escalated, raids on our chat rooms have escalated and more. It's rough right now and support is appreciated.
Nov 24, 2025 · 7:16 PM UTC
193
1,278
7,692
1,193,442
‘A step-change’: tech firms battle for undersea dominance with submarine drones
Guardian
www.theguardian.com
2025-11-28 06:00:59
As navies seek to counter submarines and protect cables, startups and big defence companies fight to lead market Flying drones used during the Ukraine war have changed land battle tactics for ever. Now the same thing appears to be happening under the sea. Navies around the world are racing to add au...
Flying drones used during the Ukraine war have
changed land battle tactics
for ever. Now the same thing appears to be happening under the sea.
Navies around the world are racing to add autonomous submarines. The UK’s Royal Navy is planning a fleet of underwater uncrewed vehicles (UUVs) which will, for the first time, take a leading role in tracking submarines and protecting undersea cables and pipelines. Australia has committed to spending $1.7bn (£1.3bn)
on “Ghost Shark” submarines
to counter Chinese submarines. The huge US Navy is spending billions on several UUV projects, including one already in use that can be launched from nuclear submarines.
Autonomous uncrewed submarines represent “a genuine step-change in the underwater battle space”, said Scott Jamieson, the managing director for maritime and land defence solutions at BAE Systems, Britain’s dominant weapons company and builder of its nuclear submarines. The new drones under development would allow navies to “scale up in ways that just weren’t available before”, at “a fraction of the cost of manned submarines”, he said.
The opportunity of a huge new market is pitting big, experienced defence companies, including BAE Systems and the US’s General Dynamics and Boeing against weapons tech startups such as the American firm Anduril – the maker of the Ghost Shark – and Germany’s Helsing. The startups claim they can move faster and cheaper.
Anduril’s Ghost Shark is an extra-large autonomous underwater vehicle (XLAUV) that has been ordered by the Royal Australian Navy.
Photograph: Rodney Braithwaite/Australian Defence Force/AFP/Getty Images
The struggle for undersea dominance has been almost constant in peacetime and war for most of the last century.
The first nuclear-powered submarine (the US’s Nautilus, named after Jules Verne’s fictional vessel) was launched in 1954, and nuclear-armed vessels are now the centrepiece of the armed forces of six countries – the US, Russia, the UK, France, China and India – while North Korea may have recently become a seventh. That is despite deep controversy over whether the weapons represent value for
huge sums of money
, and whether such a destructive arsenal
truly acts as a useful deterrent
.
Those armed forces are playing a constant game of hide and seek on the oceans. In order to avoid detection, the submarines rarely surface: maintenance problems on other vessels recently forced some British submariners
to spend a record nine months underwater
, carrying Trident nuclear missiles that are theoretically ready to strike at any time.
Tracking Russia’s underwater nuclear arsenal – which has become quieter in recent years – is a key focus of the Royal Navy, particularly focusing on the Greenland-Iceland-UK (GIUK) gap, a “chokepoint” that allows Nato allies to monitor Russian movements in the north Atlantic. A weapons executive said the South China Sea was another promising market, as China and its neighbours face off in a
tense, long-running territorial dispute
.
Underwater drones offer the promise of making it easier to track rivals’ submarines. Some sensors are designed to be dropped by other UUVs to lurk on the seabed for months at a time, according to an executive hoping to sell to the UK.
The second spur has been the increasing number of apparent attacks on oil and gas pipelines, such as the Nord Stream attack in 2022, for which Germany
has identified a Ukrainian suspect
, and damage in 2023 to the Balticconnector pipeline between Finland and Estonia. Undersea power and internet cables are also crucial to the global economy. An underwater
power cable between Finland and Estonia
was hit last Christmas, two months after
two telecommunications cables
in Swedish waters in the Baltic Sea were cut.
The UK government last week
accused Russia’s Yantar surveillance ship
of entering British waters to map undersea cables. It
said
the UK had seen a 30% increase in Russian vessels threatening UK waters in the past two years.
Parliament’s defence select committee has raised concerns over the UK’s vulnerability to undersea sabotage, known as “grey zone” actions that can cause big disruption but are likely to fall short of acts of war. Damage to the 60 undersea data and energy cables around the British Isles “could have devastating consequences for the UK”, the committee said.
Andy Thomis, the chief executive of Cohort, a British manufacturer of military technology including sonar sensors, said the crewed ships, planes and submarines used up to now to track nuclear-armed subs or sabotage vessels were “very, very capable and very, very expensive”. But, he said, “by combining those with uncrewed vessels, you get the decision-making abilities that humans can give you without putting them in very dangerous proximity”.
BAE has already tested the Herne underwater drone.
Photograph: BAE Systems
Cohort hopes that some of its towed sensors (named Krait, after a sea snake) could be used on smaller autonomous vessels.
The newest vessels contain as many as five times more sonar sensors than submarines in service. Lower power requirements are particularly important for smaller, uncrewed vessels, which do not have the luxury of a nuclear reactor on board. Passive sensors – which do not send out a sonar “ping” – make it harder to detect and destroy.
The Royal Navy, and the armed forces in general, are not known for getting the latest technology into action quickly. However, Ukraine’s forces have learned that speed and low cost are crucial when it comes to building drones for the air and the sea. For the undersea drones, the Ministry of Defence is trying to learn that lesson, asking for rapid development of tech demonstrators under “Project Cabot”.
BAE has already tested a possible contender, called Herne. Helsing is building a facility in Portsmouth, the home of the Royal Navy, to produce underwater drones. Anduril, which is run by the Donald Trump fundraiser Palmer Luckey, is eyeing up UK manufacturing sites.
The initial contract awards are expected this year, followed by testing, likely to be in north-west Scotland by the defence company QinetiQ, and a full-scale order for one or two companies, named Atlantic Net, to fill the GIUK gap with sensors.
The Royal Navy has described the project as “anti-submarine warfare as a service”, riffing on the much more common “software as a service”, according to a
£24m tender notice
published in May.
Anduril’s Dive LD autonomous underwater vehicle. The US company is eyeing up UK manufacturing sites.
Photograph: Hollie Adams/Reuters
Sidharth Kaushal, a senior research fellow on sea power at the Royal United Services Institute thinktank, said the submarine-hunting strategy of recent decades “doesn’t scale in a conflict” because it required an expensive mix of big “exquisite assets”.
Warships tow cables more than 100 metres long containing arrays of sonar sensors to try to pick up the faintest and lowest-frequency sounds. Planes such as the UK’s Boeing P-8 fleet drop disposable sonobuoys to detect submarines through the depths of the sea, satellites scour the surface for signs of wake left by a submarine communications mast, and a scattering of hunter-killer submarines patrol beneath the waves.
The idea of cheap drones taking on a chunk of this work is attractive. Yet Kaushal warned that the price advantage “remains to be seen”. Industry figures warn that a big fleet of UUVs would still come with significant maintenance costs.
For protecting undersea cables, too, it could be a double-edged sword: sabotage will be cheaper and easier as well. The prospect of drones firing on one another underwater is “absolutely realistic”, one executive said.
The Ministry of Defence described it as “contractor owned, contractor operated, naval oversight” – meaning that privately owned vessels will be charged with anti-submarine warfare for the first time, potentially making them military targets.
“The first thing the Russians will do is go out and test this, and push it,” said Ian McFarlane, the sales director for underwater systems at Thales UK, which already supplies the Royal Navy with sonar arrays towed by submarine-hunting ships, uncrewed surface boats and flying drones and hopes to play a part in Cabot, integrating that data.
However, the attraction of getting companies on board, McFarlane said, was that the Royal Navy and its allies were looking for “mass and persistence now” to counteract “an aggressor who is ramping up”.
One in 10 UK parents say their child has been blackmailed online, NSPCC finds
Guardian
www.theguardian.com
2025-11-28 06:00:58
Harms include threats to release intimate pictures as charity warns against parents sharing photos or details of children online Nearly one in 10 UK parents say their child has been blackmailed online, with harms ranging from threatening to release intimate pictures to revealing details about someon...
Nearly one in 10 UK parents say their child has been blackmailed online, with harms ranging from threatening to release intimate pictures to revealing details about someone’s personal life.
The NSPCC child protection charity also found that one in five parents know a child who has experienced online blackmail, while two in five said they rarely or never talked to their children about the subject.
The National Crime Agency has said that it is receiving more than 110 reports a month of child sextortion attempts,
where criminal gangs
trick teenagers into sending intimate pictures of themselves and then blackmail them.
Agencies across the UK, US and Australia have
confirmed a rising number of sextortion cases
involving teenage boys and young adult males being targeted by cyber-criminal gangs based in west Africa or south-east Asia, some of which have ended in tragedy.
Murray Dowey
, a 16-year-old from Dunblane, Scotland, killed himself in 2023 after becoming a victim of sextortion on Instagram and Dinal De Alwis, 16, killed himself in Sutton, south London, in October 2022
after being blackmailed over nude photographs
.
The NSPCC based its findings on a survey of more than 2,500 parents and said that tech companies were continuing to “fall short in their duty to protect children”.
Rani Govender, a policy manager at NSPCC, said: “Children deserve to be safe online, and that must be built into the very fabric of these platforms, not bolted on after harm has already been done.”
The NSPCC’s definitions of blackmail included threatening to release an intimate image or video of the child, or something the victim wanted to keep private, such as their sexuality. The information could have been obtained consensually or it could have been obtained by coercion, manipulation or created using artificial intelligence.
Perpetrators can be strangers, such
as sextortion gangs
, or someone the child knows, such as a friend or schoolmate. The blackmailer can also demand a range of things in exchange for not sharing information, such as money, more images or staying in a relationship.
The NSPCC said its definition of blackmail overlapped with sextortion but included a wider range of scenarios. “We chose the term ‘blackmail’ in our research because it allowed us to include extortion using other types of information that a child might want to keep private (eg their sexuality, images showing them without a religious garment) as well as a range of both sexual and non-sexual demands and threats,” the charity said.
The report also advised against “sharenting”, which refers to parents sharing photos of and information about their children online.
Experts recommend
telling children what a sextortion threat looks like and being aware of who they are talking to online. They also advise creating everyday opportunities for children to talk to adults, such as during meals together or regular car trips, so that teenagers feel they have a space where they can reveal they have been targeted by blackmail.
“Knowing how to talk about online blackmail in an age-appropriate way and creating an environment where children feel safe to come forward without fear of judgment can make all the difference,” said Govender.
The NSPCC also interviewed young people about why they might choose not to reveal a blackmail attempt to their parent or carer. The reasons for not disclosing the crime, according to the responses, include feeling embarrassed, preferring to talk to a friend first and feeling they “can handle it themselves”.
Show HN: Ray-BANNED, Glasses to detect smart-glasses that have cameras
So far fingerprinting specific devices based on bluetooth (BLE) is looking like easiest and most reliable approach. The picture below is the first version, which plays the legend of zelda 'secret found' jingle when it detects a BLE advertisement from Meta Raybans.
I'm essentially treating this README like a logbook, so it will have my current approaches/ideas.
Optics
By sending IR at camera lenses, we can take advantage of the fact that the CMOS sensor in a camera reflects light directly back at the source (called 'retro-reflectivity' / 'cat-eye effect') to identify cameras.
This isn't exactly a new idea. Some researchers in 2005 used this property to create 'capture-resistant environments' when smartphones with cameras were gaining popularity.
Now we have a similar situation to those 2005 researchers on our hands, where smart glasses with hidden cameras seem to be getting more popular. So I want to create a pair of glasses to identify these. Unfortunately, from what I can tell most of the existing research in this space records data with a camera and then uses ML, a ton of controlled angles, etc. to differentiate between normal reflective surfaces and cameras.
I would feel pretty silly if my solution uses its own camera. So I'll be avoiding that. Instead I think it's likely I'll have to rely on being consistent with my 'sweeps', and creating a good classifier based on signal data. For example you can see here that the back camera on my smartphone seems to produce quick and large spikes, while the glossy screen creates a more prolonged wave.
After getting to test some Meta Raybans, I found that this setup is not going to be sufficient. Here's a test of some sweeps of the camera-area + the same area when the lens is covered. You can see the waveform is similar to what I saw in the earlier test (short spike for camera, wider otherwise), but it's wildly inconsistent and the strength of the signal is very weak. This was from about 4 inches away from the LEDs. I didn't notice much difference when swapping between 940nm and 850nm LEDs.
So at least with current hardware that's easy for me to access, this probably isn't enough to differentiate accurately.
Another idea I had is to create a designated sweep 'pattern'. The user (wearing the detector glasses) would perform a specific scan pattern of the target. Using the waveforms captured from this data, maybe we can more accurately fingerprint the raybans. For example, sweeping across the targets glasses in a 'left, right, up, down' approach. I tested this by comparing the results of the Meta raybans vs some aviators I had lying around. I think the idea behind this approach is sound (actually it's light), but it might need more workshopping.
IR Circuit
For prototyping, I'm using:
Arduino uno
a bunch of 940nm and 850nm IR LEDs
a photodiode as a receiver
a 2222A transistor
TODO:
experiment with sweeping patterns
experiment with combining data from different wavelengths
collimation?
Networking
This has been more tricky than I first thought! My current approach here is to fingerprint the Meta Raybans over Bluetooth low-energy (BLE) advertisements. But,
I have only been able to detect BLE traffic during 1) pairing 2) powering-on
. I sometimes also see the advertisement as they are taken out of the case (while already powered on), but not consistently.
The goal is to detect them during usage when they're communicating with the paired phone, but to see this type of directed BLE traffic it seems like I would first need to see the
CONNECT_REQ
packet which has information as to what which of the communication channels to hop between in sync. I don't think what I currently have (ESP32) is set up to do this kind of following.
For any of the bluetooth classic (BTC) traffic, unfortunately the hardware seems a bit more involved (read: expensive). So if I want to do down this route, I'll likely need a more clever solution here.
When turned on or put into pairing mode (or sometimes when taken out of the case), I can detect the device through advertised manufacturer data and service UUIDs.
0x01AB
is a Meta-specific SIG-assigned ID (assigned by the Bluetooth standards body), and
0xFD5F
in the Service UUID is assigned to Meta as well.
capture when the glasses are powered on:
[01:07:06] RSSI: -59 dBm
Address: XX:XX:XX:XX:XX:XX
Name: Unknown
META/LUXOTTICA DEVICE DETECTED!
Manufacturer: Meta (0x01AB)
Service UUID: Meta (0xFD5F) (0000fd5f-0000-1000-8000-00805f9b34fb)
Manufacturer Data:
Company ID: Meta (0x01AB)
Data: 020102102716e4
Service UUIDs: ['0000fd5f-0000-1000-8000-00805f9b34fb']
IEEE assigns certain MAC address prefixes (OUI, 'Organizationally Unique Identifier'), but these addresses get randomized so I don't expect them to be super useful for BLE.
I’ve recently been experimenting with various ways to construct Linux VM images, but for these images to be practical, they need to interact with the outside world. At a minimum, they need to communicate with the host machine.
vsock
is a technology specifically designed with VMs in mind. It eliminates the need for a TCP/IP stack or network virtualization to enable communication with or between VMs. At the API level, it behaves like a standard socket but utilizes a specialized addressing scheme.
In the experiment below, we’ll explore using
vsock
as the transport mechanism for a gRPC service running on a VM. We’ll build this project with Bazel for easy reproducibility. Check out
this post
if you need an intro to Bazel.
There are many use cases for efficient communication between a VM and its host (or between multiple VMs). One simple reason is to create a hermetic environment within the VM and issue commands via RPC from the host. This is the primary driver for using gRPC in this example, but you can easily generalize the approach shown here to build far more complex systems.
GitHub repo
The complete repository is hosted
here
and serves as the source of truth for this experiment. While there may be minor inconsistencies between the code blocks below and the repository, please rely on GitHub as the definitive source.
Code breakdown
Let’s break down the code step by step:
External dependencies
Here are the external dependencies listed as Bazel modules:
This is largely self-explanatory. The
protobuf
repository is used for C++ proto-generation rules, and
grpc
provides the monorepo for Bazel rules to generate gRPC code for the C++ family of languages.
gRPC library generation
The following Bazel targets generate the necessary C++ Protobuf and gRPC libraries:
We want a statically linked binary to run on the VM. This choice simplifies deployment, allowing us to drop a single file onto the VM.
The code is largely self-explanatory:
#include<iostream>#include<memory>#include<string>#include<grpc++/grpc++.h>#include"proto/vsock_service.grpc.pb.h"using grpc::Server;using grpc::ServerBuilder;using grpc::ServerContext;using grpc::Status;using popovicu_vsock::VsockService;using popovicu_vsock::AdditionRequest;using popovicu_vsock::AdditionResponse;// Service implementationclassVsockServiceImplfinal : public VsockService::Service {StatusAddition(ServerContext*context, constAdditionRequest*request,AdditionResponse*response) override {int32_t result =request->a() +request->b();response->set_c(result); std::cout <<"Addition: "<<request->a() <<" + "<<request->b()<<" = "<< result << std::endl;return Status::OK; }};voidRunServer() { // Server running on VM (guest) // vsock:-1:9999 means listen on port 9999, accept connections from any CID // CID -1 (VMADDR_CID_ANY) allows the host to connect to this VM server std::string server_address("vsock:3:9999"); VsockServiceImpl service; ServerBuilder builder;builder.AddListeningPort(server_address, grpc::InsecureServerCredentials());builder.RegisterService(&service); std::unique_ptr<Server>server(builder.BuildAndStart()); std::cout <<"Server listening on "<< server_address << std::endl;server->Wait();}intmain() {RunServer();return0;}
The only part requiring explanation is the
server_address
. The
vsock:
prefix indicates that we’re using
vsock
as the transport layer. gRPC supports various transports, including TCP/IP and Unix sockets.
The number
3
is the CID, or
Context ID
. This functions similarly to an IP address. Certain CIDs have special meanings. For instance, CID 2 represents the VM host itself; if the VM needs to connect to a
vsock
socket on the host, it targets CID 2. CID 1 is reserved for the loopback address. Generally, VMs are assigned CIDs starting from 3.
The
9999
is simply the port number, functioning just as it does in TCP/IP.
#include<iostream>#include<memory>#include<string>#include<grpc++/grpc++.h>#include"proto/vsock_service.grpc.pb.h"using grpc::Channel;using grpc::ClientContext;using grpc::Status;using popovicu_vsock::VsockService;using popovicu_vsock::AdditionRequest;using popovicu_vsock::AdditionResponse;classVsockClient {public:VsockClient(std::shared_ptr<Channel> channel) : stub_(VsockService::NewStub(channel)) {}int32_tAdd(int32_ta, int32_tb) { AdditionRequest request;request.set_a(a);request.set_b(b); AdditionResponse response; ClientContext context; Status status =stub_->Addition(&context, request, &response);if (status.ok()) {returnresponse.c(); } else { std::cout <<"RPC failed: "<<status.error_code() <<": "<<status.error_message() << std::endl;return-1; } }private: std::unique_ptr<VsockService::Stub> stub_;};intmain() { // Client running on host, connecting to VM server // vsock:3:9999 means connect to CID 3 (guest VM) on port 9999 // CID 3 is an example - adjust based on your VM's actual CID std::string server_address("vsock:3:9999"); VsockClient client( grpc::CreateChannel(server_address, grpc::InsecureChannelCredentials()));int32_t a =5;int32_t b =7;int32_t result =client.Add(a, b); std::cout <<"Addition result: "<< a <<" + "<< b <<" = "<< result<< std::endl;return0;}
Running it all together
Bazel shines here. You only need a working C++ compiler on your host system. Bazel automatically fetches and builds everything else on the fly, including the Protobuf compiler.
To get the statically linked server binary:
bazelbuild//server
Similarly, for the client:
bazelbuild//client
To create a VM image, I used
debootstrap
on an
ext4
image, as described in this post on X:
Stop downloading 4GB ISOs to create Linux VMs.
You don't need an installer, a GUI, or a "Next > Next > Finish" wizard. You just need a directory of files.
Here is how I build custom, hacky, bootable Debian VMs in 60 seconds using debootstrap.
As shown in the last line, a virtual device is attached to the QEMU VM acting as
vsock
networking hardware, configured with CID 3.
The QEMU output shows:
[ 1.581192] Run /opt/server as init process[ 1.889382] random: crng init doneServer listening on vsock:3:9999
To send an RPC to the server from the host, I ran the client binary:
bazel run //client
The output confirmed the result:
Addition result: 5 + 7 = 12
Correspondingly, the server output displayed:
Addition: 5 + 7 = 12
We have successfully invoked an RPC from the host to the VM!
Under the hood
I haven’t delved into the low-level system API for
vsock
s, as frameworks typically abstract this away. However,
vsock
s closely resemble TCP/IP sockets. Once created, they are used in the same way, though the creation API differs. Information on this is readily available online.
Conclusion
I believed it was more valuable to focus on a high-level RPC system over
vsock
rather than raw sockets. With gRPC, you can invoke a structured RPC on a server running inside the VM. This opens the door to running interesting applications in sealed, isolated environments, allowing you to easily combine different OSes (e.g., a Debian host and an Arch guest) or any platform supporting
vsock
. Additionally, gRPC allows you to write clients and servers in many different languages and technologies. This is achieved without network virtualization, resulting in increased efficiency.
I hope this was fun and useful to you as well.
Please consider following me on
X
and
LinkedIn
for further updates.
There have been some recent concerns about ML-KEM, NIST’s standard for encryption with Post-Quantum Cryptography, related standards of the IETF, and lots of conspiracy theories about malicious actors subverting the standardization process. As someone who has been involved with this standardization process at pretty much every level, here a quick debunking of the various nonsense I have heard. So let’s get started, FAQ style.
Did the NSA invent ML-KEM?
No. It was first specified by a team of various European cryptographers, whom you can look up
on their website
.
Okay, but that was Kyber, not ML-KEM, did the NSA change Kyber?
No. The differences between Kyber and ML-KEM are pretty minute, mostly editorial changes by NIST. The only change that could be seen as actually interesting was a slight change to how certain key derivation mechanics worked. This change was suggested by Peter Schwabe, one of the original authors of Kyber, and is fairly straightforward to analyze. The reason for this change was that originally, Kyber was able to produce shared secrets of any length, by including a KDF step. But applications usually need their own KDF to apply to shared secrets, in order to bind the shared secret to transcripts and similar, so you would end up with two KDF calls. Since Kyber only uses the KDF to stretch the output, removing it slightly improves the performance of the algorithm without having any security consequences. Basically, there was a feature that turned out to not actually be a feature in real world scenarios, so NIST removed it, after careful consideration, and after being encouraged to do so by the literal author of the scheme, and under the watchful eyes of the entire cryptographic community. Nothing untoward happened here.
Okay but what about maybe there still being a backdoor?
There is no backdoor in ML-KEM, and I can prove it. For something to be a backdoor, specifically a “Nobody but us backdoor” (NOBUS), you need some way to ensure that nobody else can exploit it, otherwise it is not a backdoor, but a broken algorithm, and any internal cryptanalysis you might have will be caught up eventually by academia. So for something to be a useful backdoor, you need to possess some secret that cannot be brute forced that acts as a private key to unlock any ciphertext generated by the algorithm. This is the backdoor in DUAL_EC_DRBG, and, since the US plans to use ML-KEM themselves (as opposed to the export cipher shenanigans back in the day), would be the only backdoor they could reasonably insert into a standard.
However, if you have a private key, that cannot be brute forced, you need to have a public key as well, and that public key needs to be large enough to prevent brute-forcing, and be embedded into the algorithm, as a parameter. And in order to not be brute forceable, this public key needs to have at least 128 bits of entropy. This gives us a nice test to see whether a scheme is capable of having cryptographic NOBUS backdoors: We tally up the entropy of the parameter space. If the result is definitely less than 128 bits, the scheme can at most be broken, but cannot be backdoored.
So let’s do that for ML-KEM:
This is the set of parameters, let’s tally them up, with complete disregard for any of the choices being much more constrained than random integers would suggest (actually, I am too much of a nerd to not point out the constraints, but I will use the larger number for the tally).
Degree of the number field: 8 bits (actually, it has to be a power of two, so really only 3 bits)
Prime: 12 bits (actually, it has to be a prime, so 10.2 bits (Actually, actually, it has to be a prime of the form
, and it has to be at least double the rank times degree, and 3329 is literally the smallest prime that fits that bill))
Rank of the module: 3 bits (well, the rank of the module is the main security parameter, it literally just counts from 2 to 4)
Secret and error term bounds: 2 + 2 bits (really these come from the size of the prime, the module rank, and the number field degree)
Compression strength: 4 + 3 bits
In total, this gives us 34 bits. Counted exceedingly generously. I even gave an extra bit for all the small numbers! Any asymmetric cryptosystem with a 34 bit public key would be brute forceable by a laptop within a few minutes, and ML-KEM would not be backdoored, but rather be broken.
There is no backdoor in ML-KEM, because there simply is no space to hide a backdoor in ML-KEM
.
And just to be sure, if you apply this same counting bits of parameters test to the famously backdoored DUAL_EC_DRBG, you indeed have multiple elliptic curve points defined in the standard without any motivation, immediately blowing our 128 bits of entropy budget for parameters. In fact, it would be trivial to fix DUAL_EC_DRBG by applying what’s called a “Nothing up my sleeves” paradigm: Instead of just having the elliptic curves points sit there, with no explanation, make it so that they are derived from digits of π, e, or the output of some hash function on some published seed. That would still not pass our test, but that is because I designed this test to be way too aggressive, as the remarks in the brackets show, there is not really any real choice to these parameters, they are just the smallest set of parameters that result in a secure scheme (making them larger would only make the scheme slower and/or have more overhead).
So no, there is no backdoor in ML-KEM.
But didn’t NIST fail basic math when picking ML-KEM?
I thought ML-KEM was broken, something about a fault attack?
There are indeed fault attacks on ML-KEM. This is not super surprising, if you know what a fault attack (also called glitch attack) is. For a fault attack, you need to insert a mistake – a fault – in the computation of the algorithm. You can do this via messing with the physical hardware, things like ROWHAMMER that literally change the memory while the computation is happening. It’s important to analyze these types of failures, but literally any practical cryptographic algorithm in existence is vulnerable to fault attacks.
It’s literally computers failing at their one job and not computing very well. CPU and memory attacks are probably one of the most powerful families of attacks we have, and they have proven to be very stubborn to mitigate. But algorithms failing in the face of them is not particularly surprising, after all, if you can flip a single arbitrary bit, you might as well just set “verified_success” to true and call it a day. Technically, this is the strongest form of fault, where the attacker choses where it occurs, but even random faults usually demolish pretty much any cryptographic algorithm, and us knowing about these attacks is merely evidence of an algorithm being seen as important enough to do the math of how exactly they fail when you literally pull the ground out beneath them.
But what about decryption failure attacks? Those sound scary!
ML-KEM has a weird quirk: It is, theoretically, possible to create a ciphertext, in an honest fashion, that the private key holder will reject. If one were to successfully do so, one would learn information about the private key. But here comes the kicker: The only way to create this poisoned ciphertext is by honestly running the encapsulation algorithm, and hoping to get lucky. There is a slight way to bias the ciphertexts, but to do so, one still has to compute them, and the advantage would be abysmal, since ML-KEM forces the hand of the encapsulating party on almost all choices. The probability of this decapsulation failure can be compute with relatively straight-forward mathematics, the Cauchy-Schwartz inequality. And well, the parameters of ML-KEM are chosen in such a way that the actual probability is vanishingly small, less than
. At this point, the attacker cannot really assume that they were observing a decapsulation failure anymore, as a whole range of other incredibly unlikely events, such as enough simultaneous bit flips due to cosmic radiation to evade error detection are far more likely. It is true that after the first decapsulation failure has been observed, the attacker has much more abilities to stack the deck in their favor, but to do so, you first need the first failure to occur, and there is not really any hope in doing so.
On top of this, the average ML-KEM key is used exactly once, as such is the fate of keys used in key exchange, further making any adaptive attack like this meaningless, but ML-KEM keys are safe to use even with multiple decapsulations.
But wasn’t there something called Kyberslash?
Yeah
. It turns out, implementing cryptographic code is still hard. My modest bragging right is that
my implementation
, which would eventually morph into BoringSSL’s ML-KEM implementation, never had this problem, so I guess the answer here is to git gud, or something. But really, especially initially, there are some rough edges in new implementations as we learn the right techniques to avoid them. Importantly, this is a flaw of the implementation, not of the mathematics of the algorithm. In fact, the good news here is that implementationwise, ML-KEM is actually a lot simpler than elliptic curves are, so these kinds of minor side channel issues are likely to be rarer here, we just haven’t implemented it as much as elliptic curves.
Okay, enough about ML-KEM, what about hybrids and the IETF?
Okay, this one is a funny one. Well funny if you likely deeply dysfunctional bikeshedding, willful misunderstanding, and drama. First of, what are hybrids? Assume you have two cryptographic schemes that do the same thing, and you distrust both of them. But you do trust the combination of the two. That is, in essence, what hybrids allow you to do: Combine two schemes of the same type into one, so that the combined scheme is at least as secure as either of them. The usual line is that this is perfect for PQC, as it allows you to combine the well studied security of classical schemes with the quantum resistance of PQC schemes. Additionally, the overhead of elliptic curve cryptography, when compared with lattice cryptography, is tiny, so why not throw it in there. And generally I agree with that stance, although I would say that my trust in lattice cryptography is pretty much equal to my trust in elliptic curves, and quite a bit higher than my trust in RSA, so I would not see hybrids as absolutely, always and at every turn, superduper essential. But they are basically free, so why not? In the end, yes, hybrids are the best way to go, and indeed, this is what the IETF enabled people to do. There are various RFCs to that extent, to understand the current controversy, we need to focus on two TLS related ones: X25519MLKEM768 aka 0x11EC, and MLKEM1024. The former is a hybrid, the latter is not. And, much in line with my reasoning, 0x11EC is the default key exchange algorithm used by Chrome, Firefox, and pretty much all other TLS clients that currently support PQC. So what’s the point of MLKEM1024? Well it turns out there is one customer who really really hates hybrids, and only wants to use ML-KEM1024 for all their systems. And that customer happens to be the NSA. And honestly, I do not see a problem with that. If the NSA wants to make their own systems inefficient, then that is their choice. Why inefficient? It turns out that, due to the quirks of how TLS works, the client needs to predict what the server will likely accept. They could predict more things, but since PQC keys are quite chonky, sending more than one PQC key is making your handshakes slower. And so does mispredicting, since it results in the server saying “try again, with the right public key, this time”. So, if everyone but the NSA uses X25519MLKEM768, the main effect is that the NSA has slower handshakes. As said, I don’t think it’s reasonable to say their handshakes are substantially less secure, but sure, if you really think ML-KEM is broken, then yes, the NSA has successfully undermined the IETF in order to make their own systems less secure, while not impacting anyone else. Congratulations to them, I guess.
But doesn’t the IETF actively discourage hybrids?
No. To understand this, we need to look at two flags that come with TLS keyexchange algorithms: Recommended and Mandatory To Implement. Recommended is a flag with three values, Yes, No, and Discouraged. The Discouraged state is used for algorithms known to be broken, such as RC4. Clearly ML-KEM, with or without a hybrid, is not known to be broken, so Discouraged is the wrong category. It is true that 0x11EC is not marked as Recommended, mostly because it started out as an experimental combination that then somehow ended up as the thing everybody was doing, and while lots of digital ink was spilled on whether or not it should be recommended, nobody updated the flag before publishing the RFC. (technically the RFC is not published yet, but the rest is pretty much formality, and the flag is unlikely to change) So yes, technically the IETF did not recommend a hybrid algorithm. But your browsers and everybody else is using it, so there is that. And just in case you were worried about that, the NSA option of MLKEM1024 is also not marked as recommended.
Lastly, Mandatory To Implement is an elaborate prank by the inventors of TLS to create more discussions on mailing lists. As David Benjamin once put it, the only algorithm that is actually mandatory to implement is the null algorithm, as that is the name of the initial state of a TLS connection, before an algorithm has been negotiated. Otherwise, at least my recommendation, is to respond with this gif
whenever someone requests a MTI algorithm you don’t want to support. The flag has literally zero meaning. Oh and yeah, neither of the two algorithms is MTI.
TigerStyle: Coding philosophy focused on safety, performance, dev experience
Tiger Style
is a coding philosophy focused on
safety
,
performance
, and
developer experience
. Inspired by the practices of
TigerBeetle, it focuses on building robust, efficient, and maintainable
software through disciplined engineering.
Tiger Style is not just a set of coding standards; it's a practical
approach to software development. By prioritizing
safety
,
performance
, and
developer experience
, you create code that is reliable,
efficient, and enjoyable to work with.
Safety
Safety is the foundation of Tiger Style. It means writing code that
works in all situations and reduces the risk of errors. Focusing on
safety makes your software reliable and trustworthy.
Performance
Performance is about using resources efficiently to deliver fast,
responsive software. Prioritizing performance early helps you design
systems that meet or exceed user expectations.
Developer experience
A good developer experience improves code quality and maintainability.
Readable and easy-to-work-with code encourages collaboration and reduces
errors, leading to a healthier codebase that stands the test of time
[1]
.
2. Design goals
The design goals focus on building software that is safe, fast, and easy
to maintain.
2.1. Safety
Safety in coding relies on clear, structured practices that prevent
errors and strengthen the codebase. It's about writing code that works
in all situations and catches problems early. By focusing on safety, you
create reliable software that behaves predictably no matter where it
runs.
Control and limits
Predictable control flow and bounded system resources are essential for
safe execution.
Simple and explicit control flow
: Favor
straightforward control structures over complex logic. Simple
control flow makes code easier to understand and reduces the risk of
bugs. Avoid recursion if possible to keep execution bounded and
predictable, preventing stack overflows and uncontrolled resource
use.
Set fixed limits
: Set explicit upper bounds on
loops, queues, and other data structures. Fixed limits prevent
infinite loops and uncontrolled resource use, following the
fail-fast
principle. This approach helps catch
issues early and keeps the system stable.
Limit function length
: Keep functions concise,
ideally under
70 lines
. Shorter functions are
easier to understand, test, and debug. They promote single
responsibility, where each function does one thing well, leading to
a more modular and maintainable codebase.
Centralize control flow
: Keep switch or if
statements in the main parent function, and move non-branching logic
to helper functions. Let the parent function manage state, using
helpers to calculate changes without directly applying them. Keep
leaf functions pure and focused on specific computations. This
divides responsibility: one function controls flow, others handle
specific logic.
Memory and types
Clear and consistent handling of memory and types is key to writing
safe, portable code.
Use explicitly sized types
: Use data types with
explicit sizes, like
u32
or
i64
, instead
of architecture-dependent types like
usize
. This keeps
behavior consistent across platforms and avoids size-related errors,
improving portability and reliability.
Static memory allocation
: Allocate all necessary
memory during startup and avoid dynamic memory allocation after
initialization. Dynamic allocation at runtime can cause
unpredictable behavior, fragmentation, and memory leaks. Static
allocation makes memory management simpler and more predictable.
Minimize variable scope
: Declare variables in the
smallest possible scope. Limiting scope reduces the risk of
unintended interactions and misuse. It also makes the code more
readable and easier to maintain by keeping variables within their
relevant context.
Error handling
Correct error handling keeps the system robust and reliable in all
conditions.
Use assertions
: Use assertions to verify that
conditions hold true at specific points in the code. Assertions work
as internal checks, increase robustness, and simplify debugging.
Assert function arguments and return values
:
Check that functions receive and return expected values.
Validate invariants
: Keep critical conditions
stable by asserting invariants during execution.
Use pair assertions
: Check critical data at
multiple points to catch inconsistencies early.
Fail fast on programmer errors
: Detect unexpected
conditions immediately, stopping faulty code from continuing.
Handle all errors
: Check and handle every error.
Ignoring errors can lead to undefined behavior, security issues, or
crashes. Write thorough tests for error-handling code to make sure
your application works correctly in all cases.
Treat compiler warnings as errors
: Use the
strictest compiler settings and
treat all warnings as errors
. Warnings often point
to potential issues that could cause bugs. Fixing them right away
improves code quality and reliability.
Avoid implicit defaults
: Explicitly specify options
when calling library functions instead of relying on defaults.
Implicit defaults can change between library versions or across
environments, causing inconsistent behavior. Being explicit improves
code clarity and stability.
2.2. Performance
Performance is about using resources efficiently to deliver fast,
responsive software. Prioritizing performance early helps design systems
that meet or exceed user expectations without unnecessary overhead.
Design for performance
Early design decisions have a significant impact on performance.
Thoughtful planning helps avoid bottlenecks later.
Design for performance early
: Consider performance
during the initial design phase. Early architectural decisions have
a big impact on overall performance, and planning ahead ensures you
can avoid bottlenecks and improve resource efficiency.
Napkin math
: Use quick, back-of-the-envelope
calculations to estimate system performance and resource costs. For
example, estimate how long it takes to read 1 GB of data from memory
or what the expected storage cost will be for logging 100,000
requests per second. This helps set practical expectations early and
identify potential bottlenecks before they occur.
Batch operations
: Amortize expensive operations by
processing multiple items together. Batching reduces overhead per
item, increases throughput, and is especially useful for I/O-bound
operations.
Efficient resource use
Focus on optimizing the slowest resources, typically in this order:
Network
: Optimize data transfer and reduce latency.
Disk
: Improve I/O operations and manage storage
efficiently.
Memory
: Use memory effectively to prevent leaks and
overuse.
CPU
: Increase computational efficiency and reduce
processing time.
Predictability
Writing predictable code improves performance by reducing CPU cache
misses and optimizing branch prediction.
Ensure predictability
: Write code with predictable
execution paths. Predictable code uses CPU caching and branch
prediction better, leading to improved performance. Avoid patterns
that cause frequent cache misses or unpredictable branching, as they
degrade performance.
Reduce compiler dependence
: Don't rely solely on
compiler optimizations for performance. Write clear, efficient code
that doesn't depend on compiler behavior. Be explicit in
performance-critical sections to ensure consistent results across
compilers.
2.3. Developer experience
Improving the developer experience creates a more maintainable and
collaborative codebase.
Name things
Get the nouns and verbs right. Great names capture what something is or
does and create a clear, intuitive model. They show you understand the
domain. Take time to find good names, where nouns and verbs fit
together, making the whole greater than the sum of its parts.
Clear and consistent naming
: Use descriptive and
meaningful names for variables, functions, and files. Good naming
improves code readability and helps others understand each
component's purpose. Stick to a consistent style, like
snake_case
, throughout the codebase.
Avoid abbreviations
: Use full words in names unless
the abbreviation is widely accepted and clear (e.g.,
ID
,
URL
). Abbreviations can be confusing
and make it harder for others, especially new contributors, to
understand the code.
Include units or qualifiers in names
: Append units
or qualifiers to variable names, placing them in descending order of
significance (e.g.,
latency_ms_max
instead of
max_latency_ms
). This clears up meaning, avoids
confusion, and ensures related variables, like
latency_ms_min
, line up logically and group together.
Document the 'why'
: Use comments to explain why
decisions were made, not just what the code does. Knowing the intent
helps others maintain and extend the code properly. Give context for
complex algorithms, unusual approaches, or key constraints.
Organize things
Organizing code well makes it easy to navigate, maintain, and extend. A
logical structure reduces cognitive load, letting developers focus on
solving problems instead of figuring out the code. Group related
elements, and simplify interfaces to keep the codebase clean, scalable,
and manageable as complexity grows.
Organize code logically
: Structure your code
logically. Group related functions and classes together. Order code
naturally, placing high-level abstractions before low-level details.
Logical organization makes code easier to navigate and understand.
Simplify function signatures
: Keep function
interfaces simple. Limit parameters, and prefer returning simple
types. Simple interfaces reduce cognitive load, making functions
easier to understand and use correctly.
Construct objects in-place
: Initialize large
structures or objects directly where they are declared. In-place
construction avoids unnecessary copying or moving of data, improving
performance and reducing the potential for lifecycle errors.
Minimize variable scope
: Declare variables close to
their usage and within the smallest necessary scope. This reduces
the risk of misuse and makes code easier to read and maintain.
Ensure consistency
Maintaining consistency in your code helps reduce errors and creates a
stable foundation for the rest of the system.
Avoid duplicates and aliases
: Prevent
inconsistencies by avoiding duplicated variables or unnecessary
aliases. When two variables represent the same data, there's a
higher chance they fall out of sync. Use references or pointers to
maintain a single source of truth.
Pass large objects by reference
: If a function's
argument is larger than 16 bytes, pass it as a reference instead of
by value to avoid unnecessary copying. This can catch bugs early
where unintended copies may occur.
Minimize dimensionality
: Keep function signatures
and return types simple to reduce the number of cases a developer
has to handle. For example, prefer
void
over
bool
,
bool
over
u64
, and so
on, when it suits the function's purpose.
Handle buffer allocation cleanly
: When working with
buffers, allocate them close to where they are used and ensure all
corresponding cleanup happens in the same logical block. Group
resource allocation and deallocation with clear newlines to make
leaks easier to identify.
Avoid off-by-one errors
Off-by-one errors often result from casual interactions between an
index
, a
count
, or a
size
. Treat
these as distinct types, and apply clear rules when converting between
them.
Indexes, counts, and sizes
: Indexes are 0-based,
counts are 1-based, and sizes represent total memory usage. When
converting between them, add or multiply accordingly. Use meaningful
names with units or qualifiers
to avoid confusion. See
Handle division intentionally
: When dividing, make
your intent clear by specifying how rounding should be handled in
edge cases. Use functions or operators designed for exact division,
floor division, or ceiling division. This avoids ambiguity and
ensures the result behaves as expected.
Code consistency and tooling
Consistency in code style and tools improves readability, reduces mental
load, and makes working together easier.
Maintain consistent indentation
: Use a uniform
indentation style across the codebase. For example, using 4 spaces
for indentation provides better visual clarity, especially in
complex structures.
Limit line lengths
: Keep lines within a reasonable
length (e.g., 100 characters) to ensure readability. This prevents
horizontal scrolling and helps maintain an accessible code layout.
Use clear code blocks
: Structure code clearly by
separating blocks (e.g., control structures, loops, function
definitions) to make it easy to follow. Avoid placing multiple
statements on a single line, even if allowed. Consistent block
structures prevent subtle logic errors and make code easier to
maintain.
Minimize external dependencies
: Reducing external
dependencies simplifies the build process and improves security
management. Fewer dependencies lower the risk of supply chain
attacks, minimize performance issues, and speed up installation.
Standardize tooling
: Using a small, standardized
set of tools simplifies the development environment and reduces
accidental complexity. Choose cross-platform tools where possible to
avoid platform-specific issues and improve portability across
systems.
Addendum
Addendum: Zero technical debt
While Tiger Style focuses on the core principles of safety,
performance, and developer experience, these are reinforced by an
underlying commitment to zero technical debt.
A
zero technical debt policy
is key to maintaining a
healthy codebase and ensuring long-term productivity. Addressing
potential issues proactively and building robust solutions from the
start helps avoid debt that would slow future development.
Do it right the first time
: Take the time to
design and implement solutions correctly from the start. Rushed
features lead to technical debt that requires costly refactoring
later.
Be proactive in problem-solving
: Anticipate
potential issues and fix them before they escalate. Early
detection saves time and resources, preventing performance
bottlenecks and architectural flaws.
Build momentum
: Delivering solid, reliable code
builds confidence and enables faster development cycles.
High-quality work supports innovation and reduces the need for
future rewrites.
Avoiding technical debt ensures that progress is true progress—solid,
reliable, and built to last.
Addendum: Performance estimation
You should think about performance early in design. Napkin math is a
helpful tool for this.
Napkin math uses simple calculations and rounded numbers to quickly
estimate system performance and resource needs.
Quick insights
: Understand system behavior fast
without deep analysis.
Early decisions
: Find potential bottlenecks early
in design.
Sanity checks
: See if an idea works before you
build it.
For example, if you're designing a system to store logs, you can
estimate storage costs like this:
1. Estimate log volume:
Assume 1,000 requests per second (RPS)
Each log entry is about 1 KB
2. Calculate daily log volume:
1,000 RPS * 86,400 seconds/day * 1 KB ≈ 86,400,000 KB/day ≈ 86.4 GB/day
3. Estimate monthly storage:
86.4 GB/day * 30 days ≈ 2,592 GB/month
4. Estimate cost (using $0.02 per GB for blob storage):
2,592 GB * 1000 GB/TB * $0.02/GB ≈ $51 per month
This gives you a rough idea of monthly storage costs. It helps you
check if your logging plan works. The idea is to get within 10x of the
right answer.
This document is a "remix" inspired by the original
Tiger Style guide
from the TigerBeetle project. In the spirit of
Remix Culture
, parts of this document are verbatim
copies of the original work,
while other sections have been rewritten or adapted to fit the goals
of this version. This remix builds upon the principles outlined in the
original document with a more general approach.
Hash-based IDs eliminate merge conflicts and collision issues!
Previous versions used sequential IDs (bd-1, bd-2, bd-3...) which caused frequent collisions when multiple agents or branches created issues concurrently. Version 0.20.1 switches to
hash-based IDs
(bd-a1b2, bd-f14c, bd-3e7a...) that are collision-resistant and merge-friendly.
What's new:
✅ Multi-clone, multi-branch, multi-agent workflows now work reliably
What changed:
Issue IDs are now short hashes instead of sequential numbers
Migration:
Run
bd migrate
to upgrade existing databases (optional - old DBs still work)
Hash IDs use progressive length scaling (4/5/6 characters) with birthday paradox math to keep collisions extremely rare while maintaining human readability. See "Hash-Based Issue IDs" section below for details.
⚠️
Alpha Status
: This project is in active development. The core features work well, but expect API changes before 1.0. Use for development/internal projects first.
Beads is a lightweight memory system for coding agents, using a graph-based issue tracker. Four kinds of dependencies work to chain your issues together like beads, making them easy for agents to follow for long distances, and reliably perform complex task streams in the right order.
Drop Beads into any project where you're using a coding agent, and you'll enjoy an instant upgrade in organization, focus, and your agent's ability to handle long-horizon tasks over multiple compaction sessions. Your agents will use issue tracking with proper epics, rather than creating a swamp of rotten half-implemented markdown plans.
Then tell your coding agent to start using the
bd
tool instead of markdown for all new work, somewhere in your
AGENTS.md
or
CLAUDE.md
. That's all there is to it!
You don't use Beads directly as a human. Your coding agent will file and manage issues on your behalf. They'll file things they notice automatically, and you can ask them at any time to add or update issues for you.
Beads gives agents unprecedented long-term planning capability, solving their amnesia when dealing with complex nested plans. They can trivially query the ready work, orient themselves, and land on their feet as soon as they boot up.
Agents using Beads will no longer silently pass over problems they notice due to lack of context space -- instead, they will automatically file issues for newly-discovered work as they go. No more lost work, ever.
Beads issues are backed by git, but through a clever design it manages to act like a managed, centrally hosted SQL database shared by all of the agents working on a project (repo), even across machines.
Beads even improves work auditability. The issue tracker has a sophisticated audit trail, which agents can use to reconstruct complex operations that may have spanned multiple sessions.
Agents report that they enjoy working with Beads, and they will use it spontaneously for both recording new work and reasoning about your project in novel ways. Whether you are a human or an AI, Beads lets you have more fun and less stress with agentic coding.
Features
✨
Zero setup
-
bd init
creates project-local database (and your agent will do it)
Beads is designed for
AI coding agents
to use on your behalf. Setup takes 30 seconds:
You run this once (humans only):
# In your project root:
bd init
# For OSS contributors (fork workflow):
bd init --contributor
# For team members (branch workflow):
bd init --team
# For protected branches (GitHub/GitLab):
bd init --branch beads-metadata
# bd will:# - Create .beads/ directory with database# - Import existing issues from git (if any)# - Prompt to install git hooks (recommended: say yes)# - Prompt to configure git merge driver (recommended: say yes)# - Auto-start daemon for sync# Then tell your agent about bd:echo"\nBEFORE ANYTHING ELSE: run 'bd onboard' and follow the instructions">> AGENTS.md
Protected branches?
If your
main
branch is protected, use
bd init --branch beads-metadata
to commit issue updates to a separate branch. See
docs/PROTECTED_BRANCHES.md
for details.
Your agent does the rest:
Next time your agent starts, it will:
Run
bd onboard
and receive integration instructions
Add bd workflow documentation to AGENTS.md
Update CLAUDE.md with a note (if present)
Remove the bootstrap instruction
For agents setting up repos:
Use
bd init --quiet
for non-interactive setup (auto-installs git hooks and merge driver, no prompts).
For new repo clones:
Run
bd init
(or
bd init --quiet
for agents) to import existing issues from
.beads/issues.jsonl
automatically.
Git merge driver:
During
bd init
, beads configures git to use
bd merge
for intelligent JSONL merging. This prevents conflicts when multiple branches modify issues. Skip with
--skip-merge-driver
if needed. To configure manually later:
.beads/README.md
- Documentation about beads for repository visitors
.beads/metadata.json
- Database metadata
Should be in
.gitignore
(local-only):
.beads/beads.db
- SQLite cache (auto-synced with JSONL)
.beads/beads.db-*
- SQLite journal files
.beads/bd.sock
/
.beads/bd.pipe
- Daemon communication socket
.beads/.exclusive-lock
- Daemon lock file
.git/beads-worktrees/
- Git worktrees (only created when using protected branch workflows)
The
.gitignore
entries are automatically created inside
.beads/.gitignore
by
bd init
, but your project's root
.gitignore
should also exclude the database and daemon files if you want to keep your git status clean.
Using devcontainers?
Open the repository in a devcontainer (GitHub Codespaces or VS Code Remote Containers) and bd will be automatically installed with git hooks configured. See
.devcontainer/README.md
for details.
Stealth Mode (Isolated Usage)
Want to use beads in your local clone without other collaborators seeing any beads-related files? Use
stealth mode
:
Stealth mode configures:
Global gitignore
(
~/.config/git/ignore
) - Ignores
**/.beads/
and
**/.claude/settings.local.json
globally
Claude Code settings
(
.claude/settings.local.json
) - Adds
bd onboard
instruction for AI agents
Perfect for:
Personal experimentation with beads
Working on repos where not everyone uses beads yet
Keeping your issue tracking private while contributing to open source projects
AI agents that should use beads without affecting the main repo
What stays invisible to others:
No
.beads/
directory tracked in git
No AGENTS.md or README.md mentions of beads
No local
.gitattributes
or
.gitignore
modifications
Your beads database and issues remain local-only
How it works:
The global git configuration handles beads merging automatically, while the global gitignore ensures beads files never get committed to shared repos. Your AI agents get the onboard instruction automatically without exposing beads to other repo collaborators.
Most tasks will be created and managed by agents during conversations. You can check on things with:
bd list # See what's being tracked
bd show <issue-id># Review a specific issue
bd ready # See what's ready to work on
bd dep tree <issue-id># Visualize dependencies
For AI Agents
Run the interactive guide to learn the full workflow:
Quick reference for agent workflows:
# Find ready work
bd ready --json | jq '.[0]'# Create issues during work
bd create "Discovered bug" -t bug -p 0 --json
# Link discovered work back to parent
bd dep add <new-id><parent-id> --type discovered-from
# Update status
bd update <issue-id> --status in_progress --json
# Complete work
bd close <issue-id> --reason "Implemented" --json
Configuring Your Own AGENTS.md
Recommendation for project maintainers:
Add a session-ending protocol to your project's
AGENTS.md
file to ensure agents properly manage issue tracking and sync the database before finishing work.
This pattern has proven invaluable for maintaining database hygiene and preventing lost work. Here's what to include (adapt for your workflow):
1. File/update issues for remaining work
Agents should proactively create issues for discovered bugs, TODOs, and follow-up tasks
Close completed issues and update status for in-progress work
2. Run quality gates (if applicable)
Tests, linters, builds - only if code changes were made
File P0 issues if builds are broken
3. Sync the issue tracker carefully
Work methodically to ensure local and remote issues merge safely
Handle git conflicts thoughtfully (sometimes accepting remote and re-importing)
Goal: clean reconciliation where no issues are lost
4. Verify clean state
All changes committed and pushed
No untracked files remain
5. Choose next work
Provide a formatted prompt for the next session with context
See the
"Landing the Plane"
section in this project's documentation for a complete example you can adapt. The key insight: explicitly reminding agents to maintain issue tracker hygiene prevents the common problem of agents creating issues during work but forgetting to sync them at session end.
The Magic: Distributed Database via Git
Here's the crazy part:
bd acts like a centralized database, but it's actually distributed via git.
When you install bd on any machine with your project repo, you get:
✅ Full query capabilities (dependencies, ready work, etc.)
✅ Fast local operations (<100ms via SQLite)
✅ Shared state across all machines (via git)
✅ No server, no daemon required, no configuration
✅ AI-assisted merge conflict resolution
How it works:
Each machine has a local SQLite cache (
.beads/*.db
, gitignored). Source of truth is JSONL (
.beads/issues.jsonl
, committed to git). Auto-sync keeps them in sync: SQLite → JSONL after CRUD operations (5-second debounce), JSONL → SQLite when JSONL is newer (e.g., after
git pull
).
The result:
Agents on your laptop, your desktop, and your coworker's machine all query and update what
feels
like a single shared database, but it's really just git doing what git does best - syncing text files across machines.
No PostgreSQL instance. No MySQL server. No hosted service. Just install bd, clone the repo, and you're connected to the "database."
Git Workflow & Auto-Sync
bd automatically syncs your local database with git:
git pull
# bd automatically detects JSONL is newer and imports on next command
bd ready # Fresh data from git!
bd list # Shows issues from other machines
How hash IDs solve this:
Hash IDs are generated from random data, making concurrent creation collision-free:
Agent A creates
bd-a1b2
(hash of random UUID)
Agent B creates
bd-f14c
(different hash, different UUID)
Git merge succeeds cleanly → no collision resolution needed
Birthday Paradox Math
Hash IDs use
birthday paradox probability
to determine length:
Hash Length
Total Space
50% Collision at N Issues
1% Collision at N Issues
4 chars
65,536
~304 issues
~38 issues
5 chars
1,048,576
~1,217 issues
~153 issues
6 chars
16,777,216
~4,869 issues
~612 issues
Our thresholds are conservative:
We switch from 4→5 chars at 500 issues (way before the 1% collision point at ~1,217) and from 5→6 chars at 1,500 issues.
Progressive extension on collision:
If a hash collision does occur, bd automatically extends the hash to 7 or 8 characters instead of remapping to a new ID.
Migration
Existing databases continue to work
- no forced migration. Run
bd migrate
when ready:
# Inspect migration plan (for AI agents)
bd migrate --inspect --json
# Check schema and config state
bd info --schema --json
# Preview migration
bd migrate --dry-run
# Migrate database schema (removes obsolete issue_counters table)
bd migrate
# Show current database info
bd info
AI-supervised migrations:
The
--inspect
flag provides migration plan analysis for AI agents. The system verifies data integrity invariants (required config keys, foreign key constraints, issue counts) before committing migrations.
Note:
Hash IDs require schema version 9+. The
bd migrate
command detects old schemas and upgrades automatically.
Hierarchical Child IDs
Hash IDs support
hierarchical children
for natural work breakdown structures. Child IDs use dot notation:
Note:
Child IDs are automatically assigned sequentially within each parent's namespace. No need to specify parent manually - bd tracks context from git branch/working directory.
Usage
Health Check
Check installation health:
bd doctor
validates your
.beads/
setup, database version, ID format, and CLI version. Provides actionable fixes for any issues found.
-t, --type
- Type (bug|feature|task|epic|chore, default=task)
-a, --assignee
- Assign to user
-l, --labels
- Comma-separated labels
--id
- Explicit issue ID (e.g.,
worker1-100
for ID space partitioning)
--json
- Output in JSON format
See
bd template list
for available templates and
bd help template
for managing custom templates.
Viewing Issues
bd info # Show database path and daemon status
bd show bd-a1b2 # Show full details
bd list # List all issues
bd list --status open # Filter by status
bd list --priority 1 # Filter by priority
bd list --assignee alice # Filter by assignee
bd list --label=backend,urgent # Filter by labels (AND)
bd list --label-any=frontend,backend # Filter by labels (OR)# Advanced filters
bd list --title-contains "auth"# Search title
bd list --desc-contains "implement"# Search description
bd list --notes-contains "TODO"# Search notes
bd list --id bd-123,bd-456 # Specific IDs (comma-separated)# Date range filters (YYYY-MM-DD or RFC3339)
bd list --created-after 2024-01-01 # Created after date
bd list --created-before 2024-12-31 # Created before date
bd list --updated-after 2024-06-01 # Updated after date
bd list --updated-before 2024-12-31 # Updated before date
bd list --closed-after 2024-01-01 # Closed after date
bd list --closed-before 2024-12-31 # Closed before date# Empty/null checks
bd list --empty-description # Issues with no description
bd list --no-assignee # Unassigned issues
bd list --no-labels # Issues with no labels# Priority ranges
bd list --priority-min 0 --priority-max 1 # P0 and P1 only
bd list --priority-min 2 # P2 and below# Combine multiple filters
bd list --status open --priority 1 --label-any urgent,critical --no-assignee
# JSON output for agents
bd info --json
bd list --json
bd show bd-a1b2 --json
Updating Issues
bd update bd-a1b2 --status in_progress
bd update bd-a1b2 --priority 2
bd update bd-a1b2 --assignee bob
bd close bd-a1b2 --reason "Completed"
bd close bd-a1b2 bd-f14c bd-3e7a # Close multiple# JSON output
bd update bd-a1b2 --status in_progress --json
Dependencies
# Add dependency (bd-f14c depends on bd-a1b2)
bd dep add bd-f14c bd-a1b2
bd dep add bd-3e7a bd-a1b2 --type blocks
# Remove dependency
bd dep remove bd-f14c bd-a1b2
# Show dependency tree
bd dep tree bd-f14c
# Detect cycles
bd dep cycles
Dependency Types
blocks
: Hard blocker (default) - issue cannot start until blocker is resolved
related
: Soft relationship - issues are connected but not blocking
parent-child
: Hierarchical relationship (child depends on parent)
discovered-from
: Issue discovered during work on another issue (automatically inherits parent's
source_repo
)
Only
blocks
dependencies affect ready work detection.
Note:
Issues created with
discovered-from
dependencies automatically inherit the parent's
source_repo
field, ensuring discovered work stays in the same repository as the parent task.
Finding Work
# Show ready work (no blockers)
bd ready
bd ready --limit 20
bd ready --priority 1
bd ready --assignee alice
# Sort policies (hybrid is default)
bd ready --sort priority # Strict priority order (P0, P1, P2, P3)
bd ready --sort oldest # Oldest issues first (backlog clearing)
bd ready --sort hybrid # Recent by priority, old by age (default)# Show blocked issues
bd blocked
# Statistics
bd stats
# JSON output for agents
bd ready --json
Labels
Add flexible metadata to issues for filtering and organization:
# Add labels during creation
bd create "Fix auth bug" -t bug -p 1 -l auth,backend,urgent
# Add/remove labels
bd label add bd-a1b2 security
bd label remove bd-a1b2 urgent
# List labels
bd label list bd-a1b2 # Labels on one issue
bd label list-all # All labels with counts# Filter by labels
bd list --label backend,auth # AND: must have ALL labels
bd list --label-any frontend,ui # OR: must have AT LEAST ONE
See
docs/LABELS.md
for complete label documentation and best practices.
Deleting Issues
# Single issue deletion (preview mode)
bd delete bd-a1b2
# Force single deletion
bd delete bd-a1b2 --force
# Batch deletion
bd delete bd-a1b2 bd-f14c bd-3e7a --force
# Delete from file (one ID per line)
bd delete --from-file deletions.txt --force
# Cascade deletion (recursively delete dependents)
bd delete bd-a1b2 --cascade --force
The delete operation removes all dependency links, updates text references to
[deleted:ID]
, and removes the issue from database and JSONL.
Configuration
Manage per-project configuration for external integrations:
# Set configuration
bd config set jira.url "https://company.atlassian.net"
bd config set jira.project "PROJ"# Get configuration
bd config get jira.url
# List all configuration
bd config list --json
# Unset configuration
bd config unset jira.url
Use
--analyze
to export candidates (closed 30+ days) with full content
Summarize the content using any LLM (Claude, GPT, local model, etc.)
Use
--apply
to persist the summary and mark as compacted
This is agentic memory decay - your database naturally forgets fine-grained details while preserving essential context. The agent has full control over compression quality.
Export/Import
# Export to JSONL (automatic by default)
bd export -o issues.jsonl
# Import from JSONL (automatic when JSONL is newer)
bd import -i issues.jsonl
# Handle missing parents during import
bd import -i issues.jsonl --orphan-handling resurrect # Auto-recreate deleted parents
bd import -i issues.jsonl --orphan-handling skip # Skip orphans with warning
bd import -i issues.jsonl --orphan-handling strict # Fail on missing parents# Manual sync
bd sync
Import Orphan Handling:
When importing hierarchical issues (e.g.,
bd-abc.1
,
bd-abc.2
), bd needs to handle cases where the parent (
bd-abc
) has been deleted:
allow
(default)
- Import orphans without validation. Most permissive, ensures no data loss.
resurrect
- Search JSONL history for deleted parents and recreate them as tombstones (Status=Closed, Priority=4). Preserves hierarchy.
skip
- Skip orphaned children with warning. Partial import.
strict
- Fail import if parent is missing.
Configure default behavior:
bd config set import.orphan_handling resurrect
A standalone web interface for real-time issue monitoring is available as an example:
# Build the monitor-webuicd examples/monitor-webui
go build
# Start web UI on localhost:8080
./monitor-webui
# Custom port and host
./monitor-webui -port 3000
./monitor-webui -host 0.0.0.0 -port 8080 # Listen on all interfaces
The monitor provides:
Real-time table view
of all issues with filtering by status and priority
Click-through details
- Click any issue to view full details in a modal
Live updates
- WebSocket connection for real-time changes via daemon RPC
Responsive design
- Mobile-friendly card view on small screens
Statistics dashboard
- Quick overview of issue counts and ready work
Clean UI
- Simple, fast interface styled with milligram.css
The monitor is particularly useful for:
Team visibility
- Share a dashboard view of project status
AI agent supervision
- Watch your coding agent create and update issues in real-time
Quick browsing
- Faster than CLI for exploring issue details
Mobile access
- Check project status from your phone
beads-ui
- Local web interface with live updates, kanban board, and keyboard navigation. Zero-setup launch with
npx beads-ui start
. Built by
@mantoni
.
bdui
- Real-time terminal UI with kanban board, tree view, dependency graph, and statistics dashboard. Vim-style navigation, search/filter, themes, and native notifications. Built by
@assimelha
.
Have you built something cool with bd?
Open an issue
to get it featured here!
Development
# Run tests
go test ./...
# Build
go build -o bd ./cmd/bd
# Run
./bd create "Test issue"# Bump version
./scripts/bump-version.sh 0.9.3 # Update all versions, show diff
./scripts/bump-version.sh 0.9.3 --commit # Update and auto-commit
Putting aside GitHub’s
relationship with ICE
, it’s abundantly clear that the talented folks who used to work on the product have moved on to bigger and better things, with the remaining losers eager to inflict some kind of bloated, buggy JavaScript framework on us in the name of progress. Stuff that used to be snappy is now sluggish and often entirely broken.
More importantly, Actions is
created by monkeys
and
completely neglected
. After the
CEO of GitHub said to “embrace AI or get out”
, it seems the lackeys at Microsoft took the hint, because GitHub Actions started “vibe-scheduling”; choosing jobs to run seemingly at random. Combined with other bugs and inability to manually intervene, this causes our CI system to get so backed up that not even master branch commits get checked.
Rather than wasting donation money on more CI hardware to work around this crumbling infrastructure, we’ve opted to switch Git hosting providers instead.
As a bonus, we look forward to fewer violations (exhibit
A
,
B
,
C
) of our
strict no LLM / no AI policy
, which I believe are at least in part due to GitHub aggressively pushing the “file an issue with Copilot” feature in everyone’s face.
GitHub Sponsors
The only concern we have in leaving GitHub behind has to do with GitHub Sponsors. This product was key to Zig’s early fundraising success, and it
remains a large portion of our revenue today
. I can’t thank
Devon Zuegel
enough. She appeared like an angel from heaven and single-handedly made GitHub into a viable source of income for thousands of developers. Under her leadership, the future of GitHub Sponsors looked bright, but sadly for us, she, too, moved on to bigger and better things. Since she left, that product as well has been neglected and is already starting to decline.
Although GitHub Sponsors is a large fraction of Zig Software Foundation’s donation income,
we consider it a liability
. We humbly ask if you, reader, are currently donating through GitHub Sponsors, that you consider
moving your recurring donation to Every.org
, which is itself a non-profit organization.
As part of this, we are sunsetting the GitHub Sponsors perks. These perks are things like getting your name onto the home page, and getting your name into the release notes, based on how much you donate monthly. We are working with the folks at Every.org so that we can offer the equivalent perks through that platform.
Migration Plan
Effective immediately, I have made
ziglang/zig on GitHub
read-only, and the canonical origin/master branch of the main Zig project repository is
https://codeberg.org/ziglang/zig.git
.
Thank you to the Forgejo contributors who helped us with our issues switching to the platform, as well as the Codeberg folks who worked with us on the migration - in particular
Earl Warren
,
Otto
,
Gusted
, and
Mathieu Fenniak
.
In the end, we opted for a simple strategy, sidestepping GitHub’s aggressive vendor lock-in: leave the existing issues open and unmigrated, but start counting issues at 30000 on Codeberg so that all issue numbers remain unambiguous. Let us please consider the GitHub issues that remain open as metaphorically “copy-on-write”.
Please leave all your existing GitHub issues and pull requests alone
. No need to move your stuff over to Codeberg unless you need to make edits, additional comments, or rebase.
We’re still going to look at the already open pull requests and issues
; don’t worry.
In this modern era of acquisitions, weak antitrust regulations, and platform capitalism leading to extreme concentrations of wealth, non-profits remain a bastion defending what remains of the commons.
Happy hacking,
Andrew
Migrating to Positron, a next-generation data science IDE for Python and R
Since
Positron
was released from beta, we’ve been working hard to create documentation that could help you, whether you are curious about the IDE or interested in switching. We’ve released two migration guides to help you on your journey, which you can find linked below.
Migrating to Positron from VS Code
Positron is a next-generation IDE for data science, built by Posit PBC. It’s built on Code OSS, the open-source core of Visual Studio Code, which means that many of the features and keyboard shortcuts you’re familiar with are already in place.
However, Positron is specifically designed for data work and includes integrated tools that aren’t available in VS Code by default. These include:
A built-in data explorer: This feature gives you a spreadsheet-style view of your dataframes, making it easy to inspect, sort, and filter data.
An interactive console and variables pane: Positron lets you execute code interactively and view the variables and objects in your session, similar to a traditional data science IDE.
AI assistance: Positron Assistant is a powerful AI tool for data science that can generate and refine code, debug issues, and guide you through exploratory data analysis.
We anticipate many RStudio users will be curious about Positron. When building Positron, we strived to create a familiar interface while adding extensibility and new features, as well as native support for multiple languages. Positron is designed for data scientists and analysts who work with both R and Python and want a flexible, modern, and powerful IDE.
Key features for RStudio users include:
Native multi-language support: Positron is a polyglot IDE, designed from the ground up to support both R and Python seamlessly.
Familiar interface: We designed Positron with a layout similar to RStudio, so you’ll feel right at home with the editor, console, and file panes. We also offer an option to use your familiar RStudio keyboard shortcuts.
Extensibility: Because Positron is built on Code OSS, you can use thousands of extensions from the Open VSX marketplace to customize your IDE and workflow.
Also, check out our migration walkthroughs in Positron itself; find them by searching “Welcome: Open Walkthrough” in the Command Palette (hit the shortcut Cmd + Shift + P to open the Command Palette), or on the Welcome page when you open Positron:
What’s next
We’re committed to making your transition as smooth as possible, and we’ll be continuing to add to these migration guides. Look out for guides for Jupyter users and more!
We’d love to hear from you. What other guides would you like to see? What features would make your transition easier? Join the conversation in our
GitHub Discussions
.
I was at QCon SF during the recent Cloudflare outage (I was hosting the
Stories Behind the Incidents track
), so I hadn’t had a real chance to sit down and do a proper read-through of their
public writeup
and capture my thoughts until now. As always, I recommend you read through the writeup first before you read my take.
All quotes are from the writeup unless indicated otherwise.
Hello
saturation
my old friend
The software had a limit on the size of the feature file that was below its doubled size. That caused the software to fail.
One thing I hope readers take away from this blog post is the complex systems failure mode pattern that resilience engineering researchers call
saturation
. Every complex system out there has limits, no matter how robust that system is. And the systems we deal with have
many, many different kinds of limits
, some of which you might only learn about once you’ve breached that limit. How well a system is able to perform as it approaches one of its limits is what resilience engineering is all about.
Each module running on our proxy service has a number of limits in place to avoid unbounded memory consumption and to preallocate memory as a performance optimization. In this specific instance, the Bot Management system has a limit on the number of machine learning features that can be used at runtime. Currently that limit is set to 200, well above our current use of ~60 features.
In this particular case, the limit was set explicitly.
thread fl2_worker_thread panicked: called Result::unwrap() on an Err value
As sparse as the panic message is, it does explicitly tell you that the problematic call site was an unwrap call. And this is one of the reasons I’m a fan of explicit limits over implicit limits: you tend to get better error messages than when breaching an implicit limit (e.g., of your language runtime, the operating system, the hardware).
A subsystem designed to protect surprisingly inflicts harm
Identify and mitigate automated traffic to protect your domain from bad bots. –
Cloudflare Docs
The problematic behavior was in the
Cloudflare Bot Management
system. Specifically, it was in the
bot scoring
functionality, which estimates the likelihood that a request came from a bot rather than a human.
This is a system that is
designed to help protect their customer from malicious bots
, and yet it ended up hurting their customers in this case rather than helping them.
As I’ve
mentioned previously
, once your system achieves a certain level of reliability, it’s the protective subsystems that end up being things that bite you! These subsystems are a net positive, they help much more than they hurt. But they also add complexity, and complexity introduces new, confusing failure modes into the system.
The Cloudflare case is a more interesting one than the typical instances of this behavior I’ve seen, because Cloudflare’s whole business model is to offer different kinds of protection, as products for their customers. It’s protection-as-a-service, not an internal system for self-protection. But even though their customers are purchasing this from a vendor rather than building it in-house, it’s still an auxiliary system intended to improve reliability and security.
Confusion in the moment
What impressed me the most about this writeup is that they documented some aspects of
what it was like responding to this incident
: what they were seeing, and how they tried to made sense of it.
In the internal incident chat room, we were concerned that this might be the continuation of the recent spate of high volume
Aisuru
DDoS attacks
:
Man, if I had a nickel every time I saw someone Slack “Is it DDOS?” in response to a surprising surge of errors returned by the system, I could probably retire at this point.
The spike, and subsequent fluctuations, show our system failing due to loading the incorrect feature file. What’s notable is that our system would then recover for a period. This was very unusual behavior for an internal error.
We humans are excellent at recognizing patterns based on our experience, and that generally serves us well during incidents. Someone who is really good at operations can frequently diagnose the problem very quickly just by, say, the shape of a particular graph on a dashboard, or by seeing a specific symptom and recalling similar failures that happened recently.
However, sometimes we encounter a failure mode that we haven’t seen before, which means that we don’t recognize the signals. Or we might have seen a cluster of problems recently that followed a certain pattern, and assume that the latest one looks like the last one. And these are the hard ones.
This fluctuation made it unclear what was happening as the entire system would recover and then fail again as sometimes good, sometimes bad configuration files were distributed to our network. Initially, this led us to believe this might be caused by an attack.
This incident was one of those hard ones: the symptoms were confusing. The “problem went away, then came back, then went away again, then came back again” type of unstable incident behavior is generally much harder to diagnose than one where the symptoms are stable.
Throwing us off and making us believe this might have been an attack was another apparent symptom we observed: Cloudflare’s status page went down. The status page is hosted completely off Cloudflare’s infrastructure with no dependencies on Cloudflare. While it turned out to be a coincidence, it led some of the team diagnosing the issue to believe that an attacker may be targeting both our systems as well as our status page.
Here they got bit by a co-incident, an unrelated failure of their status page that led them to believe (reasonably!) that the problem must have been external.
I’m still curious as to what happened with their status page. The error message they were getting mentions
CloudFront
, so I assume they were hosting their status page on AWS. But their writeup doesn’t go into any additional detail on what the status page failure mode was.
But the general takeaway here is that even the most experienced operators are going to take longer to deal with a complex, novel failure mode, precisely because it is complex and novel! As the resilience engineering folks say,
prepare to be surprised!
(Because I promise, it’s going to happen).
A plea: assume local rationality
The writeup included a screenshot of the code that had an unhandled error. Unfortunately, there’s nothing in the writeup that tells us what the programmer was thinking when they wrote that code.
In the absence of any additional information, a natural human reaction is to just assume that the programmer was sloppy. But if you want to actually understand how these sorts of incidents actually happen, you have to fight this reaction.
People always make decisions that make sense to them in the moment, based on what they know and what constraints they are operating under. After all, if that wasn’t true, then they wouldn’t have made that decision. The only we can actually understand the conditions that enable incidents, we need to try as hard as we can to put ourselves into the shoes of the person who made that call, to understand what their frame of mind was at the moment.
If we don’t do that, we risk the problem of
distancing through differencing
. We say, “oh, those devs were bozos, I would never have made that kind of mistake”. This is a great way to limit how much you can learn from an incident.
Detailed public writeups as evidence of good engineering
The writeup produced by Cloudflare (signed by the CEO, no less!) was impressively detailed. It even includes a screenshot of a snippet of code that contributed to the incident! I can’t recall ever reading another public writeup with that level of detail.
Companies generally err on the side of saying less rather than more. After all, if you provide more detail, you open yourself up to criticism that the failure was due to poor engineering. The fewer details you provide, the fewer things people can call you out on. It’s not hard to find people online criticizing Cloudflare online using the details they provided as the basis for their criticism.
Now, I think it would advance our industry if people held the opposite view: the more details that are provided an incident writeup, the
higher esteem we should hold that organization
. I respect Cloudflare is an engineering organization a lot more precisely because they are willing to provide these sorts of details. I don’t want to hear what Cloudflare
should have done
from people who weren’t there, I want to hear us hold other companies up to Cloudflare’s standard for describing the details of a failure mode and the inherently confusing nature of incident response.
Pocketbase – open-source realtime back end in 1 file
Cheap Chinese battery electric heavy trucks are no longer a rumor. They are real machines with real price tags that are so low that they force a reassessment of what the global freight industry is willing to pay for electrification. Standing in a commercial vehicle hall in Wuhan and seeing a 400 kWh or 600 kWh truck priced between €58,000 and €85,000, as my European freight trucking electrification contact
Johnny Nijenhuis recently did
, changes the frame of the entire conversation. These are not diesel frames with a battery box welded underneath. They are purpose built electric trucks built around LFP packs, integrated e-axles and the simplified chassis architecture that becomes possible when the engine bay, gearbox, diesel tank, emissions controls and half of the mechanical complexity of a truck disappear. Anyone who has worked with heavy vehicles knows the cost structure of diesel powertrains. Removing that entire system while building at very large scale produces numbers that do not match Western experience.
China’s low price electric trucks do not arrive as finished products for Europe or North America. They need work. Western short haul freight fleets expect certain features that Chinese domestic buyers usually skip. Tires need to carry E-mark or FMVSS certification. Electronic stability controls must meet UNECE R13 or FMVSS 121. Cab structures need to meet R29 or similar requirements. Crash protection for battery packs needs to satisfy R100 or FMVSS 305. European drivers expect better seats, quieter cabs and stronger HVAC. Even in short haul work, fleets expect well understood advanced driver assistance (ADAS) features to handle traffic and depot work. However, inexpensive Chinese leaf springs are just fine for short haul trucking given the serious upgrade to driver comfort and truck performance of battery electric drivetrains.
When these adjustments are added into the bill of materials and spread across a production run, the upgrades land in the €20,000 to €40,000 range for short haul duty, per my rough estimate. That moves the landed price up to roughly €80,000 to €120,000. The comparison with Western OEM offerings is stark because Western battery electric trucks today often start near €250,000 and can move far higher once options and charging hardware are included. A short haul operator looking at the difference between a €100,000 truck and a €300,000 truck will ask which one meets the actual duty cycle. For operators with depot charging and predictable delivery routes, the cheaper truck is credible in a way that few expected even three years ago.
The long haul story is different. European and North American long haul operators require far more from a truck than a Chinese domestic short range tractor offers. Axle loads need to support 40 to 44 ton gross combined weight. Suspension needs to manage high speed stability for many hours a day on roads built for 80 to 100 km/h cruising. Cab structures must handle fatigue and cross winds on long corridors. Drivers spend nights sleeping in the cab and expect western comfort standards. Trailer interfaces require specific electrical and pneumatic systems that have to meet long established norms. Battery safety systems need to be built for high speed impacts and rollover events. All of that requires a larger budget. The gap between a domestic Chinese tractor and a European or North American long haul tractor is roughly €80,000 to €120,000 once all mechanical, safety and comfort systems are brought to the required levels per my estimate. That does not erase the cost advantage, because even a €180,000 Chinese based long haul electric truck is cheaper than many Western models, but it does shift the choice from simple purchase price to service expectations and lifetime durability.
Most freight is not long haul.
French and German economic councils
have both looked at freight movements through national data and concluded that the majority of truck trips and ton kilometers occur in short haul service. This includes urban deliveries, regional distribution, logistics shuttles between depots and ports, construction supply and waste collection. These trips are usually under 250 km, begin and end at the same depot and involve repeated stop-start movement where electric drivetrains perform well. The idea that the heavy trucking problem is a long haul problem has shaped Western investment priorities for a decade, but national economic councils in Europe now argue that solving short haul electrification first delivers most of the benefit. The fact that low cost Chinese battery electric trucks map almost perfectly onto these duty cycles suggests that they will find receptive markets once import pathways are established.
Chart of heavy truck sales in China assembled by author
China’s shift away from diesel in the heavy truck segment is dramatic. The country sold more than 900,000 heavy trucks in 2024. Diesel’s share fell to about 57% that year. Natural gas trucks rose to around 29%. Battery electric trucks reached 13%. Early 2025 data points to battery electric share rising again to about 22% of new heavy truck sales, with diesel falling close to the 50% mark. These shifts are large movements inside a very conservative sector. Natural gas trucks saw a rapid rise between 2022 and 2024 as operators chased lower fuel prices and simpler emissions compliance, but the price war in battery electric trucks has made electric freight attractive for many of the same operators. Gas trucks still fill some niches, but the pattern suggests that they may face the same pressure that diesel trucks face. Electric trucks with low running costs and high cycle life begin to look compelling to operators once the purchase price falls into a familiar range.
Western OEMs entered China with hopes of capturing a share of the largest truck market in the world, but the results have been mixed. Joint ventures like Foton Daimler once offered a bridge into domestic heavy trucking, yet the rapid rise of low cost local manufacturers in both diesel and electric segments has eroded that position. Western models arrived with higher prices and platforms optimized for different regulations and freight conditions. As domestic OEMs expanded capacity and cut costs, the market shifted toward local brands in every drivetrain category. The impact is clear. Western firms now face reduced market share, weaker margins and strategic uncertainty about long term participation in China’s truck sector.
Underlying these drivetrain transitions is a heavy truck market that is smaller and more complicated than it was five years ago. The peak in 2020, with roughly 1.6 million heavy trucks sold, was not a normal year. It was driven by a large regulatory pre-buy that pulled forward sales before tighter emissions rules arrived. The freight economy was also stronger at that time and the construction sector had not yet entered its recent slowdown. As those drivers faded, the market returned to what looks like a long term equilibrium between 800,000 and one million trucks per year. Several confounding factors overlap in this period. Freight volumes shifted. Rail took a larger share of bulk transport as China achieved what North America and Europe have only talked about, mode shifting. Replacement cycles grew longer. Real estate and construction slowed. Diesel’s loss of share is partly driven by these economic factors and partly driven by the arrival of cheaper alternatives. It is difficult to separate the exact contribution of each. The net result is a natural market size that is much lower than the 2020 peak and a much more competitive fight inside the remaining market.
Hydrogen heavy truck sales in China show a pattern of stalling growth followed by early signs of decline in 2025. Registration data and industry reports indicate that fuel cell heavy trucks were less than 1% of the heavy truck market in 2024, amounting to low single digit thousands of vehicles, and most of these were tied to provincial demonstration subsidies rather than broad fleet adoption. In the first half of 2025 the number of registered hydrogen trucks rose slightly on paper, but analysts inside China noted that real world operation rates were low and that several local programs were winding down as subsidies tightened. At the same time battery electric heavy trucks climbed from 13% of new sales in 2024 to 22% in early 2025. Hydrogen heavy trucks are losing ground inside a market that is moving quickly toward lower cost electric models, and operators are stepping away from fuel cell platforms as more credible electric options appear. I didn’t bother to include hydrogen on the truck statistics chart as it’s a rounding error and not increasing.
One indicator that connects these pieces is diesel consumption. China’s diesel use dropped by about 11% year over year at one point in 2024, which is not a small shift in a country with heavy commercial transport. Part of the drop was due to economic slowing in trucking dominant sectors, but the rise of LNG trucks and electric trucks also contributed. When a truck that once burned diesel every day is replaced by a gas or battery electric truck, national fuel consumption reacts quickly. The fuel market sees these changes earlier than the headline truck sales numbers because thousands of trucks operating every day create a measurable signal in fuel demand. The data is consistent with a freight system that is changing in composition and technology at a pace that would have seemed unlikely a few years earlier.
Western operators have to look at this landscape with practical questions in mind. The leading electric bus manufacturer in Europe is Chinese because it built functional electric buses at lower prices before Western firms did. There is no reason the same pattern will not repeat in trucks. Once the cost of a short haul electric truck falls near the cost of a diesel truck, operators will start to buy them. If the imported option is much cheaper than the domestic option, early fleets will run the numbers and make decisions based on cash flow and reliability. Western OEMs face challenges in this environment because their legacy designs and cost structures are not tuned for the kind of price war that emerged in China. They need to match cost while preserving safety and service expectations, which is difficult while shifting from a century of diesel design to a new electric architecture.
Western OEMs entered the electric truck market with the platforms they already understood. Most began by taking a diesel tractor frame, removing the engine and gearbox and adding batteries, motors and the associated power electronics. This approach kept production lines moving and reduced near term engineering risk, but it produced electric trucks that carried the compromises of diesel architecture. Battery boxes hung from ladder frames, wiring loops wound through spaces never designed for high voltage systems and weight distribution was optimized for a drivetrain that no longer existed. Several OEMs even explored hydrogen drivetrains inside the same basic frames, which locked in the limitations of a platform built around an internal combustion engine. The results were heavier trucks with less space for batteries, higher costs and lower overall efficiency.
The shift toward purpose built electric tractors is only now underway among the major Western OEMs. Volvo’s FH Electric and FM Electric, Daimler’s eActros 300 and 600, Scania’s new battery electric regional tractor and MAN’s eTruck all represent clean sheet or near clean sheet electric designs with integrated drivetrains and optimized battery packaging. These models move Western OEMs closer to the design philosophy that Chinese manufacturers adopted earlier, where the entire platform is built around the electric driveline from the start.
China has moved faster toward battery electric heavy trucks than any other major market. It built supply chains for motors, inverters, LFP cells, structural packs and integrated e-axles. It created standard designs and cut costs through volume. It encouraged competition. It is now exporting electric trucks into Asia, Latin America and Africa. Europe and North America are watching this unfold while debating the right charging standards and duty cycle models. The arrival of low cost electric trucks from China raises uncomfortable questions for Western OEMs and policymakers, but it also provides an opportunity. If freight electrification can happen at one third the expected cost, then the pace of decarbonization can be much faster. The challenge is deciding how to integrate or respond to the cost structure that China has already built.
The story of heavy trucking is no longer a slow migration from diesel to a distant alternative. The transition is already underway at scale inside the world’s largest heavy truck market. It does not look like the long haul hydrogen scenario that dominated Western modelling for the last decade. It looks like battery electric trucks built cheaply and deployed quickly into short haul service. The economic logic is straightforward. The operational fit is strong. The supply chain is built. The lesson for Western operators and policymakers is that the cost curve has shifted. The decisions that made sense even in 2024 do not match the realities of 2025. The market is moving toward electric freight because it is becoming cheaper than diesel across the majority of real world duty cycles. From the short haul electric trucks will come the new generation of long haul trucks, as night follows day. The arrival of low cost battery trucks from China marks the beginning of a new phase in freight decarbonization.
1. Introduction: The Algorithm That Broke the Future
Peter Shor writes a few pages of math.
The entire world’s public-key cryptography dies on those pages.
RSA. Diffie-Hellman. Elliptic Curve crypto. All of it collapses the moment someone runs Shor’s algorithm on a big enough quantum machine.
This isn’t Grover’s “square-root speedup” that just makes brute force a bit faster.
This is exponential turned into polynomial. Game over.
Governments already record encrypted traffic today betting on Shor tomorrow.
Your cloud backups, your TLS sessions, your Bitcoin private keys. All harvestable now, decryptable later.
Shor’s algorithm is not a future threat. It is a retroactive one.
2. The Core Problem: Factoring and Discrete Logs
Today’s internet runs on two “hard” problems:
Factoring: given a giant number N = p × q, find p and q.
→ RSA lives here.
Discrete logarithm: given g^x mod p (or a point multiplication on an elliptic curve), find x.
→ DH, ECDH, ECDSA live here.
Classical computers suck at both. Best attacks are sub-exponential and painfully slow for 2048-bit numbers.
Quantum computers with Shor don’t care.
Theorem
(The RSA Problem)
RSA Security Assumption:
Given a modulus
where
and
are large prime numbers, and given the public exponent
and ciphertext
, it is computationally infeasible for a classical computer to recover the plaintext
from
without knowing the private exponent
.
Mathematical Statement:
The RSA problem reduces to factoring: if you can factor
, you can compute
and derive
.
Theorem
(The Discrete Logarithm Problem)
Discrete Logarithm Assumption:
Given a generator
of a finite group
of prime order
, and given
, it is computationally infeasible to find the exponent
.
Mathematical Statement:
For
, find
such that
.
Elliptic Curve Variant:
Given points
and
on an elliptic curve, find the scalar
.
3. The Quantum Trick: Period Finding
Classical computers try divisors one by one. Stupid.
Shor’s genius: turn factoring into a period-finding problem.
Pick random a, compute a to the power k mod N in superposition across all k at once.
Quantum Fourier Transform spits out the period r of the sequence.
If r is even, gcd(a to the power (r/2) minus 1, N) gives you the factors with high probability.
That’s it. Polynomial time. Done.
Same trick murders discrete logs on finite fields and elliptic curves.
Theorem
(Shor's Period-Finding Algorithm)
Shor’s Algorithm:
Given a periodic function
, Shor’s algorithm finds the period
of
in
time using a quantum computer.
Key Steps:
Initialize superposition:
Apply modular exponentiation:
Apply Quantum Fourier Transform to first register
Measure to obtain approximation of
for some integer
Use continued fractions to extract
Complexity:
vs. classical
Lemma
(Period Finding for Factoring)
If
has period
, and
is even, then:
will be a non-trivial factor of
with high probability.
Proof
(Proof Sketch)
If
is the order of
modulo
, and
is even, then
or
. In the first case,
shares a common factor with
. In the second case, we need to try different
.
Example
(Shor's Algorithm Implementation)
Classical Simulation of Shor’s Algorithm:
Here’s a simplified classical implementation that demonstrates the period-finding step:
import math
import random
defshors_period_finding(a, N):
"""
Classical simulation of Shor's period finding
(In quantum computers, this would be done with QFT)
"""
# For demonstration, we'll use a simple approach
# Real quantum Shor's uses quantum Fourier transform
# Find period r such that a^r ≡ 1 mod N
r =1
whilepow(a, r, N) !=1:
r +=1
if r > N: # No period found
returnNone
return r
defshors_factor(N):
"""
Shor's algorithm for factoring
"""
if N %2==0:
return2
whileTrue:
# Pick random a < N
a = random.randint(2, N-1)
# Check if a and N are coprime
if math.gcd(a, N) !=1:
return math.gcd(a, N)
# Find period r
r = shors_period_finding(a, N)
if r isNoneor r %2!=0:
continue
# Compute a^(r/2) mod N
x =pow(a, r//2, N)
if x == N -1: # Trivial case
continue
# Compute factors
p = math.gcd(x -1, N)
q = math.gcd(x +1, N)
if p * q == N and p !=1and q !=1:
return p, q
# Example: Factor 77 = 7 × 11
N =77
factors = shors_factor(N)
print(f"Factors of {N}: {factors}")
Note:
This is a classical simulation. True Shor’s algorithm requires quantum computers for the period-finding step, which runs in
time vs. classical factoring’s exponential time.
4. Breaking RSA Step by Step
Here’s exactly how RSA-2048 dies:
The Complete RSA Breaking Process:
Public Key Acquisition:
Attacker obtains your RSA public key
Random Base Selection:
Choose random
where
(coprime to
)
Quantum Period Finding:
Run Shor’s algorithm to find the period
of
Initialize superposition:
Apply modular exponentiation:
Apply Quantum Fourier Transform (QFT) to first register
Measure to obtain approximation of
for some integer
Use continued fractions to extract period
Factor Recovery:
If
is even, compute:
and
One of these will be a non-trivial factor
or
of
Private Key Derivation:
Once factors are known, compute
Private exponent:
Full private key:
Decryption:
Use private key to decrypt all intercepted ciphertext
Time Complexity:
vs. classical factoring’s
Result:
RSA-2048 broken in minutes on a decent quantum computer. Centuries become minutes.
Example
(RSA Breaking Example)
Small RSA Example:
Let’s break RSA-8 (for illustration - real RSA uses 2048+ bits).
Public key:
,
Private key:
(computed from
)
Shor’s Algorithm Attack:
Pick
(coprime to 77)
Compute period of
Find
(the period)
Since
is even, compute
So
, and
Compute
(factor found!)
Compute
(factor found!)
Success:
Result:
RSA broken in polynomial time!
# Classical RSA implementation (vulnerable to Shor's algorithm)
People think “ECC is smaller, safer.” Wrong.
Shor solves the elliptic curve discrete log problem in basically the same runtime.
A 256-bit ECC key falls to roughly the same size quantum computer as RSA-2048.
Smaller keys = fewer qubits needed = ECC breaks first.
Exercise
(ECC Discrete Logarithm)
Consider the elliptic curve
over a finite field, with generator point
and public key
. Use Shor’s algorithm to find the private key
.
Solution
(Solution)
Shor’s algorithm for elliptic curve discrete logarithms works by finding the period of the function
where
is the order of the curve. The quantum algorithm finds
such that
in
time, same as for factoring.
Key insight:
Elliptic curve scalar multiplication
is periodic with period equal to the order of
. Shor’s algorithm finds this period, revealing the discrete logarithm.
6. Harvest Now, Decrypt Later
Intelligence agencies are already doing it.
Every TLS handshake, every VPN session, every encrypted email crossing international fiber is archived.
When the quantum day comes, they press “run Shor” and read twenty years of secrets.
Your medical records uploaded in 2025? Readable in 2035.
Your 2017 Bitcoin transactions? Stealable the day ECDSA breaks.
The threat is here today.
Problem
(Quantum Harvesting Scale)
A nation-state adversary captures 1TB of encrypted TLS traffic per day. If quantum computers become available in 2030, how much historical data becomes decryptable?
Solution
(Back-of-Envelope Calculation)
Annual encrypted data captured:
1TB/day × 365 days = 365TB/year
From 2010-2030:
20 years × 365TB = 7,300TB ≈ 7.3 petabytes
Decryptable content:
All RSA/ECC-based encryption from that period becomes readable instantly upon quantum computer availability.
Impact:
Every HTTPS session, every VPN connection, every encrypted email, every financial transaction using classical crypto becomes retroactively exposed.
7. How Many Qubits Do We Actually Need?
Real numbers (2025 estimates with full error correction):
RSA-2048: ~4096 logical qubits, 20-30 million physical qubits
256-bit ECC: ~2330 logical qubits, 12-15 million physical qubits
RSA-3072: ~6000 logical qubits, 30-40 million physical qubits
We’re not there yet. IBM, Google, PsiQuantum, and nation-states are racing.
The first useful cryptographically relevant quantum machine will be built in secret.
You will not get a press release.
8. “Just Use Bigger Keys” Is Dead
Doubling RSA to 8192-bit buys you nothing.
Shor’s runtime is O((log N)^3). Key size barely moves the needle.
Same for ECC. There is no “quantum-resistant curve size.” The whole family is dead.
9. What Actually Survives: Post-Quantum Crypto
Only new math works:
Lattices (Kyber, Dilithium)
Codes (Classic McEliece)
Multivariate (Rainbow – dead now)
Hash-based signatures (SPHINCS+)
Isogenies (SIKE – also dead)
NIST standardized round: Kyber (KEM), Dilithium + Falcon (signatures), SPHINCS+ (backup).
No efficient quantum algorithm known against hard lattice problems. Yet.
Theorem
(Learning With Errors (LWE) Problem)
LWE Hardness Assumption:
Given many samples
where
is a secret vector,
are small noise terms, and
are random, it is computationally infeasible to recover
.
Quantum Resistance:
No efficient quantum algorithm is known for solving LWE. The best known attacks are classical lattice reduction algorithms.
Applications:
Kyber (key exchange), Dilithium (signatures) are based on LWE or its ring variant RLWE.
Example
(Post-Quantum Security Levels)
NIST Security Categories:
Category 1:
Protects against attacks with computational resources comparable to AES-128
Category 3:
Protects against attacks with computational resources comparable to AES-192
Category 5:
Protects against attacks with computational resources comparable to AES-256
Example Algorithms:
Kyber-512: Category 1 (AES-128 equivalent)
Kyber-768: Category 3 (AES-192 equivalent)
Kyber-1024: Category 5 (AES-256 equivalent)
Example
(Kyber Key Exchange Example)
Post-Quantum Key Exchange with Kyber:
Kyber uses lattice-based cryptography that’s believed to be quantum-resistant. Here’s how it works:
# Install: pip install mlkem
from mlkem.ml_kem importML_KEM
from mlkem.parameter_set importML_KEM_768
# Pure Python mode (works everywhere, no compilation needed)
kem = ML_KEM(parameters=ML_KEM_768, fast=False)
# Expected sizes according to NIST FIPS 203 Table 1
print(f"Public key size : 1184 bytes")
print(f"Private key size : 2400 bytes")
print(f"Ciphertext size : 1088 bytes")
print()
# Alice generates keypair
pk, sk = kem.key_gen()
print(f"Generated public key : {len(pk)} bytes")
print(f"Generated private key : {len(sk)} bytes")
# Bob encapsulates → gets shared secret + ciphertext
ss_bob, ct = kem.encaps(pk)
print(f"Ciphertext size : {len(ct)} bytes")
# Alice decapsulates
ss_alice = kem.decaps(sk, ct)
print(f"Shared secrets match : {ss_alice == ss_bob}")
print(f"Shared secret (hex) : {ss_alice.hex()}")
Key Differences from RSA:
No factoring:
Based on lattice problems, not integer factorization
Key sizes:
Kyber-768 public key is ~1184 bytes (vs RSA-2048’s 256 bytes)
Performance:
Encapsulation/decapsulation is fast (~10-100μs)
Quantum resistance:
No known quantum algorithm breaks lattice problems efficiently
Hybrid Approach (Recommended Today):
# Use both classical and post-quantum crypto
from cryptography.hazmat.primitives.asymmetric import x25519
from cryptography.hazmat.primitives import hashes
from cryptography.hazmat.primitives.kdf.hkdf importHKDF
Replacing it all before the quantum bomb goes off is a multi-decade nightmare.
Most companies still think “we’ll upgrade when quantum arrives.”
That’s like waiting for the asteroid to hit the atmosphere before building the bunker.
11. Blockchain Gets Rekt
ECDSA dies → any Bitcoin or Ethereum address that ever sent coins (public key exposed) can be drained instantly.
Billions of dollars sitting in p2pk and reused p2pkh addresses become free for all.
Quantum-resistant chains (QRL, some layer-2 ideas) exist but almost nobody uses them.
The great crypto migration will be biblical.
12. What You Must Do Right Now
Actionable list. No excuses.
Never create new systems with plain RSA or ECC for long-term secrets.
Use hybrid cryptography today: Kyber + X25519, Dilithium + ED25519.
Rotate everything that lasts >10 years to pure PQC now.
Stop exposing raw ECC public keys on blockchains.
Re-encrypt old archives with Kyber if they must stay secret past 2035.
Pressure every vendor: Cloudflare (they’re already PQ-safe), AWS, Apple, banks. Ask “when is your PQC rollout?”
Run post-quantum TLS in production today (Cloudflare and Google already support it).
Waiting is not an option.
Exercise
(Quantum Migration Planning)
You’re responsible for a company’s cryptographic infrastructure. Create a 5-year migration plan from RSA/ECC to post-quantum cryptography.
Classical public-key cryptography is already dead.
We just haven’t held the funeral yet.
The encryption you deploy in 2025 will be broken retroactively.
The only question is whether you care enough to act before the quantum hammer falls.
Shor didn’t break cryptography.
He proved we were never safe to begin with.
“We show that these problems [discrete logarithms and the problem of factoring integers] can be solved in polynomial time on a quantum computer… These problems are generally thought to be hard on a classical computer and have been used as the basis of several proposed cryptosystems.”
—
Peter Shor, “Algorithms for quantum computation: discrete logarithms and factoring,” 1994.
“My guess has been that we’ll see a practical quantum computer within 30 to 40 years… Even so, we should all follow the NSA’s lead and transition our own systems to quantum-resistant algorithms over the next decade or so.”
—
Bruce Schneier, “NSA Plans for a Post-Quantum World,” 2015.
“For those partners and vendors that have not yet made the transition to Suite B elliptic curve algorithms, we recommend not making a significant expenditure to do so at this point but instead to prepare for the upcoming quantum resistant algorithm transition.”
—
NSA/CNSS Advisory Memorandum 02-15, August 2015
Building something that needs to survive the quantum era?
Ellipticc Drive already runs pure post-quantum Kyber + Dilithium everywhere. No legacy baggage.
After the Online Safety Act’s onerous internet age restrictions took effect this summer, it didn’t take long for Brits to get around them. Some methods went viral, like
using video game
Death Stranding
’s photo mode to bypass face scans
. But in the end, the simplest solution won out: VPNs.
Virtual private networks have proven
remarkably effective
at circumventing the UK’s age checks, letting users spoof IP addresses from other countries so that the checks never appear in the first place. The BBC
reported
a few days after the law came into effect that five of the top 10 free apps on the iOS App Store were VPNs.
WindscribeVPN shared data
showing a spike in its user figures, NordVPN
claimed
a 1,000 percent increase in purchases that weekend, and ProtonVPN reported an even higher 1,800 percent increase in UK signups over the same period.
This has not gone unnoticed in the halls of power. Murmurings have begun that something needs to be done, that the UK’s flagship child safety law has been made a mockery, and that VPNs are the problem.
The OSA
became UK law in 2023
, but it took until July for its most significant measures to take effect. It requires websites and online service providers to implement “strong age checks” to prevent under-18s from accessing a broad swathe of “harmful materials,” mostly meaning pornography and content promoting suicide or self-harm. In practice, it means everything from porn sites to Bluesky now require UK users to pass age checks, usually through credit card verification or facial scans, to get full access. You can see why so many of us signed up for VPNs.
Children’s Commissioner Rachel de Souza, a figure appointed by the government to represent children’s interests,
told the BBC
in August that access to VPNs was “absolutely a loophole that needs closing.” Her office
published a report
calling for the software to be gated behind the same “highly effective age assurance” that people are using them to avoid.
“Nothing is off the table.”
De Souza isn’t alone. The government has
faced calls
in the House of Lords to ask why VPNs weren’t taken into account in the first place, while a
proposed amendment
to the Children’s Wellbeing and Schools Bill would institute de Souza’s age-gating requirement. Even as far back as 2022, long before the Labour Party came into power, Labour MP Sarah Champion
predicted
that VPNs would “undermine the effectiveness” of the OSA, and called for the then-government to “find solutions.”
A recent article by
Techradar
added to speculation that the government is considering action, reporting that Ofcom, the UK’s media regulator and enforcer of the OSA, is “monitoring VPN use” in the wake of the act.
Techradar
couldn’t confirm exactly what form that monitoring takes, though Ofcom insisted fears that individual usage is being tracked are unfounded. An anonymous spokesperson for Ofcom would only confirm to the site that it uses “a leading third-party provider,” and that the data is aggregated, with “no personally identifiable or user-level information.” (Anonymized data
often isn’t
, but of course, we don’t know whether that’s the case here.)
Still, that research might be an important piece of the puzzle. While VPN use has clearly increased in the country since July, it’s less certain how much of that is coming from kids, and how much from adults reluctant to hand over biometric or financial data to log into Discord. Ofcom is researching children’s VPN use, but that work will take time.
The government has always insisted that it isn’t banning VPNs, and so far that hasn’t changed. “There are no current plans to ban the use of VPNs, as there are legitimate reasons for using them,” Baroness Lloyd of Effra, a minister in the Department for Science, Innovation and Technology,
told the House of Lords
last month. Then again, she shortly added that “nothing is off the table,” leaving the specter of VPN restrictions still at large.
“It’s very hard to stop people from using VPNs.”
A full ban, such as by requiring internet service providers to block VPN traffic at the source, would be unlikely in any case. There’s no serious political outcry for one, and as the government itself admits, there are plenty of good reasons to use a VPN that have nothing to do with age restrictions on porn.
“VPNs serve many purposes,” Ryan Polk, director of policy at the Internet Society, told me. “Businesses use them to enable secure employee logins; journalists rely on them to protect sources; members of marginalized communities use them to ensure private communication; everyday users benefit from online privacy and security; and even gamers use them to improve performance and reduce latency.”
Besides, everyone I’ve asked about it agrees that banning VPNs would be an uphill battle. “Blocking VPN usage is technically complex and largely ineffective,” Laura Tyrylyte, Nord Security’s head of public relations, told me. James Baker, platform power and free expression program manager at the Open Rights Group, put it even more simply: “It’s very hard to stop people from using VPNs.”
Some have suggested that the government could require sites covered by the OSA restrictions to block all traffic from VPNs, just as many streaming services already do. That brings its own complications though.
“Websites that offer the content would face an impossible choice,” says Polk, because there’s no reliable way to tell if a VPN user is originally from the UK or somewhere else. “They would either have to block all users from the UK (abandoning the market) or block all VPN users from accessing their website.”
That leaves age-restricting VPNs themselves as the likeliest outcome. The OSA already prohibits online platforms from promoting VPNs to children as a way of circumventing age checks, so extending the act to encompass VPNs themselves might not be too much of a stretch. Technically speaking, this would be the easiest option to implement, but it still comes with downsides.
Both Tyrylyte and Baker warn that any attempt to limit VPN usage would push people toward riskier behavior, whether that be less reputable VPNs with bad privacy practices, or simpler forms of direct file-sharing, like USB sticks, that introduce new security risks. In a sense, that’s happened already — both point out that Nord and other paid VPNs require a credit card, meaning underaged users are likely flocking to free options, which Baker calls a privacy risk, “as they are likely just selling your personal data.”
The UK was one of the first countries to implement online age restrictions, but just as other countries and states have followed in its footsteps there, we can expect more governments to put VPNs under scrutiny before long. Australia has banned social media for under-16s, the EU is trialling its own restrictions, and various US states have implemented age limits on the internet. As long as VPNs remain the most effective workaround, VPN restrictions will be a point of debate. In the US, they already are. Republicans in Michigan have
proposed an ISP-level ban on VPNs
, while Wisconsin lawmakers are
debating a proposal
to require adult sites to block VPN traffic entirely.
Wherever you live, the VPN panic is only getting started.
Follow topics and authors
from this story to see more like this in your personalized homepage feed and to receive email updates.
I appreciate that someone else understands that being a GUI has some basic requirements and “draws to the screen” is not the interesting one. The bar is about 20cm off the floor but everyone forgets to jump.
Despite mostly not working on GUIs in my career, I have strong opinions about consistency and about affordances for beginners and power users alike. So today, let’s take a simple case study: a button.
No one
really
agrees these days on what a button should look like, but we can figure that out later. For now, we can take an icon and draw a border around it and that probably counts:
A disclaimer: we’ll be working in HTML and CSS, and you can view the page source to see how I did each of the examples, but I’m not trying to make the best HTML or CSS here. Instead, I’m trying to demonstrate how someone might try to build a UI element from scratch, in or out of a browser. (It’s up to you to judge how successful I am.)
Buttons are specifically UI elements that do things, so let’s add an action when you click it:
Perfect! We’re basically done, right?
(Note to voice control users: for this article I have specifically hidden the first several examples from readers/tools
so you don’t have to wade through iterations of the same boring thing. It’s just bringing up a dialog. Unfortunately, when we get to the point of talking about focused elements you’ll start hearing some incomplete descriptions.)
There are lots of ways to submit an action
If you’re reading this on a phone, you probably noticed the first wrong thing: I made it work if you
click
the button with a cursor, but not if you tap it with a finger. Let’s fix that:
Are we done? Well, no. Users sometimes misclick, and so OSs back to
System 1.0 for the Mac
(possibly even earlier) have a feature called
drag cancellation
or
pointer cancellation,
where if you drag away from the button before letting go of the click/tap, it doesn’t fire. This behavior can’t be provided with only one of “mouse down on my UI element” and “mouse up on my UI element”; you need both, or some higher-level operation.
1
To be fair I had to go a little out of my way to do this section, for demonstration purposes. Modern web browsers pack up everything we’ve been talking about in a single “click” event that handles both clicks and taps as well as drag-cancellation. But if you’re trying to make a button from scratch
outside
of a web browser, you might have made this mistake.
Keyboard navigation
Some people don’t use a mouse—whether to keep their hands efficiently on the keyboard, or because they don’t have the manual dexterity to use pointing devices, or both, or some other reason. To facilitate this, desktop OSs let you tab between all the interactable items in a window—not just form fields, but
everything
you might need to interact with. (These days it’s often tucked away in a setting called “Full keyboard access” or something, but sometimes even without hunting for that you might be able to get it to work with Option-Tab or Ctrl-Tab. Adding Shift goes backwards.) Our button should support this.
We also didn’t say anywhere to draw a ring when
focused
(in fact, maybe your web browser doesn’t, but mine does). Let’s fix that by adding some explicit style information:
Even here we’re still relying on the web browser to track this “focused” state for us; ultimately
someone
has to know what UI element is focused, and which the “next” and “previous” elements should be.
While we’re here, let’s fix one more display issue from the previous section: buttons should show some feedback when you start to press them, to go with the drag-cancellation feature and also so you can tell that the click happened at all. We’re again going to lean on the web browser for this
active
or
highlighted
state, but it really does make the experience much better.
Voice control
Some people don’t use a mouse
or
keyboard to control their computers. Instead, they use voice recognition programs that can translate commands into UI actions. For yet another time, the web browser assists us here: if you can manage to select one of the “focusable” button above, you can activate it.
However, voice control is often paired with screen reading: a navigation for people who can’t use a display (usually because of visual impairment, but I have on a handful of occasions used it to work with a computer I had no monitor for). There are a number of ways interactable UI elements show up there, but in this case let’s just see what happens when the user focuses the button (using keyboard navigation, or a scroll wheel, or something):
Oh dear. We used an icon with no
alt text
, so the user has no idea what this button does. This is more about
images
than about
buttons
specifically, but even with a text button you may still want your screen-reader label / “accessibility label” to be different from the displayed text, since the user may not have the same contextual information when they navigate to it.
Okay, fixed. Ish. Again, web browsers are helping out a lot here; every OS has a different accessibility interface, and every GUI toolkit has to plug into it. Or not, and frustrate users.
Key Equivalents
We’re finally approaching something that behaves like a button from 1990. It’s still not very pretty, but most of the changes we’ve been making have been to things other than appearance. Here’s one more: did you know many buttons on your system have keyboard shortcuts, just like menu items? On Windows these are usually Alt-something; on a Mac they’re Cmd-something like menu items. (Most of the time this only matters for dialog boxes; the rest of the time most buttons have equivalent menu items anyway.)
This is less common in web pages in general. It’s not even that common in general, beyond standard shortcuts of Return for default actions, Escape for Cancel, and maybe Cmd-Delete for “confirm remove”. But we’ll add it here anyway: if you’re a power user, this button can now be pressed with Ctrl-B, or perhaps Ctrl-Option-B, or maybe Alt-Shift-B, or… (
It depends on your browser and OS
, and the chart doesn’t even seem to be up to date.)
Consistency
Even with all this, we
still
don’t have something that behaves like a normal button. It doesn’t “highlight” when pressing Space, because I didn’t explicitly write code for it. VoiceOver, the macOS screenreader, calls it a “group” or “image” rather than a button. It doesn’t support a “disabled” state.
It turns out this has been the wrong approach from the start, at least for an application or web page. We don’t need to do any of this. We can start with a system-provided button:
And then customize it to do our own drawing.
It’s still a chunk of the work we had before, but
now
we and the web browser agree that it’s a button, and we get most of what we discussed for free. As well as anything else I forgot in making this blog post.
Conclusion
All that for a button. Lists, sliders, and tables are more complicated, and text editing is so complicated that even a number of apps that otherwise roll their own UI (usually games) still use the system text input.
The general moral here is that we, as an industry, have had
40 years
to establish what desktop GUIs are like, over 30 to establish what web pages are like, and nearly 20 to establish what mobile GUIs are like. Not every tradition is good, but if you’re a sighted mouse-user, you might not even know what you’re omitting when you diverge from conventions. Which translates to people being frustrated with your app or web page, or in the worst case not being able to use it at all.
If you’re an app developer, the lesson is simple: if you want to customize a UI element, start with a standard one and change how it draws, rather than building one up from scratch and trying to match the system in every little way. If you are a GUI library author, things might be trickier. You’re
trying
to rebuild things, and there’s a good chance you
do
know more than me about GUIs. But please at least consider if it’s going to work with a screenreader.
P.S. Let this not discourage
exploration,
taking apart existing UI elements and building new ones to see how they work. I just want people to stop thinking “works for me” is good enough.
Subscribe to comments on this entry
Some random database url, from its memory
![TCL 43PF650K 43-inch 4K Ultra HD, HDR TV, Smart LED Fire TV (Dolby Vision, Dolby Atmos, DTS, HDR 10, Alexa built-in, Airplay2, Miracast) [Energy Class F]](https://i.guim.co.uk/img/media/ff68166ed5579e79af50a09d08f5c98fb088c405/0_0_5000_4000/master/5000.jpg?width=620&dpr=2&s=none&crop=none)
![TCL 50C6KS-UK QD-Mini LED TV, 4K HDR Premium, Powered by Google TV (Dolby Vision & Atmos, Hands-Free Voice Control, Apple AirPlay 2 & Alexa，2025 New Model) [Energy Class F]](https://i.guim.co.uk/img/media/1a455d11e59d8ffcf814ad884723cf9a04d98946/0_0_5000_4000/master/5000.jpg?width=620&dpr=2&s=none&crop=none)
![Sony BRAVIA, KD-55X75WL, 55 Inch, LED, Smart TV, 4K HDR, Google TV, ECO PACK, BRAVIA CORE, Narrow Bezel Design [Energy Class F]](https://i.guim.co.uk/img/media/851e2aebf29887ce1e416c917da25056797ddfd3/0_0_5000_4000/master/5000.jpg?width=620&dpr=2&s=none&crop=none)
![TCL 55T8C-UK 55” 4K 144Hz Gaming QLED Onkyo 2.1 Dolby Atmos Google TV with Dolby Vision & HDR 10+ BBC iPlayer (2025 New Model) [Energy Class E]](https://i.guim.co.uk/img/media/036877593b7c6872c8aa99abceddec4a58ce1c5f/0_0_5000_4000/master/5000.jpg?width=620&dpr=2&s=none&crop=none)
![PHILIPS Ambilight 65PUS8200 4K LED Smart TV - 65 Inch Display with Pixel Precise, Ultra HD, Titan OS Platform and Dolby Atmos Sound, Works with Alexa and Google Voice Assistant [Energy Class F]](https://i.guim.co.uk/img/media/91d24091c2d68ff7b41082b533c4a1ad7a0d83b9/0_0_5000_4000/master/5000.jpg?width=620&dpr=2&s=none&crop=none)
![Philips Ambilight 65OLED760 4K OLED Smart TV - 65 Inch Display with P5 AI Perfect Picture Engine, Ultra HD, Titan OS, Dolby Vision and Atmos Sound, Works with Alexa and Google Voice Assistant [Energy Class F]](https://i.guim.co.uk/img/media/40ae5cfd9f8c6803ab01031fb966838781ad85b3/0_0_5000_4000/master/5000.jpg?width=620&dpr=2&s=none&crop=none)
, and it has to be at least double the rank times degree, and 3329 is literally the smallest prime that fits that bill))
. At this point, the attacker cannot really assume that they were observing a decapsulation failure anymore, as a whole range of other incredibly unlikely events, such as enough simultaneous bit flips due to cosmic radiation to evade error detection are far more likely. It is true that after the first decapsulation failure has been observed, the attacker has much more abilities to stack the deck in their favor, but to do so, you first need the first failure to occur, and there is not really any hope in doing so.
#/media/File:Baryon-decuplet.svg){kind=link}
